Product container including digital marking of an article for recycling

Abstract

To increase the proportion of plastic articles to be reused or recycled by providing a product container containing digital marking.SOLUTION: In a method, a plastic article such as a beverage bottle holds two separate digital watermarks coded using two separate signal protocols. A first print label watermark holds a retailing payload containing a global trade item number (GTIN), which is used by a POS scanner for settlement. A second plastic texture watermark holds a recycle payload containing data that identifies a composition of the plastic. A recycling device uses both of the two types of watermarks to identify the plastic composition of the article (for example, a related database is used to associate a GTIN with a plastic type), thus increasing the proportion of articles that are accurately identified for sorting and recycling.SELECTED DRAWING: Figure 8

Description

Related Application Data

[0001]

[0001] In the United States, this application is a partial continuation application of Patent Application No. 16 / 435,292 filed on June 7, 2019 (published as US Patent Application Publication No. 20190306385), and further claims priority to US Provisional Patent Application No. 62 / 818,051 filed on March 13, 2019, US Provisional Patent Application No. 62 / 830,318 filed on April 5, 2019, US Provisional Patent Application No. 62 / 836,326 filed on April 19, 2019, US Provisional Patent Application No. 62 / 845,230 filed on May 8, 2019, US Provisional Patent Application No. 62 / 854,754 filed on May 30, 2019, No. 62 / 923,274 filed on October 18, 2019, US Provisional Patent Application No. 62 / 956,493 filed on January 2, 2020, US Provisional Patent Application No. 62 / 967,557 filed on January 29, 2020, and US Provisional Patent Application No. 62 / 968,106 filed on January 30, 2020. Background

[0002]

[0002] There is an urgent need to increase the proportion of plastic articles that are reused or recycled.

[0003]

[0003] U.S. Patent Application Publication No. 20040156529, which is an applicant's document, teaches that the plastic surface of a 3D (three-dimensional) object can be textured by thermoplastic molding to form a machine-readable digital watermark that holds a multi-bit payload.

[0004]

[0004] U.S. Patent Application Publication No. 20040086151, which is an applicant's document, teaches how a 3D object with an electronic watermark can be produced using injection molding (e.g., using vacuum molding or pressure molding).

[0005] U.S. Patent Application Publication No. 20020099943, a document of the applicant, teaches that an object can hold two watermarks, one formed in the topology of the object's surface and the other formed by printing.

[0006] U.S. Patent Application Publication No. 20150016712, a document of the applicant, teaches that a 3D object can be identified using a watermark or image fingerprint data, and this identification data can be linked to the recycling information of the object. For example, the identification data can be linked to a recycling code indicating whether the object is formed of polyethylene terephthalate, high-density polyethylene, polyvinyl chloride, etc.

[0007] U.S. Patent Application Publication No. 20150302543, a document of the applicant, similarly teaches that the payload of a watermark formed on a plastic object can hold or be linked to the object's recycling code. Further, U.S. Patent Application Publication No. 20150302543 teaches that a waste sorting device equipped with a camera can sort an incoming material stream based on the decoded watermark data. U.S. Patent Application Publication No. 20180345323, a document by FiliGrade B.V., also discloses sensing recycling information from a watermarked plastic bottle and separating the waste stream based on the decoded information.

[0008] The present study by the applicant improves on the above technologies and provides many additional features and advantages. Introduction

[0009]

[0009] In one aspect, the present technology involves defining a data pattern that includes binary elements spaced at positions within a regular 2D (two-dimensional) position grid. The pattern defines a first fixed reference signal and a first variable data signal. The first reference signal facilitates the geometric alignment and extraction of the first variable data signal by a decoder presented with a camera-captured image showing the physical counterpart of the data pattern. This configuration further includes shaping a three-dimensional surface topology pattern of a mold according to a smoothed counterpart of the data pattern. The topology pattern includes peaks or depressions with smooth cross-sections to facilitate the release of the molded part from the mold.

[0010]

[0010] In a particular embodiment, the 2D position grid defines M candidate positions where the binary elements can be located. The binary elements are arranged at 25%, 20%, 10% or less of those M candidate positions.

[0011]

[0011] In a further particular embodiment, the 2D position grid consists of an N×N grid of positions, each position having a grayscale or floating-point value of the first reference signal corresponding to that position. The first variable data signal includes an M×M array of positions, where M < N, and each position in the M×M array of positions has a two-tone value of the variable data signal corresponding to that position. The M×M first variable data signal values are interpolated to generate an N×N array of interpolated values. By doing so, the first variable data signal is converted from a two-tone form to a grayscale or floating-point form. At each of the N×N positions, the corresponding values of the first reference signal and the interpolated first variable data signal are combined at a weighted ratio to generate an N×N combined array of values. A thresholding operation is applied to the N×N combined array of values to identify values, and then the positions corresponding to those extreme values are marked by binary elements in the N×N grid.

[0012]

[0012] In another specific embodiment, the 2D position grid defines M positions, and each of the M positions has a corresponding value of the first reference signal. Each value represents the relative darkness of the reference signal at that position. The first variable data signal includes binary symbols, and each of the binary symbols is associated with a corresponding position within the regular 2D position grid. To generate a data pattern, the values of the first data signal are sorted to generate a ranking of the N darkest positions. P of the darkest positions within this ranking of the N darkest positions are identified (the other Q positions are left as they are). Each of the P positions is marked by a binary element. Depending on whether the corresponding binary symbol of the variable data signal has a first value or a second value, each of the remaining Q positions is either marked (by the binary element) or not marked.

[0013]

[0013] A mold can be made using the configuration described above and can be used to mold a plastic container. In such a container, the molded plastic holds a first variable data signal that may be read by a suitable decoder such as a waste recycling facility.

[0014]

[0014] In some configurations, such a plastic container further has a label. The label can include a printed pattern that defines a second fixed reference signal and a second variable data signal. The second reference signal facilitates the geometric alignment and extraction of the second variable data signal by a decoder presented with an image of the camera capture showing the printed pattern. Usually, the second fixed reference signal is different from the first fixed reference signal and / or the second variable data signal is different from the first variable data signal.

[0015]

[0015] A recycling system for processing two different such plastic containers can process one based on a plastic texture pattern and the other based on a printed label pattern. That is, a computer, which is a processor of such a recycling system, geometrically aligns the first variable data signal of the first container using a first alignment signal (of the texture-processed plastic pattern), extracts the first variable data signal, and can sort the first container for recycling based on the extracted first variable data signal. Further, a second alignment signal (of the printed label pattern) is used to geometrically align the second variable data signal of the second container, the second variable data signal is extracted, and the second container can be sorted for recycling based on the extracted second variable data signal.

[0016]

[0016] In a further aspect, the plastic container is shaped to hold information. More particularly, the container is shaped to hold a texture pattern of elements spaced at positions within a regular 2D position grid. The pattern defines a first fixed reference signal and a first variable data signal. The first reference signal facilitates the geometric alignment and extraction of the first variable data signal by a decoder presented with a camera-captured image of the container. In such a container, the 2D position grid defines M candidate positions where elements can be located. The elements are arranged at 25%, 20%, 10%, or less of those M candidate positions.

[0017]

[0017] Typically, the remaining 75%, 80%, 90%, or more of the M candidate positions follow the nominal contour of the plastic container, for example remaining smooth and simply following the cylindrical shape of the bottle. That is, at the boundaries of the texture pattern, most of the surface area of the container remains unchanged.

[0018]

[0018] In another aspect, the plastic container holds both a plastic texture pattern and a printed label pattern. The plastic texture pattern includes elements spaced at positions within a regular 2D position grid. This plastic pattern defines a first fixed reference signal and a first variable data signal. The first reference signal facilitates the geometric alignment and extraction of the first variable data signal by a decoder presented with a camera-captured image of the container. Similarly, the printed label pattern includes elements spaced apart at positions within a regular 2D position grid. Again, the printed label pattern defines a second fixed reference signal and a second variable data signal. The second reference signal facilitates the geometric alignment and extraction of the second variable data signal by a decoder presented with a camera-captured image of the container. In such a configuration, the second fixed reference signal is different from the first fixed reference signal, and / or the second variable data signal is different from the first variable data signal.

[0019]

[0019] In a particular configuration, the second fixed reference signal is different from the first fixed reference signal. In another particular configuration, the second variable data signal is different from the first variable data signal.

[0020]

[0020] In yet another aspect, the plastic container holds a texture pattern including elements spaced at positions within a regular 2D position grid. The pattern defines a first fixed reference signal and a first variable data signal. The first reference signal facilitates the geometric alignment and extraction of the first variable data signal by a decoder presented with a camera-captured image of the container. In this configuration, the 2D position grid defines M candidate positions where the elements can be located. The elements are actually arranged at 20%, 20%, 10%, or less of those M candidate positions. Again, in most of the M candidate positions, the container nominally follows its contour as described above.

[0021]

[0021] A further aspect relates to a method of marking a container to hold a plurality of symbol payloads. The method includes the step of generating an encoded data pattern of the payload. The pattern includes elements spaced at positions within a regular 2D position grid. The pattern defines a fixed reference signal and a variable data signal. The reference signal facilitates the geometric alignment and extraction of the variable data signal by a decoder presented with a camera-captured image representing the physical counterpart of the data pattern.

[0022]

[0022] The method further includes the step of forming a physical counterpart of the pattern on the container by printing or texture processing. In such a configuration, the 2D position grid defines M positions, and each of the M positions is associated with a corresponding value of the reference signal. Each value represents the relative darkness of the reference signal at that position. The variable data signal includes binary symbols, and each of the binary symbols is associated with a corresponding position within the regular 2D position grid.

[0023]

[0023] More particularly, the step of generating the data pattern begins by sorting the values of the reference signal to generate a ranking of the N darkest positions. P of the darkest positions within this ranking of the N darkest positions are then identified (leaving Q other positions unchanged). Each of these P positions is marked with a binary element. Depending on whether the corresponding binary symbol of the variable data signal has a first value or a second value, the remaining Q positions are marked or not marked with binary elements.

[0024]

[0024] A further aspect of the present technology involves a recycling system comprising an optical reader and a plastic bottle. The plastic bottle has a label. A first digital pattern encoding a first identifier is applied in ink to the label. The plastic is texture - processed with a second digital pattern encoding a second different identifier. In this case, by decoding any of the identifiers by the optical reader, the recycling system can sort the bottles by plastic type.

[0025]

[0025] Yet another aspect of the present technology also relates to a recycling system comprising an optical reader and a plastic bottle. The bottle is at least partially wrapped with a sleeve. A mark applied in ink to the sleeve is printed beforehand, i.e., between flat configurations. The printed sleeve is then wrapped around the bottle and adhered to the bottle by heat - shrinkage. The mark applied in ink to the packaging sleeve includes a machine - readable code geometrically distorted by the heat - shrinkage and is still readable by the optical reader, controlling the sorting of the plastic bottle for recycling.

[0026]

[0026] Yet another aspect of the present technology relates to a point - of - sale (POS) system that may be used for checkout in a retail store. The POS system comprises an optical reader and a plastic bottle. The plastic bottle has a printed label with a first digital pattern encoding a first identifier applied in ink, and the plastic is texture - processed with a second digital pattern encoding a second identifier. The optical reader is configured to decode the first identifier but not the second identifier.

[0027]

[0027] A further aspect of the present technology relates to a bottle made of a plastic container that serves as a substrate for an ink - printed label. The ink - printed label includes a machine - readable code that enables sorting by the plastic type of the underlying plastic container.

[0028]

[0028] The above configuration and other configurations will be described in detail in the following detailed description with reference to the accompanying drawings.

Brief Description of the Drawings

[0029]

FIG. 1A - G

FIG. 1H - L

FIG. 1M - Q

FIG. 2A

FIG. 2B

FIG. 3A

FIG. 3B

FIG. 3C

FIG. 3D

FIG. 4

FIG. 4A

FIG. 5A

FIG. 5B

FIG. 5C

FIG. 6A

FIG. 6B

FIG. 6C

FIG. 7

FIG. 8

FIG. 9

FIG. 10

FIG. 11

FIG. 12

FIG. 13

FIG. 14

FIG. 15

FIG. 16

FIG. 17

FIG. 18

FIG. 19

FIG. 20

FIG. 21A

FIG. 21B

FIG. 21C

FIG. 21D

FIG. 22

FIG. 23

FIG. 24

FIG. 25

FIG. 26A

FIG. 26B

FIG. 27

FIG. 28

FIG. 29

FIG. 30

FIG. 31

FIG. 32A

FIG. 32B

FIG. 33A

FIG. 33B

FIG. 33C

FIG. 33D

FIG. 34

FIG. 35

FIG. 36A

FIG. 36B

FIG. 37

[0030]

[0030] For example, in order to sort waste streams, there is an increasing need for highly reliable identification of plastic articles.

[0031]

[0031] Since digital watermarks are applicable to various types and shapes of substances, they are beneficial for the above purposes. Furthermore, the watermark can spread over the container and / or its label so as to improve readability even when the object is damaged, soiled, or partially blocked.

[0032]

[0032] To identify the type of substance in each object and accordingly sort waste streams, the digital watermark provides a 2D optical code signal that enables machine vision in a waste sorting system. As will be described later, the coded signals imparted to the containers by 3D printed molds, laser machined molds, and etched molds are usable to sort the containers in various recycling environments.

[0033]

[0033] According to one aspect of the present technology, plastic articles are coded using two different watermarks. One watermark is printed (or printed on the article itself) on a label affixed to the article with normal ink, and the other watermark is formed by 3D texture processing of the plastic surface.

[0034]

[0034] The printed watermark is generally designed for use by a POS scanner originally, for example, to identify an article and check the price at checkout in a retail store, and holds or indicates a retail payload that includes or indicates product name, price, weight, expiration date, packing date, etc. The texture watermark is generally effective for recycling and includes or indicates a payload that includes, for example, data related to the plastic. Each watermark usually lacks some or all of the information held by the other watermark.

[0035]

[0035] In most embodiments, it is important that the above two watermarks (retail watermark and recycling watermark) use different signal protocols. The applicant has discovered that the normal retail POS scanner has a very short time interval for reading the retail watermark before the next frame of the image arrives for analysis. When the retail watermark and the recycling watermark are shown in the same image frame, they must be distinguishable at high speed; otherwise, the POS scanner may not succeed in decrypting the retail watermark before the next image frame arrives. A part of this specification teaches how to make the above two watermarks quickly distinguishable, so as not to waste the precious few milliseconds when the POS scanner tries to decrypt the recycling watermark, thereby helping to ensure a reliable retail settlement operation.

[0036]

[0036] The basic method of making the retail watermark and the recycling watermark quickly distinguishable is by using different signal protocols (including, for example, different reference signals, different encoding protocols, and / or different output formats). Such differences enable the POS scanner to recognize the retail watermark with high reliability, while the recycling system can recognize the recycling watermark with high reliability, without the risk of the POS scanner accidentally attempting to decrypt the payload from the recycling watermark and causing confusion.

[0037]

[0037] Despite the differences in the watermark signal protocols, it is also desirable that the recycling system be configured to have a watermark processing module that reads the retail watermark (and the recycling watermark) and recognizes the information available for plastic recycling purposes from the retail watermark (generally, by referring to a database that associates the retail watermark payload data with plastic information). Thereby, regardless of which watermark is read from the article by the recycling system, the system obtains the information for controlling the appropriate sorting of articles by plastic type.

[0038] As described above, the two watermark signal protocols can differ in a number of ways, such as including a reference signal and / or an applied coding algorithm. The reference signal for each watermark (which may be referred to as a calibration signal, synchronization signal, grid signal, or alignment signal) serves as a synchronization component that enables accurate extraction of the payload by enabling recognition of the geometric pose of the watermark shown within the captured image. An exemplary reference signal is a set of multiple peaks in the spatial frequency domain. The first of the two watermarks described above may include a first reference signal in the absence of the second watermark. (The latter watermark may include a different reference signal, e.g., peaks of different frequencies, peaks of different phases, and / or a different number of peaks.)

[0039]

[0039] The coding algorithm can differ in the process by which data is encoded and / or in the format in which the encoded data is represented. For example, the coding algorithm for a printed watermark may use a signal protocol where the resulting watermark format is a square block that holds data structured as a 128×128 array of element arrangements, having a side length of 0.85 inches and a resolution of 150 pixels per inch. In contrast, the signal protocol used by a texture watermark coding algorithm may produce a square block that has a different size (usually less than 0.85 inches on a side), a pixel resolution different from 150 pixels per inch, and / or holds data configured other than as a 128×128 array. The two different signal protocols used in the two watermarks may have different payload capacities. For example, one may have a variable message portion capable of holding 48 bits, while the other may have a variable message portion capable of holding exactly half or one-third of that payload capacity.

[0040]

[0040] The above two encoding algorithms may alternatively differ, if used, by the error correction encoding method used, the redundancy rate used, the number of bits in the signature list output by the error corrector, the CRC method used if any, the scrambling key used to scramble the signature list output from the error corrector to generate a scrambled signature list, the spreading key used to generate a number of randomized "chips" from each bit of the scrambled signature list, the scatter table data defining the spatial arrangement of each of those "chips" of the output watermark pattern, etc. The decoding algorithm may correspondingly differ.

[0041]

[0041] That one type of watermark (e.g., a recycling watermark) cannot be read by one type of watermark reader (e.g., a retail POS watermark reader) may be due to any of the above-described differences between watermarks regarding their geometric reference signals, output formats, signal protocols, encoding / decoding algorithms, etc.

[0042]

[0042] Each watermark payload typically includes fixed and variable message parts. The fixed part typically includes data identifying the signal protocol used. The variable message part generally includes a plurality of fields. For a printed retail watermark, one field typically holds a Global Trade Item Number (GTIN), and other fields may hold application business identifier codes (e.g., indicating weight, expiration date, etc.) as defined by GS1. Such application business identifier codes are not currently part of the GS1 standard, but plastic identification information may be held in a printed retail watermark in the form of such application business identifier codes.

[0043]

[0043] Some recycling systems use two watermark readers, where the first reader is configured to apply a first watermark reading algorithm (e.g., to read a retail watermark using a first signal protocol), and the second reader is configured to apply a different second reading algorithm (e.g., to read a recycling watermark using a second signal protocol). Each of such readers is unable to read the other type of watermark. Other recycling systems use a single reader configured to read both types of watermarks. Still other systems use a hybrid configuration where certain components are shared (e.g., performing a common FFT operation), and other components are dedicated to one type of watermark or the other type of watermark.

[0044]

[0044] To ensure a high-confidence reading of the watermark regardless of the position of the item in the waste stream, it is preferable for the watermark to be visible from multiple perspectives of the item. For example, the recycling texture watermark preferably is formed on several surfaces including the front and back of each item. Similarly, the retail printed watermark preferably is formed on the side surfaces of each item located opposite to each other, e.g., on labels on both the front and back.

[0045]

[0045] To most effectively apply the watermark reading operation, certain embodiments of the present technology examine image pixel blocks for clues indicating the presence of watermark data. Further watermark analysis is performed only on those image blocks in which such clues are found. Many such clues include detecting glare spots (regions of pixels having values exceeding respective thresholds), aggregates of spatial image frequencies corresponding to the watermark reference signal, classifier outputs indicating that the pixel block may represent a plastic article, classifier outputs indicating that the pixel block may not represent a conveyor belt, determining that the pixels from a majority of the sub-blocks of the block have an average value within 1, 2, 3, or 4 digital numbers of the histogram peak based on the previous image, detecting a signal associated with the marking of the conveyor belt, detecting a sesame seed marking, and various other techniques for distinguishing promising image blocks from others. Once a patch of a promising image is identified, it is typically analyzed to confirm the presence of both the retail watermark and the recycling watermark.

[0046]

[0046] In some embodiments, if one image block is found to be promising, that determination also triggers the examination of multiple neighboring image blocks. The incoming image frame may first be divided into blocks at a first density (e.g., having a first pixel spacing or overlap). When a promising block is found, other blocks that are, for example, arranged away from the promising block at a smaller pixel spacing or more significantly overlapping are examined at a higher density. In connection therewith, if a promising block is found in one frame, based on the conveyor speed and the frame capture speed, watermark data in subsequent frames corresponding to the predicted movement of the article shown in the promising block can be sought and analyzed.

[0047]

[0047] Due to image analysis for verifying both types of watermarks, two instances of the same type of object (e.g., two identical 12-ounce Pepsi bottles) may be separated based on the reading results of two different watermarks. That is, the plastic type of the first bottle may be identified by its printed watermark, and the plastic type of the second bottle may be identified by its texture watermark. Regardless of the reading results of the different watermarks, both can be sent to the same recycling destination. Specific configuration

[0048]

[0048] Electronic watermark information is read from image data indicating plastic objects in the waste stream. This information can indicate the type of plastic (e.g., polyethylene terephthalate, high-density polyethylene, low-density polyethylene, polypropylene, polycarbonate, etc.), or can hold other information useful in recycling. The diverters and other mechanisms of the automatic sorting system are controlled according to such watermark information to send the plastic objects to the appropriate sorting destination for recycling or reuse.

[0049]

[0049] An electronic watermark (hereinafter referred to as a watermark) is printed on the packaging of many products and generally serves to encode a Global Trade Item Number or GTIN (resembling the widely used 1D (one-dimensional) UPC barcode) in a visually unobtrusive manner. POS scanners in retail stores can detect and decode the watermark data, use it to look up the identification and price of the product, and add them to the shopper's receipt. The watermark data is usually organized in the form of a plurality of square blocks that are edge-matched and tiled redundantly over part or all of the printing on the product. Since the watermark data is spatially dispersed, the POS scanner can read the data from different views of the product (e.g., from the front and back views of a beverage bottle).

[0050]

[0050] Most commonly, the watermark data is hidden as a slight change in the luminance and / or chrominance of the pixels that include the artwork of the package. In some cases, the watermark may have the form of an unobtrusive pattern of dots that may extend across, for example, an adhesive label affixed to a plastic fresh food container.

[0051]

[0051] To keep costs down, POS scanners typically use a simple processor. Such POS scanners generally operate mainly in finding and decoding 1D barcodes for most of their operation, and watermark reading may be a supplementary part. A POS scanner that captures 30 frames per second has only 33 milliseconds to process each frame, and most of that time is used for barcode reading. Only a few milliseconds are available for watermark reading.

[0052]

[0052] Watermark reading has two parts: discovery of the watermark and decoding of that watermark.

[0053]

[0053] In an exemplary embodiment, discovery of the watermark (which may also be referred to as watermark detection) involves analyzing a frame of the captured image to identify the location of a known reference signal. This reference signal can be a set of characteristics of peaks in the 2D Fourier amplitude region (known as the spatial frequency region). In the spatial (pixel) region, such a reference signal has the form of an aggregate of 2D sine waves of different spatial frequencies across the watermark block. FIG. 5A is a diagram showing an exemplary reference signal in the Fourier amplitude region, and FIG. 6A is a diagram showing such the same reference signal in the spatial region. To ensure continuity along the edges of the watermark block, it is desirable for the frequencies to be integer values. When an object with such a known reference signal is shown in the captured image, that particular representation clearly shows the scaling, rotation, and translation of the watermark payload data that is also present in that image.

[0054]

[0054] This watermark payload data is generally encoded by watermark elements ("waxels") that occupy positions in a 2D array typically consisting of 128×128 elements. This array may extend over an area, for example, having a side of 0.85 or 1.7 inches, depending on whether the watermark is formed at a resolution of 150 or 75 waxels per inch (WPI). Such blocks, along with a reference signal, are tiled in a repeating array across the entire package.

[0055]

[0055] Once the scaling, rotation, and translation of the watermark are known from the analysis of the reference signal as shown in the captured image, the watermark payload is decodable. The decoder samples the captured image at positions corresponding to the originally encoded 128×128 array of data and uses those sample values when decoding the original watermark payload. (For example, convolutional encoding is commonly used because a 48-bit payload is converted into a string of 1024 data, and then the 1024 data string is redundantly distributed over 16,384 positions of a 128×128 element watermark block).

[0056]

[0056] The above and other details of the watermarking technology, including details from the patent documents described herein, are well known to those skilled in the art.

[0057]

[0057] In a particular embodiment of this technology, the plastic container holds two watermarks, one watermark formed by label printing and the second watermark formed by texturing the plastic surface, such as by molding. (This label can include a substrate that is affixed to the printed container or can include printing applied directly to the container).

[0058]

[0058] Plastics can be molded by a variety of techniques including blow molding, injection molding, rotational molding, compression molding, and thermoforming. In each of such processes, the heated plastic resin is molded according to a mold. By molding the surface of the mold with a pattern, an interpattern is formed on the surface of the resulting plastic product. When the pattern of the mold is adjusted to have the shape of a watermark pattern (having a change in luminance / chrominance that is converted into a change in the height, depth, angle, reflectance, or local curvature of the mold), the resulting plastic product can have a surface texture corresponding to the watermark. Such a pattern on the plastic surface can be sensed by the optical methods detailed below.

[0059]

[0059] Figures 1A through 1Q are diagrams of representative surface textures.

[0060]

[0060] Most of the textures shown are illustrated as 2D cross-sections of 3D surfaces, showing modulation in only one dimension. For clarity of the figures, the textures are shown on flat surfaces. Of course, most plastic containers are curved in at least one dimension.

[0061]

[0061] Also, for clarity, most of the figures in FIGS. 1A - 1Q show marks having only two values. Specific examples include the "sparse" dot marks detailed in U.S. Patent Application Publication Nos. 20170024840, 20190139176, and International Publication No. 2019 / 165364, which are patent documents. Other binary marks include line drawing patterns such as Voronoi, Delaunay, traveling salesman, and brick, as detailed in International Publication No. 2019 / 113471, which is a published application, and U.S. Patent Application Publication No. 20190378235, and are shown in FIGS. 3A, 3B, 3C, and 3D, respectively. (The Voronoi pattern is realized by forming a mesh of grunts (here triangles) having vertices at positions corresponding to a sparse array of dots. The Delaunay pattern is the dual of the Voronoi pattern where the grunts have the shape of polygons with different numbers of sides. The traveling salesman pattern is realized by defining a traveling salesman path that visits each dot in a sparse array of dots. The brick pattern is realized by placing vertical line segments at the dot positions of an array of sparse dots and forming a horizontal line at an intermediate position to define a rectangular grunt).

[0062]

[0062] FIG. 1A is labeled to show the binary state of each pixel. "1" is represented here by a relatively protruding portion, and "0" is represented by the nominal reference height of the plastic surface (which may appear relatively recessed compared to the illustrated "1" state). The nominal height of the plastic surface is indicated by the dashed line.

[0063]

[0063] FIG. 1B is similar to FIG. 1A, but the corners having acute angles are rounded (e.g., by low - pass filtering) to assist in the release of the molded plastic from the mold. Rounding in this way can be used in any embodiment to flatten the acute angles.

[0064]

[0064] FIG. 1C shows an embodiment where the transition between states has a slope, and consecutive "1" values include a slight return to the nominal surface level on the opposite side. FIG. 1D is a variation of FIG. 1C. Making the transition sloped can further assist in demolding and optical detection in response to illumination.

[0065]

[0065] In some embodiments, the raised protrusions in FIGS. 1B, 1C, and 1D can each be a rounded protrusion having only a slight flat portion at the peak or no flat portion at all.

[0066]

[0066] FIG. 1E shows that the "1" state can be characterized by a surface non-parallel to the nominal plane characterizing the "0" state. FIG. 1F is a variation of FIG. 1E, showing that the "1" state does not need to be high and can simply be sloped.

[0067]

[0067] FIG. 1G shows a configuration where the "1" state and the "0" state are each sloped in different directions with respect to the nominal surface of the plastic. (The slope directions may be 180 degrees apart as shown, or may differ by only 90 degrees.) Such slopes cause light to be preferentially reflected in different directions, making the mark more prominent to the watermark reader.

[0068]

[0068] Although FIGS. 1A - 1G have been described and illustrated as including portions raised above the nominal surface, it will be recognized that such encoding can (perhaps more generally) equally well include portions recessed below the nominal surface. (Watermark encoding / reading typically does not depend on the polarity of up or down.) One example is in the formation of the lines used in the patterns of FIGS. 3A - 3D and FIG. 7. Combinations of raised and recessed portions can of course also be used.

[0069]

[0069] FIG. 1H is a diagram showing a beneficial scattering phenomenon associated with a curved surface. In most arrangements of the camera and the light source with respect to the curved surface, the incident light (indicated by the large arrow) is reflected from the surface at various angles (indicated by the small arrows), and a part of it is reflected toward the camera, generating a bright glint. In contrast, for a flat surface, almost all of the incident illumination is reflected in a single direction, often away from the camera. As a result, the flat surface usually appears dark to the camera, while the curved surface is usually characterized by a bright glint. (Occasionally, when the flat surface is reflected toward the camera, an "inversion" occurs, and the flat surface becomes brighter than the curved surface.)

[0070]

[0070] FIG. 1I is a diagram showing the scattering and focusing phenomena associated with a surface having both convex and concave portions. The convex portion acts as described above, dispersing the incident light over a wide angular range. In contrast, the curved concave portion acts as a focusing element. Compared to the dispersion caused by the convex portion, the focusing caused by the concave portion reflects a large amount of light in the overall direction of the light source. Assuming that the camera is relatively close to the light source (e.g., within 10 degrees as viewed from the illumination surface), the concave portion appears brighter than the convex portion in the image captured by the camera. (The dashed line indicates the nominal plastic surface.)

[0071]

[0071] (It will be understood that, in this specification and elsewhere, the light source and the camera can be arranged other than as shown in the figures. The light source and the camera can be arranged close to each other (e.g., within a single-digit angle), or can be arranged further apart. The light can illuminate the surface directly downward (90° incidence), or obliquely with an incident angle of 80°, 60°, 30°, or less.)

[0072]

[0072] The configuration of FIG. 1I extends to three surface features: convex, concave, and flat, as shown in FIG. 1J. The flat surface reflects as described in connection with FIG. 1H. Thus, the configuration of FIG. 1J is an example of a surface that can be used for ternary signal encoding that reflects intermediate amounts of light (i.e., the glint caused by the convex portion), large amounts of light (i.e., the focused reflection caused by the concave portion), and extreme values (usually dark but sometimes bright, caused by the flat portion) in various ways.

[0073]

[0073] FIG. 1J further shows another aspect of surface shaping that can be used in any embodiment, where the protrusions need not be dimensionally similar to the depressions. In this example, the raised convex portion is higher than the depth of the concave portion. In relation, the raised convex portion has a smaller radius of curvature than the sunken concave portion. The reverse can also be the case.

[0074]

[0074] FIG. 1K is a diagram showing that even if modulating the surface height has an effect on the reflected light pattern, it is possible that the effect is not very significant. Often important is the Transition , that is, the Differential coefficient of the function that defines the surface height.

[0075]

[0075] In FIG. 1K, the light incident at point B on the plastic surface is reflected in the same direction and with the same luminance as the light incident at point D. The two surfaces are parallel but at different heights. In contrast, the light incident at point A is reflected with a different luminance and direction than the light incident at point C. At point A, the differential coefficient of the surface is negative (the height decreases with rightward movement). At point C, the differential coefficient of the surface is positive. Assuming that the camera is placed near the light source, almost no incident light is reflected back from point A towards the camera, while almost all of the incident light is reflected back from point C towards the camera. The recess having a flat bottom shown in the cross-sectional view of FIG. 1K thus has three reflection zones: one zone along the flat bottom, one zone with a negative differential coefficient, and one zone with a positive differential coefficient. When the light source is arranged as shown (and the camera is also placed nearby), the glint of reflection is sensed by the camera from the latter zone, and there is no reflection from the first two zones.

[0076]

[0076] As shown in FIG. 1L, a similar phenomenon similarly occurs from a raised convex portion having a flat top. The leftmost side of the convex portion has a positive differential coefficient and reflects the glint of light back towards the camera. The flat top does not reflect light back towards the camera, nor does the rightmost side of the convex portion (due to its negative reflection coefficient).

[0077]

[0077] (It will be understood that the above-described results depend on the light source being arranged on the left side of the molded surface. If the light source is arranged on the right side, some of the results will be reversed.)

[0078]

[0078] Naturally, the curved shape can be adjusted to optimize performance, for example, such that reflection in a specific direction with respect to the incident light source is preferred.

[0079]

[0079] Generally speaking, in contrast to a basic shape having a linear edge, convex portions and dots with a circular planar appearance are preferred because they tend to reflect light in more directions.

[0080] Notwithstanding the foregoing, another beneficial approach to surface texture processing is to use retroreflective features, such as indentations in the shape of 3D corner reflectors, in plastics. A 3D corner reflector has the property that light is reflected back to its source over a wide range of incident angles. FIG. 1M shows this property in two dimensions, and the property extends further into three dimensions.

[0081]

[0081] Corner-shaped indentations can be formed in the plastic surface where the watermark should be visible brightly (e.g., in the "1" state) and not formed at positions where the watermark should be visible dimly (e.g., in the "0" state). The deepest "point" of the indentation can have a rounded shape, and the important thing is that most of the surface area is perpendicular to each other.

[0082]

[0082] FIG. 1N shows a part of a square 128×128 pixel watermark block, showing 16 pixels. Some are indented by retroreflective 3D corner reflectors according to the encoded data (e.g., representing a "1" signal), and others remain flat (e.g., representing a "0" signal). FIG. 10 shows a part using triangular pixels and arranged in a hexagonal array. In this case as well, some are indented by retroreflective 3D corner reflectors according to the encoded data, and others are not indented.

[0083]

[0083] In a variant embodiment, the two states of the signal tiles are not represented by corner reflectors or planes. Instead, corner reflectors are formed at the positions of each voxel. The two states are distinguished by machining three orthogonal surfaces (facets) that provide a recessed reflector. The "1" state is characterized by a smooth surface that reflects light with relatively little scattering. The "0" state is characterized by a textured surface (e.g., a rough or matte surface) that reflects light with relatively more scattering. The first type of reflector is fabricated to be efficient, and the second type of reflector is fabricated to be inefficient. Further, for a human observer, the two characteristics are substantially indistinguishable and give a texture that appears uniform on the surface. FIG. 1P is a diagram showing such a configuration (having a rough corner reflector indicated by the gray voxels).

[0084]

[0084] When square voxels are formed in plastic at a density of 75 per inch, each voxel extends over an area with a side of 0.0133 inches. Thus, the recess of each corner reflector has a width below this value. As the density of the voxels increases, the dimensions decrease.

[0085]

[0085] Of course, in a retroreflective configuration, the camera should be placed as close as possible to the light source, so that the angular distance between the two (when viewed from the conveyor) is desirably less than 10 degrees.

[0086]

[0086] The texturing of some surfaces in the configuration of FIG. 1P can be used in other configurations, including the other configurations shown. That is, some regions of the plastic surface may be roughened or given a matte finish to increase scattering, while other regions can remain smooth to increase specular reflection. In some embodiments, the plastic surface does not have depressions or protrusions for encoding watermark data. Rather, encoding can be achieved by globally performing scattering texture processing on various regions without disturbing the nominal shape of the article.

[0087]

[0087] FIG. 1Q shows, in a 3D view, a part of a plane marked with three sparse dots having the form of depressions on the surface here.

[0088]

[0088] Many of the surfaces shown can encode two signal states, some can encode three states, but more generally, M - value encoding can be used.

[0089]

[0089] FIG. 2A shows another form of ternary encoding where the signals consist of elements of - 1, 0, and 1. "-1" is represented by an inclination in one direction, "1" is represented by an inclination in another direction, and "0" is represented by an intermediate inclination between the other two. By including, for example, depressions from a nominal plastic surface that reflect the protrusions of FIGS. 1A - 1F, many other such forms can of course be conceived. Quaternary encoding can be realized using four different surface inclinations at consecutive 90 - degree angles. Quaternary encoding can be realized by using four inclinations of orthogonal encoding and a fifth state which is the nominal surface of the plastic. By expanding the set of inclinations, higher - order M - value encoding can be realized.

[0090]

[0090] (As in other figures, the surface of FIG. 2A can be roughened, for example, by a matte finish or a translucent finish, to scatter some light in a direction that is not at all a specular direction).

[0091]

[0091] In addition to M-value encoding, the present technology is also suitable for use with so-called "continuous tone" watermarks that have various intermediate states between two extreme values. In many cases, the reference signal has continuous values (or values in a number of quantized steps), and by combining such a reference signal with an M-value payload pattern representation, a continuous tone watermark is generated. The continuous values of the voxels of such a mark can be represented by the degree of local surface height or slope. Such a mark is conceptually illustrated by FIG. 2B.

[0092]

[0092] The above-described pattern suggests that the shaping extends to both sides of the plastic medium, for example, top and bottom (or the inside and outside of the bottle). In some cases, the shaping is done on only one side (e.g., the outer surface), and the other side is smooth.

[0093]

[0093] Plastic texture processing using a shaped mold is the most common, but other forming approaches can also be used. Laser or chemical etching is an example, resulting in a surface marked with depressions corresponding to amplitude or slope in the spatial variation of the watermark signal. (Laser etching is very suitable for serialization where each instance of an article is encoded differently).

[0094]

[0094] In some embodiments, the plastic surface is locally treated to achieve a matte or translucent finish rather than a gloss finish. In such cases, the watermark itself can be formed as a pattern consisting of matte and gloss voxels. The matte texture is achieved, for example, by molding or surface finishing to achieve a certain surface roughness such as a vertical variation on the order of 1 / 10 or 1 / 2 micrometer or more.

[0095]

[0095] In an exemplary embodiment, the plastic watermark is adjusted to avoid confusion by a POS scanner. As described above, such a scanner has limited processing capabilities and limited time for extracting watermark identifiers. It is possible to take several measures to help prevent the POS scanner from attempting to read the plastic watermark, i.e., an operation that wastes precious processing time, and the system may also prevent decoding of the product GTIN from the product label shown in the same frame.

[0096]

[0096] One measure to help avoid confusion by a POS scanner is to use a plastic watermark reference signal that is unlikely to be mistaken for the reference signal used in a printed label watermark. Such a reference signal can be generated randomly from a plurality of candidate signals (e.g., by selecting a set of random peak positions in the spatial frequency domain and assigning a random phase to each), and can be experimentally generated by testing each candidate to evaluate the likelihood that a POS watermark reader will mistake such a signal for a printed label watermark reference signal. Thereby, the candidate reference signal with the lowest likelihood of confusion is used.

[0097]

[0097] Another approach is similar but involves mathematically calculating the theoretical amount of confusion (correlation) showing the similarity between the random reference signals of different candidates and the printed label reference signal, and selecting the candidate with the lowest correlation.

[0098]

[0098] The applicant prefers the first approach because, as the wise man says, although there is no difference between theory and practice in theory, there is actually a difference in practice.

[0099]

[0099] The process for discovering candidate reference signals for plastic watermarks can be facilitated by imposing different constraints in the signal generation or selection process. One is that it is desirable that the peaks in the plastic reference signal should not be the same as any of the peaks in the printed label reference signal. Any randomly generated candidate plastic reference signal having such an attribute may be discarded.

[0100]

[0100] FIG. 4 is a diagram showing the peaks of the printed label watermark reference signal in the 2D Fourier amplitude domain. It is desirable that the reference signal for the plastic watermark does not have common peak positions.

[0101]

[0101] In connection with this, in the printed label reference signal, each frequency peak exists on different radial lines from the origin. A few peaks are shown in the enlarged view of FIG. 4A. It is desirable that none of the peaks in the plastic reference signal are arranged on any of those radial lines. (Depending on the scale at which the watermarked object is viewed, the peaks of the reference signal move concentrically towards the origin, move away from the origin, proceed along those radial lines, and there is a risk of confusion if both reference signals have peaks on the same radial line).

[0102]

[0102] In such a configuration, it may be desirable that none of the peaks in the plastic reference signal are on the vertical axis 31 or the horizontal axis 32 of the spatial frequency plane. Many other features of the captured image may have signal energy focused along those axes, and thus peaks along such axes are best avoided.

[0103]

[0103] The reference signal for the printed label is quadrant symmetric and mirror imaged about the vertical and horizontal frequency axes, and such a configuration may be used for the plastic reference signal for reasons of detector efficiency. However, this is not essential, and a reference signal for a plastic watermark that does not exhibit this attribute may be less likely to cause confusion.

[0104]

[0104] Peaks along the vertical and horizontal axes are best avoided, while peaks for the plastic reference signal are generally desirably arranged on radial lines at various angles. In each quadrant of the quadrant-symmetric reference signal, 1 / 4 to 1 / 3 of the peaks may be present on different radial lines within 30 degrees of the horizontal axis, 1 / 4 to 1 / 3 may be present on different radial lines within 30 degrees of the vertical axis, and 1 / 3 to 1 / 2 may be present on different radial lines between those two ranges.

[0105]

[0105] It is similarly desirable for the peaks for the plastic reference signal to differ in distance from the origin. Since resizing may move low-frequency points to positions where the watermark reading software does not look for peaks (resulting in a blank area at the center of FIG. 4), low-frequency points (e.g., less than 20 or 25 cycles per block) are undesirable and are exchanged for high frequencies (e.g., greater than 50 or 60 cycles per block). However, to ensure a nearly uniform distribution, as in the above paragraph, in the middle donut band (shown by the dashed line in FIG. 4), a spatial budget for assigning peaks is available.

[0106]

[0106] Another measure to help avoid confusion by the POS scanner is to use a reference signal in the plastic watermark that has fewer peaks than the reference signal in the printed label watermark. The fewer the number of peaks, the lower the likelihood of being mistaken for the peaks of the printed label watermark.

[0107]

[0107] The resulting advantage is that for a plastic watermark reference signal with fewer peaks, since the available signal energy budget is dispersed among fewer features, more energy can be used to encode each of the peaks. A plastic reference signal consisting of a small number of strong peaks is less likely to cause confusion as a result than a reference signal consisting of more weak peaks.

[0108]

[0108] A further defense against confusion between print watermarks and plastic watermarks is to form the marks at different scales. As described above, print watermarks are generally formed at 75 or 150 voxels per inch (i.e., watermark blocks of 1.7 or 0.85 square inches). Plastic watermarks may be formed at different resolutions, such as 200, 250, or 300 voxels per inch (i.e., 0.64, 0.51, and 0.43 square inches). Thus, since the obvious edge distortion of the watermark pattern due to curvature decreases as the watermark block size decreases, doing so helps detection from a curved container surface.

[0109]

[0109] One algorithm for generating candidate plastic reference signals is to take the Fourier amplitude plot of the label reference signals that must avoid confusion and add two circumcircles (such as those shown in FIG. 4) that provide the circular space where all points should exist. Then, as in FIG. 4A, radial lines extending from the center of the plot through each label reference signal peak are added to the outer circle. Finally, the circular space is divided into triangles using the label reference signal peaks as vertices to provide the largest triangle that does not enclose any other peaks. Then, the points in the annulus farthest from the nearest straight lines (i.e., the radial lines, the triangulation lines, and the horizontal and vertical axes) are identified and added to a set of candidate points. Repeat until the desired number of points is identified.

[0110]

[0110] To each candidate plastic reference signal, different random distortions such as tilt, rotation, and magnification / minification as well as additive Gaussian noise are applied, and by determining how often the reference signal detection stage of the POS watermark reader mistakes the distorted signal for the reference signal of the label watermark, different candidate plastic reference signals can be examined for possible confusion with the label reference signal. After each candidate reference signal has been tested with hundreds of different distortions, usually one candidate signal emerges as being better than the others. (This signal may be examined in the spatial domain by a human reviewer to confirm that it does not have subjectively undesirable attributes, but such a review may also be omitted.)

[0111]

[0111] Some candidate plastic watermark reference signals are shown in FIGS. 5A, 5B, and 5C by Fourier amplitude plots. FIGS. 6A, 6B, and 6C are diagrams showing their corresponding spatial domain representations.

[0112]

[0112] Confusion with the printed label watermark reference signal tends to decrease with the "flatness" of the spatial domain representation of the plastic reference signal. Thus, according to another aspect of the present technology, each candidate reference signal for the plastic watermark is varied by trying different phase assignments for different peaks in the Fourier amplitude plot in order to identify a set of phase assignments that minimizes the standard deviation of the pixels in the spatial domain representation. This is a task well suited for computer automation where hundreds of thousands or millions of different sets of phase assignments are tried to find a set that produces a spatial domain pattern with the minimum standard deviation.

[0113]

[0113] For example, using the method detailed above, the reference signal patterns of FIGS. 5A-5C (and FIGS. 6A-6C) were experimentally generated. By checking the correlation with the printed label watermark reference pattern, over the entire range of affine transformation, i.e., in the range of magnification / reduction between 0.5 and 2.0 in 0.02 increments, and in the rotation in the range between -90 degrees and +90 degrees in 1-degree increments, and in each pixel of possible translations, when a confusion test with the printed label watermark reference pattern was performed, 0.2> r >-0.2 (0.1> r >-0.1 in some cases) a very small correlation degree with a maximum value r was discovered.

[0114]

[0114] The correlation between two images f 1 and f 2, both having a size of P×P pixels, can be expressed as follows.

Equation

[0115]

[0115] It will be understood that the detailed reference signal consists of sine curves of equal amplitude. In other embodiments, the sine curves can have different amplitudes, generating a more prominent "woven" pattern for their spatial region representation.

[0116]

[0116] As described above, the reference signal is one of the two elements of the watermark signal, and the other is the encoded representation of the payload message. This representation can be generated by convolutional encoding the symbols of the payload message to produce a very long string of symbols (e.g., 1024 bits), sometimes called a signature. The signature may be randomized by performing an exclusive OR operation using a scramble key of the same length. The chips redundantly representing each of the scrambled signature bits are randomly spatially distributed, for example, at the positions of a square array of 128×128 (16,384) elements to form a signature array.

[0117]

[0117] By scaling the spatial region reference signal so that its average pixel value is 128 and adding or subtracting an offset value that depends on whether the chip assigned to that position is 1 or 0 to each component pixel value, a continuous-tone watermark can be generated.

[0118]

[0118] A sparse watermark can be generated by various methods involving generating output patterns of dots that are generally spaced apart. Some methods are detailed in the literature cited above and are described in the section titled "Examination of Example Watermark Creation Methods" below.

[0119]

[0119] As described above and illustrated in FIGS. 3A to 3D, the sparse pattern can be converted into various two-tone line-based representations. A further such pattern called a "snake" is shown in FIG. 7. The snake is generated from a continuous-tone watermark by the following algorithm executed using Adobe Photoshop (registered trademark) and Illustrator.

[0120] That is, a 300 DPI monochrome (grayscale) block with a white background is filled with 50% gray and encoded with a continuous-tone watermark (reference signal and payload signal). This image is then adjusted on Photoshop using the controls Adjustment→Exposure:Default / Exposure:1.05 / Offset:-0.075 / Gamma Correction:0.3. Next, filtering is applied using the Photoshop controls Filter→Blur→Gaussian Blur:Radius:3 pixels, followed by Filter→Stylize→Wind:Method=Wind / Direction:Right or Left (either one is fine). After that, the image is binarized at the threshold by the Photoshop control Image→Adjustment→Threshold:Threshold Level:140 (+ / -5). The resulting file is saved and then opened in Adobe Illustrator. The image immediately after editing is selected from inside the layer. After the "Image Trace" button in the main upper frame of the Illustrator user interface is clicked and a preview is displayed, the Image Trace Panel next to the Drop Down frame showing "Default" is clicked. From the top line of the icons, the Outline button is clicked. After the preview is displayed, the "Expand" button next to the Drop Down frame showing "Tracing Result" is clicked. This provides a UI that allows the size of the pattern strokes to be made thicker or thinner. Some bolding is applied to generate a pattern like the pattern in FIG. 7.

[0121]

[0121] It will be appreciated that such a pattern consists of a plurality of curved segments (many segments being complexly curved, i.e., having multiple changes in orientation along their length) where such segments are dispersed throughout the region, some segments crossing others, and other segments being independent without crossing others.

[0122]

[0122] To describe a larger system, a recycling apparatus according to one embodiment of the present technology uses one or more cameras and a light source to capture an image showing a plastic container with an applied watermark advancing on a conveyor in a waste stream. Depending on the embodiment, the conveyor region imaged by the camera system (i.e., its field of view) may be as small as about 2 inches by 3 inches, or as large as about 20 inches by 30 inches, or more, basically depending on the sensor resolution of the camera and the focal length of the lens. In some embodiments, multiple imaging systems are used to capture images collectively aligned in the width direction of the conveyor. (The conveyor may have a width of up to 5 feet or 2 meters in a mass feed system. A single feed system where articles are supplied to the conveyor one at a time has a narrower width, for example, having a width of 12 inches or 50 cm. A conveyor speed of 1 to 5 meters per second is common.)

[0123]

[0123] FIG. 8 shows a simple configuration in which the camera and the light source are arranged at substantially the same location, that is, the illumination is applied from a position less than 10 degrees away from the projection of the camera's optical axis onto the conveyor of the waste stream (i.e., the camera object). In another configuration, the light source is arranged to irradiate the camera object obliquely, that is, the light source is directed in a direction more than 50 degrees away from the direction of the camera's lens axis as shown in FIG. 9. In yet another configuration (FIG. 10), opposing illumination is used. That is, the axis of the light source has a direction more than 140 degrees away from the direction of the camera lens. In the latter configuration, surface texture processing can generate local shadowing processing for the plastic surface. For example, each plastic protrusion blocks light, and adjacent areas with relatively lower luminance than the area where light enters are imaged.

[0124]

[0124] The positions of the cameras and light sources in FIGS. 8 - 10 may be interchanged. In other embodiments, multiple light sources can be used. Naturally, the exposure interval should be short enough to avoid motion blur. A stroboscopically illuminated light source is useful for avoiding blur. The light source can be placed close to the conveyor to about the same size as the article so that the article can pass under it, or it can be placed at a greater distance, for example, 2 or 4 feet apart.

[0125]

[0125] FIG. 11 is a diagram showing a configuration in which light sources of two different colors, namely red and blue, irradiate a camera object from the opposite side of the camera at an oblique angle (>50 degrees) in this example. The green light source is arranged at the same location as the camera. The camera in FIG. 11 is an RGB camera and includes a 2D CMOS sensor covered with a Bayer pattern color filter. Its raw output includes pixels processed by a red filter, pixels processed by a green filter, and pixels processed by a blue filter. Three different monochrome (grayscale) images are thus formed from the corresponding pixels in the sensor array, one showing the waste stream in the red part of the visible light spectrum, one showing the waste stream in the blue part of the spectrum, and one showing the waste stream in the green part of the spectrum.

[0126]

[0126] The configuration of FIG. 11 shows light sources arranged along the moving direction of the conveyor (waste stream). In an alternative embodiment, the light sources are arranged across rather than in line with the moving direction of the conveyor. In yet another embodiment, a first pair of red / blue light sources is arranged along the moving direction of the conveyor (as shown), and a second pair is arranged across the moving direction. Those pairs of light sources are activated for alternate frames of image capture by the camera (so that, for example, frame capture may be performed at 60 or 150 frames per second). One frame is irradiated by the red / blue light sources arranged along the moving direction, the next frame is irradiated by the red / blue light sources arranged across, and so on. Each frame is irradiated by the green light source.

[0127]

[0127] Each of the resulting image frames is analyzed to determine the watermark data, examining both the printed label watermark and the plastic watermark. In some embodiments, a fourth image frame is generated by calculating the difference between the red pixel values and the blue pixel values of each Bayer cell. The resulting difference values are divided by two and added to the offset value of 128 to ensure that the elements of the difference image are in the range of 0 to 255. This difference image is also processed to decode any existing printed label watermark or plastic watermark. Such a configuration is shown in FIG. 13.

[0128]

[0128] Eight separate watermark reading systems are shown in FIG. 13, although some portions of the image processing may be integrated and the results may be shared between the label reading watermark system and the plastic reading watermark system. For example, each red frame of data may be subjected to a fast Fourier transform by a common FFT stage, and the results may be used for both label reading and plastic reading (synchronization). However, if the resolutions of the two watermarks are different (e.g., 150 WPI and 250 WPI), a completely separate processing path may be preferred.

[0129]

[0129] In a particular embodiment, images are captured using an f / 8 lens with an exposure interval of 20 microseconds. The imaging distance is set so that each captured pixel corresponds to an area of approximately 1 / 150 inch of the focusing band located 3 inches above the conveyor. Thereby, each pixel corresponds to a single waxel at 150 WPI. The camera gain (or the distance from the light source to the conveyor) is adjusted so that a pure white article on the conveyor is depicted with an 8-bit pixel value of 250 when captured.

[0130]

[0130] The effective dynamic range of the imaging system can be expanded by using different illumination bands. The normal illumination band can be irradiated as described above, so that a white object realizes a camera pixel value of 250. The adjacent high-illumination band can be irradiated at twice or more its illumination. By doing so, bright areas become overexposed, but dark articles can thus be resolved with better luminance gradation (i.e., contrast enhancement). For example, the formation of a watermark pattern of a dark printed label that may appear as a pixel value in the range of 5 to 10 under the former luminance condition can appear with an expanded range such as 10 to 20 (or 50 to 100 may also be acceptable) under the latter irradiation.

[0131]

[0131] In a specific embodiment, such illumination change is a design parameter of the lens in a single light source. For example, a linear array of LEDs may be provided with a linear lens that projects a variable luminance pattern with a high-luminance band at the center and normal luminance bands adjacent on either side. Since the conveyor moves the article through the projected light, each point of the article first passes through a normal luminance band, then through the high-luminance band, and then through another normal luminance band. Depending on the conveyor speed, frame rate, and irradiation area, each point of the article may be imaged once, twice, or more times when passing through each of the above luminance bands.

[0132]

[0132] In another configuration, in order to provide a similar effect for the high-luminance light band and the low-luminance light band, two or more different light sources can be used.

[0133]

[0133] In yet another configuration shown in FIG. 12, a linear light source 120 (shown in a side view) designed to output a substantially uniform luminance over the entire irradiation area is inclined with respect to the conveyor 122, whereby the path lengths of the light to different areas of the belt are different. In such a case, a gradation effect is produced by the attenuation of the illumination with distance, and the area 124 of the conveyor closest to the light is irradiated with high luminance, and the more distant area 126 is irradiated with gradually decreasing luminance.

[0134]

[0134] In certain embodiments, the captured image frames span both the more brightly illuminated regions and the more dimly illuminated regions of the belt. In a first single frame, the bright regions of the article are overexposed, while the dark regions have contrast enhancement. In another single frame, the bright regions are properly exposed, while the dark regions are relatively underexposed. Since the overexposed regions do not have pixel value changes that can serve as a clue for the selection of patches for analysis, the decoder tends to ignore the overexposed regions, and thus such patches are not analyzed. Similarly, due to the lack of pixel changes, the decoder also tends to ignore regions that are too dark. Thus, in a series of frames showing a single article passing through variable illumination, regions that are darker in color from one frame are analyzed (when more bright illumination is done), and regions that are darker in color from another frame tend not to be analyzed (when more dim illumination is done). Similarly, regions that are lighter in color from one frame tend to be analyzed (when more dim illumination is done), and regions that are lighter in color from another frame tend not to be analyzed (when more bright illumination is done).

[0135]

[0135] In another configuration, red light sources and white light sources are used. The red light source(s) and the white light source(s) can irradiate a common region, or can irradiate overlapping or adjacent regions. All of such irradiated regions can be within the field of view of a common imaging camera.

[0136]

[0136] In yet another configuration, polarization is used for illumination. Additionally or alternatively, one or more polarization filters can be used in the image sensor to attenuate light of orthogonal polarization.

[0137]

[0137] In many applications, glare, i.e., specular reflection of light from a surface, is an obstacle. In certain embodiments of the present technology, in contrast, such specular reflection can be important when transmitting watermark information as a signal. Instead of removing glare with a filter, a polarizing filter can be used to enhance the signal-retaining glare.

[0138]

[0138] Some embodiments of the present technology use a novel image sensor having a polarizing filter array. One example is Sony's Polarsens image sensor. The pixel array is covered by a spatially corresponding polarizer array consisting of four polarizers at different angles (90°, 45°, 135°, and 0°). An image frame consisting only of data from the 90° polarizing sensor is analyzable for watermark data. The same is true for each of the other three polarization states. Further, differences such as between the 90° "image" and the 45° "image" can be calculated, and such difference images are likewise analyzable to obtain watermark data.

[0139]

[0139] This Sony sensor can be used in various configurations. IMX250MZR is an example. IMX250MZR is a monochrome CMOS sensor having 2464 × 2056 pixels. A color CMOS sensor is Sony's IMX250MYR.

[0140]

[0140] Since the sensitivity of human vision is particularly sharp in the green spectrum, when the goal is an imperceptible state, it is unlikely that digital data will be encoded in the green channel. Better is a camera optimized to sense that the digital data is using wavelengths away from green, such as wavelengths of blue and red (which may extend into ultraviolet and infrared).

[0141]

[0141] One sensor optimized for detecting electronic watermarks at visible wavelengths other than green is detailed in applicant's U.S. Patent No. 10,455,112. One particular embodiment detailed in that patent uses a color filter array in which there are three photo cells with magenta filtering for each green-filtered photo cell, rather than a monochrome sensor.

[0142]

[0142] Once the plastic article is identified, it can be sent from the conveyor to an appropriate collection location or to a further conveyor by known means such as an electromagnetic plunger, a stepper motor control arm, a forced air jet, etc. Exemplary separation and sorting mechanisms are known to those skilled in the art from the patent publications of, for example, U.S. Patent No. 5,209,355, U.S. Patent No. 5,485,964, U.S. Patent No. 5,615,778, U.S. Patent Application Publication No. 20040044436, U.S. Patent Application Publication No. 20070158245, U.S. Patent Application Publication No. 20080257793, U.S. Patent Application Publication No. 20090152173, U.S. Patent Application Publication No. 20100282646, U.S. Patent Application Publication No. 20120168354, and U.S. Patent Application Publication No. 20170225199. These mechanisms are here referred to as "sorting diverters", or simply "diverters" for short, and their operation is controlled according to the type of plastic identified.

[0143]

[0143] Figure 14 is a diagram showing a part of the relevant data in particular detail.

[0144]

[0144] In the illustrated embodiment, each plastic watermark holds a 32-bit payload. This payload can be divided into various fields. One field identifies the type of plastic by class (e.g., ABS, EPS, HDLPE, HDPE, HIPS, LDPE, PA, PC, PC / ABS, PE, PET, PETG, PLA, PMMA, POM, PP, PPO, PS, PVC, etc.). Another field identifies the subtype of the plastic, for example, by its average molecular weight, solution viscosity value, or recommended solvent, or by whether the plastic was used as a food container or a non-food container. A third field identifies the color of the plastic. (The color may be optically perceivable. However, plastic consumer packaging increasingly includes printed shrink sleeves on the container that hide the color.) A fourth field identifies the day the plastic was manufactured, for example, by year and month. A fifth field identifies the country of manufacture. A sixth field identifies the manufacturing company. Of course, more or fewer fields can be used. Additional fields include whether the packaged item is food (or non-food), whether it is multilayer (or single-layer), and whether it can be composted (or only recycled). Some fields hold indicator values or flag (yes / no) values. Optionally, each indicator value can be decomposed into a value (or range of values) by referring to a character text, date sequence, or data structure such as a table or database.

[0145]

[0145] In an exemplary embodiment, the sorting diverter responds to the first three fields of the data and operates to separate plastics by type, subtype, and color. All of the decoded watermark data is logged to provide statistics regarding the waste stream being processed.

[0146]

[0146] The printed label payload typically holds a long payload, e.g., 48 or 96 bits. Its content can vary by item, but each typically starts with a GTIN, followed by one or more application business identifier key-value pairs (e.g., indicating expiration date, lot code, item weight, etc.). In some configurations, none of the payloads represent the type of plastic used in the item container.

[0147]

[0147] To determine the plastic type, a data structure 121 such as a table or database can be used. The data structure 121 serves to associate the GTIN of the item with corresponding information regarding the plastic used for the item container. That is, this data structure is queried using the GTIN identifier decoded from the printed label watermark payload, so that the system accesses pre-stored data to identify the plastic type, subtype, and color (if available) of the product having that GTIN. This plastic material information is supplied to the logic circuit that controls the sorting diverter so as to be made using data from the plastic watermark.

[0148]

[0148] From the above, it will be recognized that the technical problem of the prior art was to ensure a highly reliable reading of the GTIN label watermark of the product packaging presented to the POS scanner within limited time and processing constraints of the environment as described above. The technical effect of the detailed configuration above is that such packaging enables the retention of a second watermark, promoting recycling without impairing the highly reliable reading of the GTIN label watermark in the POS scanner due to the difference in the signal protocols used for the two watermarks.

[0149]

[0149] A further technical problem was to ensure a highly reliable optical reading of the watermark data from the articles in the fast-moving waste stream. In some embodiments, the reliability is improved by the imaging device used to capture the representation of the articles in the waste stream. In some embodiments, the reliability is improved by the shape of the texture marks applied to the surface of the plastic containers in the waste stream.

[0150]

[0150] It will be appreciated that the present technology is applicable in waste sorting systems of the types sold by Pellenc ST, MSS Inc., Bulk Handling Systems, National Recovery Technologies LLC, Rofin Australia PTY, Ltd., Green Machine Sales LLC, EagleVizion, BT-Wolfgang Binder GmbH, RTT Steinert GmbH, S+S Separation and Sorting Technology GmbH, and Tomra Systems ASA. The optical sorting used in such machines (e.g., based on near-infrared spectroscopy or visible spectroscopy, based on the different absorption spectra of different plastics, etc.) can be replaced by the present technology, or the present technology can be used in combination with other methods thereof. Block analysis

[0151]

[0151] In an exemplary embodiment, the conveyor belt is covered by an array of cameras, each of which provides an image frame at a rate of 150 per second. Each frame is 1280×1024 pixels in size and covers a field of view of approximately 8 inches by 6 inches of the conveyor belt. The analysis blocks are arranged over the entire captured image, and each block is analyzed to find a watermark clue such as a watermark reference signal. If a watermark reference signal is found, it is used to identify the posture of the watermarked object on the conveyor belt (using, for example, the techniques detailed in U.S. Patent Nos. 9,959,587 and 10,242,434). Using the posture information, the image is resampled in the area where the reference signal was detected, and after the waxel data is extracted, it is provided to a decoder that attempts to extract the watermark payload.

[0152]

[0152] This specification commonly refers to processing blocks or patches of images that are 128×128 pixels (or waxels) in size, but the applicant has found that the above detailed configuration is often better realized by processing smaller sets of data such as 96×96, 88×88, 80×80, 64×64, etc. (Due to the curvature and fragmentation of the articles seen in the waste stream, there are not many planar surfaces. However, geometric synchronization usually proceeds on the premise of planarity. This is why processing small patches of an image can produce excellent results, that is, the non-planar effects of physical distortion are minimized thereby). Therefore, the reader should understand that the reference to 128×128 in relation to the watermark reading operation is merely exemplary, and smaller data sets are contemplated and often preferred. (In contrast, watermark encoding may still be performed based on a 128×128 block size, but the decoder can extract the watermark payload from the analysis of smaller image blocks. Alternatively, the encoding can also proceed based on smaller blocks).

[0153]

[0153] The analysis blocks arranged across each image frame for watermark reading may be arranged uniformly or randomly spaced apart, they may be butted at their edges and tiled, or they may overlap (for example each block may overlap the next adjacent block by 20% to 80%). Figure 15 is a diagram showing an exemplary block pattern, where a 1280 x 1024 image frame is analyzed using 96 x 96 pixel blocks, and each block overlaps the adjacent block by 25%. The tiling pattern shows some blocks darker to obscure the boundaries of the individual blocks.

[0154]

[0154] In some embodiments, if a watermark reference signal or other clue (such as those detailed below) is found in one of the analysis blocks, a higher density cluster of analysis blocks is examined in its vicinity to find the reference signal, and if successful, the payload data is sought and analyzed. Figure 16 is a diagram showing an example. The original block at the center is shown with a thick line. The other blocks are arranged around it with 75% overlap (the block positions analyzed in the original block position array are omitted). Again, for clarity, some of the blocks are shown with thick dashed lines. Figure 17 is a diagram showing the region of blocks arranged at a higher density in the context of the frame of Figure 15, placed at the location where the watermark reference signal or other clue was found among the initially inspected blocks.

[0155]

[0155] In some recycling systems, the conveyor belt is empty in places and there are no articles in some parts of the camera view. Clues regarding the presence or absence of such empty spots are detectable and allow processing resources to be applied to more promising images. Similarly, the watermark processing of the captured image may be triggered only if a clue indicating that plastic may be present (or that something other than the conveyor belt is indicated) is found by a rapid evaluation of the image.

[0156]

[0156] Plastic is often characterized by areas of specular reflection or glare where the plastic surface specularly reflects incident illumination towards the camera. This glare is perceivable and can serve as a cue for activating (triggering) the watermark process. For example, multiple blocks of an input sequence of image frames (e.g., 150 frames per second) can each be analyzed to find 2×2 pixel regions where the pixel intensity is in the upper 5%, 10%, or 20% of the output range of the sensor (or within a similar percentile of previously sensed pixels from a previous block indicating the area of the conveyor belt). Frames meeting this criterion are analyzed to find watermark data. (Since plastic can extend well beyond such points, it is desirable to also analyze portions of the image other than those near the glare.)

[0157]

[0157] In a particular embodiment, no portion of the frame is processed until a glare pixel is detected. When this event occurs, the analysis of the entire frame is not triggered. Instead, a 7×7 array of pixel blocks overlapping based on the glare location is placed, and each of those blocks is analyzed to confirm the presence of a watermark reference signal. Those blocks may overlap by 50% or more of their width, i.e., more than normal block overlap. FIG. 18 is a diagram showing an example where the blocks overlap by 75% of their width. The glare location is identified by a '+' mark in the middle of the densely overlapping blocks. Again, since the boundaries of the constituent blocks are not distinct, some blocks are identified by particularly thick dashed lines.

[0158]

[0158] Additionally, or alternatively, metrics other than glare are used to determine whether the image may be subject to watermark processing.

[0159]

[0159] One method, called the block trigger method, provides a cue to help distinguish between empty and non-empty portions of the conveyor belt based on a comparison of input pixel values against a previous criterion.

[0160]

[0160] A specific one-block trigger algorithm compiles a histogram of pixel values selected from sub-blocks of an analysis block (such as one of the blocks indicated in bold in FIG. 15) for a number of captured image frames. Each block may be 96×96 pixels (when the pixel scale = 1). The block is logically divided into a 4×4 array of sub-blocks, each with a side of 24 pixels (i.e., 16 sub-blocks per block, each containing 24 2 or 576 pixels). The values from 25 randomly selected but static pixels from each sub-block are averaged together to calculate one average pixel value per sub-block (i.e., a value between 0 and 255 in 8-bit grayscale). Such new sub-block average pixel values are generated for each frame.

[0161]

[0161] The 256 of their average pixel values for a specific sub-block are finally compiled into a histogram (i.e., for 256 frames). These values show a sharp peak corresponding to the average pixel value of the empty conveyor belt at the belt position corresponding to that specific sub-block (and having that specific illumination).

[0162]

[0162] When a new frame is captured, the values are recalculated for the 16 sub-blocks within the block. Each value is judged in light of the histogram of that block. If the new value falls within some number of digital counts (e.g., 1, 2, 3, or 4) of the pixel values when the histogram shows a sharp peak, it is counted as one vote in favor of the conclusion that the sub-block image shows an empty belt. The 16 votes obtained for the 16 sub-blocks of that block are recorded. If a vote threshold (e.g., 11 out of 16 votes) concludes that the sub-block image shows an empty belt, then that block is concluded to show an empty belt. In such a case, the analysis of that block is skipped. Otherwise, that block is analyzed to find the watermark data.

[0163]

[0163] This process is executed for every frame for all the blocks within the camera view (e.g., all the blocks shown in FIG. 15).

[0164]

[0164] (When the image is captured at a high resolution, i.e., a resolution exceeding 1 pixel per voxel, the 25 values from each sub-block can be determined by sub-sampling, for example, by averaging the values of 4 or 9 of the smaller pixels in the vicinity of the 25 static positions. Alternatively, the value of a single smallest pixel closest to each of the 25 static positions can be used).

[0165]

[0165] FIG. 19 is a diagram showing an exemplary histogram for an exemplary sub-block after 219 frames have been processed. The x-axis shows the average pixel values for different frames calculated for that sub-block. The y-axis shows the number of frames (``bin count'') having different average pixel values for that sub-block. The histogram reaches a peak at 20. In the relevant part, the relevant bin counts for different average pixel values are as follows.

Table 1

[0166]

[0166] When the next frame of the image is captured, if the average value calculated from the 25 static pixel positions of this sub-block is equal to 18, 19, 20, 21, or 22 (i.e., peak value 20 + / - 2), then that sub-block is determined to indicate an empty conveyor belt. If 10 out of the 16 sub-blocks of that block match, this is used as a clue that the block indicates an empty conveyor belt. As a result, no watermarking process is performed on that block. Or, if no such match is obtained, this serves as a clue that a plastic article may be indicated by that block, and further processing is triggered.

[0167]

[0167] To ensure space for more data, each histogram is kept in a new state by periodically discarding data. For example, when the frame counter associated with the histogram indicates that 256 frames have been processed, the 256 average values for that sub-block are read into the histogram, and the contents of the histogram are decimated by half to become 128 values. This can be achieved by using the bin count for each average pixel value in the histogram and dividing by 2 (rounding down). Thereby, this frame counter is reset, i.e., to 128 frames. Thereafter, the count of the average pixel values from the next 128 frames is recorded in the histogram, and at the same time, the decimation is repeated. With this configuration, past pixel values rapidly lose importance, and the histogram can reflect the most recent data.

[0168]

[0168] By using the newly captured frame, this block triggering method provides clues regarding whether to trigger a watermark reading operation for each block position in that frame. The average pixel value derived from the newly captured frame serves to update the corresponding histogram so that it can be used when evaluating blocks in subsequent image frames.

[0169]

[0169] (It will be recognized that watermark reading can be triggered when sufficient sub - blocks have average pixel values above (brighter) and / or below (darker) the peak of the histogram (i.e., the most recent belt luminance). That is, a plastic object may contain regions of dark pixels as well as light pixels. Both help to signal the execution decision of the trigger).

[0170]

[0170] The related art proceeds similarly, but is based on the statistics of color distribution rather than luminance distribution.

[0171]

[0171] In certain embodiments of the block - trigger algorithm, when further analysis of a block is triggered, the analysis detects a watermark reference signal from that block (or decodes the watermark payload), and then the average sub - block pixel value data of that block is added to its respective histogram Cannot be added (or, if added earlier, such counts are removed). Thus, the histogram is not corrupted by data from an image known to represent an empty conveyor belt. Not present This is not impaired by data from an image known to represent an empty conveyor belt.

[0172]

[0172] Many recycling systems set a limit, i.e., a processing budget, on the number of image blocks that can be analyzed during the processing of each frame. For example, the limit is 200 blocks. Due to the analysis of blocks closely arranged around a block where a watermark reference signal or other clue is detected (e.g., as described above in connection with FIGS. 15 - 17), a fraction of this total, such as 50 - 75 blocks, may be reserved as a reserve. If the close arrangement of additional analysis blocks due to clues detected from several blocks exceeds the 200 - block limit, additional blocks can be assigned according to the value of the clue (e.g., the intensity of the detected watermark reference signal), and the blocks judged to be the most promising receive the largest allocation of neighboring analysis blocks.

[0173]

[0173] If not all 200 blocks are analyzed for each frame, the power consumption decreases, and the heat output from the computer processor(s) (which often has to be offset by air conditioning) also decreases.

[0174]

[0174] In the modified block trigger method, a total processing budget (e.g., 150 block analyses) is used per frame. That is, 11 out of 16 sub - blocks (or more generally, K out of L sub - blocks) have average pixel values (25, or more generally more than the selected pixel of N) within the peaks of the few digital counts of their respective histograms, so some block analyses are triggered as described above. Then, any remaining analysis blocks are assigned according to the difference between the above - mentioned average sub - block pixel value and the peak of each histogram summed over all 16 sub - blocks of the block. Until the total budget of 150 analysis blocks is reached, those blocks with the smallest aggregated difference are triggered for watermark analysis.

[0175]

[0175] When the loading amount of the conveyor belt exceeds any threshold, some systems enable the above - modified method automatically or manually. In extreme cases, by covering the object, the conveyor belt may be almost entirely hidden for a period of hundreds of consecutive frames. In this case, the prominent peak associated with the background belt luminance does not appear in the histogram. However, still, each histogram has a peak at some position. To allocate the total budget of analysis blocks to the image frames, the modified block trigger method uses the above - described procedure. Substantially, this results in a mostly random selection of blocks for analysis. However, since the belt is clearly congested with objects, this is not an unreasonable block - selection strategy.

[0176]

[0176] Other clues for recognizing an image that may be subject to watermark processing use image statistics such as the average value, standard deviation, and / or variance.

[0177]

[0177] FIG. 20 is a diagram showing an image frame field of view covering the entire conveyor by a large rectangle. The dashed rectangle shows the arrangement of the linear LED light sources covering the entire conveyor as well. Due to the orientation of the light source or its lens (or reflector), the illumination has a spatial luminance characteristic as shown in the chart immediately to the left, shows the maximum luminance in the area of the lamp, rapidly attenuates in one direction, and attenuates gently in the other (shown on a scale of 0 to 100).

[0178]

[0178] Arranged across the traveling direction of the belt are a plurality of strips of image blocks each having a size of 128×128 pixels. Only two columns are shown in the figure, but similar strips extend across the entire image frame. Due to the irradiation characteristics of the lamp, adjacent strips may be irradiated differently.

[0179]

[0179] (In FIG. 20, the blocks are not adjacent and do not overlap, but this is for clarity of illustration. In reality, the blocks generally are adjacent or overlap).

[0180]

[0180] A metric is derived from each of the above blocks in each strip and used as a clue to determine the similarity of the image blocks to an image showing an empty belt.

[0181]

[0181] In an exemplary embodiment, a feature quantity f is calculated for each block and used to identify an area that can be the subject of further watermark analysis. Generally, f(·) is a function of each pixel in the block. While the belt is moving but empty, for example when the sorting system is first powered on, an initialization phase is executed. The feature quantity f is calculated for each block over a plurality of frames, and the quantity is grouped for each strip. For example, for each strip, the population mean and standard deviation are estimated from the corresponding group of sample feature quantities obtained over a plurality of frames.

[0182]

[0182] Subsequently, when a new image frame is captured, feature amounts are calculated for each block of the new frame. For each feature amount, a normalized feature amount is calculated using a pre-estimated average value and a standard deviation value for a strip including the block for which the feature amount was calculated. The normalized feature amount is calculated as follows.

Equation

[0183]

[0183] To identify the block most likely to indicate the object with the watermark, the normalized feature metric values are sorted from the maximum to the minimum. Thereby, a priority order for watermark reading is constructed. If the system processing budget allows analysis of 150 blocks per frame, data from the 150 blocks with the highest first metric is sent to the watermark processing.

[0184]

[0184] As a result, various basic features f with different effectiveness in watermark processing can be used. Exemplary embodiments include block average and block standard deviation.

[0185]

[0185] The effectiveness of specific features that distinguish an image block including only belt pixels from other image blocks depends on the conditional distribution of the features for those two types of blocks. For some non-belt image blocks, the feature f AOn the other hand, it may not be useful for distinguishing the block from the belt block, while the feature f B may be effective when distinguishing the block. For other non-belt blocks, the situation may be reversed, and f A may be a suitable feature. This leads to a further type of embodiment that utilizes multiple features.

[0186]

[0186] In an embodiment of multiple features, in the initialization phase, a separate set of mean and standard deviation estimations is calculated for each feature, and a normalized feature value corresponding to the feature of each block of the new image frame is calculated. The normalized feature values are combined into a single metric value using a combination function. The resulting combined metric value is sorted, and the sorted list of metric values forms a watermarking priority list.

[0187]

[0187] An example of the combination function is the sum of the normalized feature values. Other embodiments include more complex functions derived from, for example, statistical analysis of the normalized feature distributions for two types of belt blocks and non-belt blocks. Polynomials for combining feature values are used in some embodiments. To utilize the fact that different image strips may result in different normalized feature distributions, further embodiments may have different combination functions for each image strip.

[0188]

[0188] It will be recognized that the above-described configuration always makes full use of the entire system processing budget. If the system budget allows for the analysis of 150 blocks per frame, 150 blocks are analyzed per frame. (As described above, there may also be a reserve budget for additional blocks that can be allocated based on the processing results from the first 150 blocks.)

[0189]

[0189] The above configuration may be determined, for example, as a type of classifier that classifies whether the image may show a belt (or glare from plastic). Many other types of classifiers can be used to provide a controllable cue for watermarking processing.

[0190]

[0190] Such an alternative trains a neural network to classify an image frame as indicating one type or the other by training the network with (a) only the belt, or (b) a large corpus of labeled images showing various images other than only the belt. Suitable networks and training methods are described in further detail in the patent documents U.S. Patent Application Publication Nos. 2016 / 0063359, 2017 / 0243085, and 2019 / 0019050, and further in Krizhevsky et al., "Imagenet classification with deep convolutional neural networks, Advances in Neural Information Processing Systems 2012", pages 1097-1105. Further information is described in co-pending application No. 15 / 726,290, filed October 5, 2017.

[0191]

[0191] If it is determined that an image or image patch may show only the conveyor belt, no further analysis of such an image is performed. (Instead, for example, freed processor cycles can be applied to further process other images by analyzing additional blocks, such as by attempting encoding using different candidate affine transforms).

[0192]

[0192] Another configuration classifies an image showing an empty conveyor belt and provides a clue to distinguish such an image from other images by sensing characteristic belt marks. For example, conveyor belts generally have scratches, dirt, and other stripe patterns that are extended in the belt axis direction (belt direction). Most of such marks detected in the image have a low frequency. The captured image can be low-pass filtered to reduce high-frequency noise, and the resulting image can then be analyzed (e.g., by the Canny algorithm or the Sobel algorithm) to evaluate the edge strength in various directions.

[0193]

[0193] In a particular embodiment, a 128×128 block of the image is low-pass filtered and then inspected using a Canny edge detector to evaluate the strength of the gradient along the belt direction and the strength of the gradient across the belt direction (e.g., by summing the gradient values in the vertical and horizontal image directions). If the patch shows the belt, the sum of the former gradients will be significantly larger than the sum of the latter gradients. A logistic regressor is trained to respond to the two strength values by classifying the image patch as either showing the belt or not. If it shows the belt, no further analysis is performed on such a block, and if it does not show the belt, further steganography analysis of the block can be initiated.

[0194]

[0194] In other embodiments, a ratio of the two summed gradient amounts is calculated and this value is compared to a threshold to determine whether the image block shows a conveyor belt.

[0195]

[0195] The sorter may be equipped with a laser system that detects the presence of an object. For example, the laser beam may be swept across the entire range of the belt using a rotating mirror device and may serve to trigger detection along the elements of a linear photodetector array on the other side. As long as each of the photodetectors detects the laser beam, it can be seen that the conveyor belt in the swept area is empty. Such checks can be used to suppress the analysis of the captured image blocks.

[0196]

[0196] Another type of cue that can trigger further watermark analysis is based on a salt-and-pepper (or pepper-and-salt) pattern metric that indicates the likelihood that the block shows a sparse dot watermark. An exemplary algorithm for calculating such a metric will now be described.

[0197]

[0197] Since the input image block is downsampled as necessary, scale = 1. That is, each voxel is represented by a size of 1 pixel. We are looking for dark pixels in a light-colored area, i.e., pixel outliers. However, the image contrast can be either high or low, and the illuminance can vary within the block. The calculated metric is preferably robust to such variable factors. For this purpose, we examine the neighborhood of the pixel and calculate a measure that also takes into account the sensor acquisition noise.

[0198]

[0198] The acquisition noise present in the captured image is a function of the pixel value, and the higher the pixel value, the higher the noise value. A polynomial function or a reference table can give the noise standard deviation value for each pixel value between 0 and 255. To identify pixels with outliers (e.g., relatively darkest pixels in a relatively light-colored pixel range), the measure, i.e., sigma, is calculated for the pixel neighborhood area around the target pixel with value x at coordinates (i,j) by the following formula.

Equation

[0199]

[0199] For pixels darker than the average of the neighborhood region, the above sigma value is negative. To be determined as a sparse marked dot, a darkness threshold is set such that the sigma value satisfies, for example, σ ij <-3. A filtered image block is generated that includes only pixels whose corresponding sigma value satisfies the above test. All other pixels are removed (for example, set to white with pixel value = 255).

[0200]

[0200] The procedures up to this point identify the darkest dots, including pixels that form the darkest edges (for example, including dark text). To focus the watermark extraction process only on the sparse marked dots, it is necessary to filter and remove pixels that are not separated from other pixels (in the form of morphological filtering). Various techniques can be used for this operation. A simple technique is to visit each dark pixel, examine the 5×5 pixel region centered at its image position, and count the number of dark pixels in that region. If there are three or more dark pixels in the 5×5 region, the central pixel is removed (for example, changed to white). The resulting processed block then consists entirely of separated dark dots.

[0201]

[0201] Finally, the processed block is inspected to count the number of dark dots remaining within the block boundary. This number serves as a metric indicating the likelihood that the block contains a sparse watermark.

[0202]

[0202] This metric may be compared to a threshold value K (e.g., K = 500) experimentally determined to identify frames that may indicate sparse watermark data. Alternatively, the blocks of the frame can be ranked based on their associated sparse metrics, and the blocks with the highest sparse metrics can then be further analyzed to extract watermark data up to the limit of the block processing budget.

[0203]

[0203] Various simplifications and modifications can be made to this particular algorithm. For example, a simpler procedure is to only identify a set of the darkest pixels within the block. (e.g., it is possible to identify the darkest 10% or 30% of the pixels within the block). This procedure then applies the morphological filtering and counting operations described above to obtain a sparse metric.

[0204]

[0204] Another variant configuration differentiates potential sparse dots from others based on learning obtained from previous image frames.

[0205]

[0205] The exemplary learning process analyzes pixel values from a sampling of a series of past frames, e.g., 10 blocks per frame. Each block is divided into sub - blocks, e.g., 5×8 pixels. For each analyzed sub - block, both the intermediate pixel value and the minimum pixel value are determined.

[0206]

[0206] In some sub - blocks, the minimum pixel value is that of a dark sparse dot. Since other sub - blocks do not contain sparse dots, the minimum pixel value is simply the minimum value pixel among the image content that is not a sparse dot (for example, a background image, an article marked with a continuous - tone watermark instead of a sparse watermark, etc.).

[0207]

[0207] From these collected statistical data, identify the maximum value of the minimum pixel value (the "max - minimum") for each associated sub - block average value. For example, considering all sub - blocks having an intermediate pixel value of 151, the maximum value of the minimum pixel value occurring in the analyzed series of frames is 145. Pixel values greater than 145 are very likely not to be sparse marked dots in sub - blocks having an average value of 151. This value and other similarly identified values can thus help set a threshold for distinguishing possible sparse marked dots (outliers) from impossible sparse dots.

[0208]

[0208] In a particular embodiment, all such sets of points are described by a best - fit line characterized by, for example, an inclination and an offset amount as follows. τ 外れ値 =0.96*μ - 1.6 where μ is the intermediate pixel value of the sub - block.

[0209]

[0209] Subsequently, when a new frame of the image is received, the average value of each 5×8 pixel sub-block is calculated, and the corresponding outlier threshold is determined by the equation of the best-fit line. Pixels of the sub-block having a value smaller than this threshold are identified as candidate sparse dots. (For example, if a sub-block has an intermediate pixel value of 82, all pixels of the sub-block having a pixel value of 77 or less are treated as candidate sparse dots). Subsequently, as described above, a morphological filter is applied to the entire block to discard the connected dots, and then the number of dots remaining in the block is counted to generate a sparsity metric. As described above, this metric can be tested against a threshold for identifying blocks that can be watermarked. Alternatively, all blocks of the frame can be ranked according to this metric and selected for processing based thereon until the block processing budget is reached.

[0210]

[0210] In an alternative embodiment, the process detailed above can be modified to generate a metric based on bright pixels in dark regions (i.e., salt and pepper). One such modification is to simply invert the grayscale of the image block before executing one of the algorithms described above.

[0211]

[0211] Some recycling systems may look for multiple cues when determining the blocks for watermark analysis. For example, a cue for a block trigger may first be obtained for all blocks of the image frame to identify blocks that only show a conveyor. Subsequently, the remaining blocks can each be evaluated to determine the sparsity metric as described above to evaluate which of the blocks other than the conveyor are most promising for watermark analysis.

[0212]

[0212] When an article is moved by the conveyor of the recycling system, it passes linearly through the camera by entering the field of view from one side of the camera sensor and exiting from the other. For example, as described in detail above, when a clue indicating an image block showing a non-empty conveyor belt is sensed in one frame, not only the image of the current frame can be analyzed, but also images showing continuously shifted areas of the camera field of view can be analyzed in the subsequent N frames. N is a function of the camera frame rate, the belt speed, and the camera field of view. For example, if the camera field of view is 15 inches and the conveyor is moving 10 feet per second, since the article on the conveyor moves through the camera field of view, it needs to enter the field of view in 1 / 8 second. If the camera captures 60 frames per second, N can be set to 6 (i.e., the corresponding blocks in a total of 7 frames are analyzed).

[0213]

[0213] In a particular embodiment, the array of overlapping analysis blocks is arranged along the side of the camera field of view where the object first enters, and each of those blocks is analyzed frame by frame to obtain a watermark reference signal. When a reference signal or other clue is found in any of those blocks, such a detection causes further analysis of the overlap cluster of the blocks arranged at the center of the detection block, as described above. This cluster is gradually sent across the field of view between frames according to the speed of the conveyor belt.

[0214]

[0214] Figures 21A to 21D are diagrams showing such a configuration. A series of blocks are analyzed on the side where the articles of each image frame enter. (Blocks inside the frame usually do not need to be analyzed). A watermark reference signal or other clue is recognized at one of those edge blocks (shown in thick lines), and in that case, a cluster of overlapping blocks in its vicinity can be analyzed to obtain the watermark reference signal. When the watermark reference signal is detected, the analysis continues to attempt to recover the watermark payload from the voxel data obtained using the affine parameters found from the reference signal. The corresponding cluster of blocks is analyzed in consecutive frames at consecutive positions until the detected object disappears from the camera's field of view.

[0215]

[0215] When one of the blocks of the advancing cluster detects a watermark reference signal or other clue (e.g., the thick-line block in Fig. 21C), a supplementary cluster of analysis blocks (shown in dotted lines) can be generated and placed at the center of the detection block. This supplementary cluster of blocks can similarly move across the field of view together with the original cluster in synchronization with the movement of the conveyor. On the other hand, the original block band arranged across the entry side of the camera's field of view continues to examine each new image frame for the watermark reference signal or other clue. Optimization

[0216]

[0216] As described above, the conveyor belt on which plastic articles are conveyed for identification / sorting moves relatively fast. In order to ensure sufficient illumination and depth of field, a smaller aperture and a longer exposure are desirable. This can lead to motion blur.

[0217]

[0217] Some embodiments of the present technology apply blur correction to a captured image before attempting to read the watermark. Various techniques can be used, including deconvolution by an inverse filter, a Wiener filter, or a Richardson-Lucy algorithm. Using conventional methods, an optimal point spread function (PSF) for 1D motion can be estimated. (The PSF substantially characterizes the amount of energy from a single point in the scene that exposes each pixel of the camera during a static exposure.)

[0218]

[0218] For example, more sophisticated techniques can be used, such as using a flutter shutter technique that samples the scene at different times at different intervals and uses the resulting images to derive a more refined estimate of the blurred scene. (See, for example, U.S. Patent Application Publication No. 20090277962.)

[0219]

[0219] In an exemplary embodiment, the blur correction is performed in the Fourier domain, and the Fourier transform of the image is divided by the Fourier transform of the blur kernel. In other embodiments, such operations can be performed in the spatial (pixel) domain.

[0220]

[0220] In convolutional decoding of the watermark payload, list decoding can be used. Instead of outputting a single decoded payload, list decoding outputs a list of possibilities, one of which may be correct. This allows handling more errors than is possible with unique decoding. The enumerated payloads can then be evaluated using CRC data or constraints within the payload itself (e.g., it is known that the values in a particular region of the data are derived from only a subset of the possible values) to identify the single correctly decoded payload.

[0221] Rather than attempting to characterize the pose of the 128×128 pixel image patch, as described above, it is desirable to analyze smaller patches, such as 96×96 pixels for example. (In a preferred embodiment, the camera sensor, lens, and imaging distance are selected such that an object is shown with a watermark applied at a scale where each pixel approximately corresponds to the area of a single pixel, in which case a 96×96 pixel patch corresponds to a 96×96 pixel patch). For this patch, a 128×128 FFT is performed by padding with zeros or processing adjacent pixel images with a rectangular or Gaussian window to focus on the central region. As shown above, the methods detailed in U.S. Pat. Nos. 9,959,587 and 10,242,434 are used to characterize rotation and scaling. Subsequently, it is possible to determine translation using the phase shift method of U.S. Pat. No. 9,959,587. In summary, rotation, scaling, and translation (affine parameters) explain how the original watermark is present within the captured image.

[0222]

[0222] The phase shift method gives a metric indicating the intensity of the detected reference signal within the analyzed patch of the image, i.e., the sum of the phase shifts between the measured phase and the estimated phase of each of the reference signals. If this phase shift metric is less than a threshold (the lower the metric the better), the image patch is determined to contain a readable watermark. Subsequently, an interpolation operation is performed to sample the image at points corresponding to the pixel positions derived by the recognized affine parameters to generate data for payload decoding.

[0223]

[0223] As described above, when it is determined that one patch of the image contains a readable watermark, for example, using the above-described procedure, adjacent patches are checked to determine that they also contain a readable watermark. For each such patch, a corresponding set of affine parameters is determined. (Normally, each patch is characterized by a different set of affine parameters). Also in this case, an interpolation operation is then performed to generate more voxel data to be used in payload decoding.

[0224]

[0224] As described above, adjacent patches may be adjacent with their edges butted together, or may be overlapped by any number of voxels.

[0225]

[0225] When an image patch smaller than 128×128 (e.g., 96×96 voxels or 64×64 voxels) is analyzed, not all of the 128×128 voxel coding positions may be shown in each patch (depending on the scale). Nevertheless, the corresponding positions are identified (using affine parameters) between the analyzed patches, and their sampled voxel data are combined (e.g., averaged or summed). A combined record of voxel data for some or all of the 128×128 coding positions is thus generated and given to the Viterbi decoder for extraction of the encoded watermark payload.

[0226]

[0226] This is schematically shown in FIG. 22. The reference signal is detected in the small image patch 141 (here shown as only 8×8 voxels), leading to the search for and discovery of the reference signal in the adjacent small image patches 142 and 143. Each has a different affine pose. The watermark signal block (not specifically shown) extends over an area larger than any of the patches.

[0227]

[0227] For some of the pixels such as pixel 144 in the watermark signal block, interpolation data from a single image patch is provided to the decoder. For other pixels such as pixel 145, interpolation data is available from each of two overlapping patches. These two interpolation values are averaged (or summed) and provided to the decoder. For still other pixels such as pixel 146, data from three patches is averaged (or summed) and provided to the decoder. For still other pixels such as pixel 147, there is no data available to the decoder.

[0228]

[0228] In some cases, data for a particular pixel is available from two different (but usually adjacent) 128×128 pixel watermark blocks. FIG. 23 is a diagram showing such two blocks in solid lines. Also shown in dashed lines are two 96×96 pixel patches that are processed as described above. From the affine pose parameters determined for such patches, it can be seen that the pixel indicated by the circle in the left patch corresponds spatially to the pixel indicated by the circle in the right patch. Both hold the same chip of signature information. In this case, the two pixel values are summed and sent to the decoder.

[0229]

[0229] The decoder processes all available data and generates an extracted payload (or a list of candidate payloads).

[0230]

[0230] In some embodiments, the pixel data contributed by each image patch is weighted according to an intensity metric for an associated reference signal. In other embodiments, different metrics (such as those detailed in U.S. Patent No. 10,506,128 and referred to in the literature as Reference Pattern Strength and Linear Reference Pattern Strength) can be used. Alternatively, each pixel data can be weighted according to a corresponding message strength coefficient as detailed in U.S. Patent No. 7,286,685.

[0231]

[0231] The above-described accumulation of wixel data from the plurality of patches of the image frame may be referred to as in-frame signature combination. Additionally or alternatively, accumulation of wixel data from the same or corresponding wixel positions over all patches shown in different image frames can be used and may be referred to as inter-frame signature combination.

[0232]

[0232] When the affine parameters of the patch (describing the appearance of the patch watermark) are known, it is possible to perform payload reading by a payload correlation technique instead of Viterbi decoding. This is particularly effective when the number of different payloads is small, on the order of, for example, dozens or hundreds. This may apply when only the payloads of interest are plastic type data and there are only a limited number of plastic types that may be encountered.

[0233]

[0233] In one particular configuration, a set of templates is generated, each showing a wixel encoding associated with a particular type of plastic. Wixel elements that are common across all plastic types (or over a significant percentage, such as 30%) can be removed from those templates to reduce the potential for confusion. The image data is correlated with the various templates to identify the one pattern to which the image data most strongly corresponds. Since it has already been determined that the image contains a reference signal (e.g., a reference signal for a plastic texture watermark), there should be one of a limited number of wixel patterns, resulting in a high-confidence correlation method for recognizing the payload.

[0234]

[0234] Plastic bottles are increasingly not being directly printed, but instead are being packaged in plastic sleeves that are printed and heat-shrunk to adhere to the bottle. This causes problems because heat-shrinkable materials generally shrink basically in one direction (circumferentially). The watermark pattern printed on such a sleeve is then differentially shrunk by the heat shrinkage, which can impede watermark reading.

[0235]

[0235] To solve this problem, one or more of the "seed" linear transformations (detailed in U.S. Patent Nos. 9,959,587 and 10,242,434) that serve as the starting point for iterative search to determine the affine transformation of the watermark are initialized to include differential expansion / shrinkage components. This enables the iterative process to reach a faster conclusion with a better estimate of the affine distortion when detecting the watermark from the heat-shrinkable plastic sleeve.

[0236]

[0236] In some cases, the watermark is not read from the marked article, and the conveyor advances without performing type identification. On the other hand, other articles are discharged from the conveyor towards sorting bins such as ABS, HDPE, PET, PETg, etc., leaving the un-identified articles behind.

[0237]

[0237] These un-identified articles are collected in their own sorting bins and may be reprocessed later. Reading failures are abnormal and are generally addressed by changing the posture of the article and the camera presented to the illumination. By collecting and reprocessing such articles, a second presentation in a different posture may enable identification.

[0238]

[0238] Alternatively, instead of collection and reprocessing, the article may be rolled (e.g., dropped from one conveyor to another) or pushed / mixed (e.g., the conveyor may pass through a curtain of suspended obstacles) to change the posture of the article, and a second camera / illumination system can then collect additional images for analysis.

[0239]

[0239] In some embodiments, the captured image correlates with light transmission through the article rather than simple light reflection from the article. Figure 24 shows the article moving "out a bit" from the conveyor to another conveyor, and the article ViaIt is a diagram schematically showing a configuration for presenting a camera view to one or more light sources of the other party. Such an approach can be used with any of the camera / lighting systems described above. The "clue" based on the detection of the conveyor belt can also be similarly based on the detection of such splash-free background.

[0240]

[0240] The direct least squares method for determining the scale and rotation transformation to characterize the appearance of the watermark in the image operates by continuously screening and refining a large number of candidate transformations until only one remains. Thereafter, the above-described phase shift processing is performed to realize the translational movement of the x and y of the watermark pattern in the image. In a specific embodiment of the present technology, the direct least squares method does not screen candidate transformations until there is only one, but rather two or more top candidates are output. The phase shift processing is applied to each, and a plurality of candidate affine poses are generated. The best pose that generates the minimum sum of the phase shifts between the measured phase and the estimated phase of each of the reference signals is selected. With such a configuration, the number of articles that remain unread when passing through the apparatus for the first time is small, minimizing the need for reprocessing.

[0241]

[0241] In some cases, the pattern of reflected light from a textured surface patch, especially one on a transparent plastic bottle, may appear reversed, with light areas dark and dark areas light. Further, the pattern may appear inverted (reversed left / right or up / down), as when a transparent textured surface is read from below. Thus, after magnification / shrinking and rotation (by direct least squares, correlation, or another approach), multiple versions of the image are presented for analysis by a process that determines the x and y translations (by phase shift or correlation). One version is white / black inverted (lighter areas inverted to darker areas). Another version is (left / right) inverted. Another version is the original image. Only one of them will properly synchronize with the known phase characteristics of the reference signal peak where the translation is determined, with no match seen for the others. Again, such measures help maximize the number of plastic articles read when first passing through the device and minimize the need for reprocessing.

[0242]

[0242] (In some embodiments, the decoder is configured to examine the polarity of the message chip encoded in each sub-tile block (e.g., 32×32 pixels with 16 sub-tiles per tile) to evaluate whether the message chip is inverted. Such a decoder performs the above examination by correlating the watermark signal of the sub-tile to check whether the watermark signal of the sub-tile has a positive or negative correlation peak. A negative peak indicates that the signal is inverted, and the decoder will invert the chips from such inverted sub-tiles before aggregating them with the chips of other sub-tiles. Correlation can be performed using a known or fixed portion of the watermark signal).

[0243]

[0243] The applicant has discovered that it may be beneficial to capture frames having different imaging parameters, each indicative of a common area of the belt. For example, a single camera alternates between short exposure intervals, such as 20 microseconds and 100 microseconds, and long exposure intervals in successive frames. Alternatively, two cameras may capture images of the common area of the belt, one having a relatively large aperture (e.g., f / 4) and the other having a relatively small aperture (e.g., f / 8). Alternatively, they may have different exposure intervals. The resulting changes in the captured images help ensure that small changes associated with watermark encoding are readily detectable, regardless of the wide range of luminance reflected from the imaged article.

[0244]

[0244] When a plastic material is molded, the first surface of the material generally abuts the molded mold surface, while the second, opposite surface does not. Since a vacuum pulls the first surface of the material against the mold and the second surface follows, this opposite surface can still be molded. However, the physical fineness of the second surface is not good and lacks high-frequency details. However, this second surface may also be imaged by a camera (e.g., when using a carbon black plastic tray with meat packaged, the tray may be presented to the camera either with the top up or the bottom up). To address this issue, some or all of the capture frames (or portions) can be processed to emphasize high-frequency details.

[0245]

[0245] In an exemplary embodiment, if a reference signal is not found in the block to be analyzed, the block is processed by an unsharp mask filter, and the analysis is repeated in the expectation that such processing will help detect a watermark reference signal shown from the back side of the molded plastic. In another exemplary embodiment, if a reference signal is detected in the block but the payload extraction fails, the block is processed by an unsharp mask filter and the payload extraction operation is attempted again.

[0246]

[0246] An exemplary lighting system is made from a circuit board module 250, one of which is shown in FIG. 25. Each module has a width of 10 cm and is configured to accommodate 75 LEDs of the Cree XP-E2 series. Since the white light LEDs of this series are determined to provide a drive current of 1 A for a light output between 220 lumens and 280 lumens, a module of 75 LEDs can generate an output luminous flux of 16,000 to 21,000 lumens. This module is designed for parallel use. For example, in order to cover a belt having a width of 1 meter, more than 10 such modules can be used, showing a total light output of 160,000 to 200,000 lumens or more.

[0247]

[0247] This circuit board module is configured to mount three LEDs as a set, for example, by means of three sets of proximity solder pads 252a, 252b, 252c. Each of such three LEDs is configured to accommodate a lens assembly 254 to focus the light output within the imaging range of the belt. The lens has an elliptical output, and it is desirable for the light to spread more in one dimension than in the vertical dimension. A suitable lens assembly is Carclo Technical Plastics (UK) part number 10510, which focuses the output from a Cree LED into a light beam having a full width at half maximum of 45 degrees by 16 degrees. The wide dimension is oriented along the width of the belt, while the narrow dimension is oriented along the length (advancing direction) of the belt. The latter measurement is usually selected based on the distance between the LED module and the belt and the range of the imaging field of view along the belt length.

[0248]

[0248] The higher the luminance of the illumination, the shorter the exposure interval (the greater the depth of field) can be. When the exposure interval is 100 microseconds and further when the frame is captured at a rate of 150 per second, the camera sensor is collecting light for only a total of 0.015 seconds per second. If the illumination system operates (emits light) only while the camera is capturing an exposure, then that illumination system is operating at a 1.5% duty cycle. In such a case, it is possible to operate the LED with a drive current well above the nominal specified value of 1A. For example, a drive current of 3A may be used. By doing so, the light output can be further increased to the order of 300,000 lumens per meter of belt width, for example. (It will be recognized that lumens is a scale based on the sensitivity of the human visual system. Usually, illumination specified in watts is more effective in machine vision. Lumens is used as a scale simply because it is well known to some people.)

[0249]

[0249] The light output of these LEDs decays with temperature. Therefore, it is desirable to maintain the LEDs at a relatively low temperature. To assist this operation, the circuit board module can have a substrate of aluminum or copper, and the module can be thermally coupled to an aluminum or copper heat sink using a suitable heat dissipation grease. This heat sink may have fins to increase its surface area and enhance its passive heat transfer characteristics to the surrounding air. Alternatively or additionally, this heat sink can be cooled by forced air flow or forced water flow.

[0250]

[0250] In some embodiments, all of the above LEDs are white. In other embodiments, all of the LEDs are red (e.g., having peak emission between 650 nm and 670 nm, Cree part number XPEBPR-L1-0000-00D01). In yet other embodiments, module 250 comprises LEDs having various spectra. Accordingly, the control circuit is provided to drive different colored LEDs (and possibly LEDs of different ranks) independently or in various combinations.

[0251]

[0251] In such an embodiment, each of the "triples" shown in FIG. 25 includes a red LED, a green LED, and a blue LED. These are arranged in three "ranks", A, B, and C as shown in the figure. The red LEDs in rank A are switched sequentially, the green LEDs in rank A are switched sequentially, and the blue LEDs in rank A are switched sequentially. The same applies to those in rank B and rank C. The set of LED colors and ranks can be operated during the exposure period alone or in combination with other sets of LED colors and ranks. This configuration enables various image frames to be captured under various spectra of light. For example, one frame can be captured using all red illumination, while the next frame can be captured using green or red+green+blue (~white) illumination.

[0252]

[0252] In another such embodiment, not all ranks have the same type of LED. For example, ranks A and C may have red / green / blue LEDs as described above, while rank B may have only white LEDs or only red LEDs.

[0253]

[0253] In another configuration, module 250 includes LEDs up to nine different spectra. Rank A is available with LEDs having spectra 1, 2, and 3. Rank B is available with LEDs having spectra 4, 5, and 6. Rank C is available with LEDs having spectra 7, 8, and 9. Some of these spectra may be outside the visible light range and may extend to ultraviolet or infrared wavelengths. This enables the acquisition of data that allows an object to be identifiable by its spectral signature, as detailed, for example, in the applicant's U.S. Patent Application Publication Nos. 20140293091 of "Spectra ID" and 62 / 956,845 of the co-pending patent application filed on January 2, 2020.

[0254]

[0254] In the case where the camera sensor is a color sensor having, for example, a color filter array overlaid on a monochrome sensor, different color photodetectors can capture images at different wavelengths. When both red and blue LEDs are energized during a frame exposure, the red-filtered photodetector senses an image at approximately 660 nm and the blue-filtered photodetector senses an image at approximately 465 nm. By subtracting the blue image from the red image, an image may be generated in which certain encoded markings may be particularly easy to detect (e.g., due to modulated color channels in printed label artwork to achieve encoding). The same is true for other color combinations.

[0255]

[0255] Since the plastic surface can have gloss, specular reflection is not abnormal. That is, light from a given position may be reflected from the surface patch mainly to a single position. Unless the camera is at that position, the surface patch may be imaged darkly, and thus it is difficult to analyze to obtain the encoded information. Therefore, it is desirable for the surface to be irradiated from various directions. An elongated light bar composed of a plurality of modules 250 mounted side by side can extend across the belt, have a wide light scattering across the belt (45 degrees in the case of using the lens mentioned in the above example), and help to realize the above spatial diversity. The diversity is further promoted by having two or more such light bars and irradiating the belt from various positions along its length.

[0256]

[0256] As shown in FIGS. 26A and 26B, other embodiments use an optical diffuser device. FIG. 26A shows a cross-section of a generally cylindrical reflector 261, and the axis of the reflector 261 extends across the width of the conveyor belt. A linear array 262a of lighting modules such as the module 250 described above extends along one edge portion of the reflector and is oriented upward to irradiate the reflector surface. A similar lighting array 262b is also the same from the other edge portion of the reflector. Lighting exceeding 500,000 lumens can be realized for a 1-meter-wide belt.

[0257]

[0257] A mirror or colored surface can be used, but the surface of the reflector 261 is usually white. A diffuser may be used in each lighting array 262a, 262b to disperse the illumination from the LED to the reflector. Alternatively, a lens device having a spread larger than 16 degrees usually mentioned above can be used. For example, a spread of 90 to 120 degrees can be used to achieve illumination over a wide range of the reflector. Although the reflector 261 is shown as a part of a circle in cross-section, different shapes can be used to increase the irradiation amount in the section imaged by the camera 264, and the light from the two linear lighting arrays is adjusted to be focused on a band 263 extending across the belt.

[0258]

[0258] Figure 26B is a cross-sectional view of an alternative configuration in which a plurality of linear arrays of LED module 262c extend across a belt. Module 262c differs from module 250 in that it does not include a lens. Instead, the LEDs illuminate a plastic diffuser 266. Suitable diffusers are available under the brand name Optix from Curbell Plastics, Inc. Using four or more arrays 262c of modules extending across the belt, lighting in excess of 1 million lumens can be achieved for a 1-meter belt width.

[0259]

[0259] Additionally or alternatively, the problem of specular reflection can be mitigated by the use of a plurality of cameras arranged at various positions not only across the width of the belt but also along its length. Two or more such cameras may be oriented to capture images from a common focus region of the belt. Due to different viewpoints, one camera system may succeed in decoding an identifier from an object on the belt while another camera taking the image may fail.

[0260]

[0260] In addition to capturing diverse views of an object to improve the reliability of decoding, the use of a plurality of cameras capturing a common area enables the extraction of 3D information regarding an object on a conveyor belt using well-known stereoscopic principles. This provides additional information by which the object can be recognized.

[0261]

[0261] Figure 27 is a diagram showing a configuration using both a plurality of light sources and a plurality of cameras along the length of a conveyor belt. Figure 27 is a schematic diagram. As described above and illustrated in connection with Figure 26B, it is desirable to use light sources with large apertures and diffusers.

[0262]

[0262] Figure 28 shows a modified configuration in which a single camera is used, but a portion of the camera's field of view (e.g., half) is occupied by different views of the belt relayed by a mirror system (shown in thick lines). The path length through the mirror is twice the path length without the mirror. Thus, the resolution of the half of the field of view reflected by the mirror is half that of the direct view and typically requires a high-resolution sensor. When the entire image is input to a common detection module, the direct-view portion of the captured image can be downsampled to match the resolution of the view reflected by the mirror. Alternatively, the two halves of the image can be supplied to two different detector modules, each optimized for a particular resolution that is half of the captured image. In either case, it is desirable to note that when there are an odd number of mirrors in the path, the specular reflection of the image is reversed, or such reflections are predicted and the reflected portion of the image is analyzed. (In this case also, a light source as described above including a dome-shaped reflector and a diffuser can be used).

[0263]

[0263] The illumination source is desirably placed as close as possible to the belt to allow for the shortest possible camera capture interval. However, sufficient clearance must be provided so that an article can pass beneath it while on the belt. A suitable compromise is a distance between 15 and 20 cm. Depending on the type of article on the belt, a high clearance of up to 25 cm may sometimes be necessary.

[0264]

[0264] As described above, specular reflection can be helpful (e.g., sensing texture encoding from black plastic) and can also be an obstacle. One beneficial configuration is a method in which multiple separately operable light sources are arranged relative to the camera such that one (or more) of the light sources is arranged to promote specular reflection while one (or more) of the light sources is arranged to avoid specular reflection.

[0265]

[0265] The embodiment illustrated in FIG. 29 is schematically shown. The light source A is arranged and oriented such that specular reflection (arrow AA) from a horizontal plane existing 7 cm above the belt (the nominal position on the upper surface of the plastic article) is reflected (at an angle of incidence = angle of reflection) onto the camera lens. In contrast, the light source B is arranged and oriented such that specular reflection (arrow BB) from such a surface does not hit the camera lens. Rather, the light from the light source B perceived by the camera is due to diffuse reflection. The light sources A and B are operated to irradiate various frame captures to generate frames of an image optimized to show specular reflection and diffuse reflection, respectively.

[0266]

[0266] It is desirable that the light source B be arranged such that its specular reflection ray BB passes through a distance D of at least 10 cm, preferably 15 or 20 cm, away from the camera lens.

[0267]

[0267] (FIG. 29 shows the specular reflection from the light source A that appears at the center of the captured image frame by entering the camera lens along the central axis of the camera lens, but this is not essential. All that is required is that the specular reflection from the light source A be somewhere within the camera's field of view).

[0268]

[0268] In another specific configuration, the light source A is set at an angle of 45 degrees (as shown in FIG. 29), while the light source B is set at an angle directly downward.

[0269]

[0269] In some embodiments, the light sources A and B are colored different colors. For example, the light source A may have any of white, red, blue, ultraviolet, and / or infrared, while the light source B may have a different color from them.

[0270]

[0270] Accurate extraction of payload signature data from an image patch depends significantly on accurate spatial alignment of the patch, i.e., on accurately assessing the affine pose of the patch, whereby voxel values can be sampled from their exact original encoded positions within the image. As described elsewhere, alignment in the exemplary embodiments is performed using a reference (grid) signal consisting of a set of peaks in the spatial frequency (Fourier) domain.

[0271]

[0271] As mentioned above, the accuracy of the alignment can be evaluated, for example, by a metric (the "grid intensity metric" or "Linear Reference Pattern Strength") that compares the Fourier amplitude at each estimated grid signal frequency to the amplitudes of four or eight neighboring estimated grid signal frequencies, e.g., by the ratio of the former amplitude to the average of the latter. Thereafter, the values of all those grid points can be summed to generate a final grid intensity metric.

[0272]

[0272] To ensure the accuracy of the extracted signature data, the applicant uses procedures identified elsewhere to characterize the affine pose of the image patch and then iteratively adjusts one or more of the pose parameters while monitoring changes to the grid intensity metric to optimize the grid intensity metric. For example, the x-translation parameter of the determined affine pose may be fine-tuned by 1 / 10 or 1 / 4 voxels to determine whether the grid intensity metric increases. If the grid intensity metric increases, such fine-tuning is further performed. If the grid intensity metric decreases, fine-tuning in the opposite direction is performed, etc. Similar procedures are then performed using the y-translation parameter until a local maximum is found in the grid intensity parameter function.

[0273]

[0273] This procedure can be based on image patches of the size of 32×32 pixels, and the orientation of each such patch is optimized to maximize the value of the associated grid intensity metric. In certain preferred embodiments, such analysis is performed on different 32×32 pixel patches of an image where 16 pixels overlap. Three such overlapping 32×32 patches, 281 (shown in thick line), 282, and 283 are shown in FIG. 30. In such an overlapping configuration, each pixel is included in four overlapping patches. Pixel 285 in FIG. 30 is an example and is included in patches 281, 282, 283, and the fourth patch is not shown (to avoid confusion in the illustration).

[0274]

[0274] By overlapping as described above, it becomes possible to obtain four estimated values of the value of pixel 284 (and other such pixels). The value of pixel 284 is first sampled according to the affine orientation parameters of patch 281, sampled a second time according to the affine orientation parameters of patch 282, sampled again according to the affine orientation parameters of patch 283, and sampled a fourth time according to the affine orientation parameters of the fourth patch.

[0275]

[0275] As described above, it is desirable that each value of such pixel data be weighted according to the grid intensity of the image patch in which the pixel is located in order to accumulate the values input to the Viterbi decoder. Since each pixel is found in four overlapping patches, the sum of the four weighted data is accumulated and given to the decoder as a reliability-weighted estimate of the value of that pixel.

[0276]

[0276] It has been found that the configuration described above significantly improves the percentage of images in which payload data extraction is successful.

[0277]

[0277] Further improvement in the percentage of images for which payload data extraction is successful is achievable by means of dark frame subtraction technology. Determining fixed pattern sensor noise by subtracting corresponding pattern residues from subsequently captured images after long exposure image captures while the lens cap is blocking sensor illumination is well known in night astronomy and other long exposure or high ISO photography. However, the applicant is not aware of any technology such as the present technology used in a very high illumination context with extremely short exposure times. However, the method has been found to achieve a significant improvement in decoding performance.

[0278]

[0278] In a particular method, the applicant places a cap on the camera lens and captures 100 images using a "dark field" sensor having an exposure interval and analog gain set to values expected to be used during normal operation. The frames are averaged to reduce thermal (shot) noise. A matrix of residual noise values is thus generated (a combination of read noise and dark noise) and can be subtracted from image frames captured later during operation to reduce such fixed sensor noise. (Dark pixel values in the range of 1 to 12 digital numbers, i.e., noise patterns that interfere with decoding in many boundary cases, have been discovered by this method).

[0279]

[0279] Further information regarding characterizing and removing fixed pattern noise prior to watermark decoding is detailed in the applicant's U.S. Patent No. 9,544,516.

[0280] Of course, the larger the sensor, the higher the sensitivity and the shorter the exposure can be. The sensor desirably has pixels with a side length greater than 3.5 micrometers, preferably pixels with a side length greater than 5 micrometers. Ideally, although cost is an issue, a sensor having pixels of 10 or 15 micrometer size can be used. (A 2K×2K sensor having a pixel size of 15 micrometers, and Princeton Instrument's SOPHIA2048B-152 is an example). An alternative is to use "binning" with a high-resolution sensor such as a 2.5K×2.5K sensor having 5 micrometer pixels, where adjacent 2×2 sets of pixels are binned to achieve performance close to that of a 1.25K×1.25K sensor having 10 micrometer pixels. However, since binning reduces the sensor resolution, it is preferable to use a sensor having appropriate sensitivity at the original resolution.

[0281] As described above, either a monochrome or color sensor can be used. Some printed labels are encoded using, for example, "chroma" watermarking where cyan and magenta inks are used in combination. These two inks have different specular reflection curves that allow the difference between the red and blue (and / or green) channel camera responses to be subtracted to produce an image where the watermark signal is enhanced when illuminated with white (red-green-blue) light. (See U.S. Patent No. 9,245,308). Despite the signal increase achieved by such techniques, the applicant has found that irradiating the above-described labels with red light only and sensing with a monochrome sensor produces a more powerful and less noisy recovered watermark signal. (Furthermore, red LEDs are, for example, more efficient than green and blue LEDs, and in some cases more than twice as efficient. This leads to less heat production and, as described above, produces a larger light beam output).

[0282]

[0282] In yet other embodiments, for encoding recycling-related data, print labels can be encoded with machine-readable data (e.g., a sparse watermark pattern) formed using yellow ink. Further explanation regarding plastic molding, etc.

[0283]

[0283] The following description further details techniques for encoding plastic containers and labels to hold machine-readable marks. Details are included for overcoming specific signal distortions incorporated into plastic container design and manufacture.

[0284]

[0284] For the sake of brevity, a watermark is generally an optical code that typically consists of a 2D pattern of a plurality of coded signal elements in the form of generally square blocks that can be butted edge-to-edge and tiled with other blocks to cover the entire surface. Each square array can be thought of as a "grid" of encoding positions. In some embodiments, each position is marked to represent one of two data, e.g., "-1" or "1". (In other embodiments, those two data can also be "0" and "1").

[0285]

[0285] U.S. Patent Application Publication No. 20040156529, the document cited by the applicant above, describes how to apply a code signal by etching a mold in a pattern that holds data. After a pattern that holds the desired data is determined, that pattern is used to texture the surface of the plastic by forming plastic in the mold. For injection molding processing, the mold is etched by a computer-driven etching device. Each cell of the output grid (array) pattern corresponds to, for example, a 250×250 micron patch on the mold. If the output grid pattern for a particular cell has a value of "1", a depression is formed in the corresponding patch on the mold surface. If the output grid pattern of the cell has a value of "-1" (or "0"), no depression is formed. The depth of the depression is determined considering aesthetic elements. A normal depression may have a depth of less than 1 / 2 millimeter and may be on the order of less than the patch size (250 microns). The resulting pattern of the mold pitching is the physical manifestation of the output grid pattern. When the mold is used to form the surface of a product container, a negative of this pattern is created, and each recess consequently becomes a raised point on the container.

[0286]

[0286] The size of the textured area depends on the patch size and the number of rows / columns of the output grid pattern. The larger the textured area, the more "signals" are available for decoding, and the lower the precision of the specifications of the reading device can be. A textured area with a side of about 1 centimeter has been found to provide sufficient signals. Depending on the requirements of the application, a smaller (or larger) textured area can be used.

[0287]

[0287] To form the mold according to the output grid signal, techniques other than a computer-controlled etching device can be used. A small computer-controlled milling machine can be used. A laser cutting device can also be used.

[0288] Although the above approach contemplates that a texture is already formed on the container in other embodiments, the container can be formed with a flat surface and then can be texture-treated, for example, by using a heated press mold on the premise that the packaging material is thermoplastic.

[0289] To emphasize the "signal" retained by the texture treatment, surface variations corresponding to both "1" and "-1" values in the output pattern grid are imparted (not simply corresponding to the value of "1" as described above). Thereby, raised regions are formed in patches corresponding to output pattern cells given a value of "1", and recesses are formed corresponding to output pattern cells given a value of "-1".

[0290] In other embodiments, texture treatment can also be imparted by an additional material layer applied to the container after the container having the desired output pattern is formed. For example, viscous ink is applicable in screen printing. The screen has openings where the corresponding cells of the output grid pattern have a value of "1" and no openings elsewhere. When the viscous ink is applied through the screen, small patches of ink are deposited where the screen has openings and not deposited elsewhere.

[0291] In such embodiments, patches larger than 250 microns may be used depending on the resolution limitations of the screen printing process. As a result, also in this case, a texture-treated surface having a pattern of raised regions retaining the binary data payload is obtained.

[0292] To form a texture-treated layer on the container, various materials other than ink are applicable. Thermoplastic and epoxy resin are just two alternative materials.

[0293]

[0293] In some such embodiments, technologies other than printing are used to impart a textured layer to the container. For example, various photolithography techniques can be used. One technique uses a photo-responsive polymer, which after being applied to the surface, is exposed through a mask corresponding to the output grid pattern. The exposed polymer is developed, thereby removing patches of material.

[0294]

[0294] In still other embodiments, the output grid pattern is printed on the container surface in two contrasting colors (e.g., black and white). Cells having a value of "1" can be printed in one color, and cells having a value of "-1" can be printed in the other color. In such embodiments, the binary payload is not recognized from the texture pattern, but rather from the pattern of contrasting colors.

[0295]

[0295] Other patent documents of the applicant recognized herein detail other procedures for physically implementing a 2D optical code on an article, as further explained in U.S. Priority Application No. 62 / 814,567.

[0296]

[0296] Various methods can be used to prevent signal distortion in the design and / or manufacture of plastic containers.

[0297]

[0297] In the first embodiment, the signal encoding is incorporated into the container mold during the 3D printing of the mold. The inner surface of the mold that contacts the outer surface of the container is printed to include a complex texture, pattern, image, or design. This texture, pattern, image, or design holds the encoded signal. For example, as detailed in the published U.S. Patent Application Publications Nos. 20170024840, 20190139176, and 20190332840, a raw sparse watermark signal is generated. Here, the term "raw" is used to mean that the sparse watermark signal is combined with the host image or surface. The raw sparse watermark is used as a template to guide the 3D printing of the inner surface of the mold. The surface of the mold includes deformed protrusions and recesses that collectively (and more often redundantly) hold the raw sparse watermark.

[0298]

[0298] The workflow will be described. The three-dimensional (3D) mold is designed with CAD software such as AutoCad, Photoshop, Solidworks, Materialise, or many other CAD software. The CAD software defines the shape of the mold. For example, the mold may be shaped to produce a water bottle, a yogurt cup, or other containers. A 2D encoded signal (e.g., a sparse watermark) is generated. At this time, the 2D watermark signal needs to be mapped to the 3D inner surface of the mold, preferably minimizing distortion of the encoded signal.

[0299]

[0299] One approach to minimizing distortion is to utilize one-direction pre-distortion based on the relative size of the expected container. As an example, a drum-shaped container is used. The radius of the middle part of such a container is smaller than the radii of the upper and lower parts. When mapping 2D rectangular watermark tiles onto this container, different enlargements / reductions may occur in the middle part compared to the upper and lower parts of the container. Therefore, the watermark tiles may be stretched more in one spatial dimension (the x-axis) compared to another spatial dimension (the y-axis). This type of distortion is sometimes called differential enlargement / reduction or differential deformation. Consider an example where the original watermark tile is square. As a result of differential enlargement / reduction, the square may be distorted into a parallelogram with unequal sides. The differential enlargement / reduction parameters define the characteristics and extent of this stretching. Differential enlargement / reduction can cause specific problems for watermark detectors. Considering an embedded tile with x and y coordinates and having a square with equal x and y sides, when applied, the x dimension decreases in the middle part of the container while the y dimension remains the same overall length. When mapped onto the surface when the radius of the middle part is about.75 relative to the radii of the upper and lower parts, the x coordinate shrinks by about.75*x while the y axis remains the same overall (1*y). As a result, differential enlargement / reduction occurs for the x and y coordinates, similar to forming an image capture angle of about 41 degrees, making it difficult to detect sparse watermarks.

[0300]

[0300] On the encoding side, one objective of the solution is to generate an encoded signal within an orientation range detectable by a decoder when drawn against the surface of the mold. For example, the signal is preferably within an enlargement / reduction, rotation, and translation state that the decoder can detect. Differential enlargement / reduction makes it particularly difficult to realign for data extraction. To solve this differential enlargement / reduction problem, an effort was made to maintain similar dimensions relative to each other for the x and y coordinates of the tiles after mapping to the 3D surface. Therefore, pre-distortion is applied to the tiles in one direction before embedding. In particular, the tiles are given a pre-distortion in the y direction by an amount similar to that predicted by some x-direction distortion. After applying the pre-distortion and mapping, the result is a smaller embedded tile, but as a result, it has similar dimensions with respect to the x and y sides. The y direction of the various tiles arranged on the surface can be individually determined by the relative size of the radius at each embedding position. The pre-distortion varies across the mold based on the position where the tile is placed on the 3D surface. (This same distortion correction process can be used, for example, when applying a label to a container such as a heat shrink wrap label for a curved container. The y direction of the embedded tile can be modified to include the same expected enlargement / reduction as the x direction after heat shrinkage).

[0301]

[0301] Another approach to minimizing distortion is to utilize so-called UV texture processing (or mapping). UV texture processing utilizes polygons that make up a 3D object that is texture processed by surface attributes from a 2D image (e.g., a "UV texture map"). This texture map has coordinates U, V, while the 3D object has coordinates X, Y, Z. UV mapping maps the pixels in the UV texture map to SurfaceAssign to the mapping. This can be achieved by copying the triangular patches of the UV texture map and pasting them onto the triangles on the 3D object. UV texture processing is an alternative mapping system that only performs mapping to the texture space instead of the geometric space of the object. The rendering operation uses the UV texture coordinates to determine how to arrange the three-dimensional surface. Using UV texture processing, a 2D sparse watermark (or other 2D encoded signal) can be held on the surface of the mold. Here, the sparse watermark is used as the UV texture map used for texture processing the surface of the mold. Varying the gray-scale levels within the UV texture map can be used to indicate the texture depth or height of the mold surface. The resulting texture-processed surface of the mold holds its watermark signal.

[0302]

[0302] When used in combination with advanced decoding techniques in a detector, as described in U.S. Patent No. 9,182,778 (including, for example, direct least squares for recovering geometric transformations within a detector), U.S. Patent No. 9,959,587 (using direct least squares for projection approximation and using phase estimation in coordinate value correction and correlation metrics), U.S. Patent No. 10,373,299 (using direct least squares to improve projection distortion (tilt) performance), and U.S. Patent No. 10,242,434 (a detector using a hybrid of a supplemental method for geometric alignment such as Log polar for low tilt angle / weak signals and direct least squares for high tilt angles), further detection improvement can be achieved when the above one-dimensional scaling or UV texture mapping is used prior to 3D printing or laser texture processing. Such detection techniques help to recover signals distorted during mold making and / or during image capture of containers manufactured using such molds. For example, mold making may introduce an initial distortion related to the 2D mapping to the surface of the 3D object, and image capture may introduce a tilt, zoom, and / or rotation from the camera angle to the container.

[0303]

[0303] In a further embodiment, the distortion cancellation technique described in the applicant's patent documents is used to compensate for the mapping of the 2D signal to the 3D template. For example, see U.S. Patent Nos. 6,122,403, 6,614,914, 6,947,571, 7,065,228, 8,412,577, 8,477,990, 9,033,238, 9,182,778, 9,349,153, 9,367,770, 9,864,919, 9,959,587, 10,242,434, and 10,373,299, which are patent documents.

[0304]

[0304] The surface texture of the inner surface of the mold is used to generate an encoded signal in a plastic container. This texture is created by creating protrusions and / or depressions on the mold surface, resulting in depressions or protrusions being formed in the container. For example, for a sparse watermark tile, each embedding position corresponds to, for example, an n×m inch patch of the mold. When the embedding position has a value of "1", a depression is formed in the corresponding patch of the mold surface. When the embedding position has a value of "-1", no depression (or recess) is formed. Thereby, a raised area is formed in the container corresponding to the embedding position having a value of "1", and an unchanged area (or depression) is formed in the container corresponding to the embedding position having a value of "-1". When an image of the marked container is analyzed, the protrusions and depressions have different reflection characteristics. These differences can be analyzed to decode the encoded signal.

[0305] Return to the workflow, define the shape of the mold, generate a 2D encoded signal (e.g., a sparse watermark), map the watermark signal to the 3D inner surface of the mold, and then create the corresponding 3D printer file format (e.g., STL, OBJ, AMF, or 3MF) to control the 3D printer. The 3D printer prints the mold including the surface texture according to its encoded signal pattern. Of course, the 3D printer needs to be able to print at the resolution corresponding to its encoded signal pattern. For example, if the encoded signal pattern corresponds to 50 or 100 dots per inch, the printer needs to be able to reproduce it.

[0306]

[0306] In other embodiments, instead of using a sparse watermark to guide surface texture processing, a signal generated by a neural network, or a signal based on Voronoi, Delaunay, or dot-dithered halftoning can be used. Such signals are described in International Publication No. WO 2019 / 113471 and U.S. Patent Application Publication No. US 2019 / 0378235, which are patent publications.

[0307]

[0307] In other embodiments, encoding is performed on the mold surface by laser engraving, etching, embossing, or ablation. A machine-readable mark (retained by surface topology changes within the mold) is applied to the plastic when the container is formed. Very fine texture patterns can be achieved by laser engraving and tool etching. Recently, laser texture processing for molds has evolved to create different depth levels. The plurality of different depth levels can be used to hold different signal information. For example, from the perspective of signal values, the first depth may represent "1", the second depth may represent "0", and the third depth may represent "-1". Similar to the above, UV texture mapping and / or one-way pre-distortion can be used to suppress the 2D to 3D conversion.

[0308] Another problem related to 3D printed molds, laser engraved molds, or etched molds is that the container surface signal must withstand formation, not degrade the final container (e.g., not create overly thin regions), and enable easy release of the container from the mold (e.g., ensure it does not stick to the mold and fail to release). For example, if the mold creates convex or raised regions on the container, the corresponding depressions in the mold should be shaped to facilitate release of the container from the mold. For example, if the mold includes sharp and deep recesses (corresponding to sharp and high convex parts of the container), the container may not be released from the mold. The unevenness of the mold can be shaped in one direction, for example, it can be shaped in a teardrop shape (or yardang shape) in the mold release direction. Alternatively, the recesses can be shaped to match the draft angle for tool removal for the tool, material type, and / or slice shape.

[0309]

[0309] Similar considerations are required for sintered metal or ceramic parts where the watermark is retained by the surface texture. The watermark texture-treated parts must be released from the mold without deforming prior to being heated, and the watermark texture deforms with the part during sintering. The expected deformation can be corrected by pre-distortion of the watermark signal.

[0310]

[0310] In yet another embodiment, the watermark texture may be formed by linear peeling ridges by modulation of the strip position. A mold having such linear ridges can be spatially moved and / or sized up or down to represent the watermark signal. Details of mold fabrication

[0311]

[0311] In a particular example, a sparse watermark signal is used and a shape or structure is placed at the dot positions. Instead of marking square dots, the 3D surface topology is preferably formed by smoothed depressions, recesses or peaks using curves such as, for example, Gaussian or sine curves. Another example forms line drawings or elemental features (circles, lines, ellipses, etc.) that coincide with the peaks and / or valleys of a continuous watermark signal. Another example, to name a few, forms 3D surface patterns of signal-rich art designs described in WO 2019 / 113471 and US Patent Application Publication No. 20190378235, including but not limited to Voronoi, stippling, Delaunay, traveling salesman patterns. In such an example, the topology is formed such that the cross-section of the pattern of peaks or depressions on the surface is smoothed (e.g., in the form of a sine curve or Gaussian curve cross-section). The achievable cross-section depends on the type of marking (CNC milling, laser marking, 3D printing) and should be designed to ensure proper part release from the mold. Smoothing of the contours will solve this problem.

[0312]

[0312] In the following example, the design challenges of converting two-dimensional data holding a signal into a mold are described.

[0313]

[0313] When selecting a signal type (e.g., continuous, binary, or sparse), various factors are involved, such as the type of plastic to be molded (PET, PP, PE, HDPE, LDPE, etc.), the type of manufacturing process (e.g., blow molding, injection molding, thermoforming, etc.), the type of mold (metal, ceramic, etc.), the mold-making process (etching, engraving, etc.), aesthetic properties, and the attributes of the camera / lighting used for detection. In particular, continuous signals typically require high resolution both in space and in depth (e.g., embossing, debossing, etching, etc.). Binary signals typically have high resolution spatially but low resolution with respect to depth. Sparse binary signals can be realized when the available spatial and depth resolutions are both low, such as in the case of thermoforming. (Blow molding and injection molding achieve better accuracy compared to thermoforming).

[0314]

[0314] Another element to be considered is the ratio of the reference (synchronization) signal strength to the message signature strength. By ensuring a message signal strength that is sufficiently strong relative to the synchronization signal component, the reliability in the recovery of the digital payload is improved. For sparse and binary marks, the synchronization signal to message signal ratio can be determined heuristically based on the specified watermark resolution, image resolution, dot size, number of dots per tile, and payload size. Different sets of heuristic methods can be created for different plastic types, mold types, etc. For example, the properties of the plastic (e.g., the intrinsic viscosity of sheet grade, bottle grade, film grade PET) may determine how effective the mold pressing plastic is in retaining the spectral characteristics (e.g., low frequency vs. high frequency) of the watermark signal. Similar considerations apply to continuous and binary signals.

[0315]

[0315] Yet another element to be considered is the watermark signal resolution. The resolution of the watermark signal in each signal block (tile) should be high enough to achieve the desired aesthetic properties while reducing the geometric distortion caused by the curvature of the object across the entire tile by making the watermark payload readable from small tiles. In one example, the recommended resolution is 200 watermark cells (waxels) per inch (WPI) or more. For a tile size of 128×128 waxels, the tile dimensions for a 200 WPI tile are thus 0.64 inches × 0.64 inches.

[0316]

[0316] In addition to the improvement in detection in objects having shapes other than rectangular, high-resolution watermarks enable the improvement of detection from flattened, crushed, deformed, or shredded objects as seen in the recycling stream.

[0317]

[0317] Reducing the dot density of each watermark tile has various advantages. For example, the visibility of the signal pattern of the formed object is reduced, which means less interference with the visual quality and aesthetic characteristics of the object. In a transparent container, this signal pattern has less visual impact on the contents of the container (e.g., the water in a transparent plastic water bottle). Furthermore, since the dots are converted into convex or concave / depressed / grooved portions on the object surface, fewer dots mean smaller dot intervals, making it easier to form corresponding shapes in the mold. Techniques for creating the topology (e.g., convex or concave / depressed / grooved portions) of the mold surface include, for example, laser engraving, etching, electrical discharge machining (e.g., so-called spark erosion), computer numerical control (CNC) milling, or 3D printing. When using the bits of CNC milling, it is possible to pay attention to ensuring sufficient resolution. A marking device with a larger marking width can be used to remove the surface material and leave convex portions with a diameter smaller than the bit width and with a curved shape. The bit shape can be changed to achieve the desired dot representation, including, for example, conical bits, triangular bits, circular cross-sections, ball mills. Furthermore, the depressions do not need to be deep, but luminance changes can be used. The smoother the contours of the convex and concave portions of the mold can be realized as the number of convex / concave portions arranged at wider intervals is smaller.

[0318]

[0318] The dot density can be expressed as the ratio of the dots to the tile having the maximum ratio of the dot range. The maximum ratio of the dot range in a watermark signal tile containing binary pattern dots or no dots per cell is 50%. This means that half of the cells (pixels) in that tile are marked with dots (e.g., dark values). The dot density preferably becomes lower as the visibility becomes lower, for example, it becomes 10 to 35 (meaning 5 to 17.5% of the tile marked with dots).

[0319]

[0319] The dot size has been described above. The dot size is a parameter that controls the size of the element dot structure within the sparse signal. The dot size for a specific image resolution is indicated by the number of dots per inch (DPI), for example 600 DPI, which means 600 pixels per inch. The dot size is an integer value that indicates the dimension of the dot along one axis of the pixels at a given image resolution. A dot size of 1 means that the dot is 1 pixel. A dot size of 2 means that the dot is 2 pixels (for example, composed of rows and columns in a two-dimensional array of pixel coordinates, or composed along a diagonal). For example, at 600 DPI, a dot size of 1 or 2 would result in a dot width of 42 or 84 microns. Since the bit only needs to be partially pushed into the surface of the aluminum mold, a depression with this dot width can be formed with a larger bit size (for example, 257 microns).

[0320]

[0320] Dots can have various shapes. Square dots can be easily represented in the form of pixels in an image, but there may be more suitable shapes and structures for encoding signals in physical materials such as plastic or metal. Examples include circles, ellipses, lines, etc. Shapes with smoother edges or corners may be easier to form than those with sharp edges or corners (for example, due to the ease of manufacturing the mold).

[0321]

[0321] Various types of plastics, molds, and mold making enable markings of various depths on the surface of the plastic, for example, deeper or shallower. Generally, the deeper the available mark, the lower the available dot density, while shallower marks usually use higher dot densities. The deeper the mark, the higher the likelihood of withstanding workflow changes such as surface wear, planarization, and pulverization.

[0322]

[0322] The image signal representation of the watermark tile provided for conversion to 3D surface topology for a mold may be in a vector or raster image file such as the SVG image format. For example, in combination with the watermark tool plugin of the applicant, Digimarc Corp, image editing tools such as Adobe Photoshop, design tools such as Adobe Illustrator, or signal processing software such as MathWorks' MATLAB can be used to create the file.

[0323]

[0323] In an electronic image file, dots can have various shapes. Square dots can be easily represented in the form of pixels in an image, but for encoding signals in physical materials such as plastic or metal, various shapes and structures are generally more suitable. Examples include circles, ellipses, lines, etc. For example, due to the ease of manufacturing a mold, smoother shapes are easier to reproduce than shapes with sharp edges or corners. The vector representation enables dots to be defined from the perspective of the beneficial dot shapes for the aesthetic characteristics of the finished molded product and further for the performance of the mold. Elements to consider regarding the performance of the mold are the tapering, smoothing, or profiling of recesses or protrusions for the release of the molded part from the mold. In a simple example, the dots have a circular shape and, for example, using a CNC device, facilitate the formation on the surface of an aluminum mold. The 3D structure of its shape is related to the ease of induced luminance changes that hold the modulation required when encoding the watermark signal, together with manufacturing (e.g., release). The form of signal-rich art (such as described in U.S. Patent Application Publication No. 20190378235 and International Publication No. 2019 / 113471) can be created by selectively placing an object of the desired shape at the dot positions and / or drawing vector art through the dot positions such that the vector art is highly correlated with the watermark signal at the dot positions.

[0324]

[0324] The resolution of the tile image (e.g., in DPI units) determines the granularity of the modulation made in the material. The use of a high resolution (e.g., 600 DPI) provides greater flexibility when designing features (e.g., dots or other structures) that can be subjected to embossing, debossing, etching, milling, electrical discharge machining, etc. The use of a high resolution also provides greater flexibility during signal formation, e.g., when creating sparse binary marks, by providing a greater margin in the selection of prohibited areas, dot shapes, sizes, etc. Examination of exemplary watermarking methods

[0325]

[0325] In the exemplary watermarking method, a plurality of symbol message payloads (e.g., 48 binary bits that may represent a product's Global Trade Identification Number (GTIN) or plastic recycling information, along with 24 associated CRC bits) are applied to an error corrector. This corrector uses an error correction method to convert the symbols of the message payload into a very long array of coded message elements (e.g., binary or M - valued elements). (Suitable coding methods include block codes, BCH, Reed - Solomon, convolutional codes, turbo codes, etc.). The output of the corrector may include hundreds or thousands of binary bits, e.g., 1024, sometimes called raw signature bits. These bits may be scrambled by performing an exclusive - OR operation using a scrambling key of the same length, thereby generating a scrambled signature.

[0326]

[0326] Each of those scrambled signature bits modulates a 16-length pseudo-random noise modulation sequence (spreading carrier wave), for example, by performing an exclusive OR operation. Each of the scrambled signature bits thus generates a modulated carrier wave sequence of 16 "chips" and generates a scrambled extended payload sequence consisting of 16,384 elements. This sequence is mapped to the elements of a square block having 128×128 embedding positions according to the data in the scrambling table, and a 2D payload signature pattern is generated. (For each of the four 64×64 quadrants of the 128×128 block, the scrambling table assigns 4 chips for each scrambled signature bit). Each position of the 128×128 block is associated with either a value of 0 and 1 or black and white, and approximately half of the positions have each state. This two-mode signal is often mapped to a larger two-mode signal centered on 128 8-bit gray-scale values having, for example, values of 95 and 161. Each of these embedding positions may correspond to a small region of pixels, such as a 2×2 patch, called a "bump", and generates a watermark message block having dimensions of 256×256 pixels.

[0327]

[0327] As described above, to assist in recognizing the parameters of any affine transformation to which a watermark has been applied prior to decoding, a synchronization component is generally included in the digital watermark, whereby the payload can be accurately decoded. A particular synchronization component has the form of a reference signal consisting of peaks of a sinusoid with a pseudo-random phase in the Fourier domain having a magnitude of tens or more. This signal is transformed (e.g., by an inverse fast Fourier transform) into a spatial domain of 256×256 block size corresponding to a 256×256 block to which the scrambled and expanded payload sequence is mapped. The spatial domain reference signal, which may contain floating point values between -1 and 1, can be scaled to a range of -40 to 40 and combined with the 256×256 pixel payload block to produce a final watermark signal block having values in a range of, for example, 55 (i.e., 95 - 40) to 201 (i.e., 161 + 40). This signal can then be added to the host image after a first reduction to make it less conspicuous.

[0328]

[0328] When such a watermark signal block is printed at a spatial resolution of 300 dots per inch (DPI), a printed block of approximately 0.85 square inches results. (Since a side dimension of 0.85 inches corresponds to 128 pixels, the result is 150 pixels per inch.) Such blocks can be tiled edge-to-edge to mark a larger surface.

[0329]

[0329] The above-described watermark signal may be referred to as a "continuous tone" watermark signal. A continuous tone watermark signal is typically characterized by multi-valued data, i.e., not just on / off (or 1 / 0 or white / black), and thus is called "continuous". Each pixel of the host image (or a region within the host image) is associated with one corresponding element of the watermark signal. Most of the pixels of this image (or image region) change their values by combination with their corresponding watermark elements. The change is usually both positive and negative, e.g., at one position it changes to increase the local luminance of the image, and at another place it changes to decrease it. Further, the changes may vary in degree, with some pixels changing by a relatively small amount while others change by a relatively large amount. Usually, the amplitude of the watermark signal is small enough (i.e., steganographic) that the presence of the watermark signal in the image is not noticed without careful viewing.

[0330]

[0330] (Due to the high redundancy of the encoding, in some embodiments, pixel changes between directions can be ignored. For example, in such an embodiment, only pixel values change in the positive direction. Pixels that usually change in the negative direction remain unchanged.)

[0331]

[0331] In a modified continuous tone watermark, the signal acts so as not to change the local luminance of the artwork pixels, but changes their color. Such a watermark is called a "chrominance" watermark (rather than a "luminance" watermark). For example, in U.S. Patent No. 9,245,308, an example is detailed.

[0332]

[0332] A "sparse" or "binary" watermark is different from a continuous-tone watermark. A "sparse" or "binary" watermark does not change most of the pixel values of the host image (or image region). Rather, it has a print density that results in markings between about 5% and 45% of the pixel positions in the image (which may be set by the user). Adjustments are usually made in the same direction, such as by reducing brightness. Sparse elements are usually binary, for example either white or black. A sparse watermark may be formed on top of other images, but is usually presented in an area of artwork that is blank or uniformly colored. In such cases, the sparse marks contrast with their background and are made visible without careful viewing. Sparse marks may have the form of an area of seemingly random dots, but as detailed elsewhere, may also have the form of a line structure. When having a continuous-tone watermark, a sparse watermark usually has the form of signal blocks tiled across an area of the image.

[0333]

[0333] A sparse watermark can be created from a continuous-tone watermark by thresholding. That is, the darkest element of the combined reference signal / payload signal block is copied to the output signal block until the desired dot density is obtained.

[0334]

[0334] U.S. Patent Application Publication No. 20170024840 details various other forms of sparse watermarks. In one embodiment, the signal generator starts with a 128×128 input. One is the payload signal block, and as described above, its position is a binary (0 / 1 or black / white) expanded and scrambled payload sequence. The other is the spatial domain reference signal block, and each position is assigned a floating-point number between -1 and 1. The darkest (most negative) "x" % of those reference signal positions are recognized and set to black, and the rest are set to white. The spatially corresponding elements of these two blocks are both logically ANDed to find the coincidence of black elements between the two blocks. These elements are set to black in the output block, and the other elements remain white. By setting "x" higher or lower, the output signal block can be made darker or lighter.

[0335]

[0335] U.S. Patent Application Publication No. 20190332840 details further sparse coding embodiments. One embodiment uses a reference signal generated at a relatively high resolution (e.g., 384×384 pixels) and a payload signature over an array of relatively low resolution (e.g., 128×128). The latter signal has only two values (i.e., two-tone), and the former signal has more values (i.e., multi-values such as binary gray scale or multi-values consisting of floating-point values). The payload signal is interpolated to the higher resolution of the reference signal and is converted from a two-tone form to multi-values in the process. These two signals are combined at high resolution (e.g., by summing at a weighted ratio), and a thresholding operation is applied to the result to identify the positions of the extreme (e.g., dark) values. These positions are marked to generate a sparse block (e.g., a 384×384 sparse block). This threshold level determines the dot density of the resulting sparse marks.

[0336]

[0336] Another embodiment classifies samples in a block of a reference signal by value (darkness) to create a ranked list of the N darkest positions (e.g., 1600 positions) each having a location (e.g., within a 128×128 element array). The darkest position among those N positions is always marked at an output block (e.g., 400 positions or P positions) so that the reference signal is strongly represented reliably. The remaining of the N positions (i.e., N - P or Q positions) are marked or not marked according to the value of the message signal data mapped to such positions (e.g., by a scatter table of an encoder). Positions of sparse blocks that do not exist among the N darkest positions (i.e., also do not exist among P or Q positions) are never marked and are consequently reliably ignored by a decoder. By setting the number N larger or smaller, sparse marks having more dots or fewer dots are generated. (This embodiment is referred to as the "fourth embodiment" in U.S. Patent Application Publication No. 20190332840 referenced above).

[0337]

[0337] When generating sparse marks, an interval constraint is applicable to candidate mark positions to prevent concentration. This interval constraint may be in the form of a prohibited region of circular, elliptical, or other (e.g., irregular) shape. The prohibited region may (or may not) have two or more or fewer symmetry axes. The implementation of the interval constraint can use a related data structure having one element for each position in the tile. When a dark mark is added to the output block, corresponding data is stored in a data structure that identifies positions made unavailable for possible marking due to the interval constraint.

[0338]

[0338] In some embodiments, by varying the relative amplitude of spatial frequency peaks, the reference signal can be adjusted to appear non-random, such that not all spatial frequency peaks have equal amplitude. Such a change in the reference signal consequently affects the appearance of the sparse signal.

[0339]

[0339] Sparse patterns can be shown in various forms. The simplest part is an apparently random pattern of dots. However, more artistic depictions are possible, including those described and illustrated above.

[0340]

[0340] Another obvious artistic pattern for holding watermark data is detailed in the patent document U.S. Patent Application Publication No. 20190139176. In one approach detailed, the designer creates a candidate artwork design or selects one from a library of designs. Vector art in the form of line forms or small discontinuous printing structures of a desired shape works well in this approach. The payload is input into a signal generator, which generates a raw data signal in the form of a two-dimensional tile of data signal elements. This method then edits the artwork at that spatial location according to the data signal elements at the spatial location. Once an artwork with the desired aesthetic quality and robustness is generated, it is applied to the object, for example, by laser marking.

[0341]

[0341] Other techniques for generating a visible artwork with a robust data signal are detailed in the assignee's patent documents U.S. Patent Application Publication No. 20190213705 and co-pending application No. 62 / 841,084 filed on April 30, 2019. In some embodiments, a neural network is applied to an image containing machine-readable code to transform its appearance while maintaining machine-readability. One particular method trains a neural network with a style image having various features. The trained network is then applied to an input pattern that encodes a plurality of symbol payloads. The network adapts features from the style image to represent details of the input pattern, thereby generating an output image in which features from the style image contribute to the encoding of the plurality of symbol payloads. This output image can then be used as a graphical component of a product package, such as a background, border, or pattern fill. In some embodiments, the input pattern is a watermark pattern, while in other embodiments, the input image is a host image that has been pre-watermarked.

[0342]

[0342] Other such techniques do not require a neural network. Instead, a watermark signal block (i.e., a reference and message signal) is decomposed into sub-blocks. The style image is then analyzed to find the sub-blocks having the highest correlation with each of the watermark signal sub-blocks. The sub-blocks from the style image are then mosaicked together to generate an output image that visually evokes the style image but has signal characteristics approximating the watermark signal block.

[0343] In addition to the reference documents cited elsewhere, details regarding watermark encoding and reading that may be included in the embodiments of the present technology are disclosed in the applicant's previous patent applications including U.S. Patent Nos. 5,850,481, 6,122,403, 6,590,996, 6,614,914, 6,782,115, 6,947,571, 6,975,744, 6,985,600, 7,044,395, 7,065,228, 7,123,740, 7,130,087, 7,403,633, 7,763,179, 8,224,018, 8,300,274, 8,412,577, 8,477,990, 8,543,823, 9,033,238, 9,349,153, 9,367,770, 9,521,291, 9,600,754, 9,749,607, 9,754,341, 9,864,919, 10,113,910, 10,217,182, and U.S. Patent Application Publication Nos. 20160364623, and U.S. Application No. 16 / 270,500 filed on February 7, 2019, No. 62 / 814,567 filed on March 6, 2019, No. 62 / 820,755 filed on March 19, 2019, and No. 62 / 946,732 filed on December 11, 2019.

[0344]

[0344] The above-described technology is often described in the context of imparting a print watermark, but similar techniques can be used for watermarking based on 3D textures / shapes. The sparse dots and line elements of the binary marks can be represented by protrusions (or depressions) on the 3D surface.

[0345]

[0345] Similarly, positive and negative changes in pixel values associated with a continuous-tone watermark can be represented by spatial changes in the 3D surface height. In some configurations, the surface is changed in only one direction by, for example, protrusions that rise from the surface. In other configurations, the surface may be changed in the reverse direction by both protrusions that rise from the 3D surface and depressions (recesses) that are recessed below the 3D surface.

[0346]

[0346] When the surface is changed only in one direction, one embodiment ignores negative changes in the watermark value, and the surface is changed only by positive changes. Another embodiment ignores positive changes in the watermark signal, and the surface is changed only by negative changes. In both such embodiments, the surface change may be in either the positive protruding direction or the negative recessed direction.

[0347]

[0347] In still other embodiments, the most negative change (extreme value) of the continuous watermark signal does not correspond to a change on the surface, while changes gradually becoming positive from this extreme value correspond to gradually increasing surface changes (either a protrusion or a recess). In still other embodiments, the most positive change of the continuous watermark signal does not correspond to a change in the surface, while changes gradually becoming negative from this value correspond to gradually increasing surface changes (also either a protrusion or a recess in this case).

[0348]

[0348] When the surface is changed in two directions, negative values of the continuous tone watermark signal can correspond to recesses in the surface (the depth depends on the negative signal value), while positive values of the watermark signal correspond to protrusions from the surface (the height depends on the positive signal value). In other embodiments, the polarity is switchable, where positive values of the watermark signal correspond to recesses in the surface and vice versa. The depth of the deepest recess and the height of the highest protrusion may be equal, but this is not essential. The same applies to the average depth of the recesses and the average protrusion height. The depth / height may be asymmetric, as if a DC offset were applied to the continuous tone watermark signal.

[0349]

[0349] When the surface is changed in two directions, it is desirable for both the recesses and the protrusions to hold the watermark payload information (different from the configuration of U.S. Patent Application Publication No. 20180345323, which teaches that only one or the other holds the payload information).

[0350] A recycling system including individually specifiable sorting bins (or categories) will be described with reference to FIGS. 32A and 32B. The recycling system can have its functions coupled to one or more stages, but includes two stages. One or more sorting units have a light source, an image capture unit, a watermark reader, and a control logic circuit. The sorting unit reads electronic watermark information from image data indicating plastic objects in a waste stream. This information can indicate the type of plastic (e.g., polyethylene terephthalate, high density polyethylene, low density polyethylene, polypropylene, polycarbonate, etc.) or can hold other information useful in recycling. A diverter and / or other mechanism is controlled according to such electronic watermark information to send the plastic object to an appropriate sorting destination for recycling or reuse. In a first stage shown on the left side of FIG. 32A, plastic objects (or other container materials) are first sorted in a binary manner, for example, into an encoded category and an unencoded category. The unencoded category includes plastics without a detectable electronic watermark. The unencoded category includes plastics that originally did not contain a watermark and plastics that have deteriorated so much that any original electronic watermark cannot be detected. The encoded category includes plastics with a detectable electronic watermark.

[0351]

[0351] The encoded plastics are further processed according to a second stage shown on the right side of FIG. 32A. Although FIG. 32A shows two separate processing stages, the first stage (left side of the figure) and the second stage (right side of the figure) can be coupled to be one or three or more stages.

[0352]

[0352] Referring to FIGS. 32A and 32B, the encoded plastics are passed or conveyed under or through the sorting unit at a speed of, for example, 1 to 3 m / s, more preferably 5 to 9 m / s, for example 5 m / s. In one embodiment, the encoded plastics are sent one by one, and in other embodiments, the encoded plastics are sent in multiples.

[0353]

[0353] The sorting unit has a light source (plural possible), an image capture unit (plural possible), a watermark reader (plural possible), and a control logic circuit. For example, the light source (plural possible) may have an LED (plural possible), and the image capture unit may have one or more cameras or an image sensor array. The watermark reader operates to decode a watermark from an image frame representing the encoded plastic. The watermark reader provides the decoded watermark data to the control logic circuit to control a sorting diverter that can be individually specified as a sorting destination along the stream sorting path.

[0354]

[0354] The configuration of FIG. 33A shows an embodiment of a sorting unit having a plurality of light sources arranged along the conveyor (waste stream) moving direction. In an alternative embodiment, the light source is arranged not in the same direction as the conveyor moving direction but in a direction crossing the conveyor moving direction. In yet another embodiment, one or more light sources are arranged along the conveyor moving direction (as shown in the figure), and one or more light sources are arranged in a direction crossing the moving direction. For alternate frames of image capture by a camera (which may capture frames at a rate of, for example, 150, 300, or 500 frames per second), different light sources can be activated. One frame is irradiated by one light source, the next frame is irradiated by another light source, and so on. Alternatively, if a plurality of image sensors are used, or if an image sensor with two or more color filters is used, the plurality of light sources are activated simultaneously, the first sensor captures an image corresponding to the first light source, the second sensor captures an image corresponding to the second light source, and so on.

[0355]

[0355] In certain examples, the three light sources of FIG. 33A include a red LED (having a peak emission, for example, between 620 nm and 700 nm, referred to as "660 nm or around 660 nm"), a blue LED (having a peak emission, for example, between 440 nm and 495 nm, referred to as "450 nm or around 450 nm"), and an infrared (or near-infrared) LED (having a peak emission, for example, between 700 nm and 790 nm, referred to as "730 nm or around 730 nm"). In a more particular example, the red LED has a narrow-band center wavelength between 650 nm and 670 nm, for example 660 nm, and the full width at half maximum value ("FWHM") of the emission is 30 nm or less. The blue LED has a narrow-band center wavelength between 440 nm and 460 nm, for example 450 nm, and the FWHM of the emission is 30 nm or less. The infrared (or near-infrared) LED has a narrow-band center wavelength between 720 nm and 740 nm, for example 730 nm, and the FWHM of the emission is 40 nm or less.

[0356]

[0356] In another particular example of the separation unit shown in FIG. 33B, the two light sources include a blue LED (having a peak emission, for example, between 440 nm and 495 nm, referred to as "450 nm or around 450 nm") and an infrared LED (having a peak emission, for example, between 700 nm and 790 nm, referred to as "730 nm or around 730 nm"). In an even more particular example, the blue LED has a peak emission between 440 nm and 465 nm, for example 450 nm, and the infrared LED has a peak emission between 710 nm and 740 nm, for example 730 nm.

[0357]

[0357] In another particular example of the separation unit, ambient light is used to irradiate the object to be coded.

[0358]

[0358] The image capture unit(s) of the separation unit has one or more cameras or image sensor arrays for capturing an image or image frame corresponding to various LED irradiations. This camera or image sensor array can be arranged centered around the LED at various positions (or vice versa), for example, as described in this patent document.

[0359]

[0359] In an alternative embodiment, referring to FIGS. 33C and 33D, each point on the belt within the field of view is preferably illuminated by each light color from a plurality of directions by a diffused light source. The LED light is preferably focused by a lens with at least 40 degrees FWHM perpendicular to the belt operating range. Along the belt operating range, the light preferably comes from at least two directions (e.g., two light bars) about 10 to 25 degrees away from the camera axis. The light can be arranged at a position at least 50 cm away from the belt to minimize the difference between the near field of view and the far field of view. Each light source can be focused to illuminate the field of view (FoV). Furthermore, light diffusion at the top of the LED lens is recommended, and a suitable depth of field (DoF) is at least about 10 cm. The use of camera gain (digital or analog) is not recommended to maximize the SNR. The captured images are monochrome and have an 8-bit dynamic range without compression, for example.

[0360]

[0360] The image capture unit may have one or more monochrome cameras with a capture speed of, for example, 8 bits or more and up to 300 to 700 frames per second (FPS). The number of frames per second depends to some extent on the conveyor belt speed. For example, at least 300 FPS is preferable for a belt speed of 3 m / s (for example, enabling 150 FPS for frames with red LED illumination + 150 FPS for frames with blue LED illumination). However, for a 5 m / s belt speed, at least 500 FPS is preferable (for example, enabling 250 FPS for red LED illumination frames + 250 FPS for blue LED illumination frames). The recommended maximum camera exposure time is approximately 60 μs for a 3 m / s belt or approximately 40 μs for a 5 m / s belt. A monochrome area scan camera with a global shutter can be used to minimize motion artifacts. In this alternative embodiment, as shown in FIG. 33C, the optical axis of the camera is perpendicular to the conveyor belt. The sampling resolution of the camera at a distance of 50 cm from the belt can be measured in pixels, for example, 150 to 600 pixels per inch, and in one example, 170 to 180 PPI, etc. The camera(s) is preferably arranged such that the FoV captures the entire width of the belt. When using multiple cameras, at least a 2 cm FoV overlap is preferable. Regarding the belt length FoV, it is preferable that at least 14 cm of the belt is captured along the belt travel direction. A lens aperture of f / 5.6 or less, such as f / 8, is recommended.

[0361]

[0361] The light source can operate in a pulsed manner and be synchronized with the camera, and can operate periodically with different combinations of color LEDs. For example, two frames are generated by irradiating the first frame with a 730 nm LED and the second frame with a 450 nm LED, or two frames are generated by irradiating the first frame with a mixture of a 730 nm LED and a 450 nm LED and irradiating the second frame with a 660 nm LED.

[0362]

[0362] In an exemplary embodiment, the sorting unit analyzes the resulting image frame to obtain watermark data, searches for both a printed label watermark and a plastic watermark as shown in FIG. 34, or more generally, searches for one or more watermarks held by plastic as shown in FIG. 35, and has a watermark reader or decoder. The watermark payload data can be used as an input (s) to the control logic circuit of the sorting unit (e.g., the diverter control logic circuit). In one embodiment, the watermark payload data is provided to a database that indexes related information, and the related information is provided to the control logic circuit. The diverter control logic circuit controls various diverters arranged along the path of FIG. 32B, such as a path including, for example, one or more conveyors, rollers, or free-fall paths.

[0363]

[0363] Referring to FIG. 36A, a watermark reader (housed within or communicating with the sorting unit shown in FIGS. 32A and 32B) decodes an electronic watermark from the captured image frame. In one example, the decoded electronic watermark includes a GTIN (and perhaps other data). To determine the sorting bin value, a data structure 122 such as a table or database can be used. The data structure 122 serves to associate the item GTIN with corresponding information regarding the coded plastic container. That is, this data structure is queried with the GTIN identifier decoded from the electronic watermark, whereby the system accesses pre-stored data to identify, for example, the sorting bin value for a product having that GTIN (and other information such as, for example, plastic type, subtype, and / or color). The sorting bin value information can be provided to the logic circuit that controls the sorting diverter. This enables the sorting bins to be individually specified along the recycling path.

[0364] Referring back to FIG. 32B, the control logic circuit uses the sorting bin value information to activate one or more sorting diverters along the waste recycling path to sort the encoded plastic articles into specific sorting bins. Consider an example where Brand X manufactures three different types of plastic containers including Container A (encoded with GTIN A), Container B (encoded with GTIN B), and Container C (encoded with GTIN C). Brand X is strongly interested in recycling those containers to help minimize their material costs. The data structure 122 is updated to include the sorting bin locations associated with the GTINs of Container A, Container B, and Container C (FIG. 36B). The sorting unit decodes the watermark including GTIN B from the plastic article on the conveyor. The sorting unit queries the data structure 122 using GTIN B to find the associated sorting bin value, in this case, "Brand X, Sorting Bin B". The control logic circuit uses that sorting bin value to activate "Sorting Diverter - Brand X, Sorting Bin B" to place the corresponding plastic article into Sorting Bin B of Brand X. The control logic circuit may also use other data related to the recycling system, such as the conveyor speed and the physical location of Sorting Bin B of Brand X along the path, to activate "Sorting Diverter - Brand X, Sorting Bin B". If no sorting bin value is associated with a particular GTIN, the corresponding plastic container can be sorted based on the material type, or subtype, or other information included in the data structure 122. The decoded watermark data and associated sorting events can be logged to provide statistics regarding the waste stream being processed.

[0365] From the above, it will be recognized that the technical problem was a binary level of sorting (e.g., encoded or non - encoded). However, by using the techniques of the present disclosure, N - value sorting (or sorting with individually specifiable sorting destinations) is achievable. This level of detailed sorting enables container - by - container recycling, ensures material purity, and helps reduce the use of non - recyclable raw materials.

[0366]

[0366] FIG. 37 illustrates an overview of an ecosystem including the recycling system shown in FIGS. 32A and 32B. This ecosystem clearly arranges the elements of the container life cycle, including supplying finely sorted bales for dedicated composite material reprocessing and ultimately producing recycled materials that can rival or replace virgin material supplies.

[0367]

[0367] The illustrated recycling system enables an increase in knowledge regarding how to design the reuse and recycling of products made of composite or multilayer materials ("circular design"). Another advantage is the increase in knowledge regarding the overall environmental footprint of the container, including the net effect on greenhouse gas emissions of improved sorting, separation, and recycling of composite and multilayer materials. Conclusion

[0368]

[0368] Although the principles of the present invention have been described and illustrated with reference to explanatory examples, it will be understood that the technology is not limited thereby.

[0369]

[0369] For example, the described embodiments involve images captured using visible illumination, but this is not essential. Other forms of illumination, such as ultraviolet or infrared light, can be used as an alternative.

[0370]

[0370] Although a plastic bottle has been described as including both a printed watermark and a texture watermark, it will be understood that the particular technology of the present technology improves the texture watermark regardless of the presence or absence of the printed watermark. Thus, for example, a plastic bottle encoded with recycling information using a pattern as shown in the figures is an improvement over the prior art markings of plastic containers with recycling data (which tend to be obtrusive and detract from the aesthetic characteristics of the package). Similarly, other improvements, such as detailed cues for distinguishing an empty conveyor from a non-empty conveyor, are applicable to watermarking in general.

[0371]

[0371] In various detailed embodiments, the print watermark and the texture watermark use reference signals that include peaks at different spatial frequencies, but this is not essential to avoid confusion. In other embodiments, both watermarks use reference signals that include peaks at the same spatial frequency, in which case it is possible to distinguish the watermarks (e.g., by a store-exclusive terminal) using other attributes of their protocols. For example, it is possible to distinguish a printed label watermark from a textured plastic watermark using the version of the bit string encoded in the variable data. (In the exemplary signal protocol, a 1024-bit message string is formed as a concatenation of (a) a 100-bit string indicating the protocol version, followed by (b) a 924-bit string based on 47 bits of payload data. The latter bits are formed by concatenating 47 bits of payload data with 24 corresponding CRC bits. The 71 bits are then convolutionally encoded at a rate of 1 / 13 to produce 924 bits. Thereby, the bit string indicating the protocol version represents approximately 10% of the signal energy). Or, the reference signal of one watermark can use peaks at spatial frequencies that are a subset of the peaks used in the reference signal of the other watermark.

[0372]

[0372] When two watermark reference signals share some or all of the spatial frequency peaks in common, to avoid confusion, the peaks of one reference signal may be assigned a different phase from the peaks of the other reference signal. If sufficient false detection behavior cannot be obtained by differentiating the two watermarks based on the peak phase, additional inspections can be performed. For example, for two different corresponding portions of the captured image, the phase may be checked twice. Those corresponding portions may be in consecutive image frames or may also be a single image frame processed to generate two images. For example, Gaussian noise can be added to generate a second image. Or, the second image can be generated by discarding even the rows and columns of pixel data from the first image. There are many such possibilities. The result is reliable only if the identification based on the two phases of the watermark signals from the two corresponding images matches.

[0373]

[0373] In still other embodiments, to avoid confusion, the two watermarks use different scrambling keys, or different diffusion keys, or different diffusion tables.

[0374]

[0374] In embodiments where the two watermark reference signals use spatial frequency peaks in common, the processing configuration can be simplified. For example, such synchronization can be performed by a common processing stage because synchronization of enlargement / reduction and rotation for both watermarks generates a common set of reference signal peaks. Such methods are detailed, for example, in U.S. Provisional Patent Application No. 62 / 834,260 filed on April 15, 2019 and U.S. Provisional Patent Application No. 62 / 834,657 filed on April 16, 2019, which are patent application documents.

[0375]

[0375] In a particular detector, a "supergrid" reference signal structure that includes all peaks from both reference signals is used. The enlargement / reduction and rotation of the input image are determined by synchronizing with a composite reference signal or the like. When such synchronization is achieved, it is easy to determine whether the input image contains one of the reference signals or the other by, for example, searching for and inspecting a unique peak or phase in one of the two reference signals.

[0376]

[0376] Although the above-described embodiments use a reference signal consisting of peaks in the Fourier amplitude domain, it should be understood that the reference signal can indicate peaks in different transformation domains.

[0377]

[0377] In this regard, the watermark signal need not include a separate reference signal for the purpose of geometric synchronization. In some cases, the payload portion of the watermark signal itself has a known pattern or structure that enables geometric synchronization independent of an individual reference signal.

[0378]

[0378] The term "watermark" generally refers to a mark that is not noticeable to humans, i.e., is steganographic. A steganographic watermark can be beneficial but is not essential. A watermark that forms an obvious, noticeable pattern to humans can be used in embodiments of the present technology.

[0379]

[0379] The GTIN payload data field from the label watermark of the embodiment of FIG. 13 is used to access information such as the corresponding plastic type from a database, but this is not essential. For this purpose, other fields of the label watermark that detail, for example, the various data types mentioned here can be used. In fact, the use of a database in combination with the label watermark is not essential, and the payload can directly hold plastic data, such as in one of the application business identifier key value pairs.

[0380]

[0380] Similarly, although GTIN information is generally encoded only in the label watermark, in some embodiments, the plastic texture watermark can also encode that information. In such cases, information regarding the plastic or sorting bin of the sorting destination that makes up the GTIN can be obtained by using a data structure (such as Table 121) that associates the GTIN with such other information.

[0381]

[0381] Although this specification has specifically described 2D image sensors, 2D sensors are not essential. Image sensing can instead be performed by a linear array sensor that captures line-scanned images at an appropriate high rate.

[0382]

[0382] The surface shaping shown in some of the figures uses mainly straight lines, partly for drafting convenience. Generally, surface texture processing realizes a curved tapered shape.

[0383]

[0383] The processing patch of the captured image having a specific size in pixel units has been described. The exact pixel size of the patch cannot be determined until its scale is evaluated (for example, using the direct least squares method mentioned above), but since the encoding scale of each watermark encountered by the system is known in advance and the imaging distance is fixed, the scale correspondence relationship between the captured pixels and the encoded pixels is roughly known and is sufficient for the purposes of this technology.

[0384]

[0384] The image processing described here is usually performed on data that has been pre-filtered in an “8-axis” (or “cross”) manner as described in the references cited above. In an exemplary embodiment, the 8-axis filtered data can have integer values in the range of -8 to 8.

[0385]

[0385] In some embodiments, after the affine parameters characterizing the article pose are recognized, the reference signal becomes mere noise later, so the estimated value of the reference signal is subtracted from the captured image. The 8-axis processing can then be applied to the remaining signal.

[0386]

[0386] Although this specification has repeatedly referred to plastic bottles, it will be appreciated that the present technology can be used in combination with any article, such as trays, bags, cups, shipping containers, etc.

[0387]

[0387] Furthermore, although recycling has been emphasized in this specification, it should be understood that the same technology can be used for sorting plastic and other containers for reuse. For example, a beverage manufacturer may serialize bottles by texture processing using unique identifiers for each. When a consumer returns a bottle for reuse, in the process of washing and refilling the bottle, the present technology can be used to sense the serialization identifier and increment a counter that keeps track of the number of times the bottle has been processed for reuse. When the bottle reaches an empirically determined lifespan (e.g., after 30 uses), it can be diverted for recycling.

[0388]

[0388] For optimal diverter performance, the center of gravity of the detected plastic article is estimated and the center of gravity position is used to control the operation of the diverter mechanism (e.g., the center of gravity position is the target of compressed air injection). Each detection of a watermark block serves as a data point in estimating the center of gravity. In one example, the coordinates of the center of each watermark block are determined in the coordinate system of the image in which the block is detected. Their x- and y-direction coordinates are each averaged to obtain the center of gravity of the object within that image frame. For example, in FIG. 31, their coordinates are averaged to indicate the position shown by the dot. The spatial relationship between the camera field of view and the diverter assembly is known, such as the speed of the belt, and the diverter can be activated at a time calculated to optimally transfer the article from the belt and be directed to such a position.

[0389]

[0389] (When the belt is congested with objects, it is possible to check the watermark blocks to verify the payload integrity before the positions of the watermark blocks are averaged. If one watermark indicates one plastic type and nearby blocks indicate different plastic types, it is found that the watermark and the nearby blocks are marking different articles, and their coordinates should not be used together in a common average.)

[0390]

[0390] The object image can also be passed to a convolutional neural network trained to classify the input image as showing an object belonging to one of a limited number of classifications, such as a bottle or something flat (e.g., a plastic shipping envelope with padding). The pressure or direction of the air released from the air jet diverter is desirably controlled according to such a classification to help ensure that the object is properly transferred. For example, a flat object can act as a sail and less air is applied to transfer the flat object than is applied to transfer a bottle by catching the air (the curved surface of the bottle generally deflects the air around the bottle).

[0391]

[0391] There is a short time interval between when an item is imaged by a camera(s) and when the item is placed for transfer from the conveyor. This interval is generally sufficient to enable cloud processing. For example, a captured image (or a derivative of such an image) can be transmitted to a remote cloud computer service such as Microsoft Azure, Google Cloud, Amazon AWS. A cloud processor(s) can perform some or all of the processing detailed herein, return the resulting data to the waste processing system, and the diverter is controlled according to the resulting data.

[0392]

[0392] Similarly, in a waste stream where some items effectively contain a plastic recycling code in their payload, the recycling codes of other items must be retrieved from a database (e.g., based on a lookup from a decoded GTIN identifier), and the short interval before transfer provides time to query the cloud database for the required recycling codes of the latter items.

[0393]

[0393] It will be recognized that a recycling system using aspects of the present technology does not inherently require a conveyor belt. For example, items can be conveyed through a camera system to a diverter system by other means such as by rollers or free fall. All such alternatives are intended to be included within the term "conveyor belt".

[0394]

[0394] Although the processing of captured images using an unsharp mask filter has been mentioned, other filters (linear or non-linear) can similarly be used to enhance the high-frequency components of the image (or, similarly, to stop enhancing the low-frequency components).

[0395]

[0395] Most of the described configurations operate with grayscale images, but certain improvements in performance (e.g., more reliable recognition of empty belts and watermark decoding in certain modes) may be achievable with higher-dimensional multi-channel images. As described above, an RGB sensor can be used. However, half of the pixels of an RGB sensor are usually green-filtered (due to the prevalence of the common Bayer color filter). Better results can potentially be obtained using a sensor that outputs four (or more) different channels such as R / G / B / ultraviolet, or R / G / B / infrared, or R / G / B / polarization, or R / G / B / white.

[0396]

[0396] Although described in the context of plastic articles, it will be recognized that many aspects of the present technology are applicable to other articles such as those formed of glass or metal.

[0397]

[0397] Similarly, although the present technology has been described in the context of digital watermarks, it will be recognized that any other machine-readable markings such as DotCode and dot peen marking can be used (although certain advantages such as readability from different perspectives may be compromised). The document of U.S. Patent No. 8,727,220 teaches 20 different 2D codes that can be embossed or molded on the outer surface of a plastic container.

[0398] As described above, in some embodiments, the image blocks are analyzed to seek clues indicating whether the block represents a conveyor belt. If the block does not represent a conveyor belt, further analysis is performed, such as block analysis to obtain a watermark reference signal. In other embodiments, the block is first analyzed to confirm the presence of a watermark reference signal, and such reference signal detection serves as a clue. Such reference signal detection triggers further analysis, such as block analysis to obtain payload data and / or analysis of spatially displaced blocks in nearby blocks or subsequent image frames to obtain a reference signal. (Typically, the type of detected reference signal indicates whether the associated watermark is likely to be of the type that has been printed or texture processed, so that the corresponding decoding algorithm is applicable).

[0399] As described above, the two watermarks contemplated by certain embodiments of the present technology differ in three aspects: shape, payload, and signal protocol. It should be understood that each of these attributes is distinct for the purpose of avoiding ambiguity. The two watermarks may differ in shape (printed or texture processed), but may be the same in signal protocol and payload. Similarly, the two watermarks may differ in payload, but may be the same in shape and signal protocol. Similarly, the two watermarks may differ in signal protocol, but may be the same in shape and payload. (The signal protocol encompasses all aspects of the watermark except its shape and payload, such as reference signals, encoding algorithms, output data formats, payload lengths, syntax, etc.).

[0400] Much has been said about watermark blocks having a square shape, but it will be recognized that the printed or texture processed surfaces can be tiled in the same manner as other shaped watermark blocks. For example, a hexagonal honeycomb shape may be composed of triangularly shaped voxels.

[0401]

[0401] Although this technique has been described with reference to the detection of a watermark synchronization (reference) signal using a direct least squares and phase shift approach, other techniques may also be used. One example is a coiled omnidirectional configuration as detailed in U.S. Patent Application Publication No. 20190266749, which is a patent document. Another option is to use an impulse matching filter approach (e.g., correlating with a template consisting of peaks) as detailed in U.S. Pat. Nos. 10,242,434 and 6,590,996, which are patent documents.

[0402]

[0402] It will be recognized that finishing the surface to achieve a matte or translucent finish is, to some extent, in the form of 3D surface shaping / texture processing. Generally, non-ink processing that changes the bidirectional reflectance distribution function (BDRF) of the surface is considered a 3D shaping / texture processing operation here.

[0403]

[0403] For example, the curved surfaces shown in FIGS. 1H to 1L are symmetric in part and / or in cross-section of a sphere, but neither is essential. A part of an ellipse, or a more complex (higher-order) surface, can be used more globally. Some surfaces cut by a plane perpendicular to the nominal surface of a plastic article may have an asymmetric shape. In fact, some such surfaces are characterized by not having a cross-section that is perpendicular and symmetric to the nominal plastic surface.

[0404]

[0404] Pay particular attention to U.S. Provisional Patent Application No. 62 / 956,845, referenced at the beginning of this specification. This application details research by another team of the present assignor, but addresses similar themes such as recycling. The application details features, methods, and configurations that the applicant intends to be incorporated into embodiments of the present technology. (Similarly, the applicant intends that the features, methods, and configurations of the present technology be incorporated into embodiments of the technology of U.S. Provisional Patent Application No. 62 / 956,845). Thus, for example, the identification of an object using both deterministic and probabilistic methods that trigger, for example, analysis routines specific to the object (such as contamination analysis) is detailed in the above-referenced application and is similarly applicable to embodiments of the present technology. That application and the present technology should be read together to achieve a more complete understanding of the present technology. (That application does not re-describe everything here in order to comply with the patent law requirement that the specification must be "concise").

[0405]

[0405] It will be understood that the methods and algorithms detailed above can be executed using a computer device with one or more processors, one or more memories (such as RAM), storage (such as a disk or flash memory), a user interface (which may include, for example, a keypad, a TFT-type LCD or OLED display screen, a touch or other gesture sensor, along with software instructions for providing a graphical user interface), an interconnection of these elements (such as a bus), and a wired or wireless interface for communicating with other devices.

[0406]

[0406] The methods and algorithms detailed above can be implemented in a variety of different hardware processors, including microprocessors, application-specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs). Hybrid configurations of such components can similarly be used.

[0407]

[0407] As a microprocessor, the applicant means a multi-purpose clock-driven integrated circuit having a specific structure, namely, integer and floating-point arithmetic logic units (ALUs), control logic circuits, a set of registers, and a scratchpad memory (known as cache memory) linked by a fixed bus interconnect. The control logic circuit fetches instruction codes from an external memory, and the ALU initiates an example of the operations necessary to execute that instruction code. The instruction codes are obtained from a limited instruction vocabulary that may be considered the native instruction set of the microprocessor.

[0408]

[0408] For a specific embodiment of one of the processes detailed above in a microprocessor, such as the recognition of affine pose parameters from a watermark reference signal of a captured image or the decoding of watermark payload data, one first defines a sequence of algorithmic operations in a high-level computer language such as MatLab or C++ (which may be referred to as source code), and then uses a commercially available compiler (such as the Intel C++ compiler) to generate machine code (i.e., instructions from among the native instruction set, which may be referred to as object code) from the source code. (Both source code and machine code are considered software instructions here.) Then, the process is executed by instructing the microprocessor to execute the compiled code.

[0409]

[0409] Currently, many microprocessors are a fusion of several simpler microprocessors (referred to as "cores"). With such a configuration, multiple operations can be executed in parallel (some elements such as the bus structure and cache memory may be shared among the cores).

[0410]

[0410] Examples of microprocessor architectures include devices from the Intel Xeon, Atom, and Core-I series, as well as various models from ARM and AMD. Since these microprocessor architectures are general-purpose components, they are an attractive option in many applications. Implementation does not require waiting for a customized design / assembly.

[0411]

[0411] The Graphics Processing Unit (GPU) has a close relationship with the microprocessor. The GPU is similar to the microprocessor in that it includes an ALU, control logic circuits, registers, caches, and fixed bus interconnects. However, the native instruction set of the GPU is optimized in common for image / video processing tasks such as moving large blocks of data to and from memory and performing the same operation on multiple data sets simultaneously. Other specialized tasks such as rotating and translating arrays of vertex data to different coordinate systems and interpolating are also commonly supported. Representative vendors of GPU hardware include Nvidia, ATI / AMD, and Intel. As used herein, the applicant intends to refer to the microprocessor that also includes the GPU.

[0412]

[0412] Due to the characteristics of the data being processed and the potential for parallelization, the GPU is an attractive structural option for the execution of certain algorithms among the detailed algorithms.

[0413]

[0413] The microprocessor can be reprogrammed by appropriate software to execute various different algorithms, but the ASIC is not reprogrammable. For example, certain Intel microprocessors may be programmed today to recognize affine pose parameters from a watermark reference signal and programmed tomorrow to prepare the user's tax return, but the ASIC structure does not have this flexibility. Rather, the ASIC is designed and assembled to perform a dedicated task. The ASIC is made for a specific purpose.

[0414]

[0414] The ASIC structure has an array of circuits specially designed to perform specific functions. There are two general classes: gate arrays (sometimes called semi-custom) and full-custom. In the former, the hardware has a regular array structure of (usually) millions of digital logic gates (such as XOR and / or AND gates) that are assembled in the diffusion layer and dispersed on the silicon substrate. Subsequently, a metallization layer with specially designed interconnects is applied to permanently link specific ones of the gates in a fixed topology. (As a result of this hardware structure, many, generally most, of the assembled gates remain unused.)

[0415]

[0415] However, in a full-custom ASIC, the gate configuration is specially designed to achieve the intended purpose (e.g., to execute a specified algorithm). The special design enables more efficient use of the available substrate space, allowing for shorter signal paths and higher performance. A full-custom ASIC can also be assembled to include analog components and other circuits.

[0416]

[0416] Generally speaking, ASIC-based embodiments of watermark detectors and decoders achieve higher performance and consume less power compared to microprocessor-based embodiments. However, the drawback is that significant time and cost are required to design and assemble a circuit tailored to one specific application.

[0417]

[0417] For example, for any particular implementation of the above processing using an ASIC, such as recognizing affine pose parameters from a watermark reference signal in a captured image or decrypting watermark payload data, this case also starts by defining the sequence of operations in source code such as MatLab or C++. However, instead of compiling to the native instruction set of a general-purpose microprocessor, the source code is compiled into a "hardware description language" such as VHDL (IEEE standard) using a compiler such as HDL Coder (available from MathWorks). The VHDL output is then applied to a hardware synthesis program such as Design Compiler by Synopsis, HDL Designer by Mentor Graphics, or Encounter RTL Compiler by Cadence Design Systems. This hardware synthesis program generates output data that specifies a particular array of electronic logic gates that implements this technology in hardware form, as a specialized machine dedicated to the purpose of implementing this technology in hardware form. This output data is then passed to a semiconductor manufacturer, who uses the output data to fabricate a specially designed silicon component. (Suitable manufacturers include TSMC, Global Foundries, and ON Semiconductors).

[0418]

[0418] A third hardware structure that can be used to execute the algorithms detailed above is an FPGA. An FPGA is similar to the semi-custom gate array described above. However, instead of using a metallization layer to provide fixed interconnections between the general-purpose arrays of gates, the interconnections are set by a network of switches that can be electrically configured (and reconfigured) to be either ON or OFF. Configuration data is stored in an external memory and read from that external memory. Due to such a configuration, the links of the logic gates, and thus the function of the circuit, can be freely changed by loading various configuration instructions from the memory that reconfigure how those interconnection switches are set.

[0419]

[0419] Furthermore, an FPGA generally differs from a semi-custom gate array in that it is not composed of simple gates as a whole. Rather, an FPGA can include several logic elements configured to perform complex combinatorial functions. Also, memory elements (such as flip-flops, but more generally complete blocks of RAM memory) can also be included. The same applies to A / D and D / A converters. Also in this case, due to the reconfigurable interconnects that characterize the FPGA, the additional elements as described above can be incorporated at the desired positions in a larger circuit.

[0420]

[0420] Examples of FPGA architectures include the Stratix FPGA from Intel and the Spartan FPGA from Xilinx.

[0421]

[0421] As in the case of using other hardware architectures, the implementation of processing on the FPGA detailed above starts by describing the processing in a high-level language. Further, as in the case of ASIC embodiments, next, the high-level language is compiled into VHDL. However, thereafter, the commands for the interconnect configuration are generated from the VHDL by software tools (such as Stratix / Spartan) specialized for the series of FPGAs being used.

[0422]

[0422] The hybrid of the structures described above can also be further used to execute the detailed algorithms. As a component of the ASIC, a microprocessor integrated on a substrate is used. Such a configuration is called a system-on-chip (SOC). Similarly, the microprocessor can be utilized, among other elements, for the reconfigurable interconnect with other elements of the FPGA. Such a configuration may be called a system-on-programmable chip (SORC).

[0423]

[0423] Yet another type of processor hardware is, for example, neural network chips such as the Intel Nervana NNP-T, NNP-I, and Loihi chips, Google Edge TPU chips, and Brainchip Akida neuromorphic SOCs.

[0424]

[0424] Software instructions for implementing the functions detailed above with the selected hardware can be described by one of ordinary skill in the art without undue experimentation from the description provided herein, and can be described, for example, in C, C++, Visual Basic, Java®, Python, Tcl, Perl, Scheme, Ruby, Caffe, TensorFlow, etc. in combination with relevant data.

[0425]

[0425] The software and hardware configuration data / instructions are generally stored as instructions in one or more data structures held by tangible media such as magnetic or optical disks, memory cards, ROMs, etc., which may be accessed via a network. Some embodiments may be implemented as an embedded system, i.e., a specialized computer system in which the user cannot distinguish between the operating system software and the application software (as is common in the case of basic mobile phones, for example). The functions detailed herein can be realized by operating system software, application software, and / or embedded system software.

[0426]

[0426] The different functions can be implemented in different devices. The different tasks can be exclusively executed by various devices or the execution can be distributed among the devices. Similarly, the description of the data stored in a particular device is also illustrative, and the data can be stored anywhere, such as on a local device, a remote device, in the cloud, distributed, etc.

[0427]

[0427] Other recycling configurations are taught in the patent documents U.S. Patent No. 4,644,151, U.S. Patent No. 5,965,858, U.S. Patent No. 6,390,368, U.S. Patent Application Publication No. 2006 / 0070928, U.S. Patent Application Publication No. 2014 / 0305851, U.S. Patent Application Publication No. 2014 / 0365381, U.S. Patent Application Publication No. 2017 / 0225199, U.S. Patent Application Publication No. 2018 / 0056336, U.S. Patent Application Publication No. 2018 / 0065155, U.S. Patent Application Publication No. 2018 / 0349864, and U.S. Patent Application Publication No. 2019 / 0030571. Alternative embodiments of the present technology use features and configurations from these references.

[0428]

[0428] This specification has described various embodiments. It should be understood that the methods, elements, and concepts detailed in connection with one embodiment can be combined with the methods, elements, and concepts detailed in connection with other embodiments. Although some such configurations have been specifically described, many configurations have not been described due to the number of substitutions and combinations. The applicant also recognizes and intends that the methods, elements, and concepts of this specification can be combined, substituted, or exchanged not only among themselves but also with those known from the referenced prior art. Furthermore, it will be recognized that the detailed techniques can be included with other current and future technologies to obtain beneficial effects. Implementing such combinations is very straightforward for those skilled in the art from the teachings provided in this disclosure.

[0429]

[0429] Although this disclosure has detailed a particular order of operations and a particular combination of elements, it will be recognized that other contemplated methods may re-sort the operations (and in some cases, some operations may be deleted and others added), and other contemplated combinations may involve deleting some elements and adding others.

[0430]

[0430] Although disclosed as an overall system, smaller combinations of the detailed configurations are also separately contemplated (for example, deleting various features of the overall system's features).

[0431] Although specific aspects of the present technology have been described by reference to exemplary methods, it will be recognized that apparatuses configured to perform the operations of such methods are also contemplated as part of the applicant's invention. Similarly, other aspects are described by reference to exemplary apparatuses, and the methodologies performed by such apparatuses are likewise within the scope of the present technology. Additionally, tangible computer-readable media containing instructions for configuring a processor or other programmable system to perform such methods are clearly contemplated.

[0432]

[0432] To achieve a comprehensive disclosure, while complying with the requirements regarding the brevity of the patent law, the applicant hereby incorporates by reference each of the documents referred to herein. (Even if a particular teaching among the teachings of the documents cited above is referred to in connection with the above, those documents are incorporated in their entirety). These references are incorporated into the configurations detailed herein and disclose the technologies and teachings that the applicant intends to be incorporated with the technologies and teachings detailed in this specification.

[0433]

[0433] Considering the various embodiments to which the principles and features described above are applicable, it will be apparent that the embodiments detailed are merely illustrative and should not be construed as limiting the scope of the present invention. [Item of the Invention] [Item 1] A method, defining a data pattern including binary elements spaced at positions within a regular two-dimensional position grid in an electronic file, the pattern defining a first fixed reference signal and a first variable data signal, the first reference signal facilitating the geometric alignment and extraction of the first variable data signal by a decoder presented with a camera-captured image representing a physical counterpart of the data pattern; A step of forming a three-dimensional surface topological pattern of a mold according to a smoothed counterpart of the data pattern, the topological pattern including peaks or depressions having a smooth cross-section so as to facilitate the release of a molded part from the mold, and the method including the step. [Item 2] The method according to item 1, wherein the two-dimensional position grid defines M candidate positions where the binary elements can be located, and the binary elements are arranged at 25% or less of the M candidate positions in the two-dimensional position grid. [Item 3] The two-dimensional position grid consists of an N×N grid of positions, and each position has a grayscale or floating-point value of the first reference signal corresponding to the position. The first variable data signal consists of an M×M array of positions, provided that M < N, and each position in the M×M array of positions has a two-tone value of the variable data signal corresponding to the position. Interpolate the M×M first variable data signal values to generate an N×N array of interpolated values, thereby converting the first variable data signal from a two-tone form to a grayscale or floating-point form. At each of the N×N positions, sum the corresponding values of the first reference signal and the interpolated first variable data signal at a weighted ratio to generate an N×N sum array of values. Apply a threshold processing operation to the N×N sum array of values to generate extreme values. Mark the positions in the N×N grid corresponding to the extreme values. The method according to item 1. [Item 4] The two-dimensional position grid defines M positions, and each of the M positions has a value of the first reference signal corresponding to the M positions, and each value represents the relative darkness of the reference signal at that position. The first variable data signal includes binary symbols, and each of the binary symbols is associated with a corresponding position within the regular two-dimensional position grid. The method includes By sorting the values of the first reference signal, a ranking of the N darkest positions is generated, identifying P of the darkest positions in the ranking of the N darkest positions, leaving Q other positions as they are, and marking each of those P positions with a binary element, marking or not marking each of the Q other positions with a binary element according to whether the corresponding binary symbol of the variable data signal has a first value or a second value The method according to item 1, comprising the step of generating the data pattern by an operation including the above. [Item 5] A template generated by the process according to item 1. [Item 6] A method further comprising the step of molding a plastic container using the molded template, wherein the molded plastic of the container holds the first variable data, according to item 1. [Item 7] A plastic container molded using the method according to item 6. [Item 8] The plastic container according to item 7, wherein the plastic container further has a label, the label includes a printed pattern defining a second fixed reference signal and a second variable data signal, the second reference signal facilitates geometric alignment and extraction of the second variable data signal by a decoder presented with an image of a camera capture showing the printed pattern, the second fixed reference signal is different from the first fixed reference signal, and / or the second variable data signal is different from the first variable data signal. [Item 9] A recycling system including a camera, a processor, and the molded first and second plastic containers according to item 8, wherein the processor geometrically aligns the first variable data signal of the first container using a first alignment signal, extracts the first variable data signal, sorts the first container for recycling based on the extracted first variable data signal, and further (b) geometrically aligns the second variable data signal of the second container using a second alignment signal, extracts the second variable data signal, and sorts the second container for recycling based on the extracted second variable data signal. A recycling system configured to process camera-captured images of the molded plastic containers. [Item 10] A method including the step of molding a plastic container to hold a texture pattern of elements spaced apart at positions within a regular two-dimensional position grid, wherein the pattern defines a first fixed reference signal and a first variable data signal, the first reference signal facilitating geometric alignment and extraction of the first variable data signal by a presented decoder from a camera-captured image of the container, the two-dimensional position grid defining M candidate positions where elements can be located, and in the two-dimensional position grid, the elements are arranged at 25% or less of the M candidate positions. [Item 11] The method according to item 10, wherein more than 75% of the remaining M candidate positions follow the nominal contour of the plastic container. [Item 12] A plastic container molded according to the method according to item 10. [Item 13] The plastic container according to item 10, wherein at the boundary of the pattern, most of the surface area of the container follows the nominal contour of the container. [Item 14] A plastic container that holds a plastic texture pattern and a printed label pattern, wherein the plastic texture pattern includes elements spaced apart at positions within a regular two-dimensional position grid, the plastic pattern defines a first fixed reference signal and a first variable data signal, the first reference signal facilitates the geometric alignment and extraction of the first variable data signal by a decoder presented with a camera-captured image of the container, the printed label pattern includes elements spaced apart at positions within a regular two-dimensional position grid, the printed label pattern defines a second fixed reference signal and a second variable data signal, the second reference signal facilitates the geometric alignment and extraction of the second variable data signal by a decoder presented with a camera-captured image of the container, the second fixed reference signal is different from the first fixed reference signal, and / or the second variable data signal is different from the first variable data signal. [Item 15] The plastic container according to item 14, wherein the second fixed reference signal is different from the first fixed reference signal. [Item 16] The plastic container according to item 14, wherein the second variable data signal is different from the first variable data signal. [Item 17] A plastic container that holds a texture pattern including elements spaced apart at positions within a regular two-dimensional position grid, the pattern defines a first fixed reference signal and a first variable data signal, the first reference signal facilitates the geometric alignment and extraction of the first variable data signal by a decoder presented with a camera-captured image of the container, the two-dimensional position grid defines M candidate positions where elements can be located, and in the two-dimensional position grid, the elements are arranged in 25% or less of the M candidate positions. [Item 18] The plastic container according to item 17, wherein 75% or more of the remaining M candidate positions follow the nominal contour of the plastic container. [Item 19] The plastic container according to item 17, wherein at the boundary of the formed pattern, most of the surface area of the container follows the nominal contour of the container. [Item 20] A method of marking a container to hold a plurality of symbol payloads, generating a data pattern encoding the payload, the pattern including elements spaced at positions within a regular two-dimensional position grid, the pattern defining a fixed reference signal and a variable data signal, the reference signal facilitating the geometric alignment and extraction of the variable data signal by a decoder presented with a camera-captured image representing the physical counterpart of the data pattern; forming a physical counterpart of the pattern on the container by printing or texture processing; the two-dimensional position grid defines M positions, each of the M positions having a value of a reference signal corresponding to the M positions, each value representing the relative darkness of the reference signal at that position, the variable data signal including binary symbols, each of the binary symbols being associated with a corresponding position within the regular two-dimensional position grid, the generating step specifically generating a ranking of the N darkest positions by sorting the values of the reference signal; identifying P of the darkest positions in the ranking of the N darkest positions, leaving Q other positions as they are, and marking each of the P positions with a binary element; marking or not marking each of the Q other positions with a binary element according to whether the corresponding binary symbol of the variable data signal has a first value or a second value. [Item 21] A recycling system comprising an optical reading device and a plastic bottle, wherein the plastic bottle has a label to which a first digital pattern encoding a first identifier is applied in ink, and the plastic is texture-processed with a second digital pattern encoding a second identifier, and the recycling system can sort the bottles by plastic type by decoding any of the identifiers, but the two identifiers are different. [Item 22] A recycling system comprising an optical reading device and a plastic bottle, wherein the bottle is at least partially wrapped in a sleeve, and the sleeve is pre-printed with a mark applied in ink between planar forms and then wrapped around the bottle and adhered to the bottle by heat shrinkage, and the mark applied in ink on the packaging sleeve includes a machine-readable code geometrically distorted by the heat shrinkage and is still readable by the optical reading device to control the sorting of the plastic bottle for recycling. [Item 23] A POS system comprising an optical reading device and a plastic bottle, wherein the plastic bottle has a printed label to which a first digital pattern encoding a first identifier is applied in ink, and the plastic is texture-processed with a second digital pattern encoding a second identifier, and the optical reading device is configured to decode the first identifier but not the second identifier. [Item 24] A bottle comprising a plastic container serving as a base for an ink-printed label, wherein the ink-printed label includes a machine-readable code that enables sorting by the plastic type of the plastic container serving as the base. [Item 25] A system for processing an object stream including a plurality of plastic objects, a conveyor belt, one or more light sources arranged to irradiate the plastic objects on the conveyor belt One or more cameras arranged to capture an image of a plastic object on the conveyor belt, Watermark reading means for decoding from the image (a) first electronic watermark information having a first signal protocol and (b) second electronic watermark information having a different second signal protocol, A sorting machine that reacts to the decoded watermark information, a system comprising. [Item 26] The system according to item 25, further comprising clue detection means for triggering the watermark reading means in response to a promising part of the image. [Item 27] A recycling system comprising plastic food and beverage containers on a conveyor, and further (a) transferring a specific container of the containers from the conveyor based on an electronic watermark printed on the container, and (b) based on an electronic watermark formed by performing a texture treatment on the plastic surface of the container Transfer means for transferring a specific container of the containers from the conveyor, at least one of the containers holding a first electronic watermark formed by printing and a second electronic watermark formed by plastic texture treatment, the first and second electronic watermarks having different signal protocols. [Item 28] A container having a first electronic watermark using a first signal protocol, wherein the payload of the first electronic watermark holds data that enables a recycling system to determine the type of plastic from which the container was manufactured, the container further comprising a second electronic watermark using a second signal protocol different from the first signal protocol, the payload of the second electronic watermark further holding data that enables the recycling system to determine the type of plastic from which the container was manufactured, the first electronic watermark and the second electronic watermark having different signal protocols, but both being useful for plastic recycling. [Item 29] The container according to item 28, wherein the first digital watermark is formed by three-dimensional surface texture processing including a plurality of corner reflector-shaped depressions. [Item 30] The container according to item 28, wherein the first digital watermark is formed by three-dimensional surface texture processing, the second digital watermark is formed by ink printing, and the first digital watermark and the second digital watermark are different in configuration and signal protocol. [Item 31] The container according to item 30, wherein the three-dimensional surface texture processing is asymmetric, and the depth of the deepest depression on the textured surface is not equal to the height of the highest protrusion on the textured surface. [Item 32] The container according to item 30, wherein the three-dimensional texture processing is asymmetric, and the average depth of the depressions on the textured surface is not equal to the average height of the protrusions on the textured surface. [Item 33] The container according to item 30, wherein the three-dimensional surface texture processing is defined according to the sum of a reference signal and a payload signal, and the reference signal is clipped. [Item 34] The container according to item 30, wherein the three-dimensional surface is textured by both depressions and protrusions, and both the depressions and the protrusions hold the payload information of the second digital watermark. [Item 35] The container according to item 30, wherein the three-dimensional surface texture processing represents a plurality of voxels, the voxels are arranged in a grid array of encoded positions, and each encoded position has a side length of 0.0133 inches or less. [Item 36] The container according to item 30, having a front side and a back side, wherein the first digital watermark formed by three-dimensional surface texture processing is present on both the front side and the back side, and the second digital watermark formed by ink printing is present on both the front side and the back side. [Item 37] The payload of the second digital watermark further holds additional data that enables a POS device to identify at least the name and price of an article contained in the container and sold, the payload of the first digital watermark lacks at least part of the additional data, and the first digital watermark and the second digital watermark differ in shape, payload, and signal protocol, the container according to item 30. [Item 38] capturing an image showing an article in a goods stream; extracting a first digital watermark payload from an image showing a first article in the goods stream; determining, based on the payload data extracted from the first digital watermark, that the first article is formed of a first type of recyclable plastic; extracting a second digital watermark payload from an image showing a second article in the goods stream; determining, based on the payload data extracted from the second digital watermark, that the second article is formed of a second type of recyclable plastic; sorting the first article and the second article from the goods stream based on the determined plastic type; A recycling method including. [Item 39] The recycling method according to item 38, wherein the first type of recyclable plastic is polyethylene terephthalate and the second type of recyclable plastic is high density polyethylene. [Item 40] The recycling method according to item 38, including extracting the first digital watermark payload from an image showing a label of the first article. [Item 41] The recycling method according to item 38, including extracting the second digital watermark payload from an image showing a three-dimensional surface texture of the second article. [Item 42] The recycling method according to item 38, wherein the first article is a bottle. [Item 43] The first electronic watermark payload extracted from the description of the first article includes a fixed message part and a variable message part, the variable message part includes a plurality of fields, and one of the fields is a Global Trade Item Number (GTIN) field. The recycling method according to item 38. [Item 44] The first watermark payload includes a code for identifying the plastic used in the first article, the second electronic watermark payload includes linking data, and the recycling method further includes, by using the linking data, obtaining from a database a code for identifying the plastic used in the second article. The recycling method according to item 38. [Item 45] The captured image includes image frames, and the recycling method includes analyzing each of a plurality of pixel blocks in each of the image frames to find clues indicating the presence of watermark data, and performing further image analysis if a clue is found. As a result of finding the first clue, the first electronic watermark payload is extracted, and as a result of finding the second clue, the second electronic watermark payload is extracted. The recycling method according to item 38. [Item 46] The recycling method according to item 45, wherein at least two of the plurality of pixel blocks in the image frame overlap each other. [Item 47] The recycling method according to item 45, wherein the first clue includes detecting a region of pixels each having a value exceeding a threshold. [Item 48] The recycling method according to item 45, wherein the first clue includes detecting an ensemble of spatial image frequencies corresponding to a watermark reference signal. [Item 49] The recycling method according to item 45, wherein the first clue includes an output from a classifier indicating that the pixel block may indicate a plastic article. [Item 50] The recycling method according to item 45, wherein the first clue includes an output from a classifier indicating that the pixel block may not indicate a conveyor belt. [Item 51] The recycling method according to item 45, wherein the first clue is based on a determination that the pixels from most of the sub - blocks within the block have an average value within 1, 2, 3, or 4 digital numbers of the histogram peak based on the previous image. [Item 52] The recycling method according to item 38, wherein the object stream is moved by a conveyor belt passing through a camera, the articles on the conveyor belt enter the camera frame field along a first edge of the camera frame, and the recycling method includes analyzing a plurality of overlapping blocks reaching the first edge of the camera frame to find a clue indicating the possibility of the existence of watermark data, and performing further image analysis when a clue is found. [Item 53] The recycling method according to item 38, wherein the first digital watermark is a watermark encoded according to a signal protocol representing payload data using tiles of 128×128 elements, and when N < 128, the second digital watermark is a watermark encoded according to a second different signal protocol representing payload data using tiles of N×N elements. [Item 54] The recycling method according to item 38, including performing an unsharp masking operation on the image from which the first digital watermark payload is subsequently extracted. [Item 55] Evaluating a first pixel patch and a second pixel patch of the captured image to determine whether the patch may indicate a conveyor belt; a step of performing a watermark process on the first pixel patch as a result of the evaluation and determination that there is a possibility that the first pixel patch does not indicate the conveyor belt; including a step of not performing a watermark process on the second pixel patch as a result of the evaluation and determination that there is a possibility that the second pixel patch indicates the conveyor belt; wherein the first digital watermark is extracted from an image including the first pixel patch; The recycling method according to item 38. [Item 56] The recycling method according to item 38, including a step of irradiating a first area of the object stream with a first light source and irradiating a second area of the object stream with a second light source of a different type from the first light source, wherein the first digital watermark is extracted from a depiction of the first article when irradiated by the first light source and the second digital watermark is extracted from a depiction of the second article when irradiated by the second light source. [Item 57] The recycling method according to item 56, wherein the first light source emits illumination of a first color and the second light source emits illumination of a second different color. [Item 58] The recycling method according to item 56, wherein the first light source emits illumination in a first polarization state and the second light source emits illumination in a second different polarization state. [Item 59] The recycling method according to item 38, wherein the step of extracting the first digital watermark payload includes determining a parameter characterizing the pose of the first article as shown in the first image, the determination using an iterative process starting with a first initial set of affine parameters, and the step of extracting the second digital watermark payload includes determining a parameter characterizing the pose of the second article as shown in the second image and using an iterative process starting with a second initial set of affine parameters different from the first initial set. [Item 60] The object stream is moved by a conveyor belt at a certain speed and includes an image sequence captured at a certain frame rate of the captured images. The sequence includes first, second, and third frames. The recycling method includes analyzing a plurality of image blocks of the first frame to find watermark clues; detecting a watermark clue in a first image block of the first frame; in the second frame, identifying one or more second image blocks for analysis based on the conveyor belt speed and the frame rate; The recycling method according to item 38, including [Item 61] The recycling method according to item 60, including identifying one or more third image blocks for analysis in the third frame based on the conveyor belt speed and the frame rate. [Item 62] The first digital watermark includes a first reference signal, and the second digital watermark includes a second reference signal. When a scrambling test is performed using the second reference signal for the entire range of the affine transformation, that is, using the scaling of the first reference signal in the range between 0.5 and 2.0 in 0.02 increments, and using the rotation of the first reference signal in the range between -90 degrees and +90 degrees in 1-degree increments, and using the translation of the first reference signal over the range of each pixel of possible relative translation, the first reference signal has a correlation of 0.2 > r > -0.2. The recycling method according to item 38. [Item 63] The recycling method according to item 38, wherein one of the digital watermarks includes a traveling salesman path that visits more than 100 points only once and changes direction at most of the points. [Item 64] The recycling method according to item 38, wherein one of the digital watermarks includes a mesh of lines that extend into a region and intersect at the vertices to define a moiré. [Item 65] The recycling method according to item 64, wherein the glints each have a triangular shape. [Item 66] The recycling method according to item 64, wherein the glints each have a rectangular shape. [Item 67] The recycling method according to item 64, wherein the glints are polygons having various numbers of sides. [Item 68] The recycling method according to item 38, wherein one of the digital watermarks includes a pattern of a plurality of curved segments, the plurality of segments being complexly curved in a plurality of directions along their lengths, some segments crossing other segments while other segments do not cross other segments. [Item 69] The recycling method according to item 38, comprising extracting the second digital watermark payload from an image showing a three-dimensional surface texture on the second article including a plurality of depressions each having a corner reflector shape, the corner reflector shape including three mutually perpendicular surfaces. [Item 70] Each of the first article and the second article is marked by two different types of watermarks, namely (a) a first type of watermark printed on either the article or a label attached to the article, the first type of watermark using a first signal protocol, and (b) a second type of watermark formed as a three-dimensional texture on the surface of the article, the second type of watermark using a second signal protocol different from the first signal protocol. The recycling method includes applying first and second different watermark reading algorithms to the captured image, the first watermark reading algorithm being configured to read the watermark using the first signal protocol, and the second watermark reading algorithm being configured to read the watermark using the second signal protocol. The recycling method can separate the first article and the second article from the article stream based on either the first type of watermark or the second type of watermark for marking the article by the payload data extracted by the respective first watermark reading algorithm or second watermark reading algorithm. The recycling method according to item 38. [Item 71] The recycling method according to item 70, further comprising the step of applying the first and second different watermark reading algorithms to a common pixel patch, wherein the common pixel patch is analyzed to confirm the presence of both the first digital watermark and the second digital watermark. [Item 72] The recycling method according to item 70, wherein the first watermark reading algorithm performs geometric synchronization using a first reference signal, and the second watermark reading algorithm performs geometric synchronization without using the first reference signal. [Item 73] The recycling method according to item 72, wherein the second watermark reading algorithm performs geometric synchronization using a second reference signal different from the first reference signal. [Item 74] The recycling method according to item 70, wherein the first signal protocol of the first type of digital watermark represents the first payload by position elements of an encoded position array of a first size, i.e., an encoded position array of 128×128, and the second signal protocol of the second type of digital watermark represents the second payload by position elements of an encoded position array of a second size different from the first size. [Item 75] The recycling method according to item 70, wherein the first signal protocol of the first digital watermark generates an encoded square block pattern having a side dimension of 0.85 inches, and the second signal protocol of the second digital watermark generates an encoded square block pattern having a side dimension smaller than 0.85 inches. [Item 76] The recycling method according to item 70, wherein the first signal protocol of the first electronic watermark has a first payload capacity, and the second signal protocol of the second electronic watermark has a second different payload capacity. [Item 77] The recycling method according to item 70, wherein the first watermark reading algorithm performs a first type of error correction decoding, and the second watermark reading algorithm performs a second type of error correction decoding. [Item 78] The recycling method according to item 70, wherein the first watermark reading algorithm uses a first key for descrambling, and the second watermark reading algorithm uses a second different key for descrambling. [Item 79] The recycling method according to item 70, wherein the first watermark reading algorithm uses a first spreading sequence for demodulation, and the second watermark reading algorithm uses a second different spreading sequence for demodulation. [Item 80] The recycling method according to item 70, wherein the first watermark reading algorithm uses a first scattering table, and the second watermark reading algorithm uses a second different scattering table. [Item 81] A method for processing images showing first and second plastic objects in an object stream, applying a first electronic watermark reading algorithm to a first image showing the first object; determining the plastic type of the first object based on first payload data decoded by the first electronic watermark reading algorithm from the first image, and transferring the first object from the object stream to a first sorting destination according to the determined plastic type of the first object; applying a second electronic watermark reading algorithm to a second image showing the second object; Determine the plastic type of the second object based on the second payload data decoded from the second image by the second electronic watermark reading algorithm, and transfer the second object from the object stream to a second sorting destination according to the determined plastic type of the second object, including the step of, wherein the second sorting destination is the same as the first sorting destination, wherein the first and second plastic objects are two instances of the same type of object, wherein each of the objects is marked with both a first electronic watermark and a second electronic watermark, the first watermark is formed by a first process, the second watermark is formed by a second process different from the first process, the first watermark uses a first signal protocol, and the second watermark uses a second signal protocol different from the first signal protocol, wherein the first electronic watermark reading algorithm is different from the second electronic watermark reading algorithm, the difference including that the first algorithm is configured to read the watermark using the first signal protocol, and the second algorithm is configured to read the watermark using the second different signal protocol, A method in which, based on the reading of either the first or second watermark by each of the respective first or second watermark reading algorithms, an object is transferable from the object stream. [Item 82] The method according to item 81, further including the step of applying the first electronic watermark reading algorithm to an image including the first pixel patch and further applying the second electronic watermark reading algorithm to the image including the first pixel patch, wherein the pixel patch is analyzed to confirm the presence of both the first and second electronic watermarks. [Item 83] The method according to item 81, wherein the formation of the first watermark is realized by printing on a label, and the formation of the second watermark is realized by three-dimensional texture processing of plastic. [Item 84] The method according to item 81, wherein the first signal protocol of the first electronic watermark includes a first reference signal, and the second signal protocol of the second electronic watermark signal does not include the first reference signal. [Item 85] The method according to item 84, wherein the second signal protocol of the second electronic watermark includes a second reference signal different from the first reference signal. [Item 86] The method according to item 81, wherein the first signal protocol of the first electronic watermark includes representing the first payload by elements of positions in an encoded position block of a first size, that is, an encoded position block of 128×128, and the second signal protocol of the second electronic watermark includes representing the second payload by elements of positions in an encoded position block of a second size different from the first size. [Item 87] The method according to item 81, wherein the first signal protocol of the first electronic watermark generates an encoded square block pattern having a side dimension of 0.85 inches, and the second signal protocol of the second electronic watermark generates an encoded square block pattern having a side dimension smaller than 0.85 inches. [Item 88] The method according to item 81, wherein the first signal protocol of the first electronic watermark has a first payload capacity, and the second signal protocol of the second electronic watermark has a second different payload capacity. [Item 89] The method according to item 81, wherein the first signal protocol of the first electronic watermark uses a first encoding algorithm, and the second signal protocol of the second electronic watermark uses a second different encoding algorithm. [Item 90] The method according to item 89, wherein the first encoding algorithm uses a first type of error correction encoder, and the second encoding algorithm uses a second different type of error correction encoder. [Item 91] The method according to item 89, wherein the first encoding algorithm uses a first scrambling key and the second encoding algorithm uses a second different scrambling key. [Item 92] The method according to item 89, wherein the first encoding algorithm uses a first spread modulation sequence and the second encoding algorithm uses a second different spread modulation sequence. [Item 93] The method according to item 89, wherein the first encoding algorithm uses a first scattering table and the second encoding algorithm uses a second different scattering table. [Item 94] Capturing a frame of an image showing a conveyor belt for objects; Inspecting a plurality of pixel blocks arranged across the frame to find a clue indicating a pixel block that may indicate watermark data, the plurality of pixel blocks including a first pixel block and a second pixel block, the pixel block being included in the plurality of pixel blocks having a first pixel interval; Discovering a watermark clue in the first pixel block, as a result, analyzing the first pixel block, and further analyzing N pixel blocks around the first pixel block, the N pixel blocks having a second pixel interval smaller than the first pixel interval; Extracting a first type of watermark from one of the analyzed pixel blocks; A method comprising. [Item 95] Discovering a watermark clue in the second pixel block among the plurality of pixel blocks, as a result, watermarking the second pixel block, and further watermarking M pixel blocks around the second pixel block, the M pixel blocks having a third pixel interval smaller than the first pixel interval; The method according to item 94, further comprising extracting a watermark of a second type different from the first type from one of the watermarked pixel blocks. [Item 96] Irradiating an object stream with a first illumination to capture a first image; Analyzing the first image to detect and decode a first digital watermark formed on a first plastic container; Sending the first plastic container for recycling according to the data decoded from the first digital watermark; Irradiating the object stream with a second illumination different from the first illumination to capture a second image; Analyzing the second image to detect and decode a second digital watermark formed on a second plastic container; Sending the second plastic container for recycling according to the data decoded from the second digital watermark; A recycling method including the above steps. [Item 97] The recycling method according to item 96, wherein the first watermark is formed by printing on the first plastic container, and the second watermark is formed by performing three-dimensional texture processing on the second plastic container. [Item 98] The recycling method according to item 96, wherein the first illumination is red light. [Item 99] The recycling method according to item 96, wherein the first illumination is white light. [Item 100] The recycling method according to item 96, wherein the first illumination includes polarization. [Item 101] The recycling method according to item 96, wherein one of the first and second illuminations includes infrared light or ultraviolet light. [Item 102] In a method of creating a machine-readable pattern for holding recyclable information encoded on the surface of a plastic container, the machine-readable pattern includes a set of multiple peaks in a conversion region, each peak having an assigned phase, and the method includes evaluating more than a hundred different sets of phases assigned to the set of peaks to identify which of the sets generates a pattern having a minimum standard deviation in a spatial region. [Item 103] A plastic container provided with a printed label, wherein the label is printed to hold a first digital watermark encoding a payload P1 for sensing by a POS scanner in a retail store, the first digital watermark including a first reference signal enabling geometric alignment of the first digital watermark for decoding by the POS scanner, the first reference signal including a first set of peaks in a two-dimensional Fourier amplitude region, in a plastic container wherein the surface of the plastic is shaped to hold a second digital watermark encoding a payload P2 different from P1 for sensing by a recycling device, the second digital watermark including a second reference signal enabling geometric alignment of the second digital watermark for decoding, the second reference signal including a second set of peaks in the two-dimensional Fourier amplitude region, A plastic container, wherein the first digital watermark is encoded as a pattern of local luminance or chrominance changes and the second digital watermark is encoded as a three-dimensional texture pattern. [Item 104] The plastic container according to Item 103, wherein the second set of peaks is different from the first set of peaks, so that a POS scanner attempting to read the payload P1 in a retail store is prevented from being confused by the presence of the second digital watermark. [Item 105] The plastic container according to Item 104, wherein some but not all of the peaks of the second set overlap the peaks of the first set. [Item 106] The plastic container according to item 104, wherein none of the peaks of the second set overlap with the peaks of the first set. [Item 107] The plastic container according to item 106, wherein each of the peaks of the first set is present on different radial lines in the two-dimensional Fourier amplitude plot of the first reference signal, and none of the peaks of the second set is present on one of the radial lines. [Item 108] The plastic container according to item 104, wherein the first set consists of M peaks, the second set consists of N peaks, and here N < M. [Item 109] The plastic container according to item 103, wherein the first digital watermark is formed with a spatial resolution of J watermark elements per inch, the second digital watermark is formed with a spatial resolution of K watermark elements per inch, and here K is greater than J. [Item 110] The plastic container according to item 103, wherein the payload P1 holds a message that is longer than the payload P2. [Item 111] The plastic container according to item 103, wherein the first watermark has a first payload-to-reference signal intensity ratio, the second watermark has a second payload-to-reference signal intensity ratio, and the first intensity ratio and the second intensity ratio are different. [Item 112] The plastic container according to item 111, wherein the second intensity ratio is smaller than the first intensity ratio. [Item 113] A plastic container having a label, wherein the plastic is shaped to encode a first watermark including a first reference signal consisting of first blocks arranged in a tile-like manner, and the label is printed to encode a second watermark including a second reference signal consisting of second blocks arranged in a tile-like manner. In the plastic container, when a scrambling test is performed using the blocks of the second reference signal over the entire range of affine transformation, that is, using the enlargement / reduction of the first reference signal blocks in the range between 0.5 and 2.0 in 0.02 increments, and using the rotation of the first reference signal blocks in the range between -90 degrees and +90 degrees in 1-degree increments, and using the translation of the first reference signal blocks over the range covering each pixel of possible relative translation, the blocks of the first reference signal have a correlation of 0.2 > r > -0.2. A plastic container characterized by this. [Item 114] In a plastic container shaped to encode machine-readable data, the plastic shaping is characterized by having the form of a traveling salesman path that visits more than 100 points only once and changes direction at most of the points. [Item 115] In a plastic container shaped to encode machine-readable data, the plastic shaping is characterized by having the form of a mesh of lines that collide at the vertices to extend into a region and define a glint. [Item 116] The plastic container according to Item 115, wherein the glints each have a triangular shape. [Item 117] The plastic container according to Item 115, wherein the glints each have a rectangular shape. [Item 118] The plastic container according to Item 115, wherein the glints are polygons having different numbers of sides. [Item 119] In a plastic container shaped to encode machine-readable data, the plastic molding has a shape of a pattern of a plurality of curved segments, the plurality of segments are complexly curved in a plurality of directions along their lengths, some segments cross other segments while other segments do not cross other segments. A plastic container characterized by this. [Item 120] A product container formed of plastic and having a label layer attached thereto, the label layer having printed artwork including text and a label digital watermark, the label digital watermark including a first synchronization component that enables a dedicated retail store floor terminal to identify and decode the position of the payload of the label digital watermark that holds a retail identifier. In a product container, The product container is The plastic is formed to encode a texture digital watermark, the texture digital watermark including a second synchronization component that enables a recycling device to identify and decode the position of the payload of the texture digital watermark that holds a recycling identifier different from the retail identifier. The product container holds two watermarks, one watermark being useful for a dedicated store floor terminal and the other watermark being useful for recycling, and the watermarks are each associated with two different synchro...

Claims

**Claim 1** A product container made of plastic and having first and second 2D digital watermarks, wherein the first 2D digital watermark adopts a first signal protocol and holds a first plurality of symbol payloads, and the second 2D digital watermark adopts a second signal protocol and holds a second plurality of symbol payloads, and each of the first and second pluralities of symbol payloads holds data indicating the type of plastic from which the product container is made, in a product container. The first 2D digital watermark is formed by three-dimensional surface texture processing, the second 2D digital watermark is formed by ink printing, and the first 2D digital watermark and the second 2D digital watermark are different in configuration and signal protocol. Each of the first and second 2D digital watermarks is present on both first and second side surfaces of the product container that are located opposite to each other. The first 2D digital watermark includes a 2D pattern composed of a plurality of first 2D signal blocks, and the plurality of first 2D signal blocks are arranged in a tiled manner with their edges butted against each other so as to cover the entire surface of the first side surface. A product container characterized by this. **Claim 2** The first signal protocol includes a first reference signal, the second signal protocol includes a second reference signal different from the first reference signal, and the first and second reference signals are capable of identifying the geometric postures of the first and second 2D digital watermarks shown in the captured image. The product container according to claim 1. **Claim 3** The first 2D digital watermark includes a 2D grid at a first position, and each of the first positions has a value of the corresponding first reference signal. The second 2D digital watermark includes a 2D grid at a second position, and each of the second positions has a value of the corresponding second reference signal. The product container according to claim 2. **Claim 4** The product container according to claim 2 or 3, wherein the first reference signal includes a first set of a plurality of peaks in the 2D Fourier amplitude region. **Claim 5** The product container according to claim 4, wherein the second reference signal includes a second set of a plurality of peaks in the 2D Fourier amplitude region, and the first set and the second set are different. **Claim 6** The product container according to claim 5, wherein the first and second sets of a plurality of peaks include different numbers of peaks.

7. The product container according to claim 5, wherein the first set includes peaks having frequencies different from those of the peaks of the second set.

8. The product container according to claim 5, wherein the first set includes peaks having phases different from those of the peaks of the second set.

9. The product container according to claim 1, wherein the first plurality of symbol payloads indicate the type of plastic with reference to a data structure.

10. The product container according to claim 1, wherein the first plurality of symbol payloads further indicate whether the product container is for food or non-food.

11. The product container according to claim 1, wherein the first plurality of symbol payloads further indicate whether the product container includes a single layer or a plurality of layers.

12. The product container according to claim 1, wherein only one of the payloads of the first and second 2D electronic watermarks encodes the global trade item number of the product.

Images