Image Registration for Printing

Abstract

A method for printing on an article may include selecting a shape model. The shape model is generated from a first image and includes a fiducial, a reference position, and a graphic marker, the fiducial being located in the first image. The reference position is based on the position of the fiducial in the first image, and the graphic marker is positioned at a preselected position relative to the reference position. The method may further include positioning the fiducial in a second image of the article by a computing device, establishing the reference position in the second image of the article, positioning the graphic marker at the preselected position in the second image of the article, and printing the graphic at a graphic position on the article, the graphic position on the article corresponding to the preselected position of the graphic marker on the second image of the article.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 579,769, filed on October 31, 2017, which is incorporated herein by reference.

[0003] Field

[0004] This disclosure generally relates to manufacturing systems, and more particularly, to flexible manufacturing systems and related methods.

[0005] Background

[0006] Apparel items, such as clothing items and / or footwear items, are typically offered in a variety of sizes, styles, materials, and / or other characteristics of the product. Additionally, styles are constantly changing, e.g., based on fashion and / or the season of the year. The diversity and frequent changes in product specifications can pose difficulties in manufacturing. Summary of the Invention

[0007] Aspects of the present disclosure may be realized in one or more of the following embodiments:

[0008] 1) A method for printing on an item, comprising:

[0009] selecting a shape model, wherein the shape model is generated from a first image and includes a datum, a reference position, and a graphic marker, wherein the datum is located in the first image, wherein the reference position is based on the position of the datum in the first image, and wherein the graphic marker is located at a pre - selected position relative to the reference position;

[0010] locating the datum in a second image of the item by a computing device;

[0011] establishing the reference position in the second image of the item;

[0012] positioning the graphic marker within the second image of the item at the pre - selected position; and

[0013] printing a graphic on the item at a graphic position, wherein the graphic position on the item corresponds to the pre - selected position of the graphic marker on the second image of the item.

[0014] 2) The method according to 1), wherein, before printing the graphic on the item, the method further comprises displaying the second image of the item with the graphic marker at the pre - selected position.

[0015] 3) The method according to 1) or 2), wherein positioning the graphic marker includes rotating the graphic marker relative to the second image of the item.

[0016] 4) The method according to any one of 1)-3), wherein the preselected position includes a first direction coordinate, a second direction coordinate, and a rotational orientation.

[0017] 5) The method according to any one of 1)-4), wherein before printing the graphic on the article, the method further includes determining the size of the article based on a second image of the article.

[0018] 6) The method according to any one of 1)-5), further including scaling the graphic mark based on the size of the article.

[0019] 7) The method according to any one of 1)-6), wherein the reference includes a logo provided on the article.

[0020] 8) The method according to 7), wherein after positioning the reference in the second image and before printing the graphic on the article, the method further includes scaling the graphic mark based on the size of the logo.

[0021] 9) The method according to any one of 1)-8), wherein the reference includes a pattern provided on a shoe last for supporting the article.

[0022] 10) The method according to any one of 1)-9), wherein the reference includes a surface pattern of the article.

[0023] 11) The method according to any one of 1)-10), wherein the article is a footwear article.

[0024] 12) The method according to any one of 1)-11), wherein the article is a clothing article.

[0025] 13) A manufactured article produced by the method according to any one of items 1)-12).

[0026] 14) One or more computer-readable storage media storing computer-readable instructions which, when executed by a computer, cause the computer to perform the method according to any one of items 1)-12).

[0027] 15) A method for printing on an article, comprising:

[0028] Locating a reference in an image of the article by a computing device;

[0029] Establishing a reference position in the image based on the position of the reference;

[0030] Locating a graphic mark within the image relative to the reference position; and

[0031] Print a graphic on the article based on the reference position of the graphic marker for positioning.

[0032] 16) The method according to 15), wherein a shape model of the article is used to perform positioning of the reference, establishing the reference position, and positioning the graphic marker.

[0033] 17) The method according to 15) or 16), wherein the reference includes a logo provided on the article, and wherein after positioning the reference and before printing the graphic on the article, the method further includes scaling the graphic marker based on the size of the logo.

[0034] 18) The method according to any one of 15)-17), wherein the reference is coupled to a printing bed of a printing device.

[0035] 19) An article manufactured by the method according to any one of items 15)-18).

[0036] 20) One or more computer-readable storage media storing computer-readable instructions that, when executed by a computer, cause the computer to perform the method according to any one of items 15)-18).

[0037] 21) A method for printing on an article, comprising:

[0038] Locate a reference in an image of the article, wherein the reference is attached to a printing bed of a printing device;

[0039] Use the reference to establish a reference point for printing on the article;

[0040] Provide a raster or vector graphic file including a graphic;

[0041] Print a representation of the graphic on the article; and

[0042] Remove the article from the printing bed.

[0043] 22) The method according to 21), wherein the raster or vector graphic file is a second raster or vector graphic file, and the method further includes:

[0044] Receive a first raster or vector graphic file including the graphic;

[0045] Transform the graphic based on the reference point; and

[0046] Generate the second raster or vector graphic file including the transformed graphic.

[0047] 23) The method according to 22), wherein the transformation includes at least one or more of the following:

[0048] Mirroring, rotating, stretching, resizing, coloring, cropping, cutting, splitting, or obscuring the transformed graphic.

[0049] 24) The method according to any one of 21)-23), wherein the raster or vector graphic file is received via a computer network together with a customer order for the item.

[0050] 25) The method according to any one of 21)-24), further comprising:

[0051] Storing the raster or vector graphic file on a computer-readable storage medium.

[0052] 26) The method according to any one of 21)-25), wherein locating the reference includes automatically detecting, by a computer, physical features of the footwear item, including one or more of a printed object, a heel edge, a bite line, a stud, a vamp edge, or any surface pattern.

[0053] 27) An item manufactured by the method according to any one of items 21)-26).

[0054] 28) One or more computer-readable storage media storing computer-readable instructions that, when executed by a computer, cause the computer to perform the method according to any one of items 21)-26). Brief Description of the Drawings

[0056] Figure 1 Depicts an exemplary embodiment of a system for implementing the disclosed technology.

[0057] Figure 2 Depicts a system for processing an exemplary footwear item Figure 1 .

[0058] Figure 3 Depicts a flowchart outlining an exemplary method of printing a graphic on an item.

[0059] Figure 4 Depicts a flowchart outlining another exemplary method of printing a graphic on an item.

[0060] Figure 5 Depicts a screenshot showing exemplary configuration parameters.

[0061] Figure 6 Depicts a flowchart outlining an exemplary method of generating a shape model profile.

[0062] Figure 7Depicts an exemplary embodiment of a fiducial marker.

[0063] Figure 8 Depicts another exemplary embodiment of a fiducial marker.

[0064] Figure 9 Depicts Figure 8 a detailed view of the fiducial marker of

[0065] Figure 10 Depicts another exemplary embodiment of a fiducial marker.

[0066] Figure 11 Depicts another exemplary embodiment of a fiducial marker.

[0067] Figure 12 Depicts another exemplary embodiment of a fiducial marker.

[0068] Figure 13 Depicts another exemplary embodiment of a fiducial marker.

[0069] Figure 14 Depicts another exemplary embodiment of a fiducial marker.

[0070] Figure 15 Depicts an exemplary device for positioning a fiducial marker.

[0071] Figure 16 Depicts an exemplary embodiment of a printing system.

[0072] Figure 17 Depicts a screenshot showing an exemplary module for generating a shape model profile.

[0073] Figure 18 Depicts a screenshot showing other exemplary modules for generating a shape model profile.

[0074] Figure 19 Depicts a screenshot showing other exemplary modules for generating a shape model profile.

[0075] Figure 20 Depicts a screenshot showing other exemplary modules for generating a shape model profile.

[0076] Figure 21 Depicts a screenshot showing other exemplary modules for generating a shape model profile.

[0077] Figure 22 Depicts an exemplary computing system for implementing the disclosed technology.

[0078] Figure 23Depicts a flowchart outlining an exemplary method of printing on an article using a shape model profile.

[0079] Figure 24 Depicts a flowchart outlining an exemplary method of printing on an article.

[0080] Figure 25 Depicts a flowchart outlining an exemplary method of printing on an article.

[0081] Figure 26 Depicts a flowchart outlining an exemplary method of printing on an article.

[0082] Figure 27 Depicts a flowchart outlining an exemplary method of printing on an article.

[0083] Figure 28 Depicts a flowchart outlining an exemplary method of printing on an article.

[0084] Figure 29 Depicts a flowchart outlining an exemplary method of printing on an article.

[0085] Detailed description

[0086] General principles

[0087] The systems, methods, and devices described herein should not be construed as limiting in any way. On the contrary, this disclosure focuses, both individually and in various combinations and sub - combinations with each other, on all novel and non - obvious features and aspects of the various disclosed embodiments. The disclosed systems, methods, and devices are not limited to any specific aspect or feature or combination thereof, and the disclosed things and methods do not require the presence of any one or more specific advantages or the solution of any one or more specific problems. Moreover, any feature or aspect of the disclosed embodiments can be used in various combinations and sub - combinations with each other, as would be recognized by one of ordinary skill in the relevant art based on the information disclosed herein.

[0088] As used in this application, unless clearly indicated otherwise in context, the singular forms "a", "an", and "the" include the plural forms. Additionally, the term "include" means "comprise". Further, the term "coupled" encompasses mechanical, electrical, magnetic, optical, and other physical ways of coupling or linking items together, and does not exclude the presence of intermediate elements between the coupled items. Also, as used herein, the term "and / or" represents any one or combination of items in the phrase.

[0089] Although the operations of some of the methods disclosed herein are described in a particular, sequential order for presentation purposes, it should be understood that this description method includes rearrangements unless the specific language stated below requires a particular order. For example, operations described in sequence may be rearranged or performed concurrently in some cases. In addition, for simplicity, the figures do not show the various ways in which the disclosed things and methods can be used in combination with other things and methods. Further, the description sometimes uses terms such as "provide", "produce", "generate", "display", "receive", "emit", "determine", and "select" to describe the disclosed methods. These terms are high-level descriptions of the actual operations performed. The actual operations corresponding to these terms will vary depending on the particular implementation and can be readily discerned by those skilled in the art while having the benefit of this disclosure.

[0090] The terms "system" and "device" are used interchangeably herein. Unless the context clearly indicates otherwise, neither of these terms implies any limitation on the type of computing system or computing device. Generally, a computing system or computing device can be local or distributed and can include any combination of dedicated hardware and / or general-purpose hardware with software that implements the functions described herein.

[0091] For purposes of illustration, the detailed description uses terms such as "determine" and "use" to describe computer operations in a computing system. In these cases, these terms are high-level descriptions of the operations performed by a computer and should not be confused with operations performed by a human. The actual computer operations corresponding to these terms vary depending on the implementation.

[0092] For better understanding, an operating theory, scientific principles, or other theoretical descriptions are provided herein with reference to the apparatus or method of the present disclosure, and are not intended to limit the scope. The apparatus and method in the appended claims are not limited to those that operate in the manner described by such operating theory.

[0093] Any disclosed method can be implemented as computer-executable instructions stored on one or more computer-readable media (e.g., computer-readable media such as one or more optical media discs, volatile memory components (such as DRAM or SRAM), or non-volatile memory components (such as a hard disk drive)) and executed on a computer (e.g., any commercially available computer, including a smart phone or other mobile device including computing hardware). Any computer-executable instructions for implementing the disclosed technology, as well as any data created and used during implementation of the disclosed embodiments, can be stored on one or more computer-readable media (e.g., computer-readable storage media). The computer-executable instructions can be, for example, part of a dedicated software application or a software application accessed or downloaded via a web browser or other software application (such as a remote computing application). Such software can be executed using one or more networked computers on, for example, a single local computer (e.g., threads executing on any suitable commercially available computer) or in a networked environment (e.g., via the Internet, a wide area network, a local area network, a client-server network (such as a cloud computing network) or other such networks).

[0094] For clarity, only certain selected aspects of the software-based implementation are described. For example, it should be understood that the disclosed technology is not limited to any specific computer language or program. For example, the disclosed technology can be implemented by software written in C, C++, Java, or any other suitable programming language. Similarly, the disclosed technology is not limited to any particular computer or hardware type.

[0095] In addition, any software-based implementation (including, for example, computer-executable instructions for causing a computer to execute any disclosed method) can be uploaded, downloaded, or remotely accessed via suitable communication means. Such suitable communication means include, for example, the Internet, the World Wide Web, an intranet, a software application, a cable (including a fiber optic cable), magnetic communication, electromagnetic communication (including RF, microwave, and infrared communication), electronic communication, or other such communication means.

[0096] As used herein, the term "module" refers to a portion of hardware and / or computer-executable instructions that can be used to implement a particular method disclosed herein. Such a module can be implemented with a processor and a memory storing computer-executable instructions that, when executed by the processor, cause the processor to execute the particular method associated with the module. In some examples, such a module can be implemented using digital logic gates implemented as an application specific integrated circuit (ASIC), a system on a chip, and / or programmable logic (e.g., in a field programmable gate array (FPGA) integrated circuit).

[0097] A computer-readable medium is any available medium that can be accessed within a computing system. By way of example, and not limitation, for a computing system, the computer-readable medium includes memory and / or storage devices. As should be readily appreciated, the term computer-readable storage medium includes media for data storage, such as memory and storage devices, and does not include transmission media, such as modulated data signals.

[0098] Any trademarks used herein are the property of their respective owners.

[0099] Introduction to the disclosed technology

[0100] For example, automated manufacturing processes can improve efficiency and quality control. Despite these and other advantages, however, typical automated manufacturing processes also have disadvantages. For example, when the product to be manufactured changes, typical automated manufacturing processes can be extremely difficult or impossible to create and / or update. As a result, many products require manual handling, which increases labor costs and production time and reduces quality control.

[0101] In particular, known automatic printing systems, such as those used in the publishing industry, rely on an initial manual or automatic alignment of the substrate to be printed relative to a physical alignment reference point or barrier, including physically abutting a stack of sheets against a raised structural barrier (e.g., an edge, wall, fence, etc.) of a loading platform, tray, rack, etc. In addition, in addition to the precise and accurate initial alignment of the substrate, accurate printing on existing printing systems must also rely on the correct and consistent shape and size of the substrate to be printed. Any variation in the shape, size, position, or orientation of the substrate will result in misalignment of the graphics relative to the substrate and / or incorrectly scaled printing.

[0102] In addition, after the initial alignment, many printing systems then reposition the substrate from a loading position to a printing position, during which the initially accurately aligned substrate may be translationally positioned due to a number of effects (e.g., sliding relative to the underlying substrate or alignment structure, physical contact with an object / obstacle, displacement due to air flow or vibration, friction along the travel path, etc.), resulting in misalignment of the substrate relative to the graphics application structure (e.g., a print head, stamp, roller, embosser, etc.) and thus misalignment of the printed graphics relative to the substrate.

[0103] Although some printing systems allow a user to specify one of several substrate size options (e.g., "letter size", "A4", "legal size", etc.) before printing and to enlarge or reduce the image size by specifying a percentage increase or decrease, such options typically must be entered by the user before running a print job (single print or batch print) and rely on the accuracy of the user input. However, such user input and the initial alignment of the substrate as described above constitute a form of "dead reckoning". Once set and printing begins, the printing system proceeds without diagnosing any changes in the size, shape, position, orientation of the substrate, or surface variations of the substrate, whether initial or subsequently introduced, any one or all of which can inherently result in dimensional, shape, position, orientation defects and / or distortion of the printed pattern relative to the substrate.

[0104] As will be recognized, existing printing systems widely used in publications are designed to print on relatively flat (e.g., two-dimensional) substrates (e.g., paper, cardboard, metal, fabric, etc.). Printing systems configured to print on three-dimensional objects rely to a large extent on the substrate / article being symmetric (e.g., circular, as in the case of Mars, Incorporated's M&M chocolate candies) to ensure consistent placement of the pattern relative to the article. Alternatively, when printing on articles of irregular size and shape, the pattern size remains constant and inconsistent positioning relative to the article is tolerated, e.g., in the case of peanut M&M candies.

[0105] However, the inventors are aware that there is currently no printing system that can automatically identify the size, shape, rotation, z-axis height, contour, or other variations of each article loaded onto the printing system, whether loaded individually or multiple (e.g., loaded in pairs or in batches), nor is there a printing system that individually, automatically, and virtually (e.g., in its digital form) specifically alters / transforms the digital pattern relative to any or all of the detected conditions of the article on which printing is to be performed, these conditions including: article type (e.g., based on the presence or absence of one or more optically detectable features, including features detectable individually or predominantly at wavelengths normally invisible to the human eye (e.g., but not limited to wavelengths in the ultraviolet or infrared ranges)), size, position (e.g., on one or both of the X and Y axes), orientation (e.g., on any or all three-dimensional rotation axes), height (on the Z axis), contour (e.g., Z-axis height variations across the visible surface of the entire substrate).

[0106] For example, the inventors have recognized that there is currently no printing system capable of receiving the simultaneous loading of two different shoes of different types (e.g., a high-top basketball shoe for the left foot and a low-top running shoe for the right foot), different sizes (e.g., a children's size 3T shoe and an adult men's size 9 shoe), where one shoe is translated relative to the other shoe along one or both of the X-axis and the Y-axis, and / or where the longitudinal axis or other axis of one of the loaded shoes is rotated off-axis relative to the respective axis / axes of the other loaded shoe and / or relative to the printing mechanism (e.g., an inkjet printhead), where the shoes present different colors and / or different aesthetic designs relative to each other, and where the graphics to be printed on one shoe differ in type, size, position, orientation from the graphics to be printed on the other shoe, and where after the manual or automatic loading of the shoes is complete, the printing system automatically identifies the respective type, size, position, orientation, and / or color condition of each shoe, digitally alters one or more graphics to be printed on each shoe in one or more of type, size, position, orientation, and / or color, and correctly prints the graphics on each shoe with respect to type, size, position, orientation, and color.

[0107] One of ordinary skill in the relevant art will recognize that, given the entire description, drawings, and claims provided herein, embodiments of the present invention provide a printing system with such capabilities. In one or more embodiments, an existing printer can be retrofitted with one or more inventive features to endow the printer with one or more capabilities it did not previously possess. In one or more other embodiments, a printer can be initially designed and produced such that it includes one or more inventive features, constituting a new printer with one or more capabilities not present in prior printers.

[0108] As used herein, the term "article" includes "wearable articles", such as clothing articles and / or footwear articles (collectively and / or alternatively referred to herein as "articles"). For example, such articles are typically made of soft goods (e.g., textiles, polymers, and / or other natural or synthetic materials) that are relatively bendable and / or stretchable compared to hard goods (e.g., metals, woods, etc.), although some articles may also include relatively rigid and / or inelastic structures (e.g., metal shoelace holes, rigid adornments, etc.). Such articles can also be produced in a variety of sizes and have styles, colors, materials, and other features that often change according to fashion, season, and / or other reasons. Such articles are examples of manufactured products with the above variations. The demand for customization and / or personalization further increases these variations. Accordingly, developing an automated manufacturing process for articles is extremely difficult and time-consuming.

[0109] This disclosure provides exemplary methods, apparatuses, and / or systems that can be used, for example, to automate and / or more rapidly automate the manufacturing process of an item. For example, the techniques disclosed herein can be used to generate a shape model that can be used in a manufacturing process (e.g., printing) to account for variations in the size, color, material, and / or other characteristics of such an item.

[0110] As used herein, a shape model is a representation of the extractable edges of an item or figure in an image that can be used to find corresponding shapes in other items or images of other items. For example, a shape model can be generated based on an item having a particular color scheme and size, and then the system can use that shape model to detect one or more other items having various other color schemes and / or sizes. In other words, the system can use a shape model of an item (e.g., generated from a men's size 9 with a black / white color scheme) to identify and process multiple different items (e.g., a men's size 12 with a red / green color scheme). For example, this can increase the ease of generating an automated process, reduce the reliance on many imprecise manual processes (e.g., loading and aligning items (including, e.g., asymmetric and three-dimensional items) onto a printing system), and the production time necessarily involved in such manual processes, and / or improve the quality control of graphics printed onto an item (including the accuracy and precision / consistency of placement, orientation, size / scale, etc.).

[0111] The system can generate a shape model based on an image of an item and can identify other items based on images of other items.

[0112] In certain embodiments, the disclosed methods, apparatuses, and / or systems can be used to generate a shape model when the target is a single color and intensity on a uniform background and / or has sharp edges with no or few interfering features, and / or when the target is uneven, has uneven lighting, has complex graphics that interfere with the target, and / or has texture. In some embodiments, the disclosed methods, apparatuses, and / or systems can be configured to, for example, help isolate the useful edges needed for an accurate shape model from noise, distortion, and / or other interference in the image.

[0113] In some embodiments, the shape model can be used to quickly and accurately align an item with manufacturing equipment (e.g., a printer). For example, in certain embodiments, the position of an item can be determined within the processing space of the manufacturing equipment.

[0114] In some embodiments, fiducials can be used to generate shape models and / or align an item with manufacturing equipment. In some embodiments, fiducials can be, for example, physical features of the item (e.g., logos, edges, seams, perforations, patterns, etc.). Additionally or alternatively, fiducials can be one or more other components coupled to or adjacent to the item and / or processing equipment (e.g., a printer).

[0115] In some embodiments, the processing equipment can include a work space that can be automatically moved to align a fiducial plane with the plane of the processing equipment.

[0116] It should be noted that although many of the examples described herein relate to printing graphics on an item, various other output devices can be used with the disclosed technology in addition to printing systems. For example, the output device can be a cutting tool, such as a laser or other type of device (e.g., a sewing machine or an adhesive applicator).

[0117] Exemplary Embodiments

[0118] Figure 1 A schematic diagram showing an exemplary embodiment of a flexible manufacturing system 100 (also referred to as system 100) is presented. In this basic configuration, system 100 can include a computing system 102, an output device 104 (e.g., a two-dimensional or three-dimensional printer, etc.), an image sensor 106 (e.g., an image capture device such as a camera and / or a still camera), and a display 108 (e.g., a visual monitor device). The computing system 102 can be coupled to the output device 104, the image sensor 106, and the display 108. The computing system 102 can include components that control the functions of the output device 104, the image sensor 106, and the display 108. One or more additional components can be included in the system.

[0119] In some embodiments, system 100 can be used with a variety of items, including wearable items such as footwear items and / or clothing items. For example, footwear items can include running shoes, soccer shoes, football shoes, rugby shoes, basketball shoes, baseball shoes, athletic shoes, hiking boots, sandals, socks, and other types of shoes. For example, clothing items can include hats, scarves, jackets, shirts, dresses, sweaters, gloves, underwear, ties, vests, shorts, trousers, and other wearable and non-wearable clothing. Items herein can also include wearable and / or portable "accessory items", including bags, wallets, backpacks, jewelry, hair retaining ornaments (e.g., hairpins, hairbands, etc.), watchbands / watches, belts, etc.

[0120] System 100 can be used with other objects that can be processed by output device 104, including "equipment items", and more specifically including "sports equipment items" such as shin guards, knee pads, elbow pads, shoulder pads, balls, bats, helmets, hockey sticks, support bands, and other forms of equipment having printable surfaces.

[0121] Although several types of items are specifically identified herein as examples of items on which printing can be performed in the disclosed inventive embodiments, the intended uses and embodiments extend to a wide range of items, including consumer and industrial goods and / or packaging, including tools, documents, medical supplies and pharmaceuticals, textiles, food, kitchen utensils, bedding, automotive parts, electronic parts or components, multimedia items (e.g., optical storage media, memory cards), etc. Broadly speaking, the inventive embodiments can include use with any item that includes a printable surface (e.g., a surface on which printing can be performed) and that can be loaded onto a printer or relative to which a printer can be positioned for printing on the item.

[0122] In some embodiments, system 100 can include various types of components for applying a graphic or any type of design or image to an item. Additionally, the process of applying the graphic can occur during the manufacture (e.g., assembly) of the item and / or after the item has been manufactured.

[0123] In some embodiments, a graphic or graphic image can be applied to a footwear item after the footwear item has been manufactured in a three-dimensional form including an upper and a sole structure. In some embodiments, a flexible manufacturing system can be used at a retail location to apply a user-selected graphic to a footwear item and / or a clothing item.

[0124] As used herein, the term "graphic" refers to any visual design feature, including: various types of photographs, logos, text, illustrations, lines, shapes, patterns, images, and any combination of these features. Moreover, the term graphic is not intended to be limiting and can incorporate any number of continuous or discontinuous visual features.

[0125] For example, in an exemplary embodiment, the graphic can include a logo applied to a small area of a footwear item. In another embodiment, for example, the graphic can include a color applied to a large area of one or more regions or the entire footwear item.

[0126] In some embodiments, output device 104 can include various components for directly applying a graphic to an item. In certain embodiments, output device 104 can be a printing system. Printing system 104 can include one or more individual printers. In some embodiments, the printing system can include two or more printers networked together.

[0127] As used herein, the term "article printing device" may refer to any type of system capable of printing on textiles, footwear articles, apparel articles, or other objects. This includes printers, plotters, 3D printers, and / or 3D printing systems. The article printing device may use any type of printable material that can be dispensed by the printer, including but not limited to natural or synthetic inks (e.g., ultraviolet ("UV") curable inks, acrylic resin inks, polyurethane inks, TPU inks, silicone inks, etc.), adhesives, sealants, and light-reactive materials (e.g., photochromic materials, photoluminescent materials, etc.).

[0128] The printing system may utilize various types of printing techniques. These techniques may include: toner-based printing, liquid inkjet printing, solid ink printing, dye-sublimation printing, inkless printing (including thermal printing and UV printing), resin deposition, microelectromechanical systems ("MEMS") jet printing technology, and any other suitable printing method. In some embodiments, the printing system may utilize a combination of two or more different printing techniques. The type of printing technique used may vary depending on factors such as: the material of the target article, the size and / or geometry of the target article, the desired properties of the printed image (such as durability, color, ink density, etc.), as well as printing speed, printing cost, and maintenance requirements (to name just a few).

[0129] In one or more embodiments, the printing system may utilize an inkjet printer in which ink droplets can be ejected onto a substrate (such as an inner or outer panel of a formed shoe upper). Using an inkjet printer may, for example, allow for variations in color and ink density. This arrangement may also allow for a certain separation between the print head and the target object, which can facilitate printing directly on an object having a certain curvature and / or surface texture.

[0130] In some embodiments, system 100 may include one or more cameras, which may include image sensor 106. For example, in one embodiment, the camera may be a "Basler ace acA4600-10uc" camera from Basler AG of Ahrensburg, Germany, which has an "MT9F002 CMOS" image sensor from ON Semiconductor of Phoenix, AZ. In other embodiments, two or more cameras may be used to generate a three-dimensional image. In other embodiments, one or more analog cameras may be used to generate an image, and the generated analog image may be converted into a digital image format.

[0131] System 100 may include components for facilitating the alignment of a printed graphic onto an item. For example, image sensor 106 may capture an image of the item. Computing system 102 may use the image from image sensor 106 to align the position of the item in the output device workspace with the coordinate system of the output device (this is sometimes referred to as creating "real-world" coordinates). The image may be represented in a digital image format (e.g., RAW, JPEG, TIFF, etc.) and may be displayed on display 108. Using computing system 102 and leveraging the coordinates of the image of the item, the coordinates of the item, and the coordinates of the output device, a user may position a graphic on the image of the item, and output device 104 may place the graphic on the item at a physical location corresponding to the image location. In other words, the position of the graphic displayed on the display relative to the item will be the position at which the graphic will be printed on the item.

[0132] The coordinate system may be a Cartesian coordinate system, a polar coordinate system, a cylindrical coordinate system, a spherical coordinate system, or other types of coordinate systems. The coordinate system may be used, for example, to determine the positioning, rotation, size, and / or other relationships of the graphic with respect to the item, as further explained below.

[0133] System 100 may include additional components for facilitating the alignment of a printed graphic on an item. In some embodiments, it may be useful to provide a way for a user to align the item with the printing system to ensure that the graphic is printed in a desired portion (i.e., location) of the item. For example, in some embodiments, system 100 may include components for pre-aligning the item with the printer in a manner that accommodates various types, shapes, and / or sizes of items. In certain embodiments, system 100 may include lasts, jigs, and / or other types of devices configured to hold and / or manipulate the item.

[0134] In some embodiments, system 100 may include audio and / or haptic devices. For example, the system may emit a tone when the graphic is aligned or misaligned.

[0135] The display 108 may include a touch screen, a monitor, and / or other viewing devices. The display 108 may provide, for example, a graphical user interface (“GUI”) that allows user interaction.

[0136] Reference Figure 2 , the system 100 may be used, for example, to apply graphics to an article, such as the footwear article 200. In certain embodiments, the article 200 may take the form of a sports shoe (such as a running shoe). However, it should be noted that the system 100 may be used with any other type of article or object having a printable surface. Additionally, although the article 200 shown is only one article, the system 100 may be used to apply graphics to two or more articles simultaneously supported by the system 100, including articles that make up a pair of shoes (e.g., a left article and a right article).

[0137] In some embodiments, the article 200 may include an upper 202 and a sole structure 204. Generally, the upper 202 may be any type of upper that includes a printable surface, regardless of design, shape, size, and / or color. For example, in an embodiment where the article 200 is a basketball shoe, the upper 202 may be a high-top upper shaped to provide a high level of ankle support. In an embodiment where the article 200 is a running shoe, the upper 202 may be a low-top upper. In other embodiments, the article 200 may be various other types of footwear (such as sandals) having other types of uppers (e.g., straps).

[0138] In some embodiments, the upper 202 may have a generally contoured shape approximating the shape of a foot. For example, the outer side portion 206 of the upper may be generally contoured rather than substantially flat. Additionally, it will be understood that the shape of the outer side portion 206 and any other portion of the upper 202 may vary from one embodiment to another. In particular, the principles described herein for applying graphics to footwear articles are not limited to articles having any particular geometry and / or shape, but are widely applicable to a variety of articles and article shapes.

[0139] In some embodiments, the upper 202 may be configured to have one or more design features. For example, the upper 202 may include a design feature 208 disposed on the outer side portion 206. In the illustrated embodiment, the design feature 208 takes the form of an oval-like design on the upper 202. However, in other embodiments, the design feature 208 may be configured as any one of a variety of indicia, graphic images, or other design features. Examples of various design features that may be incorporated into the upper 202 include logos, numbers, letters, graphics, decorative elements, and other kinds of design features. Additionally, in some embodiments, the design feature 208 may be applied to the upper 202 using ink, for example, by using a printer. In other embodiments, the design feature may include a separate layer of material attached to the substrate of the upper 202.

[0140] As described above, the system 100 can be used to place a graphic on an item 200, for example. Figure 3 An exemplary method 300 for placing a graphic on an item that can be performed by the system 100 is shown. The method 300 may include selecting a profile for the item (process block 302), performing image registration of the item (process block 304), and printing a graphic on the item (process block 306).

[0141] Figure 4 An exemplary method 400 for placing a graphic on an item that can be performed by the system 100 is shown. The method 400 may include selecting an operation mode (decision block 402), such as automatic (process block 404) or manual (process block 406).

[0142] If the user selects the automatic mode, the user may, for example, scan a product identification image linked to the profile (e.g., a barcode, a quick response (“QR”) code, a watermark, etc.). The system 100 may extract features from the profile (process block 408). The features of the profile may include information about the item, such as style, color, and / or size (to name a few). The system 100 may perform a database lookup (process block 410). If no configuration parameters are found, the user may reselect the mode (decision block 402) and attempt to identify and / or construct the configuration again. If configuration parameters are found, a configuration build may be provided (process block 412). The user may preview the configuration parameters (process block 414). The configuration parameters may be stored as, for example, a comma-separated value (CSV), Excel, database, or other suitable file type.

[0143] If the user selects the manual mode, the user can provide input (e.g., via a keyboard, touch screen, mouse, and / or microphone) or select a configuration file. System 100 can search for and locate the configuration file (e.g., an.xml file stored locally or accessed from a network file location). If a configuration file is found, the user can preview the configuration parameters (process block 414).

[0144] In either the automatic or manual mode, system 100 can perform image registration (process block 416). If the image registration fails, the user can re-select the mode (decision block 402) and reconstruct the configuration. Alternatively, the user can view the preview (process block 414) and attempt image registration again (process block 416). If the image registration is successful, system 100 can print a graphic on the article (process block 418).

[0145] In some embodiments, the configuration parameters can include a shape model, XY offset, scale, angle, print configuration, transfer configuration number, and graphic producer, among other parameters. For example, Figure 5 A screenshot of an exemplary configuration parameter 500 is shown. The parameter 500 can include, for example, a base name (e.g., model number), style, color number, size, transfer parameters, print configuration, and / or the number of layers of the left article and / or the right article. For each layer (e.g., both the left article and the right article have four layers), the parameters can include an X position, a Y position, a scale, and a rotation orientation (collectively referred to as “configuration”). Although for each layer in the illustrated embodiment, each layer has the same configuration, each layer can have the same or different configurations as one or more other layers of the left article or the right article.

[0146] Shape Model Generation and Benchmark

[0147] System 100 can be used, for example, to generate a shape model. As described above, the shape model can be a representation of the extractable edges of an object or portrait desired in an image, which can be used to find corresponding shapes in other images.

[0148] Figure 6 An exemplary method 600 for generating a shape model is shown. Method 600 can include generating an image (process block 602), scaling the image (process block 604), locating a reference in and / or on the image (process block 606), determining a region of interest (“ROI”) (process block 608), setting an origin (process block 610), placing a graphic marker (process block 612), and / or saving the shape model (process block 614).

[0149] In some embodiments, for example, an item (e.g., item 200) can be placed in the working space (e.g., print bed) of the output device 104. The image sensor 106 can be used to generate an image of the item. The image can be in any suitable digital image format (e.g., RAW, PNG, etc.). The image can then be scaled to a 1:1 ratio. For example, in some embodiments, the image can be scaled such that 1 mm on the item image equals 1 mm on the item.

[0150] The system 100 can locate a fiducial marker in the image. Generally, a fiducial marker (or fiducial) is an object or marker that is placed in the field of view of the image sensor 106, appears in the generated image, and is used as a reference point. In some embodiments, it can be a component or characteristic of the imaging object (e.g., item). In other embodiments, it can be placed in or on the imaging object, and / or can be a marker or set of markers in the field of view of the image sensor 106. Thus, the fiducial can be used, for example, to establish one or more reference points for use by the output device 104 (e.g., printer).

[0151] The fiducial marker can be any characteristic or attribute that can be identified in the image generated by the image sensor 106. In some embodiments, the fiducial marker can include a single point. With a single point, a two-way orientation (e.g., x-y coordinates) can be obtained.

[0152] In other embodiments, the fiducial marker can include more than one point (e.g., two or more points). For more than one point, a two-way orientation (e.g., x-y coordinates), an angle, and / or a scale can be obtained. The fiducial marker can be used to identify the exact positioning of the item in the working space. This can, for example, make the positioning, alignment, and scaling of graphics more efficient (e.g., faster and easier) than having to perform these determinations manually each time the item is placed in the working space for processing by the output device 104.

[0153] The fiducial marker can be used, for example, during the original manufacturing process of the item, such as when manufacturing or assembling one or more components of the item. In other embodiments, the fiducial can be used, for example, during a post-manufacturing process (e.g., customization and / or modification).

[0154] In certain embodiments, the fiducial marker can be an integral part of the item, visible or invisible. For example, the fiducial marker can be a component and / or part of a component of the item that is a design feature on the finished product, such as a logo and / or surface pattern. In other embodiments, the fiducial marker can be a removable temporary component, or a part of an item component that is invisible on the finished product.

[0155] For example, in some embodiments, system 100 may include a computer-readable storage medium storing computer-readable instructions that cause a processor to identify a specific region or portion of an item and / or locate a logo as a reference. For example, Figure 7 shows a portion of an exemplary item 700, where logo 702 (e.g., swoosh) is a reference mark. In other embodiments, the reference mark can be any graphic printed on the item. The reference mark does not have to be a continuous shape. For example, the reference mark can be three stripes spaced apart.

[0156] In some embodiments, system 100 may include instructions configured to identify a specific region or portion of an item and position a pattern in and / or on the item as a reference (e.g., lace holes, studs, bite line profile, end of the heel portion, edge of the upper, collar portion of the shoe, and / or one or more surface patterns). For example, Figures 8 to 9 shows an exemplary item 800 that includes a plurality of perforations or holes 802 on the upper 804 of the item that can be used as a reference.

[0157] In a particular embodiment, a pattern change in a knitted and / or textile item can be detected and used as a reference. For example, Figure 10 shows a portion of an exemplary item 1000 having a pattern change 1002 that can be used as a reference. This embodiment can improve printing accuracy because an item made of knitted material and / or textile material shrinks and / or the fabric translates when the shoe last tends not to change its position. Accordingly, the process can, for example, accommodate such shrinkage and / or positional translation. The process can, for example, allow the output device 104 system to operate in the correct area after alignment with the reference (e.g., print artwork), which thus improves the quality and / or consistency of the item.

[0158] In some embodiments, as Figure 11 shown, the reference mark can include one or more positioning pins 1100 of a knitting machine (e.g., 12 in the shown embodiment), and the knitting machine can be used, for example, to manufacture an exemplary knitted item 1102.

[0159] In some embodiments, the reference mark can be set or positioned on a temporary and / or sacrificial member that is coupled to the item (e.g., a tab or a swatch) and can be removed from the item once processed by the output device 104. For example, Figure 12An exemplary article 1200 is shown having a sacrificial tab 1202 coupled to a sole structure 1204. The tab 1202 can include fiducial marks 1206 (e.g., six dots). Although not shown, the tab 1202 can have features (e.g., perforations) to facilitate removal of the tab 1202 after the article has been processed by the output device 104.

[0160] The sacrificial tab can be coupled to the article in various ways. For example, in some embodiments, the sacrificial tab can be integrally formed as a single unitary piece with one or more portions of the article (e.g., the upper and / or sole structure). This can be achieved, for example, by co-molding the sacrificial tab and one or more portions of the article. In other embodiments, the sacrificial tab can be formed as a separate component that is coupled to one or more portions of the article (e.g., the upper and / or sole structure) by, for example, adhesives, fasteners, and / or other means for making the coupling.

[0161] In some embodiments, the fiducial marks can be visible to the system 100 only under certain lighting conditions, e.g., under light that is not within the visible spectrum (e.g., light having a wavelength of approximately 380 nm - 750 nm). For example, in some embodiments, the fiducial marks are visible to the system 100 only under UV, infrared, and / or other non-visible lighting conditions. In such embodiments, the fiducial marks can include UV fluorescent inks and / or UV fluorescent materials (e.g., threads, yarns, textiles, etc.). The fiducial marks can then be exposed to UV light before and / or during image generation to make the fiducial marks visible to the image sensor 106. The system 100 can then use the fiducials as reference points to align the output device 104.

[0162] In some embodiments, the fiducial marks can be components of the output device and / or can be coupled to the output device 104. For example, Figure 13 A first fiducial mark 1300 disposed on a first shoe last 1304 and a second fiducial mark 1302 disposed on a second shoe last 1306 are shown, respectively. In the illustrated embodiment, each fiducial mark 1300, 1302 includes 6 dots. Five dots are arranged in a partial hexagonal pattern (i.e., a complete hexagon is missing one dot), and the remaining dot is located at the center of the partial hexagon. This dot configuration can, for example, allow the system 100 to determine the rotation, orientation, and / or relative angle of the fiducial mark. In other embodiments, various other dot patterns and / or other types of indicators can be used.

[0163] Still referring to Figure 13 , in some embodiments, the article 1308 can be placed on the shoe lasts 1304, 1306, and the fiducials 1300, 1302 coupled to the shoe lasts can be used as reference points.

[0164] Additionally or alternatively, the fiducial may be coupled to a structure (e.g., a print bed) within the workspace of the output device 104.

[0165] In some embodiments, the articles 1308 disposed on the shoe lasts 1304, 1306 may be a left - right pair (e.g., articles of the same style, color, and / or size). In other embodiments, the articles on the shoe last 1304 and the articles on the shoe last 1306 may be of different styles, colors, and / or sizes. In any case, the system may be configured to position one or more fiducial marks on the print bed and / or on one or more articles, and the printing device may print one or more graphics on both articles.

[0166] In another embodiment, the article may be placed into the system at the same location relative to the initial location of the fiducial mark each time. The fiducial mark may then be moved from the initial location to a second location where it contacts the article. Based on the change in location between the initial location and the second location, the system may determine one or more characteristics of the article. For example, referring Figure 14 , the article 1400 may be placed on the stationary shoe last 1408. The fiducial mark 1402 may be set at an initial location that is known to the system 100 and is spaced a first distance from the shoe last 1408, which first distance is also known to the system 100. The fiducial mark 1402 may be attached to a movable support member (e.g., an arm or a column) and may then be moved from the initial location to a position where the fiducial mark 1402 contacts the article (e.g., the bottom plate 1406 of the article). The system may then determine the distance that the fiducial mark has moved from the initial position to the second position. Based on the determined distance that the fiducial mark 1402 has moved from the first position to the second position, the system 100 may determine, for example, by determining the distance between the fiducial mark 1402 and the shoe last 1408, the thickness between the inner side of the insole 1404 of the article 1400 and the outer side of the bottom plate 1406 of the article 1400. Based on the determined thickness, the system 100 may, for example, identify certain other parameters (e.g., model and / or size) of the article 1400 corresponding to the determined thickness.

[0167] In some embodiments, the system 100 may be configured with a movable member such that the fiducial mark and the article may be moved relative to each other to align the height or z - coordinate of the fiducial mark in a plane that is the same as the height or z - coordinate of the article. This may be achieved in various ways.

[0168] For example, in a particular embodiment, the fiducial mark 1500 may be attached to a fiducial support member 1502 (e.g., a column, a rod, a bar, a strut, etc.), as Figure 15As shown. The reference support member 1502 can move in the height or z - direction relative to the shoe last on which the article 1504 is disposed (e.g., into and out of the page plane as shown in Figure 15 ). In some embodiments, the reference support member 1502 can have a reset configuration in which the surface of the reference support member 1502 (e.g., the surface on which the reference mark 1500 is disposed) protrudes upward beyond the uppermost surface of the article 1504. The reference support member 1502 can move from the reset configuration to a set configuration in which the surface of the reference support member 1502 is aligned with the uppermost surface of the article 1504 (e.g., in the same plane). By pressing down on the reference support member 1502 (e.g., using a plate), the reference support member 1502 can move from the reset configuration to the set configuration.

[0169] The reference support member 1502 can be selectively locked in place with a locking mechanism. For example, the locking mechanism can include a biasing element (e.g., a spring) that biases the reference support member to the reset configuration and a retaining mechanism (e.g., teeth and grooves) that holds the reference support member in the set configuration. In a particular embodiment, the locking mechanism can be configured to operate in a manner similar to a "clickable" retractable pen. Once the article 1504 has been processed, the reference support member 1502 can move from the set configuration to the reset configuration. This can be achieved automatically or manually.

[0170] In other embodiments, the shoe last and / or the article support member 1506 (e.g., a printing bed) coupled to the shoe last can be movable and the reference support member 1502 can be fixed. For example, the system 100 can include a sensor device having a sensor (e.g., a sensing laser and / or an electric or magnetic field) that determines the height or z - coordinate of the article 1504. The system 100 can include one or more motors, actuators, and / or other mechanisms for adjusting the article support member 1506 such that the uppermost portion of the article 1504 is aligned with the reference mark 1500 (e.g., in the same plane).

[0171] In another exemplary embodiment, a grid can be formed on the shoe last in the ankle region of the shoe last. The grid can be visible on the portion of the shoe last that extends upward from the article mouth. The appearance of the grid can vary along the height of the shoe last such that the grid can be used to identify the shoe collar portion of the article as a reference. The reference can be used to provide a reference point for an output device.

[0172] In other embodiments, a light source (e.g., a light-emitting diode (“LED”)) or a laser (e.g., a UV laser) can generate a fiducial. For example, the light source can be placed behind and / or inside the article to “backlight” the article such that the light or pattern generated by the light source passes through the article and is visible to the image sensor 106. The system 100 can detect and locate the light and use the image projected onto the article as a fiducial. In other embodiments, the light source can be used to display a fiducial on the outer surface of the article. This can be achieved, for example, by directly illuminating the outer surface of the article. The system 100 can detect and locate the reflected light and use the received image as a fiducial.

[0173] In short, a fiducial can be any consistent and repeatable feature visible to the image sensor and can be used by the system before, during, and / or after article assembly.

[0174] In some embodiments, multiple fiducials can be used for one or more of several beneficial purposes. For example, the article can have a first fiducial mark (e.g., a logo), and the workspace of the output device can have a second fiducial mark (e.g., coupled to a shoe last). In some embodiments, the system 100 can search for the first fiducial mark (e.g., a logo), and if the first fiducial is not found, the system 100 can use the second fiducial mark.

[0175] In other embodiments, multiple fiducials can be used to provide the scale (e.g., dimensions) of the article. This can be achieved, for example, by determining the relationship (e.g., distance, offset, rotation) of the first fiducial relative to one or more other fiducials. For example, the article can have a first fiducial located towards the toe portion of the article and a second fiducial located towards the heel portion of the article. The system 100 can identify the first fiducial and the second fiducial and measure the distance and / or angle between the first fiducial and the second fiducial. The system 100 can use the distance and / or angle, for example, to determine the dimensions of the article and / or the orientation of the article relative to the output device 104.

[0176] In some embodiments, the contrast available along the surface of the article can be increased by illumination to facilitate fiducial identification. In certain embodiments, various lighting types (e.g., LED, UV, etc.) and / or lighting positions (e.g., side lighting, backlighting, etc.) can be provided. In certain embodiments, the system 100 can include a diffusion dome 1600 having high-brightness LED illumination coupled to the image sensor 106 and disposed above the print bed 1602, as Figure 16 shown. The diffusion dome 1600 can be used to change the lighting conditions. For example, changing the lighting conditions can improve the system's ability to locate fiducials, for example, by providing enhanced image contrast.

[0177] As an alternative to or in addition to the shoe last mentioned herein, in one exemplary embodiment, one or more holding components (e.g., shoe lasts) and / or methods described in U.S. Pat. Nos. 9,301,576 and 9,456,651 can be used to support footwear items, which patents are incorporated herein by reference. Various other types of shoe lasts, jigs, holding components, etc. can be used to hold other types of items.

[0178] Region of Interest Module

[0179] When creating a shape model and / or positioning fiducials (as explained above), system 100 can determine a region of interest (“ROI”) in the image of the item. The ROI can include an object (e.g., a logo) for creating the shape model. In some embodiments, the ROI can be the portion of the image where fiducials are set.

[0180] When determining the region of interest (“ROI”) (e.g., Figure 6 608 in), system 100 can use one or more of various types of imaging modules to reduce or eliminate noise in the image, and / or more clearly define the edges around the target shape (e.g., fiducials on or adjacent to the item). To limit the extraction to the desired edges, an ROI can be specified around the target shape. Once the ROI is generated, system 100 can begin extracting and displaying the edges of the target shape. To enhance and / or clarify the extracted target edges, the user can modify the view of the model image.

[0181] The ROI can be generated automatically, manually, or a combination of both. System 100 can include various tools, modules, and / or components for improving the quality of the ROI. This can, for example, reduce the time and / or improve the consistency with which system 100 can locate fiducials and / or generate or utilize a shape model.

[0182] In some embodiments, system 100 can include various imaging tools, modules, and / or components for filtering the generated image. For example, in one embodiment, system 100 can include modules for adjusting the view of the model image, such as selecting a filter or “grayscale” option, and including modules for adjusting the amount of smoothing, and / or adjusting gray windowing. For example, the ROI can be adjusted to eliminate unwanted edges.

[0183] Figure 17 A user interface 1700 (e.g., a graphical user interface or “GUI”) is shown that displays an exemplary module for removing noise from an image. One or more features of the user interface are described herein by Figure 17identified by the specific terms shown therein (e.g., "Load Image", "Set Origin", etc.), and can be understood as corresponding to one or more executable functions / actions, and a set of machine-executable instructions stored on a data storage medium and corresponding to that function. The functions can include user-selectable actions, or user-selectable conditions used with respect to the performance of one or more actions. When selected, these functions can individually affect or cause the operation of a computing device that executes the encoded instructions, or can also affect or cause the physical actions of a printing device. However, in Figure 17 the specific terms shown in the GUI of Figure 17 and the specific terms shown in other GUIs of the invention embodiments shown in several figures should be considered illustrative and exemplary of these features, and do not limit the embodiments to the specific appearance of the depicted GUI, nor to the exact functions corresponding to these specifically depicted terms. For example, the feature identified by the term "Save" in the "Profile" section of the GUI of

[0184] These modules can include one or more of the grayscale option module 1702, the automatic detection custom ROI module 1704, the ROI tool module 1706, the view interaction module 1708, the grayscale windowing module 1710, and the image smoothing module 1712.

[0185] The grayscale ratio image can be extracted from the color image in various ways. For example, the image can include a red channel, a green channel, and a blue channel. In a particular embodiment, the system 100 can combine the red channel, the green channel, and the blue channel by using the formula 0.299 * red + 0.587 * green + 0.114 * blue to combine the three color intensity values of a pixel into a single grayscale ratio pixel to generate a grayscale filter. In some embodiments, the system 100 can generate various other filters, such as red - green ("R - G"), red - blue ("R - B"), and green - blue ("G - B").

[0186] Depending on the selected filter (e.g., grayscale, R-G, R-B, G-B), the desired edges can be blurred, enhanced, or in some cases simplified to show a consistent dark-light polarity. For example, the selected filter also helps reduce background noise. The user can utilize different options to determine which filter best highlights the desired edges of the target shape. If needed, the filter can be changed later (e.g., after generating the ROI and the extracted edges are visible).

[0187] System 100 can be configured with an automatic detection custom ROI module 1704. If the object (e.g., logo) is darker than the background of the item, the user can select "Dark". For other items, for example, the user can select "Light". In some embodiments, selecting either button can put system 100 into an automatic detection mode that shows all automatically detected shapes. Clicking on the desired shape can select the desired shape, exit the automatic detection mode, and / or generate a custom ROI just outside the shape. To exit the automatic detection mode without selecting a shape, the user can select "Cancel".

[0188] In some cases, the initially detected shape may not be as well-defined as desired. If this occurs, the image refinement modules (such as the grayscale options module 1702, the image smoothing module 1712, and / or the grayscale windowing module 1710) can be adjusted during the detection process to improve the results. Additional details regarding the image smoothing module 1712 and the grayscale windowing module 1710 are provided below.

[0189] As described above, the ROI can be manually selected by the user. For example, if the target shape is too complex, incomplete, and / or if there are interfering features on the item that prevent the automatic detection custom ROI module 1704 from isolating the target shape as needed, the user can utilize the ROI tool module 1706. The ROI tool module 1706 can be used, for example, to move the ROI and / or adjust the size of the ROI.

[0190] The ROI tool module 1706 can include boundary generation tools and controls to remove, receive, and / or store them. In some embodiments, the boundary generation tools can include "Rectangle 1", "Rectangle 2", "Circle", and / or "Ellipse".

[0191] In a particular embodiment, "Rectangle 1" can be a bounding box that can be resized with control points at the corners and moved with a control point at the center. In some embodiments, "Rectangle 1" cannot be rotated.

[0192] In some embodiments, "Rectangle 2" can be a bounding box that can be rotated using the control points on the arrow. In some embodiments, "Rectangle 2" can also be resized and moved.

[0193] In a particular embodiment, "Circle" can be a circular boundary that can be moved through the center and resized through the edge points.

[0194] In some embodiments, "Ellipse" can be an elliptical boundary that can be rotated using the control points on the arrow and resized using the edge points.

[0195] In some embodiments, the ROI tool module 1706 can be used alone or in various combinations to manually create an ROI.

[0196] As the shape bends away from the image sensor due to changes in the height of the item's surface (e.g., near one side of the item), the shape may become distorted. This distortion is different for various orientations of the item. Additionally, the illumination in these areas may be more oblique, which may give the edges different characteristics. To reduce the distortion, the user can use a negative ROI to "clip off" the portion of the ROI closest to the dropping edge of the item. In some embodiments, a model using a clearer and flatter portion of the target shape performs better in the recognition phase than a model that includes all the edges. In some embodiments, the contour generated by the system 100 during recognition can fill in the excluded edges.

[0197] In some embodiments, by selecting an area and then clicking the add (+) or subtract (-) option, an area can be added to the ROI or subtracted from the ROI. In a particular embodiment, the positive (+) area can be displayed on the display 108 with a first color boundary line, and the negative (-) area can be displayed on the display 108 with a second color boundary line. The system 100 can generate a final ROI by combining all the positive (+) areas and removing all the negative (-) areas. The final ROI can be displayed on the display 108 with a third color boundary.

[0198] In some embodiments, an ROI can be saved to a storage medium by selecting "Save" on the ROI module group and selecting a file name in a dialog box. By selecting "Load", a stored ROI can be received for reuse.

[0199] The view interaction module 1708 can be used to, for example, examine the extracted edges of a target object. For example, once the ROI is generated, the edges can be automatically extracted with the current settings and displayed in a first color. In some embodiments, the optimal edges of the model can be simple lines. To improve edge extraction, the user can utilize the grayscale windowing module 1710 and / or the image smoothing module 1712. To facilitate edge inspection, the view interaction module 1708 can include options for zooming and / or moving the image using an input device (such as a mouse, keyboard, microphone, button, pedal, and / or touch screen, etc.).

[0200] The grayscale windowing module 1710 can have one or more adjustment members (such as, for example, the two "sliders" shown Figure 17 . The first adjustment member can control the darkest gray level allowed in the working image. The first adjustment member can have a default position (such as, all the way to the left). The second adjustment member can control the lightest gray level allowed in the working image. The second adjustment member can have a default position (such as, all the way to the right). In some images, the background outside the shape can include a mixture of gray values. In some images, the interior of the shape is uneven. In certain embodiments, this can make the edges indeterminate, resulting in ill-defined edges with many intertwined lines and segments. In some embodiments, the grayscale windowing module 1710 can optionally convert all pixels below the lower limit to the lower limit and all pixels above the upper limit to the upper limit, thus leaving only the gray values that include the edge transitions of interest.

[0201] In some embodiments, the user can adjust the first adjustment member until the dark part of the image is as uniform as possible at the edges of the shape. The user can adjust the second adjustment member until the light part of the image is as uniform as possible at the edges of the shape. While doing so, the contour lines can become smoother and simpler. For example, this can improve the matching ability of the model and / or speed up the discovery process.

[0202] The image smoothing module 1712 can be used to, for example, remove irrelevant details from noisy images. In some embodiments, edge detection can be very sensitive. In certain cases, this sensitivity can pick up unwanted edges on the unmodified model image. In some embodiments, sliders and / or digital controls can define the level of smoothing (for example, a higher number can correspond to more smoothing). A default level can be selected. If too much detail is removed from the model image and there are only few details that complicate the edges of the target shape, the smoothing can be reduced to capture the weaker edges. In images with a large amount of interfering texture or with small and narrow objects in contact with the shape, the smoothing can be increased to avoid unwanted edges.

[0203] Once the ROI is determined, system 100 may define an origin. In some embodiments, the origin may be the geometric center of the ROI. In other embodiments, the origin may be moved to a specific location on a shape or some other feature. For example, the origin may be used as a reference point relative to which graphical markers may be placed and / or the output device 104 may be aligned.

[0204] A graphical marker is a graphical user object that defines the position, scaling, and / or orientation of a graphic to be placed (e.g., printed) on an item. In some embodiments, a graphical marker may be generated by receiving a desired graphic from an image file (e.g.,.jpg,.tiff,.gif, etc.) or from a Microsoft Word (.doc) or Adobe's Portable Document Format (.pdf) file. The graphical marker may be positioned relative to the item image. For example, Figure 18 A graphical marker 1800 located on an image of an item 1802 is shown.

[0205] In some embodiments, the graphical marker 1800 may be resized, moved, and / or rotated as needed. In certain embodiments, the graphical marker 1800 may be offset, scaled, and / or rotated based on a detected shape model. For example, if a target shape is found in a rotated orientation, the graphical marker offset may reflect such a change in X and Y distances (e.g., triangulation).

[0206] This process may, for example, improve the consistency and efficiency of throughput because the system may, for example, automatically position graphical markers and / or resize graphical markers regardless of the size or positioning of the item on the print bed.

[0207] In other embodiments, the graphical marker 1800 may be resized and / or oriented independently of the shape model. This may, for example, cause the position (e.g., X and Y offsets) of the graphical marker 1800 to shift from the new origin but reflect the original orientation of the marker.

[0208] As Figure 19 shown, in some embodiments, a grid 1900 may be displayed over the image. The grid 1900 may facilitate the positioning and / or alignment of the graphical marker 1902. As shown, in some embodiments, system 100 may include a "Marker Grid Dialog" interface to allow the configuration of the grid to be changed in various ways (e.g., horizontal and vertical scales, colors, etc.).

[0209] In some embodiments, system 100 may identify the outline of a target shape (e.g., the edge of a logo). System 100 may then print a graphic on the outline of the target shape.

[0210] In some embodiments, settings and one or more graphical markers can be stored as a profile. The profile can be written to a storage device (e.g., as an.xml file). In some embodiments, while saving the profile, additional files containing shape models can be stored.

[0211] For footwear items and other items that include more than one item (e.g., a left item and a right item), separate profiles (e.g., a left profile and a right profile) can be generated and stored for each item.

[0212] As Figure 20 shown, in some embodiments, system 100 can include additional modules 2000 to further define the edges of the ROI of the shape model. These modules can be adjusted to achieve the desired accuracy and / or speed of the system in identifying the target shape in the item image. Modules 2000 can include one or more of contrast module 2002, minimum feature size module 2004, scale range module 2006, angle range module 2008, polarity module 2010, pyramid level module 2012, optimization module 2014, display image pyramid module 2016, angle step module 2018, scale step module 2020, and minimum contrast module 2022. Each of these modules 2000 is further described below.

[0213] The contrast module 2002 can be used, for example, to improve the edge detection of a target shape (e.g., a logo). If the edge is incomplete, the low contrast can be reduced until the edge is clearly visible. If unwanted edges persist, the low contrast or high contrast can be increased. The change in contrast can interact with smoothing and gray level windowing. The contrast module can have an "Auto Mode" in which system 100 can automatically select appropriate values as the ROI and image view change. Manually changing the settings when the auto mode is "On" can automatically turn off the auto mode.

[0214] For example, if edges persist around small objects within or surrounding the target shape (such as those from patterns in textiles and / or graphics), the minimum feature size module 2004 can be used. These unwanted edges can be eliminated by increasing the value of the minimum feature size module 2004. If this also removes small edge segments from the target shape, this may indicate a need to adjust the gray scale windowing. In some cases, when strings or other objects cross the edges of the target shape or create internal sub-shapes, small negative ROIs can be used to clip these features from the model. It should be noted that it is not necessary to include all edges in the model for it to work well. The minimum feature size module 2004 can have an "auto mode" in which the system 100 can automatically select an appropriate value as the ROI and image view change. In some embodiments, manually changing these settings when the auto mode is "on" can turn off the auto mode.

[0215] For example, if artifacts (such as graphics, features, etc.) that are sufficiently similar to the target shape and whose sizes fall within a ratio range are present on an item, the ratio range module 2006 can be used. The maximum and minimum ratio limits can be adjusted to eliminate these artifacts.

[0216] The angle range module 2008 can be used to find a desired target object, for example, when the desired target object is one of multiple symmetric objects found in an image. In embodiments where the left and right versions of a graphic are mirror images of each other, this is usually not a problem. However, if symmetric graphic shapes are used, the angle range module 2008 can be adjusted to limit the angle to avoid finding the wrong one. The starting angle and ending angle can be adjusted until only the desired target shape is found.

[0217] The polarity module 2010 can be used, for example, to determine the gray values at the edges of a target shape where there is a transition from bright to dark or from dark to bright. This can include various functions such as "use_polarity", "ignore_global_polarity", and "ignore_local_polarity". The functions identified herein by specific terms (e.g., "use_polarity", "ignore_global_polarity", and "ignore_local_polarity", etc.) can be understood as corresponding to one or more executable functions / actions, as well as a set of machine-executable instructions stored on a data storage medium and corresponding to that function. The functions can include user-selectable actions, or user-selectable conditions for use in relation to performing one or more actions. When selected, these functions can individually affect or cause the operation of a computing device that executes the encoded instructions, or can also affect or cause physical actions of the computing device and / or a printing device. However, the specific terms should be considered illustrative and exemplary of these features, and do not limit the listed embodiments, nor the exact functions corresponding to these specifically described terms. Thus, for the intended embodiments, the specific terms used are exemplary rather than exclusive.

[0218] For example, when the target is dark (e.g., black) and is surrounded by light (e.g., white), the use_polarity setting can increase the success of finding the shape model in the correct location, and vice versa. For use_polarity, the target shape in the image and the model can have the same contrast. For example, if the target shape in the model based on a first item is a dark object on a bright background (e.g., a black logo on a white background), then the target shape can be found if the target shape in the second item image is also darker than the background (e.g., a blue logo on a yellow background).

[0219] For example, the ignore_global_polarity setting can cause the search to examine all edges regardless of polarity. This may result in a slower search, but can be useful in specific cases where the target and the background are the same or similar (e.g., black against black, white against white, etc.). For ignore_global_polarity, the target shape can also be found in the image if the contrast is globally reversed (e.g., a model with a dark logo on a brighter background can also be used to find a brighter logo on a darker background).

[0220] For example, when the polarity of some regions is different from that of other regions, ignore_local_polarity can be used. For example, if a target shape is partly bounded by some transitions from dark to light and partly by some transitions from light to dark. For ignore_local_polarity, the model can be found even if the contrast changes locally. For example, if the target shape consists of parts with medium gray values that contain darker or brighter sub-objects, this option can be useful.

[0221] In some embodiments, the edges in the image that fit the model can be searched first at a very low resolution, and then, if necessary, gradually increased to a finer resolution to determine the fitness of the detected edges. The pyramid level module 2012 can, for example, vary the number of steps in the search. In certain cases (e.g., the edges correspond to very fine details), the number of steps can be increased. Reducing the number of steps can reduce the search time but may reduce the accuracy. The pyramid level module 2012 can include an "automatic mode" in which the system 100 can automatically select appropriate values as the ROI and the image view change.

[0222] The optimization module 2014 can be used to, for example, adjust the noise of the target edges. The model can be (e.g., indirectly) based on the detected edges. In some embodiments, the edges may sometimes be quite noisy, which can lead to many direction changes and / or short runs deviating from the main edge. In addition, the edges may contain more points than the system 100 can effectively use. Therefore, the contour of the edge can be smoothed internally. The "point_reduction_medium" setting (e.g., the default setting) can be used to balance precise and effective detection. For example, if the edges are very noisy and / or the search takes a relatively long time, the "point_reduction_high" setting can be used. For example, if the number of shape model points is too low to detect the target, the "point_reduction_low" setting can be used. The "none" setting can be used to use all the points. The optimization module 2014 can include an "automatic mode" in which the system 100 can automatically select appropriate values as the ROI and the image view change.

[0223] The display image pyramid module 2016 can be used to, for example, display a specific image pyramid level. For example, this is useful for diagnostic purposes, such as for determining why the detection of the target shape takes longer than expected.

[0224] The angle step module 2018 can, for example, determine how many degrees (e.g., one-tenth of a degree) the model has rotated during matching. Setting a larger step size can result in faster matching but may be less accurate. In the illustrated embodiment, for convenience, the units are shown as 10 times the internally set value. Thus, setting 1 is equal to 0.1 degree. In some embodiments, adjusting the "starting angle" and / or "ending angle" of the angle range module 2008 can cause this value to be recalculated. In some embodiments, the angle step module 2018 can take effect if it changes after the angle range module 2008 changes. The angle step module 2018 can include an "automatic mode" in which the system 100 can automatically select an appropriate value as the ROI and the image view change.

[0225] The scale step module 2020 can, for example, determine the increment by which the system 100 steps through the scale range during matching. In the illustrated embodiment, for convenience, the units are shown as 1000 times the internally set value. In some embodiments, when set to 2 (i.e., 0.002), a 10% scale range (0.95 - 1.05) will be stepped through in 50 steps of 0.002. If a small scale range is set and the model is not found at the correct scale, this value can be decreased (e.g., from 2 to 1). If scaling is not an important factor for a particular model, the larger the step size, the shorter the matching duration (e.g., 3 vs. 2). The scale step module 2020 can include an "automatic mode" in which the system 100 can automatically select an appropriate value as the ROI and the image view change.

[0226] The minimum contrast module 2022 can, for example, set a minimum contrast that defines the edges to be considered during matching. In certain situations (e.g., having very weak edges and / or significant interfering factors), adjusting the minimum contrast module 2022 can improve the matching result. For example, the minimum contrast value of the minimum contrast module 2022 can be several units lower than the low contrast value of the contrast module 2002. The minimum contrast module 2022 can include an "automatic mode" in which the system 100 can automatically select an appropriate value as the ROI and the image view change.

[0227] In some embodiments, the settings can be stored as profiles. The profiles can be written to a storage device (e.g., as an.xml file). In some embodiments, additional files containing shape models can be stored while saving the profiles.

[0228] Shape Model Testing

[0229] System 100 can be used to test a shape model. In some embodiments, system 100 can test a shape model by attempting to find a target shape (e.g., a fiducial) in images other than the one (preferably several) images used to create the shape model. Generally, the more images tested, the more robust the shape model (also referred to as a "shape model profile" or "profile"). Exemplary methods for testing a profile are described further below.

[0230] To test a shape model, system 100 can generate a shape model stored in SHM format and a profile file stored in XML format. Among other things, the data described in the shape model can include fiducial dimensions, fiducial types, and / or fiducial locations, etc. The profile file can include image files of articles that differ from the shape model in one or more aspects such as the dimensions, style, color scheme, and / or other characteristics of the article. Additionally or alternatively, system 100 can receive a previously generated model from various other sources such as a storage device and / or a network.

[0231] Images for testing a profile can be received from various sources. For example, one or more images can be received from an image sensor 106, a storage device, and / or a network.

[0232] Figure 21 An exemplary user interface 2100 for testing a profile is shown. Image file names can appear in a list box 2102, and a status column 2104 (e.g., Pass / Fail) can display the status of the received images. Initially, the status column 2104 can display "Untested".

[0233] An image name can be selected and the image scanned to detect the shape model. If the shape model is found in the image, the status column 2104 can display "Pass". If the test does not find the target shape, "Fail" is displayed in the status column 2104. The position and orientation can be displayed in an output window (not shown) on the display 108. The edges of the model found can be displayed on the image (e.g., in a first color), for example. Any graphical markers in the profile can be displayed at their relative positioning with respect to the target shape found. A refined output profile that traces the entire edge of the target shape can also be shown on the image (e.g., in a second color). Additionally, the origin and orientation can be displayed. In some embodiments, all of the received images can be tested at once by selecting "Scan All".

[0234] The display simplified profile module 2105 can be used to identify, for example, the differences between the model outline and the edges of the target shape in the image. For well-defined models, such as those generated by careful gray-level windowing, this option may make little difference. In less robust models, the simplified display may show some discrepancies between the model and the target.

[0235] In some embodiments, refining the ROI can improve the profile. For example, if unwanted contours interfere with the matching process, the ROI can be edited to reduce or eliminate the unwanted contours. After editing the ROI, the test can be retried to determine if the image passes.

[0236] For example, if the detected edges diverge from the target shape near the descending curve of the article, negative ROI can be added and / or adjusted to remove the edges near the descending curve.

[0237] For example, if the detected edges are connected to the contrasting background surrounding the target, appropriate refined edges can be generated. This can be achieved, for example, by adjusting the ROI to move it away from those curved regions and / or adjusting the gray-level windowing (e.g., using the gray-level windowing module 1710).

[0238] If unwanted details in the background and / or inside the target shape are picked up in the test image, it may be necessary to change and / or adjust the module (e.g., the gray-level options module 1702).

[0239] If one or more images still fail, one or more other modules can be used to see if the image can pass. These modules can include, for example, the minimum score module 2106, the greediness module 2108, the maximum deformation module 2110, the last pyramid level module 2112, the sub-pixel module 2114, and the tight refined profile module 2116.

[0240] In some embodiments, the maximum deformation module 2110 can be adjusted first. For example, the maximum deformation module 2110 can be used to adjust the tolerance in which the accepted edges can drift from the model. For example, a value of 4 can allow the accepted edges to drift 4 pixels from the model. If the deformation looks larger, the maximum deformation can be increased. In certain embodiments, it is preferred that the maximum deformation is not increased more than necessary, as it can cause the detected edges to be less precise. In some embodiments, for example, increasing the maximum deformation too much can cause the system 100 to find the wrong shape, or cause a failure score in an image that has previously passed when the detected shape is rejected due to other criteria.

[0241] The minimum score module 2106 can be used to, for example, adjust the percentage of edges in the model that must match for the target to pass. For example, in some embodiments, 80% can be the default value. In certain embodiments, settings up to 98% will continuously eliminate most models. However, in some embodiments, lower values can be used to achieve a more robust profile. However, in some embodiments, settings below 50% may result in unpredictable matches for portions of other shapes. In some embodiments, it may be advantageous to find the value that allows all test images to pass and set the minimum value to be a few points lower than a safety or buffer value.

[0242] The greedy module 2108 can be used to, for example, determine the amount of edges required for the target to pass. If the model is defined with very few available edges, reducing this value can help.

[0243] In some embodiments, shapes are searched in the image at increasingly fine resolutions, starting from a downscaled version and working upwards to accelerate the search. The final pyramid level module 2112 can, for example, determine the level at which the search will stop. In certain embodiments, if the search finds the target shape at a slightly lower level and then rejects it at a higher level (e.g., due to too much detail in the detected edges), stopping at the lower level by adjusting the pyramid level module 2012 can, for example, allow the lower-resolution shape to pass. For example, this setting can also be adjusted to reduce the search time. In some embodiments, the levels can be sorted such that the lowest resolution is searched first. In other embodiments, the highest resolution is searched first.

[0244] The sub-pixel module 2114 can be varied to adjust the sub-pixel accuracy. For example, in some embodiments, the search results can be set to return the location and orientation with "least squares" sub-pixel accuracy. For example, if increased accuracy of the found location is needed, the "least squares high" setting or the "least squares very high" setting can be selected. In some embodiments, the "interpolation" setting or the "none" setting can be selected (e.g., if the search takes too long). However, in certain embodiments, these settings may result in lower accuracy.

[0245] In some embodiments, while refining and finalizing the outline of the target shape, other edges near the true edge of the target shape may be picked up. For example, this may cause the refined outline to follow the wrong edge. If this occurs, the tight refinement contour module 2116 can be selected. For example, this can force the refined edge to be followed only very close to the target shape.

[0246] The result, including the refined contour, can be stored in a format compatible with the format of the output device (e.g., a printing device).

[0247] Image Sensor Calibration

[0248] To transform the received and generated images into the real-world coordinate system, system 100 can be used to generate the calibration of image sensor 106 and / or system 100 can receive a previously generated calibration (e.g., from a storage device and / or a network). The calibration can be used to determine the intrinsic camera parameters and lens distortion. The calibration can be generated by inputting camera parameters, calibration images, and calibration parameters.

[0249] The camera parameters that can be input into system 100 can include: focal length, sensor size X, sensor size Y, and image sensor type. For example, in some embodiments, the focal length of telecentric lenses can be set to zero. The initial value can be the nominal focal length of the lens used (e.g., in millimeters). For example, the sensor size X of a pinhole camera can correspond to the horizontal distance between two adjacent units on the sensor. For a telecentric camera, the sensor size X can represent the horizontal size of a pixel in the real-world coordinates (e.g., in microns). For example, the sensor size Y of a pinhole camera can correspond to the vertical distance between two adjacent units on the sensor. For a telecentric camera, the sensor size Y can represent the vertical size of a pixel in the real-world coordinates (e.g., in microns). The image sensor type (e.g., area scan division) can be specified.

[0250] The calibration images can be pre-generated (e.g., using image sensor 106 or another device), or they can be generated during the calibration process (e.g., using image sensor 106). In some embodiments, multiple calibration images (e.g., 1 - 25) can be used. In certain embodiments, 6 - 20 calibration images can be used. In an exemplary embodiment, 15 calibration images can be used.

[0251] The calibration parameters can include the corrected image width, corrected image height, calibration scale (e.g., dots per inch (“DPI”), such as 300 DPI), and / or world pose index.

[0252] Once the calibration parameters have been entered and a calibration image has been generated or received, the calibration map can be stored. Together, the calibration map and the information file can constitute the calibration. For example, the calibration map and the information file can be stored at the same location with different extensions using a path and a name. For example, in one or more embodiments, the calibration map can have a.tif file extension, while the information file can have an.xml file extension.

[0253] System 100 can also receive the calibration map and / or the information file. For example, system 100 can receive the map and the file from a storage device and / or a network.

[0254] System 100 can test the calibration by receiving an image, correcting the image, and storing the corrected image. The corrected image can apply the calibration map to the image. The image can be corrected for lens and parallax distortion and mapped to a specified calibration scale.

[0255] Image Acquisition

[0256] Image acquisition can be implemented in various ways. System 100 can acquire an image from image sensor 106 and / or from a storage device, a network, or other locations. The acquired image can be used as a model image, a calibration image, and / or a test image. Additionally, if a calibration image is to be acquired during the calibration process, the acquisition interface can be initialized by selecting an interface of a specific image sensor.

[0257] In some embodiments, the acquisition interface can be selected (e.g., from a drop-down list of all interfaces found on the system). In a particular embodiment, one or more fields can depend on the selected acquisition interface, and / or one or more fields can be independent of the selected acquisition interface.

[0258] In certain embodiments, the fields can include horizontal resolution, vertical resolution, image width, image height, starting row, starting column, field, bits per channel, color space, general, external trigger, image sensor type, device, port, line input, initialization, snapshot, start / stop live video, storage, corrected image, automatic gain, and / or other fields.

[0259] In a particular embodiment, the horizontal resolution can include a real pixel value such as 800 or a preset value, where 1 represents full resolution, 2 represents half resolution, and 4 represents quarter resolution.

[0260] In some embodiments, the vertical resolution can include a real pixel value such as 600, or a preset value such as 1 represents full resolution, 2 represents half resolution, and 4 represents quarter resolution.

[0261] In certain embodiments, the desired width of the acquired image can be input. For example, this value can be less than or equal to the actual pixel width of the image sensor. In some embodiments, a preset value of 0 can be input to use the entire width. In some embodiments, the image width can be less than the actual pixel width of the image sensor. Accordingly, a column offset (e.g., from the left side) can be set to correspond to the left side of the desired image portion.

[0262] In certain embodiments, the desired height of the acquired image can be input. For example, this value can be less than or equal to the actual pixel height of the image sensor. In some embodiments, a preset value of 0 can be input to use the entire height. In some embodiments, the image height can be less than the actual pixel height of the image sensor. In this way, a row offset (e.g., from the top) can be set to correspond to the top of the desired image portion.

[0263] In some embodiments, the acquisition interface can support interlaced fields and / or progressive fields.

[0264] In certain embodiments, the number of bits transferred per pixel per channel can be set, e.g., 8.

[0265] In some embodiments, the desired color space of the acquired image can be set. For example, in certain embodiments, gray can be used for single-channel images. In other embodiments, red-green-blue (“RGB”) or luminance-chrominance (“YC B C R ” (sometimes referred to as “YUV”) can be used for color images.

[0266] In some embodiments, a common field can be used for various available options specific to the selected acquisition interface.

[0267] In some embodiments, an external trigger can be used to initiate image acquisition.

[0268] In certain embodiments, the image sensor type can be selected to search for the desired image file to receive when generating an image.

[0269] In some embodiments, a device can be selected based on the selected interface. In certain embodiments, the device can have multiple output connections. A desired port can be selected for the selected device.

[0270] In certain embodiments, a multiplexer can be used to connect to multiple cameras. In this way, the desired input line of the multiplexer can be selected.

[0271] In some embodiments, initial parameters can be defined for the acquisition interface. If successful, the acquisition tool can be enabled. If not successful, diagnostic information can be displayed in the output area.

[0272] In a particular embodiment, a snapshot can be set up to receive and display a single image. In some embodiments, the image can be streamed from the acquisition interface to the display 108.

[0273] In some embodiments, a desired automatic gain setting of the acquisition interface can be selected.

[0274] Example computing system

[0275] Figure 22 FIG. depicts a general example of a suitable computing system 2200 in which the described innovations may be implemented. Computing system 2200 is not intended to impose any limitation as to scope of use or functionality, since the innovations may be implemented in various general-purpose or special-purpose computing systems. For example, computing system 2200 may be used to implement hardware and software.

[0276] Reference Figure 22 , computing system 2200 includes one or more processing units 2210, 2215, non-volatile memory 2220, and memory 2225. In Figure 22 , this basic configuration 2230 is shown within the dashed lines. The processing units 2210, 2215 execute computer-executable instructions, including instructions for generating shape models, locating fiducials in images, and / or aligning output devices with articles as disclosed herein. The processing unit can be a general-purpose central processing unit (“CPU”), a processor in a special-purpose integrated circuit (“ASIC”), or any other type of processor. In a multiprocessing system, multiple processing units execute computer-executable instructions to increase processing power. For example, Figure 22 illustrates a central processing unit 2210 as well as a graphics processing unit (“GPU”) or co-processing unit 2215. Tangible memory 2225 can be volatile memory (e.g., registers, caches, RAM) accessible by the processing unit, non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.), or some combination of both. Memory 2225 stores software 2280 implementing one or more of the innovations described herein in the form of computer-executable instructions suitable for execution by the processing unit.

[0277] The computing system may have additional features. For example, the computing system 2200 includes a storage device 2240, one or more input devices 2250, one or more output devices 2260, and one or more communication connections 2270. An interconnection mechanism (not shown), such as a bus, a controller, or a network, interconnects the components of the computing system 2200. Typically, an operating system software (not shown) provides an operating environment for other software executing in the computing system 2200 and coordinates the activities of the components of the computing system 2200.

[0278] The tangible storage device 2240 may be removable or non-removable and includes magnetic disks, tapes, or cassettes, CD-ROMs, DVDs, or any other medium that can be used to store information and can be accessed within the computing system 2200. The storage device 2240 stores instructions for software 2280 that implements one or more of the innovations described herein.

[0279] The input device 2250 may be a touch input device such as a keyboard, a mouse, a pen, or a trackball, a voice input device, a scanning device, a microphone, a button, a pedal, or another device that provides input to the computing system 2200. For video encoding, the input device 2250 may be a camera with an image sensor, a video card, a TV tuner card, or a similar device that accepts video input in analog or digital form, or a CD-ROM, a CD-RW, a DVD, or a Blu-ray disc that reads video samples into the computing system 2200. The output device 2260 may be a display, a printer, a speaker, a CD burner, or another device that provides output from the computing system 2200.

[0280] The communication connection 2270 enables communication with another computing entity via a communication medium (e.g., a connected network). The communication medium conveys information in a modulated data signal, such as computer-executable instructions, compressed graphic information, video, or other data. The communication connection 2270 is not limited to wired connections (e.g., gigabit or gigabit Ethernet, InfiniBand on electrical or fiber-optic connections, Fibre Channel over electrical or fiber-optic connections), but also includes wireless technologies (e.g., RF connections via Bluetooth, WiFi (IEEE 802.11a / b / n), WiMax, cellular, satellite, laser, infrared) and other suitable communication connections for providing network connectivity for the disclosed agents, bridges, and agent data clients. In a virtual hosting environment, the communication connection may be a virtual network connection provided by the virtual host.

[0281] Some embodiments of the disclosed methods may be implemented using computer-executable instructions that implement all or part of the disclosed technology in a computing cloud 2290. For example, the disclosed computer-readable instructions may be executed by a processor located in a computing environment 2230, or the disclosed computer-readable instructions may be executed on a server located in a computing cloud 2290.

[0282] A computer-readable medium is any available medium that can be accessed within a computing system 2200. By way of example and not limitation, for a computing system 2200, computer-readable media include memory 2220 and / or storage device 2240. It should be readily understood that the term computer-readable storage medium includes media for data storage, such as memory 2220 and storage device 2240, but does not include transmission media such as modulated data signals or other transient signals.

[0283] The innovation may be described in the general context of computer-executable instructions, such as those included in program modules executed in a computing system on a target real or virtual processor. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular data types. In various embodiments, the functions of program modules may be combined or divided as needed among program modules. Computer-executable instructions for program modules may be executed in a local or distributed computing system.

[0284] In some embodiments, computing system 102 may be configured with components and / or functionality of computing system 2200.

[0285] Additional examples of the disclosed technology

[0286] In accordance with the examples discussed above, additional examples of the disclosed subject matter are discussed herein. Unless otherwise noted, any feature of any example may be combined with or separated from any feature of any example.

[0287] Figure 23 An exemplary method 2300 for printing on an article using a shape model profile is depicted. The system may generate an image of the article (process block 2302). The system may provide a configuration file (process block 2304), a shape model (process block 2306), and / or a profile (process block 2308). The system may transform the image using the configuration file, shape model, and / or profile (process block 2310). The system may display the transformed image (process block 2312). The system may align the transformed image with an output device (process block 2314). The system may optionally receive input from an operator (process block 2316). The system may actuated the output (process block 2318). The output may include printing a graphic image on the article.

[0288] Figure 24 Depicts an exemplary method 2400 for printing on an item. One or more steps of the method can be performed using a computer. Method 2400 can include generating an image of the item (process block 2402), locating a fiducial in the item image (process block 2404), establishing a reference position in the image using the fiducial (process block 2406), locating a graphic marker within the image relative to the reference position (process block 2408), and printing on the item based on the position of the located graphic marker (process block 2410).

[0289] In some embodiments of any of the methods described in paragraphs 225 - 226, the action of printing on the item is performed by a 2D printing device. In other embodiments, the action of printing on the item is performed by a 3D printing device.

[0290] In some embodiments of any of the methods described in paragraphs 225 - 227, the item is a plurality of items. In certain embodiments, the plurality of items includes a first item and a second item. The first item and the second item can have the same or similar characteristics (e.g., style, color, size, left - right, etc.). Alternatively, the first item can have one or more characteristics that are different from the corresponding characteristics of the second item.

[0291] In some embodiments of any of the methods described in paragraphs 225 - 228, the method can further include generating an image of the item using an image sensor. In some embodiments, the method can further include generating an image and a graphic of the item.

[0292] In some embodiments of any of the methods described in paragraphs 225 - 229, the method can further include receiving an image of the item. In some embodiments, the method can further include receiving an image and a graphic of the item.

[0293] In some embodiments of any of the methods described in paragraphs 225 - 230, the fiducial includes a logo set on the item. In some embodiments, the fiducial includes an image printed on the item.

[0294] In some embodiments of any of the methods described in paragraphs 225 - 231, the fiducial includes a surface pattern of the item. In some embodiments, the surface pattern includes one or more shoelace eyelets. In some embodiments, the surface pattern includes one or more studs. In some embodiments, the surface pattern includes a bite line profile. In some embodiments, the surface pattern includes one or more perforations.

[0295] In some embodiments of any of the methods described in paragraphs 225-232, the fiducial includes a pattern variation in a knitted article. In some embodiments, the fiducial includes a pattern variation in a textile article. In some embodiments, the fiducial includes a locating pin of a loom.

[0296] In some embodiments of any of the methods described in paragraphs 225-233, the fiducial is disposed on a sacrificial member that is coupled to the article during manufacturing and removed from the article after assembly.

[0297] In some embodiments of any of the methods described in paragraphs 225-234, the fiducial is only visible to the system under certain lighting conditions. In some embodiments, the method may further include exposing the fiducial to the lighting conditions under which the fiducial is visible prior to the action of positioning the fiducial. In some embodiments, the fiducial includes an ultraviolet fluorescent material and the lighting condition is ultraviolet light. In some embodiments, the ultraviolet fluorescent material is an ultraviolet fluorescent ink. In some embodiments, the ultraviolet material is an ultraviolet fluorescent yarn.

[0298] In some embodiments of any of the methods described in paragraphs 225-235, the fiducial includes light visible on the article. In some embodiments, a light source is disposed in an interior portion of the article and the light is visible on an exterior portion of the article.

[0299] In some embodiments of any of the methods described in paragraphs 225-236, the fiducial includes a laser visible on the article. In some embodiments, the laser is disposed in an interior portion of the article and the laser light is visible on an exterior portion of the article.

[0300] In some embodiments of any of the methods described in paragraphs 225-237, the fiducial is disposed on a shoe last. In some embodiments, the fiducial is a grid on the shoe last. In some embodiments, the shoe last includes one or more lights that enable the grid to be visible through the article.

[0301] In some embodiments of any of the methods described in paragraphs 225-238, a shape model of the article is used to perform the positioning of the fiducial.

[0302] In some embodiments of any of the methods described in paragraphs 225-239, a shape model of the article is used to perform the positioning of the graphic.

[0303] In some embodiments of any of the methods described in paragraphs 225-240, the article is a footwear article. In some embodiments, the article is a clothing article.

[0304] In some embodiments of any of the methods described in paragraphs 225-241, the article can be manufactured according to any disclosed method.

[0305] Figure 25 Depicts an exemplary method 2500 for printing on an item. One or more steps of the method can be performed using a computer. Method 2500 can include locating a fiducial attached to a print bed of a printing device in an image of the item (process block 2502), using the fiducial to establish a reference point for printing on the item (process block 2504), providing a raster or vector graphics file including a graphic (process block 2506), printing a representation of the graphic on the item (process block 2508), and removing the item from the print bed (process block 2510).

[0306] In some embodiments of the method described in paragraph 243, the method further includes: receiving a first raster or vector graphics file including a graphic, transforming the graphic based on the reference point, and generating a second raster or vector graphics file including the transformed graphic.

[0307] In some embodiments of any of the methods described in paragraphs 243-244, the transformation includes at least one or a combination of the following: mirroring, rotating, stretching, distorting, resizing, coloring, cropping, trimming, splitting, or obscuring the graphic or certain portions of the graphic. Transforming the graphic results in a transformed graphic, which can then be printed onto an item specifically corresponding to such transformation.

[0308] In some embodiments of any of the methods described in paragraphs 243-244, the graphic is customer-selected and is received with a customer order for the item.

[0309] In another representative embodiment, a method for printing onto an item can include automatically detecting physical features of a footwear item (including one or more of the object to be printed, heel edge, bite line, studs, upper edge, or any surface pattern), aligning a printing device to the detected features, and printing on the item.

[0310] In some embodiments, one or more computer-readable storage media storing computer-readable instructions cause a computer to perform any of the disclosed methods when the instructions are executed by the computer.

[0311] In another representative embodiment, a system can include one or more image sensors, one or more processors, one or more output devices, and one or more computer-readable storage media storing computer-executable instructions that, when executed by the processor, cause the system to perform Figure 26An exemplary method 2600 of processing an article as depicted, the instructions including instructions to receive an article image from one or more image sensors (process block 2602), instructions to determine one or more fiducial locations in the article image (process block 2604), and instructions to send a signal to one or more output devices to process the article using the one or more fiducials as reference points (process block 2606).

[0312] In some embodiments of the system described in paragraph 249, one or more output devices include a printing device configured with a print head for applying graphics to an article.

[0313] In some embodiments of the system described in any one of paragraphs 249 - 250, one or more output devices include a cutting device configured with a laser for cutting an article.

[0314] In some embodiments of the system described in any one of paragraphs 249 - 251, one or more fiducials are attached to the output device, wherein the system further includes one or more lasts to which the output device is coupled and on which the article is disposed, wherein the one or more lasts are configured to position the article relative to the output device, and wherein the instructions further include instructions to determine the distance between the one or more fiducials attached to the output device and the article, and instructions to establish one or more reference points of the output device based on the positions of the fiducials.

[0315] In some embodiments of the system described in any one of paragraphs 249 - 252, one or more fiducials are movable relative to one or more lasts. In some embodiments, one or more lasts are movable relative to one or more fiducials.

[0316] In some embodiments of the system described in any one of paragraphs 249 - 253, the system includes a plurality of image sensors.

[0317] In some embodiments of the system described in any one of paragraphs 249 - 254, the plurality of image sensors includes a first image sensor and a second image sensor, wherein the first image sensor has a first orientation relative to the article, wherein the second image sensor is spaced apart from the first image sensor and has a second orientation relative to the article, and wherein the instructions further include instructions to generate a three - dimensional image of the article based on a first image generated by the first image sensor and a second image generated by the second image sensor.

[0318] In some embodiments of the systems described in any of paragraphs 249-255, the plurality of image sensors include a first image sensor and a second image sensor, where the first image sensor has a first orientation relative to the workspace of the output device, where the second image sensor is spaced apart from the first image sensor and has a second orientation relative to the workspace of the output device, and where the instructions further include instructions for generating a three-dimensional image of the workspace of the output device based on a first image generated by the first image sensor and a second image generated by the second image sensor.

[0319] In some embodiments of the systems described in any of paragraphs 249-256, the output device includes a print bed. In some embodiments, one or more fiducials are coupled to the print bed.

[0320] In some embodiments of the systems described in any of paragraphs 249-257, the output device is a three-dimensional printing system.

[0321] In some embodiments of the systems described in any of paragraphs 249-258, the instructions further include instructions for determining the distance from one or more fiducials to a feature of an item and instructions for scaling a graphic based on the determined distance.

[0322] In some embodiments of the systems described in any of paragraphs 249-259, the feature of the item is an edge of the item. In some embodiments, the feature of the item is an insole of the item.

[0323] In some embodiments of the systems described in any of paragraphs 249-260, the one or more fiducial points include a reference position. In some embodiments, the reference position is the center of gravity of the fiducial. In some embodiments, the reference position is an edge of the fiducial. In some embodiments, the reference position is the center of the bounding box of the fiducial.

[0324] In another representative embodiment, the system includes one or more image sensors, one or more displays, one or more processors, one or more output devices, and one or more computer-readable storage media storing computer-executable instructions that, when executed by the processor, cause the system to perform Figure 27 the exemplary method 2700 of processing an item depicted in, the instructions including instructions for displaying a shape model of the item (the shape model including graphic markers) on the display (process block 2702), and instructions for printing a graphic on the item at a location corresponding to the positioning of the graphic markers (process block 2704).

[0325] In some embodiments of the system described in paragraph 262, the instructions further include instructions to display an area of interest of the shape model on a display. In some embodiments, the instructions further include instructions to display a fiducial.

[0326] In some embodiments of the system described in any one of paragraphs 262-263, the instructions further include instructions to display a print bed.

[0327] In some embodiments of the system described in any one of paragraphs 262-264, the instructions further include instructions to display graphical markers used in the shape model.

[0328] In some embodiments of the system described in any one of paragraphs 262-265, the instructions further include instructions to display a list of parameters of the shape model.

[0329] Figure 28 An exemplary method 2800 for printing on an item is depicted. Method 2800 may include displaying an image of the item (process block 2802), displaying a fiducial in the image of the item (process block 2804), displaying an area of interest of the item in the image of the item (process block 2806), displaying a shape model of the item (process block 2808), displaying a list of parameters of the shape model (process block 2810), and providing an indication of alignment or misalignment of the item based on the position of the fiducial and the parameters of the shape model (process block 2812).

[0330] Figure 29 An exemplary method 2900 for printing on an item is depicted. Method 2900 may include compiling an image library (process block 2902), generating a shape model of the item (process block 2904), and compiling an item profile including item parameters based on the item dimensions (process block 2906) and storing the image, shape model, and profile in a computer-readable medium (process block 2908).

[0331] In some embodiments of any of the methods described in paragraphs 267-268, the image library and the shape model are based on an item having a first dimension, and the method further includes scaling a graphic to be printed on the item to a second dimension.

[0332] In some embodiments of any of the methods described in paragraphs 267-269, a table is used to scale the graphic. In some embodiments, an equation is used to scale the graphic.

[0333] In some embodiments of any of the methods described in paragraphs 267-270, the method further includes transferring data from a first location to a second location.

[0334] In some embodiments of any of the methods described in paragraphs 267-271, a method for printing on an article includes receiving data from a party performing any of the disclosed methods. In some embodiments, the method further includes printing a graphic on the article using the data.

[0335] In some embodiments of any of the methods described in paragraphs 267-272, the method further includes obtaining a configuration based on a bar code. In some embodiments, the method further includes obtaining a profile based on a quick response code. In some embodiments, the method further includes obtaining a configuration from a watermark printed on the article.

[0336] In some embodiments of any of the methods described in paragraphs 267-273, the method further includes identifying a region of interest on the article. In some embodiments, the region of interest is selected by a user. In some embodiments, the region of interest is selected by a computing device.

[0337] In another representative embodiment, the system includes a display of an image of the article, a fiducial, and a region of interest of the article.

[0338] In some embodiments of the system described in paragraph 275, the graphical user interface further includes a display of a graphical marker placed on the image of the article and a display of the position of the graphical marker relative to the fiducial.

[0339] In some embodiments of the system described in any of paragraphs 275-276, the user can move the graphical marker relative to the article.

[0340] In some embodiments of the system described in any of paragraphs 275-277, the image is a black and white image. In some embodiments, the image is a grayscale image. In some embodiments, the image is a color image. In some embodiments, the image is filtered with a color filter of an image sensor.

[0341] In some embodiments of the system described in any of paragraphs 275-278, the fiducial is a sacrificial material on the article. In some embodiments, the graphical marker represents a graphic to be printed on the article and the graphic is provided by the user. In some embodiments, the graphical marker represents a graphic to be printed on the article and the graphic is provided by a third party.

[0342] Given that the disclosed principles can be applied to many possible embodiments, it should be recognized that the illustrated embodiments are merely preferred examples and should not be considered to limit the scope of the claims. Instead, the scope of the claimed subject matter is defined by the appended claims and their equivalents.

Claims

1. A method for printing on an article, comprising: selecting a shape model, wherein the shape model is based on a first image of a first article and includes a reference, a reference position, and a graphic marker, wherein the reference is located in the first image of the first article, wherein the reference position is based on the position of the reference in the first image of the first article, and wherein the graphic marker is located at a preselected position relative to the reference position; locating the reference in a second image of a second article by a computing device, wherein the second article is different from the first article; establishing the reference position in the second image of the second article; locating the graphic marker at the preselected position within the second image of the second article; and printing a graphic at a graphic position on the second article, wherein the graphic position on the second article corresponds to the preselected position of the graphic marker on the second image of the second article.

2. The method according to claim 1, wherein, before printing the graphic on the second article, the method further includes displaying the second image of the second article having the graphic marker at the preselected position.

3. The method according to claim 1 or claim 2, wherein, locating the graphic marker includes rotating the graphic marker relative to the second image of the second article.

4. The method according to claim 1 or claim 2, wherein, the preselected position includes first direction coordinates, second direction coordinates, and a rotation orientation.

5. The method according to claim 1 or claim 2, wherein, before printing the graphic on the second article, the method further includes determining the size of the second article based on the second image of the second article.

6. The method according to claim 1 or claim 2, further comprising scaling the graphic marker based on the size of the second article.

7. The method according to claim 1 or claim 2, wherein, the reference includes a logo provided on the first article.

8. The method according to claim 7, wherein, after locating the reference in the second image and before printing the graphic on the second article, the method further includes scaling the graphic marker based on the size of the logo.

9. The method according to any one of claims 1-2 and 8, wherein, the reference includes a pattern provided on a shoe last for supporting the second article.

10. The method according to any one of claims 1-2 and 8, wherein, the reference includes a surface pattern of the second article.

11. The method according to any one of claims 1-2 and 8, wherein, the first article and the second article are footwear articles.

12. The method according to any one of claims 1-2 and 8, wherein, the first article and the second article are clothing articles.

13. A manufactured article made by the method according to any one of claims 1-12.

14. One or more computer-readable storage media storing computer-readable instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1-12.

15. A method for printing on an article, comprising: locating a fiducial in an image of the article by a computing device based on a shape model, wherein the shape model is based on one or more images of another article, and wherein the another article is different from the article; establishing a reference position in the image of the article based on the position of the fiducial; locating a graphic marker within the image of the article relative to the reference position; and printing a graphic on the article based on the reference position of the located graphic marker.

16. The method according to claim 15, wherein the shape model of the another article is used to perform locating the fiducial, establishing the reference position, and locating the graphic marker.

17. The method according to claim 15 or claim 16, wherein the fiducial includes a logo provided on the article, and wherein after locating the fiducial and before printing the graphic on the article, the method further includes scaling the graphic marker based on the size of the logo.

18. The method according to claim 15 or claim 16, wherein the fiducial is coupled to a print bed of a printing device.

19. An article manufactured by the method according to any one of claims 15-18.

20. One or more computer-readable storage media storing computer-readable instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 15-18.

21. A method for printing on an article, comprising: locating a fiducial in an image of the article based on a shape model, wherein the fiducial is attached to a print bed of a printing device, wherein the shape model is based on one or more images of another article, and wherein the another article is different from the article; using the fiducial to establish a reference point for printing on the article; providing a raster or vector graphic file including a graphic; printing a representation of the graphic on the article; and removing the article from the print bed.

22. The method according to claim 21, wherein the raster or vector graphic file is a second raster or vector graphic file, and the method further includes: receiving a first raster or vector graphic file including the graphic; transforming the graphic based on the reference point; and generating the second raster or vector graphic file including the transformed graphic.

23. The method according to claim 22, wherein the transformation includes at least one or more of the following: mirroring, rotating, stretching, resizing, coloring, cropping, trimming, splitting, or masking the transformed graphic.

24. The method according to any one of claims 21-23, wherein the raster or vector graphic file is received via a computer network together with a customer order for the article.

25. The method according to any one of claims 21-23, further comprising: storing the raster or vector graphic file on a computer-readable storage medium.

26. The method according to any one of claims 21-23, wherein, locating the reference includes automatically detecting, by a computer, physical features of the footwear item, including one or more of a printed object, a heel edge, a bite line, a stud, a vamp edge, or any surface pattern.

27. An article manufactured by the method according to any one of claims 21-26.

28. One or more computer-readable storage media storing computer-readable instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 21-26.

Images