Footwear characteristics evaluation method and system

Abstract

The disclosure relates to a method and a system of determining the characteristics of a footwear assembly. The method comprises processing a footwear assembly at a first subset of one or more footwear manufacturing stations according to a sequence of processing steps; establishing and storing a footwear identification representation associated with the footwear assembly; receiving from an imaging system an output of image data of the footwear assembly; determining a characteristics evaluation quantifying the characteristics of the footwear assembly by analyzing image data on the basis of a characteristics evaluation model; associating the characteristics evaluation with a second subset of one or more footwear manufacturing stations based on the footwear identification representation.

Description

FOOTWEAR CHARACTERISTICS EVALUATION METHOD AND SYSTEMField of the invention

[0001] The present invention relates to associating determined characteristics of footwear assemblies with one or more footwear manufacturing stations.Background of the invention

[0002] The manufacture of footwear is a relatively unique and challenging process in that it involves the processing of many different materials according to highly specialized processing steps which are required to obtain a finished article of footwear.

[0003] Manufacture of a finished article of footwear typically involves the processing of footwear assemblies at one or more footwear manufacturing stations according to a sequence of processing steps. A footwear assembly typically comprises footwear parts which are processed at footwear manufacturing stations through processing steps such as cutting, trimming, roughing, stitching, shaping, molding, lacing, brushing, or gluing. A footwear assembly is typically processed according to a predefined sequence of processing steps at one or more predefined footwear manufacturing stations. A different footwear design or size may require a different sequence of processing steps at predefined footwear manufacturing stations.

[0004] Because of the relative complexity of manufacturing an article of footwear, it is often desirable to evaluate the characteristics of footwear assemblies as they are being manufactured as well as finished articles of footwear. Each processing step must be carefully controlled to ensure that the characteristics of the footwear assembly are in accordance with e.g. established quality standards. Characteristics of a footwear assembly may include properties such as the color of a material, the shape of a part, the thickness of a sole, the location of a stitch, or the absence of parts.

[0005] Characteristics of a footwear assembly may be identified via inspection by a human operator. This operator may e.g. inspect finished articles of footwear as they come off the manufacturing line before they are boxed to ensure that the finishedarticles of footwear are not defective and live up to quality control standards. Inspection by a human operator may for example be performed visually or with the aid of instruments or sensors. Human operators may also be stationed on the footwear manufacturing line and evaluate footwear assemblies as they are being processed at footwear manufacturing stations or as they are being transported between footwear manufacturing stations.

[0006] Characteristics of a footwear assembly may also be determined automatically via one or more sensors capturing data related to footwear assemblies and determining corresponding characteristics based on that data. Such a technology is described for example in US 9,939,803 B2 wherein an automatic quality control is established via the use of cameras and the implementation of a 2-D and 3-D recognition technology to ensure that footwear parts or shapes are properly formed or aligned with each other.

[0007] If an automatic or manual quality control system finds that the characteristics of a footwear assembly do not conform to established quality standards, then several actions may be taken. The affected footwear assembly or finished article of footwear may simply be discarded and the footwear manufacturing line may continue unimpeded. The footwear manufacturing line may also be subject to a stoppage during which an attempt to identify the cause of the unacceptable quality of footwear assemblies may be performed.Summary of the invention

[0008] The inventors have identified the above-mentioned problems and challenges related to footwear manufacturing systems and have developed the invention and embodiments described below in relation to determining a characteristics evaluation quantifying the characteristics of a footwear assembly by analyzing image data on the basis of a characteristics evaluation model and associating said characteristics evaluation with one or more footwear manufacturing stations based on a footwear identification representation.

[0009] In an aspect, the invention relates to a method of determining the characteristics of a footwear assembly, comprising: processing said footwear assemblyat a first subset of one or more footwear manufacturing stations according to a sequence of processing steps; establishing and storing a footwear identification representation associated with said footwear assembly; receiving from an imaging system an output of image data of said footwear assembly; determining a characteristics evaluation quantifying said characteristics of said footwear assembly by analyzing said image data on the basis of a characteristics evaluation model; associating said characteristics evaluation with a second subset of said one or more footwear manufacturing stations based on said footwear identification representation.

[0010] A method of determining the characteristics of footwear assemblies according to the invention may in various embodiments make it possible to determine the characteristics of footwear assemblies automatically by providing one or more images of a footwear assembly as input to a characteristics evaluation model and subsequently associate a characteristics evaluation with one or more footwear manufacturing stations.

[0011] The characteristics evaluation model may be in the form of an image processing algorithm which, based on raw image data, outputs a characteristics evaluation identifying and quantifying the characteristics of the footwear assembly. This automatic identification and quantification of the characteristics of footwear assemblies offers several advantages such as reduced cost compared to other methods which may require manual inspection of footwear assemblies by an operator in order to identify and quantify the characteristics, such manual inspection being both slower and more expensive than the proposed method.

[0012] A method of determining the characteristics of footwear assemblies according to the invention may in various embodiments be able to determine the characteristics of a footwear assembly in terms of the presence of one or more defects. Defects which impact the characteristics of a footwear assembly may according to the method be identified as belonging to a certain class such as e.g. excessive or insufficient application of glue, scuff marks on leather or textile, asymmetry in construction, aberrant stitching, and the presence of unwanted objects such as needles or nails.

[0013] Common defects often seen during the production of footwear assemblies are for example physical damage on upper leather such as cuts, bubbles on the sole both inside and outside, back height and inside height of an upper related to the position of the upper on the last, wrinkles on the upper, inconsistency in the color of sole parts, missing parts on the footwear assembly such as eyelets and stickers, mould pinching causing marks on the footwear assembly, sole material on undesired part of the upper, inconsistent application of coating on uppers or soles, trimming of sole parts either too much or too little, and roughing quality being either roughed too little or too much. One or more of the preceding examples of defects of footwear assemblies may advantageously be visible on image data as provided by an imaging system and may be formalized as a characteristics evaluation which may be used to associate the indicated defects with the one or more footwear manufacturing stations at which the footwear assembly has been processed.

[0014] The method for determining characteristics of a footwear assembly may therefore involve the automatic classification of defects based on image data of a footwear assembly, said image data being processed by a characteristics evaluation model whereby the characteristics evaluation model identifies the type of defect automatically. Such a method provides advantages over existing systems such as manual inspection in that the characteristics of the footwear assembly is automatically determined in terms of the presence or absence of defects without requiring manual inspection, processing, or logging of data by a human operator.

[0015] A method of determining the characteristics of footwear assemblies according to the invention may in various embodiments determine the characteristics of footwear assemblies by providing one or more characteristics measures which quantify the characteristics of the footwear assembly in terms of e.g. the location of one or more defects and assigning one or more numbers identifying the severity of the identified defects. A characteristics evaluation model according to the invention may for example provide the location or extent of a defect such as coordinates identifying the location of a scuff mark, the deviation in centimeters of an aberrant stitching, or geometric measures quantifying the asymmetry of a footwear assembly.

[0016] A method of determining the characteristics of footwear assemblies according to the invention may in various embodiments make it possible to associate characteristics evaluations with one or more footwear manufacturing stations. The process of association involves relating the characteristics of the footwear assembly to the various footwear manufacturing stations at which the footwear assembly has been processed and evaluate which footwear manufacturing stations may have caused the footwear assembly to exhibit e.g. a defect.

[0017] This automatic association between the characteristics of a footwear assembly and the footwear manufacturing stations at which it was processed provides advantages such as being able to identify footwear manufacturing stations which manufacture defective footwear assemblies and being able to take steps to prevent the manufacture of further defective footwear assemblies. Existing systems may not provide this automatic link between determining the characteristics of a footwear assembly and identifying the relationship between the characteristics and the individual footwear manufacturing stations of the footwear manufacturing line.

[0018] The present invention also provides advantages in terms of reducing run-in time when batches of footwear are manufactured at a footwear manufacturing line, such as for the purpose of replenishment. The present invention provides additional means by which manufacture may be optimized by identifying e.g. defects of footwear assemblies and associating them with one or more footwear manufacturing stations such that footwear assemblies can be e.g. routed away from particular footwear manufacturing stations or such that the problems pertaining to a particular footwear manufacturing station may be corrected.

[0019] Modern footwear manufacturing lines must often satisfy minimum order quantities of particular models or designs of footwear that trend towards becoming smaller as compared to the past. Consequently, setup time and run-in time of a footwear manufacturing line is extremely important and is an area in which this invention provides advantages in terms of quickly identifying defects of footwear assemblies as well as localizing the cause of the defect to one or more footwear manufacturing stations.

[0020] A footwear assembly within the context of the present invention is understood as a gathering of one or more footwear parts which are processed according to a sequence of processing steps at one or more footwear manufacturing stations to obtain a finished article of footwear.

[0021] The definition of a footwear assembly may in some special cases extend to include footwear manufacturing equipment which is brought into close physical proximity with footwear parts and which undergoes processing along with the footwear parts.

[0022] Such footwear manufacturing equipment may include a mold in which footwear parts may be molded and wherein the footwear part may be stored. In such an example, the mold and the footwear part in the mold together may constitute a footwear assembly.

[0023] Another example of a footwear assembly which comprises footwear manufacturing equipment is when an upper is mounted on a last. The upper and the last together may according to the definition be considered as a footwear assembly.

[0024] A footwear assembly may be said to be one of a plurality of footwear assemblies wherein each footwear assembly is associated with a respective footwear identification representation. The word each does not necessarily imply that all footwear assemblies of a plurality of footwear assemblies have a corresponding footwear identification representation. Instead, only some of the footwear assemblies may have corresponding footwear identification representations such that e.g. footwear identification representations are established and stored for some footwear assemblies.

[0025] Footwear parts are gathered in different ways for different purposes during the different manufacturing steps which comprise the manufacture of a finished article of footwear. Footwear parts that are brought into close physical proximity constitute a footwear assembly such as when different textile or leather elements constituting the footwear parts of an upper are placed side by side on a tray to prepare for subsequent manufacturing steps.

[0026] Footwear parts that are brought into physical contact also constitute a footwear assembly such as when different textile or leather elements are arranged on top of each other for subsequent stitching at a footwear manufacturing station performing e.g. automatic stitching.

[0027] Footwear parts that are attached using a form of mechanical fastening or bonding also form a footwear assembly. Mechanical fastening involves the use of e.g. stitching, nails, rivets or lacing to attach two or more footwear parts. Bonding for example involves the use of glue, thermal bonding, or chemical bonding. Footwear parts are understood to be any of the components at any point of the manufacture of a footwear which constitute a part of a manufactured article of footwear.

[0028] Footwear parts may therefore constitute any component of an article of footwear such as a piece of textile or leather, a waterproof membrane, a lining, a foam midsole, a plastic or rubber outsole. A footwear assembly may therefore also be understood to be a finished article of footwear wherein the footwear assembly comprises all the different components which make up the manufactured article of footwear.

[0029] The characteristics of a footwear assembly within the context of the present invention is understood as a quantitative measure of a manufactured footwear assembly which defines how the footwear assembly conforms to standards defined for footwear assemblies at different stages of manufacturing required for the manufacturing of an article of footwear.

[0030] The characteristics of a footwear assembly may be established in relation to any measurable quantity by which the footwear assembly may be evaluated against a defined quantitative standard for the manufacturing of footwear assemblies. The characteristics of a footwear assembly are therefore established through one or more characteristics measures which quantify different aspects of the footwear assembly in terms which may be compared against an established quantitative standard.

[0031] Characteristics of a footwear assembly may for example be used when left and right shoes are paired to form a pair of footwear, i.e. matching may be performedbased on the color of the right and left pair which may vary due to e.g. the quality of leather used.

[0032] A characteristics measure may for example be the enumeration and classification of the defects of a footwear assembly such as e.g. excessive or insufficient glue, undesired openings or holes in the upper or sole, missing, malformed or undesired parts, or scuff marks on leather or textile parts. A characteristics measure may also quantify the location and severity of an identified defect. This may be done via quantitative measures associated with each defect such as the width of a stitching or the thickness of a sole edge. These quantitative measures may then be compared with an established acceptable range such as an acceptable stitch width or sole edge thickness.

[0033] Characteristics of a footwear assembly may also be termed footwear assembly characteristics.

[0034] One or more of the characteristics of a footwear assembly may change as it undergoes a sequence of processing steps at one or more footwear manufacturing stations. The characteristics of a footwear assembly may thus continuously change as the footwear assembly is being manufactured at one or more footwear manufacturing stations. New characteristics may be added as footwear parts are added to or removed from the footwear assembly or as footwear parts are modified according to e.g. mechanical or chemical processes. The number, types and associated quantitative measures of the characteristics of a footwear assembly are thus subject to change at any point during the manufacturing process.

[0035] A characteristics measure may also be understood as a measurable deviation from an established statistical measure of some aspect of a footwear assembly which can be derived from images. A characteristics measure may thus be formulated with reference to e.g. color, texture, roughness, shape, curvature, thickness, stitch length, buffing or surface pattern of a footwear assembly or a part of a footwear assembly. Established statistical measures allow for the characteristics measure to be used in identifying footwear assemblies which deviate in some way from the standardproduction e.g. in terms of the color, surface texture or other characteristics of a footwear assembly.

[0036] Different characteristics of a footwear assembly may be identified via the use of image data. Characteristics may for example relate to the position of a particular footwear part of the footwear assembly i.e. the image data may be processed to determine whether the position of a particular footwear part is as desired. Characteristics may also relate to unwanted elements of a footwear assembly such as footwear parts that are identified from the image but should not be present according to the manufacturing specifications of the footwear assembly. Image data may also be used to identify characteristics that relate to the relative position or orientation of different parts of the footwear assembly such as the relative position of two or more elements. Particular types of image data may also be used to identify characteristics that relate to the structure of a material such as the presence or size of air bubbles in a polyurethane outsole.

[0037] Processing of a footwear assembly is understood to encompass the one or more manufacturing steps that a footwear assembly undergoes from e.g. initial processing of raw materials through to the final steps required to manufacture a finished article of footwear. Processing may refer to physical modifications or alterations to the footwear assembly or to footwear parts that comprise the footwear assembly i.e. where material is added to or removed from the footwear assembly. Processing may therefore involve activities such as cutting, stitching, shaping, molding, lacing, or brushing. Processing may also refer to manufacturing steps that do not add or remove material such as rearranging of the footwear parts of a footwear assembly, or gathering of different footwear parts to form a footwear assembly.

[0038] Processing of a footwear assembly is typically performed at one or more footwear manufacturing stations which each perform one or more steps in the processing.

[0039] A processing step within the context of the invention is understood to be an individual step in the processing which is delimited such that it forms a part of theprocessing of a footwear assembly. Processing of a footwear assembly may thus be split into one or more processing steps which may be performed according to a mode of operation such as sequentially, in parallel or overlapping.

[0040] Processing steps may be performed on the footwear assembly at one or more footwear manufacturing stations. Each footwear manufacturing station may perform one or more processing steps such that for example a single footwear manufacturing station may perform both a process step of cutting and a process step of stitching of a footwear assembly.

[0041] A sequence of processing steps within the context of the present invention is understood to define the order in which the footwear assembly undergoes processing at one or more footwear manufacturing stations. The sequence of processing steps may define the order in which processing steps are performed. The sequence of processing steps may also equivalently define the order of footwear manufacturing stations at which the footwear assembly is processed.

[0042] A footwear identification representation within the context of the present invention is understood as a unique ID which defines the identity of a footwear assembly. The footwear identification representation thus defines a means for associating a tangible, storable and readable code which is an unambiguous reference to a particular footwear assembly or at least defines an identity of a particular footwear assembly.

[0043] The footwear identification representation typically comprises a digital number or code through which a given footwear assembly is uniquely identified. The identification of a footwear assembly may therefore rely on linking the footwear identification representation with the footwear assembly that it identifies.

[0044] The linking between a footwear assembly and its corresponding footwear identification representation may be done through means such as reading a radio frequency identity RFID tag forming a part of the footwear assembly as it is being processed at various footwear manufacturing stations. The RFID tag may hold thefootwear identification representation in the form of a unique digital number or code which may be obtained when the RFID tag is scanned.

[0045] The footwear identification representation may also be represented by a digital representation which is stored centrally in a database. A link between the footwear identification representation and the footwear assembly can then be obtained through the scanning of a visual representation of the footwear identification representation which forms a part of the footwear assembly. This visual representation of the footwear identification representation may for example be a barcode or a quickresponse QR code which can be scanned by an optical scanner to obtain the digital representation which is then compared to its counterpart digital representation stored centrally in a database.

[0046] The footwear identification representation provides the means through which the sequence of processing steps for a footwear assembly can be identified at any point during the manufacture of the footwear assembly at one or more footwear manufacturing stations. The sequence of processing steps of a footwear assembly may be stored in digital form such that they are linked to the footwear identification representation. Upon obtaining the footwear identification representation of the footwear assembly through means such as scanning a visual representation of the footwear identification representation, the sequence of processing steps can then be identified.

[0047] Footwear identification representation associated with a footwear assembly may also comprise information indicative of model design, size, color, materials, footwear type, male / female / unisex model, unique assembly ID, unique footwear assembly ID, or any combination thereof of the footwear assembly or the footwear which is to be produced from the footwear assembly. Such information may for example be stored in a database, e.g. a footwear identification representation database.

[0048] The information may be accessible via the footwear assembly, e.g. via reading an RFID on the footwear assembly. Such a reading can directly provide the information or link the footwear assembly to the information stored in a footwearidentification representation database. However, embodiments of the invention do not necessarily rely on reading information via the footwear assembly. Instead, a system controller may keep track of each individual footwear assembly, such that it is capable of linking the each footwear assembly to its associated footwear identification representation.

[0049] The footwear assembly’s current position in the sequence of processing steps may also be identified in some embodiments of the invention. Based on the footwear assembly’s position in the sequence, it may be possible to discriminate between processing steps which have already been performed on the footwear assembly and processing steps which have not yet been performed on the footwear assembly.

[0050] Linking a footwear assembly with its corresponding unique footwear identification representation may not involve reading a visual representation of the footwear identification representation, but instead simply employ a means by which for example a system controller keeps track of each footwear assembly on the footwear manufacturing line such that each footwear assembly can be linked to its corresponding footwear identification representation at any point during which the footwear assembly is being processed according to a sequence of processing steps. The footwear identification representation may thus be made accessible through active or passive means through which the unique identity of a footwear assembly can be established.

[0051] The footwear identification representation may therefore provide a digital record of how its corresponding footwear assembly has been processed i.e. according to which sequence of processing steps.

[0052] A footwear manufacturing station within the context of the present invention is understood as a location at which one or more processing steps are performed on footwear assemblies to achieve the manufacture of articles of footwear. A footwear manufacturing station may comprise equipment for performing the necessary manufacturing operations. The footwear manufacturing station may comprise a robotwith a robot arm and various tools which may perform specific manufacturing operations such as cutting, stitching, molding, lacing, and brushing.

[0053] A footwear manufacturing station may comprise multiple pieces of equipment capable of performing multiple different processing steps. This is achieved e.g. by the footwear manufacturing station comprising a robot arm with interchangeable tools used for applying different manufacturing operations to the footwear assembly. A footwear manufacturing station may also comprise tools for manipulating the orientation or position of the footwear assembly such as a vacuum pick-and-place tool.

[0054] A subset within the context of the present invention refers to a non-empty division or portion of a set of objects. The term subset is strictly used in relation to the one or more footwear manufacturing stations at which the footwear assembly is processed. A subset of one or more footwear manufacturing stations therefore refers to a non-zero number of footwear manufacturing stations selected from a total number of one or more footwear manufacturing stations which is equal to or larger than the number of footwear manufacturing stations in the subset. A subset may for example consist of all the footwear manufacturing stations such as when only one footwear manufacturing station is present and the subset therefore necessarily consists of this one footwear manufacturing station. A subset of footwear manufacturing stations may also consist of all footwear manufacturing stations such that the subset is equal to the total number of footwear manufacturing stations.

[0055] A first subset of one or more footwear manufacturing stations includes the footwear manufacturing stations at which a footwear assembly is processed. The subset thus defines the footwear manufacturing stations at which the footwear assembly is processed according to a sequence of processing steps. A subset of one or more footwear manufacturing stations may be different for different footwear assemblies which are subject to different sequences of processing steps.

[0056] A second subset of one or more footwear manufacturing stations includes the footwear manufacturing stations for which an association with a characteristicsevaluation has been established on the basis of the footwear identification representation corresponding to the footwear assembly.

[0057] Different footwear assemblies used in e.g. different footwear designs or sizes may be subject to some of the same processing steps whereas other processing steps may be different or the order of processing steps may be different.

[0058] An imaging system within the context of the present invention is understood as any mechanical, digital or electronic viewing device, camera or any other instrument capable of recording and transmitting image data. The imaging system comprises at least an image sensor capable of converting light into electronic signals which comprise recorded image data. The imaging system comprises an output through which the image data may be transmitted to other systems. The imaging system is typically a digital system that records image data in a machine-readable digital format which may be transmitted to other digital systems via a digital interface.

[0059] The imaging system may comprise a number of other components depending on the required performance and specific conditions under which the imaging system must function. These components may include lighting, filters, optics, processing capabilities, or communication module. Such additional components allow the imaging system to improve the performance of its function of capturing image data of footwear assemblies according to different parameters which may be relevant according to a specific embodiment.

[0060] The imaging system may comprise components such as cameras which are physically distributed i.e. the imaging system may comprise more than one image sensor and these image sensors may be located in different places such that they are able to record image data of footwear assemblies at different stages of manufacture. The image sensors may also be located with different points of origin in space, such that they are able to record image data of the same footwear assembly from different angles.

[0061] The imaging system may be used to provide image data of each footwear assembly manufactured at a footwear manufacturing line. However, it may also beused selectively such as by providing image data of only some of the footwear assemblies manufactured at a footwear manufacturing line.

[0062] The imaging system may at times provide image data that are unsuitable as input to a characteristics evaluation model. Image data that are unsuitable or of inferior quality may be caused by different factors such as hardware problems or variable lighting conditions.

[0063] Examples of imaging systems using different types of image sensors and sensor inputs are digital camera systems, magnetic resonance imaging systems, computed tomography scan systems, ultrasound imaging systems, X-ray imaging systems, infrared imaging systems, thermal imaging systems, or laser scan imaging systems.

[0064] Image data within the context of the present invention is understood as a data representation of a visual representation of an object which is provided by an imaging system. Image data are typically obtained by converting a physical quantity such as light into a machine-readable format such as a digital image via the use of an image sensor. Image data is usually stored as a number of bytes i.e. in a machine-readable digital format.

[0065] Image data may be a visual representation of many different types of objects, and image data may be obtained via the conversion of many different types of physical quantities into a machine-readable format such as a digital data format.

[0066] Image data may be of several different types and formats such as a two- dimensional color or grayscale digital image. Image data may also be in the form of point clouds representing points of an object in e.g. a three-dimensional space. Image data may be in the form of sequences of images that may for example be spatially or temporally related.

[0067] Image data of a footwear assembly may within the context of the present invention be obtained using an imaging system such that image data constituting for example one or more digital images forming a visual representation of the footwearassembly are generated. A footwear assembly for which image data has been recorded may be referred to as an imaged footwear assembly. Image data of a footwear assembly may be provided at any point during the manufacturing process required to manufacture a finished article of footwear when an imaging system is available to provide an output of image data. Thus, the footwear assembly may at or between manufacturing steps be brought into the field of view of one or more imaging devices, upon which image data of the footwear assembly can be recorded, stored, and provided as an output from the imaging system.

[0068] An example of an imaging system and associated image data is a thermal camera and associated thermal images. In an example, a footwear sole may be sprayed with a coating as part of a processing step at a footwear manufacturing station. The thermal camera may be placed in the vicinity of the footwear manufacturing station and provide thermal images of the sole. As the coating and the sole material absorb heat differently, it is possible via analyzing the thermal image data to determine whether the coating properly covers the desired surfaces of the sole. If such coating defects are determined, the particular footwear manufacturing station may be put offline for repair or the processing step may be adapted to ensure that the coating covers the appropriate parts of the sole for sole parts subsequently processed at the footwear manufacturing station.

[0069] Different quantities and types of image data may be required for different types of footwear assemblies depending upon the complexity of their footwear parts or on the requirements of the manufacturing step to which they belong. Providing an output of image data may also involve the use of multiple imaging devices as well as the physical manipulation of the orientation of the footwear assembly. The footwear assembly may for example be conveyed to the field of the view of the imaging device, or it may be rotated such that a specific part of the footwear assembly becomes visible to the imaging device such that image data that visualize that specific part may be generated. Other types of deformation of the footwear assembly may be necessary to reveal the defects of the footwear assembly to the imaging device such as by pullingor pushing on a part of the footwear assembly before image data is recorded by the imaging system.

[0070] The volume of image data may require specialized infrastructure and hardware to store, process, and handle. Such infrastructure may be connected locally with the imaging system or the characteristics evaluation model i.e. the image data may be stored in one or more hard drives as e.g. a database. The image data or characteristics evaluation may also be stored on a cloud server i.e. on a remote service that provides data handling and storage.

[0071] Management of large volumes of image data may require the implementation of a data management method according to which aspects of image data such as metadata, storage, documentation, and data quality.

[0072] A characteristics evaluation within the context of the present invention is understood as data which is obtained as the output from a characteristics evaluation model to which image data visualizing an imaged footwear assembly has been provided as input. A characteristics evaluation may include processed image data and one or more characteristics measures established for the imaged footwear assembly. Characteristics measures may be indicated both directly on output images as well as in associated data structures forming a part of the characteristics evaluation.

[0073] A characteristics evaluation model within the context of the present invention is understood as a computer-implemented model for evaluating the characteristics of a footwear assembly. The characteristics evaluation model takes as its input image data depicting a footwear assembly obtained from the imaging system. The image data provided at the input of the characteristics evaluation model are processed according to one or more algorithms implemented by the characteristics evaluation model. The characteristics evaluation model in turn provides as its output a characteristics evaluation which quantifies the characteristics of the footwear assembly based on the provided image data. The characteristics evaluation model may be implemented using various image processing algorithms, or machine learning algorithms.

[0074] Associating within the context of the present invention is understood as establishing a relationship between the characteristics evaluation given as an output of the characteristics evaluation model and one or more footwear manufacturing stations at which the footwear assembly has been processed according to a sequence of processing steps. Associating the characteristics evaluation with one or more footwear manufacturing stations may involve establishing a causal relationship which relates the processing steps performed at one or more footwear manufacturing stations to particular characteristics measures of the characteristics evaluation. This association may be done on the basis of the footwear identification representation corresponding to the footwear assembly for which the characteristics evaluation has been determined. The characteristics evaluation may for example be compared directly with the footwear identification representation based on which an association between one or more footwear manufacturing stations and the characteristics evaluation may be established.

[0075] This is possible because the footwear identification representation contains at least a record of the first subset of footwear manufacturing stations at which the footwear assembly was processed and because it may also contain more detailed information such as at which footwear manufacturing station each of the processing steps were performed upon the footwear assembly.

[0076] Association between the characteristics evaluation and one or more footwear manufacturing stations therefore makes it possible to determine a footwear manufacturing station and a processing step which caused a footwear assembly to exhibit particular characteristics as quantified by the characteristics evaluation and its one or more characteristics measures.

[0077] The association between the characteristics evaluation and one or more footwear manufacturing stations may be performed through a simple table look-up whereby the data structures of the footwear identification representation and the characteristics evaluation are compared directly. The association may also involve more sophisticated algorithms which perform the association based on an estimate of how one or more characteristics measures of the characteristics evaluation may belinked with particular footwear manufacturing stations and corresponding processing steps.

[0078] The association between the characteristics evaluation and one or more footwear manufacturing stations may also be selective in that it is only applied for certain characteristics measures e.g. certain characteristics measures may deviate from established standards which in turn triggers a selective association of only those particular characteristics measures with one or more footwear manufacturing stations. This approach may be particularly relevant if there are concerns that a footwear assembly has one or more characteristics which may be characterized as defects i.e. that the footwear assembly has not been manufactured to an acceptable level of quality wherefore it must typically be discarded.

[0079] A footwear manufacturing line within the context of the present invention is understood as designating a footwear assembly line comprising a number of footwear manufacturing stations at which footwear assemblies are processed for the manufacturing of articles of footwear. A footwear manufacturing line may be understood as a conventional inline footwear assembly line by which footwear assemblies are processed at footwear manufacturing stations and conveyed between them according to a fixed sequence of processing steps.

[0080] A footwear manufacturing line in the context of the present invention may also be understood to include dynamic manufacturing whereby footwear assemblies are routed individually between footwear manufacturing stations according to the sequence of processing steps which must be performed for each individual footwear assembly. A footwear manufacturing line may therefore be understood to be adaptable such that e.g. the routing of a footwear assembly may be adapted continuously according to for example the availability of the different footwear manufacturing stations.

[0081] A footwear manufacturing line according to the present invention typically comprises transport measures for transporting the footwear assemblies between thedifferent footwear manufacturing stations. Such transport measures may include e.g. conveyors, mobile carriages, drones, or automatic guided vehicles.

[0082] The routing of a footwear assembly indicates that a footwear assembly has a transportation path through the footwear manufacturing line from an input footwear manufacturing station to a final output footwear manufacturing station. Routing of a footwear assembly may be done on the basis of the design and model of the footwear that the footwear assembly will become after completion of manufacture, or the routing may take into account factors such as availability of footwear manufacturing stations.

[0083] In an embodiment of the invention, said footwear assembly comprises one or more footwear parts.

[0084] The footwear assembly may comprise one or more footwear parts which may for example already be joined together, or which may be arranged for subsequent joining via a processing step whereby e.g. glue or stitching is applied. The number of footwear parts of a footwear assembly may change before, during or after each processing step. This may be due to a processing step whereby one or more footwear parts are joined together, one or more footwear parts are added to the footwear assembly, or one or more parts are removed from the footwear assembly.

[0085] In an embodiment of the invention, said footwear assembly comprises an identification marking corresponding to said footwear identification representation, and wherein said identification marking is attached to said footwear assembly.

[0086] The footwear assembly may advantageously comprise an identification marking attached to the footwear assembly which can be used to identify the footwear assembly via its corresponding footwear identification representation as it is processed at one or more footwear manufacturing stations. As the identification marking is attached to the footwear assembly, it is possible to read or scan the identification marking at any point during which the footwear assembly is processed at one or more footwear manufacturing stations. As the identification marking corresponds to the footwear identification representation, it becomes possible to update the footwearidentification representation as the identification marking is scanned at different footwear manufacturing stations and thereby indicate e.g. which footwear manufacturing stations the footwear assembly has been processed at or which processing steps the footwear assembly has been processed according to.

[0087] In an embodiment of the invention, said footwear assembly comprises footwear.

[0088] The footwear assembly may advantageously comprise footwear. A footwear in the context of the present invention is understood as a footwear assembly that comprises at least an upper and a sole part. Footwear may therefore for example be taken to mean a finished article of footwear, or it may be understood as an upper to which an outsole has been bonded. Footwear may still be subject to processing steps even if all footwear parts that constitute the footwear have already been joined to the footwear. Certain characteristics are particularly relevant for articles of footwear such as the relative orientation or position of different parts of the footwear. Such parts are only identifiable if image data are provided of the footwear after a number of processing steps have been performed.

[0089] In an embodiment of the invention, said footwear assembly comprises footwear mounted on a last.

[0090] A footwear assembly may advantageously comprise a footwear mounted on a last. The last is required for certain processing steps involving e.g. cementing of an outsole onto the upper or the direct injection of an outsole to an upper. A last in the context of the present invention is understood to be a mechanical form roughly in the shape of a human foot onto which a footwear upper may be mounted. While providing the mechanical support required for certain processing steps, the last may also be used when transporting the footwear between footwear manufacturing stations i.e. such that the footwear is transported or conveyed between footwear manufacturing stations while still being mounted on the last.

[0091] In an embodiment of the invention, said footwear assembly comprises an upper mounted on a last.

[0092] The footwear assembly may comprise an upper which is mounted on a last. Such a footwear assembly is typically encountered during the manufacture of footwear because of the need for mechanically supporting, shaping and stretching the upper. A lasting of the upper onto the last may be performed as a processing step at a footwear manufacturing station.

[0093] An upper which is mounted on a last may provide for the identification of characteristics of the footwear assembly which may not otherwise be visible on image data provided by the imaging system. These characteristics may for example be related to the shape of the upper or to the way in which the upper is stretched across the last. When such image data are provided as an input to the characteristics evaluation model, characteristics measures as described that are particularly relevant to the upper may be provided as part of the determined characteristics evaluation.

[0094] In an embodiment of the invention, said footwear assembly comprises a sole or sole part.

[0095] The footwear assembly may advantageously comprise a sole or a part of a sole. A sole may be manufactured individually at one or more footwear manufacturing stations by e.g. a process of press molding. Image data provided of a sole or sole part may be used to determine a characteristics evaluation providing characteristics measures that are specifically related to the sole. These characteristics measures may relate to the dimensions of the sole, the patterning, holes in the sole and so on.

[0096] In an embodiment of the invention, said footwear assembly comprises an upper or upper parts.

[0097] The footwear assembly may advantageously comprise an upper or upper parts. Image data of an upper may be used to determine unique characteristics of the upper. These characteristics may relate to a Strobel stitching of the outer layer, lining or membrane of the upper to the insole. This stitching is particularly important for waterproof and water permeable footwear wherein the holes made by the stitching must subsequently be filled out such that the footwear is rendered waterproof. A characteristics evaluation associated with footwear manufacturing stations at whichsuch a Strobel stitching is performed may be obtained by supplying the characteristics evaluation model with image data of an upper or upper parts.

[0098] In an embodiment of the invention, said upper is made from an upper material, wherein said upper material is one or more of the upper materials selected from the list comprising: cotton, polyester, nylon, propylene, lycra and / or wool.

[0099] In an embodiment of the invention, said footwear assembly comprises a midsole or midsole parts.

[0100] The footwear assembly may advantageously comprise a midsole or midsole parts such that characteristics of midsole or midsole parts may be determined via a characteristics evaluation obtained by analyzing image data on the basis of a characteristics evaluation model. Such characteristics may relate to the shape of the midsole or midsole parts or to gaps or holes in the midsole. Certain imaging systems would also allow for characteristics measures which quantify the inner structure of the material used when molding the midsole or midsole parts. Such characteristics may for example involve the identification of bubbles in an ethylene vinyl acetate material used in the midsole.

[0101] The imaging system may in an example be a millimeter wave radar capable of emitting electromagnetic radiation such as in a range between 30 to 300 GHz. The millimeter wave radar or scanner may enable the formation of images whereby the inner structure of a midsole or outsole material may be ascertained. The millimeter wave radar may for example be utilized to identify via image processing the number and size of air bubbles inside a polyurethane material as the presence, number, and size of such air bubbles may be classified as a defect and for example be associated with a midsole or outsole that is less durable. If such a defect is determined by a method according to the invention, the problem may be addressed by for example adapting the ratio of a resin and a hardener in a polyurethane mixer unit to address the formation of air bubbles in e.g. direct injection molded midsoles or outsoles.

[0102] In an embodiment of the invention, said midsole or midsole parts are made from a midsole material, wherein said midsole material is one or more of the midsolematerials selected from the list comprising: thermoplastic rubber, polyurethane, and / or ethylene vinyl acetate.

[0103] In an embodiment of the invention, said footwear assembly comprises an outsole or outsole parts.

[0104] The footwear assembly may advantageously comprise an outsole or outsole parts such that characteristics of outsole or outsole parts may be determined via a characteristics evaluation obtained by analyzing image data on the basis of a characteristics evaluation model. Such characteristics may relate to various aspects of the outsole such as the shape, thickness, width, holes or gaps in the outsole.

[0105] In an embodiment of the invention, said outsole or outsole parts are made from an outsole material, wherein said outsole material is one or more from the outsole materials selected from the list comprising: thermoplastic rubber, polyvinyl chloride, ethylene vinyl acetate, thermoplastic polyurethane, or thermoplastic elastomer.

[0106] In an embodiment of the invention, said footwear assembly is an insole or insole parts.

[0107] The footwear assembly may advantageously comprise an insole or insole parts whereby a characteristics evaluation comprising characteristics measures related to the insole or insole parts may be obtained. The determined characteristics evaluation may in turn be associated to relevant footwear manufacturing stations at which the insole or insole parts were processed.

[0108] In an embodiment of the invention, said footwear assembly comprises joined footwear parts.

[0109] The footwear assembly may advantageously comprise footwear parts that are joined together. Joined parts may be associated with certain characteristics related to how the bonding is performed, whether this bonding involves stitching, gluing, thermal adhesion or other bonding methods. Image data of joined footwear parts may be used to obtain a characteristics evaluation which quantifies certain characteristics measures that are particularly relevant to joined footwear parts and allow forassociating the characteristics evaluation with one or more footwear manufacturing stations at which processing steps involving bonding were performed.

[0110] In an embodiment of the invention, said footwear assembly comprises loosely gathered footwear parts.

[0111] A footwear assembly may advantageously comprise loosely gathered footwear parts. Loosely gathered footwear parts that have not been adhered or bonded in any way may provide for image data from which a characteristics evaluation may be determined that quantifies characteristics that are particularly relevant for footwear parts that are loosely gathered. Such characteristics may relate to e.g. shape, position, orientation or surface characteristics of the loosely gathered footwear parts, based on which the characteristics evaluation may be associated with the one or more footwear manufacturing stations at processing steps related to the gathering of footwear parts were performed.

[0112] In an embodiment of the invention, said footwear assembly undergoes at least one state change from a first type of footwear assembly to a second type of footwear assembly as it is processed according to said sequence of said processing steps at said first subset of said one or more footwear manufacturing stations.

[0113] A footwear assembly manufactured according to a sequence of processing steps at a first subset of footwear manufacturing stations may advantageously belong to different footwear assembly types as it is being processed i.e. a footwear assembly may initially comprise an upper part but after one or more processing steps it may be processed into an upper and subsequently into a footwear when the outsole is bonded to the upper. The image data provided by the imaging system will therefore reveal different characteristics of the footwear assembly depending on which state the image data captures the footwear in.

[0114] In an embodiment of the invention, said footwear assembly is transported between said one or more footwear manufacturing stations by use of a footwear assembly carrier.

[0115] In an embodiment of the invention, said method further comprises the step of providing image data of said footwear assembly carrier.

[0116] In an embodiment of the invention, said footwear assembly carrier is a last.

[0117] A footwear assembly may be transported mounted on a last. The last may provide for image data which can be used to quantify characteristics associated with the last as determined by the characteristics evaluation model.

[0118] In an embodiment of the invention, said footwear assembly carrier is a mold.

[0119] A footwear assembly may be contained in a mold. Image data of the mold may be used to quantify characteristics associated with the mold as determined by the characteristics evaluation model. Image data of the mold may be used to quantify characteristics associated with processing steps that involve cleaning of the mold which may be performed after it has been used for press molding or direct injection molding of footwear parts.

[0120] In an embodiment of the invention, said type of footwear assembly includes one or more selected from the list comprising upper, upper parts, midsole, midsole parts, outsole, outsole parts, insole, insole parts, joined footwear parts, loosely gathered footwear parts, last or mold.

[0121] In an embodiment of the invention, said footwear assembly corresponds to one out of a plurality of footwear sizes.

[0122] In an embodiment of the invention, said footwear assembly corresponds to one out of a plurality of footwear designs.

[0123] In an embodiment of the invention, said sequence of said processing steps is predetermined for said footwear assembly.

[0124] The sequence of processing steps according to which a footwear assembly is processed may be predetermined such that it is known at a time before the footwear assembly is processed at a footwear manufacturing station. This advantageouslyallows for improved planning as the availability of footwear manufacturing stations can be used to map out at which footwear manufacturing stations the footwear assembly must be processed such that it is subject to the correct sequence of processing steps.

[0125] Such predetermined information about the sequence of processing steps for a footwear assembly may also be used to obtain information about footwear manufacturing stations which may not be used to process the footwear assembly. Some footwear manufacturing stations may for example perform specific processing steps which are not required by the predetermined sequence of processing steps for a given footwear assembly.

[0126] The footwear assembly may therefore advantageously skip certain footwear manufacturing stations at which no processing of the footwear assembly is required such that these footwear manufacturing stations are available to process other footwear assemblies that are being manufactured according to their own predetermined sequence of processing steps.

[0127] In an embodiment of the invention, said sequence of said processing steps is uniquely determined for said footwear assembly.

[0128] The sequence of processing steps according to which a footwear assembly is processed may be uniquely determined according to different considerations. The sequence of processing steps may for example be determined based on which design or size of footwear is ultimately desired. One design may for example require a processing step which comprises an engraving whereas another design may not. The sequence of processing steps may also depend on considerations relating to optimizing the manufacture of footwear assemblies, such as availability or redundancy of footwear manufacturing stations. A footwear assembly may for example be subject to a different sequence of processing steps if one or more footwear manufacturing stations are not operational or available for immediate processing of the footwear assembly.

[0129] A uniquely determined sequence of processing steps for a footwear assembly may also be updated in real time to accommodate dynamic changes such as the availability of footwear manufacturing stations which may change as the footwear assembly is being processed. The uniquely determined sequence of processing steps for a footwear assembly may therefore be updated one or more times during which the footwear assembly is being processed according to a sequence of processing steps.

[0130] Changes to the sequence of processing steps may also be reflected in a corresponding change in the first subset of footwear manufacturing stations at which the footwear assembly is processed. Each footwear assembly is therefore processed according to a uniquely determined sequence of processing steps such that the first subset of footwear manufacturing stations at which a footwear assembly is processed may vary across different footwear assemblies of the same or of different types. Two consecutively manufactured footwear assemblies of the same type may therefore be processed at different footwear manufacturing stations even if the sequence of processing steps required for one footwear assembly is identical to that required for the other footwear assembly i.e. the sequence of processing steps is uniquely determined for each footwear assembly.

[0131] A uniquely determined and adaptable sequence of processing steps provides many advantages related to modern footwear manufacture in which there can be a high degree of variance of footwear designs and therefore types of footwear assemblies being processed. The sequence being unique and adaptable provides for opportunities in terms of optimizing the manufacture of footwear assemblies according to for example the availability of footwear manufacturing stations.

[0132] In an embodiment of the invention, one or more of said processing steps of said sequence of processing steps are different.

[0133] One or more of the processing steps performed on the footwear assembly may advantageously be different. This provides for the different types of processing steps typically required for manufacture of a footwear assembly. A list of differentprocessing steps may for example comprise cutting, stitching, shaping, molding, roughing, lacing, gluing, and injecting.

[0134] In an embodiment of the invention, said footwear identification representation comprises a representation of said first subset of said one or more footwear manufacturing stations at which said footwear assembly has been processed according to said sequence of said processing steps.

[0135] The footwear identification representation may advantageously comprise an enumeration of the identity and sequence of footwear manufacturing stations at which it was processed. The footwear identification representation may be continuously updated as the footwear assembly is processed at a first subset of footwear manufacturing stations according to a sequence of processing steps. Accordingly, the footwear identification representation therefore contains a record of the footwear manufacturing stations at which it has been processed. Such a record may be employed when associating a provided characteristics evaluation with one or more footwear manufacturing stations because the footwear manufacturing stations at which the footwear assembly was processed can be identified from it.

[0136] In an embodiment of the invention, said footwear identification representation comprises a representation of said sequence of said processing steps according to which said footwear assembly has been processed.

[0137] The footwear identification representation may advantageously comprise a representation which enumerates the sequence of processing steps according to which the footwear assembly has been processed. This representation may be continuously updated as the footwear assembly is processed sequentially at the different footwear manufacturing stations. Accordingly, the footwear identification representation therefore contains a record of the footwear manufacturing stations at which it has been processed. Such a record may be employed when associating a provided characteristics evaluation with one or more footwear manufacturing stations because the processing steps performed upon the footwear assembly may be identified from the footwear identification representation.

[0138] In an embodiment of the invention, said footwear identification representation comprises a timestamp indicating the time at which one or more of said processing steps in said sequence occurred.

[0139] Advantageously, the footwear identification representation may comprise a timestamp which may indicate the time at which different processing steps in the manufacture of a footwear assembly occurred. The timestamp may thus comprise the time at which a processing step in a sequence of processing steps was performed at a footwear manufacturing station. The timestamp may be continuously updated as the footwear assembly undergoes processing at different footwear manufacturing stations according to a sequence of processing steps. The timestamp enables the tracking of when different processing steps are performed and additionally how much time each processing step takes. This allows for a more detailed tracking of a footwear assembly as it is processed. This information may be used to monitor the manufacturing process of footwear assemblies, and it may also be used to optimize the manufacturing process.

[0140] In an embodiment of the invention, said footwear assembly is automatically identified by said footwear identification representation at one or more of said first subset of said one or more footwear manufacturing stations.

[0141] The footwear assembly may advantageously be identified by its corresponding footwear identification representation at one or more of the footwear manufacturing stations at which it is processed. Identification is the process by which a link is established between the footwear assembly and its corresponding footwear identification representation, such that the unique identity of the footwear assembly is established. This provides the advantage that the footwear assembly’s identity can be established at several points during manufacture of the footwear assembly such that the identity of the footwear assembly is unambiguously defined.

[0142] The identification of the footwear assembly at different footwear manufacturing stations is important to ensure that the corresponding footwear identification representation is updated correctly such as with information describing the sequence of processing steps that have been performed on the footwear assembly.

[0143] The identification of a footwear assembly may be performed by reading e.g. an RFID tag or a barcode which is physically attached to the footwear assembly. This reading may be performed by an RFID or barcode scanner. Identification of a footwear assembly may also be performed by the system tracking the position and / or location of the footwear assembly indirectly by other means such as by tracking how each footwear assembly is conveyed from one footwear manufacturing station to another.

[0144] In an embodiment of the invention, said footwear identification representation is automatically updating as said footwear assembly is processed at said first subset of said one or more footwear manufacturing stations by reading a footwear identification code using a footwear identification reader, and wherein said footwear identification code is attached to said footwear assembly.

[0145] In an embodiment of the invention, said footwear identification representation is automatically updating as said footwear assembly is processed at said first subset of said one or more footwear manufacturing stations by reading a footwear identification code using a footwear identification reader, and wherein said footwear identification code is attached to a footwear assembly carrier to which said footwear assembly is connected.

[0146] In an embodiment of the invention, establishing and storing said footwear identification representation is performed on a server.

[0147] In an embodiment of the invention, said image data are of said footwear assembly at the end of said sequence of processing steps.

[0148] An imaging system may advantageously be employed at the end of a sequence of processing steps performed on a footwear assembly.

[0149] The end of a sequence of processing steps is understood as after the footwear assembly has been processed according to the last processing step in the sequence of processing steps. The imaging system may thereby provide image data of the footwear assembly which can be used to determine a characteristics evaluation which may beassociated with all the footwear manufacturing stations at which the footwear assembly has been processed.

[0150] The image data of a footwear assembly which has undergone all processing steps in a sequence of processing steps can be used by the characteristics evaluation model to provide a characteristics evaluation which may provide characteristics measures that relate to one or more of the footwear manufacturing stations at which the footwear assembly was processed. In this way, the image data so obtained provides a form of aggregated or compounded information which relates to a sequence of processing steps as well as several different footwear manufacturing stations. The information contained in the image data is aggregated in the sense that the characteristics evaluation that may be obtained from it may contain characteristics measures which relate to processing steps performed at any point during the sequence of processing steps.

[0151] Based on such image data, it may therefore be possible to identify interactions between different processing steps that have caused particular characteristics of the footwear assembly to emerge. Associating the characteristics evaluation with the one or more footwear manufacturing stations at which these processing steps were performed is a unique advantage of providing image data of a footwear assembly which are recorded at the end of a sequence of processing steps.

[0152] In an embodiment of the invention, said imaging system is located such that it provides said image data of said footwear assembly that has been processed according to a final processing step in said sequence of said processing steps.

[0153] Advantages of placing the imaging system such that it provides image data of a fully processed footwear assembly are obtained because of the aggregate information contained in such image data. When such image data is input to the characteristics evaluation model, it can be used to determine a characteristics evaluation which in turn can be related to all processing steps in the sequence of processing steps and the characteristics evaluation may therefore be associated with any of the footwear manufacturing stations at which the footwear assembly was processed.

[0154] In an embodiment of the invention, said image data are of said footwear assembly at the beginning of said sequence of said processing steps.

[0155] The image data may be provided of a footwear assembly which has not undergone processing according to any processing steps in the sequence of processing steps at any footwear manufacturing stations. Such image data may in turn be used to identify parts of the footwear assembly which may be obscured by later processing steps such as e.g. when footwear parts are joined together or layered such that some parts of the footwear assembly would no longer be visible if image data were provided after such processing steps had been performed.

[0156] In an embodiment of the invention, said imaging system is placed such that it provides said image data of said footwear assembly after said footwear assembly has been processed at one or more of said first subset of said one or more footwear manufacturing stations.

[0157] Placing the imaging system such that it provides image data of the footwear assembly after it has already been processed at one or more footwear manufacturing stations may provide several advantages. Certain characteristics of the footwear assembly may only be relevant after the footwear has undergone certain processing steps at one or more footwear manufacturing stations. The characteristics evaluation determined by providing the image data as an input to the characteristics evaluation model may therefore be suitable for identifying characteristics of the footwear assembly which are dependent upon the interaction between two or more processing steps performed at different footwear manufacturing stations.

[0158] In an embodiment of the invention, said imaging system is distributed across two or more of said one or more footwear manufacturing stations.

[0159] The imaging system may advantageously be distributed such that it comprises cameras at or between two or more footwear manufacturing stations. The imaging system may thereby provide image data of a footwear assembly at different points in the sequence of processing steps. These image data may consequently be provided tothe characteristics evaluation model such that it can provide a characteristics evaluation for each processing step for which image data have been provided.

[0160] Having image data of the footwear assembly corresponding to different processing steps is advantageous in that it may only be possible to identify certain characteristics of the footwear assembly before or after certain processing steps. This may for example be due to certain footwear parts of the footwear assembly being processed or occluded in such a way that the imaging system makes it impossible to identify certain relevant characteristics. Having a distributed imaging system therefore allows for the input to the characteristics evaluation model to facilitate the characteristics evaluation model in providing a potentially more useful characteristics evaluation.

[0161] In an embodiment of the invention, said imaging system is integrated with a third subset of said one or more footwear manufacturing stations.

[0162] The imaging system may advantageously be integrated with one or more of the footwear manufacturing stations which thereby correspond to a third subset. This provides the advantage that the imaging system can provide image data of the footwear assembly from the time that it arrives for processing at the footwear manufacturing system until such time that it has been processed at the footwear manufacturing station. Thereby the imaging system can provide image data of the footwear assembly as it’s being processed or in between the one or more processing steps which are performed at the footwear manufacturing station.

[0163] In an embodiment of the invention, said imaging system provides said image data of said footwear assembly after said footwear assembly has been processed according to one or more of said processing steps at said third subset of said one or more footwear manufacturing stations with which said imaging system is integrated.

[0164] An imaging system integrated with a footwear manufacturing station may be configured to provide image data of the footwear assembly after it has been processed according to one or more processing steps at the footwear manufacturing station. Such image data may provide for the determination of a characteristics evaluation whichquantifies characteristics of the footwear assembly which are not identifiable before the footwear assembly has undergone the processing steps performed at the footwear manufacturing station.

[0165] In an embodiment of the invention, said imaging system provides said image data of said footwear assembly before said footwear assembly has been processed according to one or more of said processing steps at said third subset of said one or more footwear manufacturing stations with which said imaging system is integrated.

[0166] An imaging system which is integrated with a footwear manufacturing station may advantageously provide image data of the footwear assembly before the footwear assembly is processed at the footwear manufacturing station. It may be necessary to provide image data before the assembly is processed, because the processing steps performed at the footwear manufacturing station may make it impossible to identify particular characteristics measures of the characteristics evaluation if such image data were to be provided as an input to the characteristics evaluation model.

[0167] Characteristics that are quantifiable by a characteristics evaluation before a processing step but not after may include characteristics that relate to parts of the footwear assembly which may be removed during the processing step or which may be obscured due to layering or bonding of different footwear parts that are added to the footwear assembly as part of the processing.

[0168] In an embodiment of the invention, said imaging system is configured to provide said image data of said footwear assembly before said footwear assembly has been processed at said first subset of said one or more footwear manufacturing stations.

[0169] The imaging system may be configured to provide image data of a footwear assembly that still has not undergone processing steps at one or more footwear manufacturing stations. The image data provided by the imaging system therefore depicts a footwear assembly which has undergone some but not all processing steps in the sequence of processing steps at which the footwear assembly is to be processed.

[0170] In an embodiment of the invention, said imaging system is configured to provide said image data of said footwear assembly according to different subsets of said one or more footwear manufacturing stations and different sequences of processing steps.

[0171] The imaging system may advantageously be configured such that it is able to provide image data of footwear assemblies that have been processed at different subsets of footwear manufacturing stations according to different sequences of processing steps. The imaging system may therefore be utilized to provide image data of footwear assemblies which require different processing such as footwear assemblies which correspond to different footwear designs or footwear assemblies which correspond to different types such as an upper or an outsole.

[0172] In an embodiment of the invention, said imaging system is one of a plurality of imaging systems each configured to provide image data of said footwear assembly.

[0173] The imaging system being one of a plurality of imaging systems may provide advantages when the footwear manufacturing stations are distributed with a considerable distance in physical space or when the footwear manufacturing stations may be thought of as constituting two or more footwear manufacturing lines which must each be equipped with an imaging system.

[0174] The volume of manufactured footwear assemblies may also necessitate a plurality of imaging systems such as when a single imaging system is not able to provide the amount of image data required.

[0175] In an embodiment of the invention, said imaging system is configured to identify said footwear assembly by linking said footwear assembly with said footwear identification representation.

[0176] In an embodiment, the imaging system may be configured to identify the footwear assembly by linking it with its corresponding footwear identification representation. The image data thereby correspond to a particular footwearidentification representation such that further processing of the image data may be referred to and identified with the correct footwear assembly.

[0177] In an embodiment of the invention, said imaging system is configured to perform one or more image processing steps as part of an image processing pipeline before providing said image data.

[0178] The image data provided as an output from the imaging system is not necessarily the raw image data as captured by an imaging sensor such as a camera.

[0179] In an embodiment of the invention, said imaging system is a digital imaging system.

[0180] A digital imaging system advantageously provides output image data in the form of a digital representation which may be stored, read and processed by a computer. Such image data are advantageously processed by image processing algorithms which make use of modern computer hardware to ensure efficiency in processing.

[0181] In an embodiment of the invention, said imaging system comprises a camera.

[0182] The imaging system may advantageously comprise a camera whereby physical properties such as light reflected from the footwear assembly may be converted into an electrical signal and used for the generation of images.

[0183] In an embodiment of the invention, said camera comprises an adjustable camera pose.

[0184] The pose of the camera may be adjustable such that the camera can capture image data from different angles. This provides the advantage that certain parts of the footwear assembly from which certain characteristics may be identified are only visible from certain angles.

[0185] In an embodiment of the invention, said imaging system comprises a lighting system.

[0186] The imaging system may advantageously comprise a lighting system whereby the lighting in the proximity of the imaging system can be controlled. This is advantageous when image data of a footwear assembly is to be collected because consistent illumination can be achieved thereby avoiding problems with poor quality of the image data or with repeatability where image data of different footwear assemblies advantageously should be able to resolve the same level of detail.

[0187] In an embodiment of the invention, said imaging system comprises optical filters.

[0188] A filter may be used to filter out certain wavelengths of light or provide certain interference patterns or polarization of light.

[0189] In an embodiment of the invention, said imaging system comprises optics.

[0190] In an embodiment of the invention, said imaging system comprises a processor.

[0191] In an embodiment of the invention, said processor is one or more of the processor types selected from the list comprising: central processing unit CPU, graphics processing unit GPU, application-specific integrated circuit ASIC, digital signal processor DSP, field-programmable gate array FPGA, or microcontroller.

[0192] In an embodiment of the invention, said imaging system comprises a manipulator configured to adjust the pose of said footwear assembly.

[0193] The imaging system may advantageously comprise a manipulator which is configured to adjust the pose of the footwear assembly. This provides the advantage that the footwear assembly can be moved or rotated such that a different view of the footwear assembly becomes visible to the imaging system which is thereby able to provide image data of the footwear assembly seen from different angles. This may be particularly relevant for imaging systems that utilize sequences of images such as e.g. a stereo imaging system or a 3D imaging system.

[0194] In an embodiment of the invention, said imaging system comprises a box configured to provide consistent illumination of said footwear assembly.

[0195] The imaging system may comprise a box which blocks ambient light from the surrounding environment such that the illumination of the footwear assembly in the box is controlled. When the footwear assembly is placed in this box, the image data provided by the imaging system is subject to a controlled level of illumination and therefore provides advantages in terms of repeatability when the image data is used as an input to the characteristics evaluation model which in turn generates the characteristics evaluation.

[0196] In an embodiment of the invention, said imaging system comprises a communication module.

[0197] The imaging system may advantageously comprise a communication module which provides the advantage that image data may be transmitted to other electronic systems for storage or processing. The communication module may be a wired standard interface through which the imaging system can be connected directly to other systems or it may be a wireless connection providing as an example a connection to an external server.

[0198] In an embodiment of the invention, said imaging system comprises one or more sensors configured to provide an output of sensor data.

[0199] It may be advantageous to augment the image data with data from one or more different sensor types. This data may be processed along with the image data by the characteristics evaluation model and may enable the determined characteristics evaluation to provide a quantification of additional characteristics that are not possible using image data on its own.

[0200] In an embodiment of the invention, said one or more sensors are one or more of the types of sensors selected from the list comprising: temperature sensors, humidity sensors, elastic displacement sensors, contact thickness gauge, vibration sensors, capacitive sensors, or inductive sensors.

[0201] Additional sensors positioned at the imaging system may advantageously be of many different types. A temperature sensor may for example be used to measure the temperature of the footwear assembly whereas other types of sensors may measure quantities such as humidity, elasticity, displacement, thickness, vibration, capacitance, or inductance. The measurements so obtained may be used as additional input to the characteristics evaluation model which may in turn determine a characteristics evaluation which quantifies additional characteristics of the footwear assembly that are possible due to the additional sensor input so provided.

[0202] In an embodiment of the invention, said image data comprises one or more of the image types selected from the list comprising: 2D images, video sequences, 3D images, 3D point clouds, 2D laser scans.

[0203] The image data provided by the imaging system may advantageously be of many different types depending on the type of footwear assembly. The characteristics of some types of footwear assemblies may be identifiable only by providing certain types of image data to the characteristics evaluation model. The image data may therefore be adapted to fit the requirements of the characteristics evaluation which must be generated by the characteristics evaluation model to provide the appropriate association with the one or more footwear manufacturing stations.

[0204] 2D image data may be relevant for characteristics that relate to the surface of a footwear assembly i.e. that which can be captured using a standard digital camera. 2D image data may be especially relevant when identifying characteristics relating to e.g. the position, orientation, size, or width of parts of a footwear assembly.

[0205] 3D image data may be relevant when a computer model of the footwear assembly is desired, and it may also be obtained from multiple 2D images using techniques such as structure from motion.

[0206] In an embodiment of the invention, said image data comprise metadata.

[0207] The image data may advantageously comprise metadata which may include technical data about the imaging system, a timestamp, or metadata generated by the imaging system itself.

[0208] In an embodiment of the invention, said image data comprise digital color images.

[0209] The image data may advantageously comprise digital color images providing color information in addition to intensity information. The digital color images may be encoded using any suitable encoding such as RGB or HSV. Digital color images may be particularly advantageous for certain applications such as when determining characteristics that relate to the color of a footwear assembly or a part of a footwear assembly.

[0210] In an embodiment of the invention, said image data comprise digital grayscale images.

[0211] The image data may advantageously comprise digital grayscale images providing e.g. intensity levels for a two-dimensional grid of pixel values. Use of grayscale image data may be advantageous for certain applications such as when characteristics related to the presence or absence of certain parts of the footwear assembly must be determined by the characteristics evaluation model.

[0212] In an embodiment of the invention, said image data are digital binary images.

[0213] The image data may advantageously comprise digital binary images providing e.g. binary values for a two-dimensional grid of pixel values.

[0214] The routing of a footwear assembly may be adapted according to a prioritized order for the manufacture of footwear assemblies. A particular footwear assembly may therefore be rerouted such that it for example can be manufactured more quickly or it may be deprioritized and put into a waiting queue.

[0215] In an embodiment of the invention, said first subset of said one or more footwear manufacturing stations each perform one or more of said processing steps from said sequence of processing steps on said footwear assembly.

[0216] The first subset of footwear manufacturing stations at which the footwear assembly is processed may advantageously each perform one or more of the processing steps in the sequence of processing steps according to which the footwear assembly must be processed.

[0217] Each processing step may require specialized machinery such as footwear manufacturing robots or automatic sewing machines such that a footwear manufacturing station may advantageously be specialized to perform one or more processing steps by including such relevant components.

[0218] In an embodiment of the invention, said second subset of said one or more footwear manufacturing stations is a subset of said first subset of said one or more footwear manufacturing stations.

[0219] The second subset of footwear manufacturing stations that are associated with the characteristics evaluation may advantageously be a subset of the one or more footwear manufacturing stations at which the footwear assembly was processed. This is due to the characteristics evaluation quantifying the characteristics of the footwear assembly which are typically related to those footwear manufacturing stations at which the footwear assembly was processed according to a sequence of processing steps.

[0220] Performing the association between the characteristics evaluation and the second subset of one or more footwear manufacturing stations will therefore typically result in the associated footwear manufacturing stations being one or more of the footwear manufacturing stations at which the footwear assembly was processed.

[0221] In an embodiment of the invention, said footwear assembly is transported between said first subset of said one or more footwear manufacturing stations according to a routing based on said footwear identification representation.

[0222] The footwear identification representation may be used to determine the routing between the first subset of footwear manufacturing stations at which it is to be processed according to a sequence of processing steps. The footwear manufacturing stations at which the footwear assembly must be processed can therefore be selected according to a routing based on the footwear identification representation.

[0223] In an embodiment of the invention, said routing is predetermined and defines said first subset of said one or more footwear manufacturing stations at which said footwear assembly is processed according to said sequence of said processing steps.

[0224] The routing of a footwear assembly between said first subset of footwear manufacturing stations may advantageously be predetermined such that it is known before the sequence of processing steps is initiated. This provides advantages of planning i.e. such that availability of the footwear manufacturing stations which are part of the routing and at which the footwear assembly must be processed is ensured.

[0225] In an embodiment of the invention, said routing is dynamic such that said first subset of said one or more footwear manufacturing stations is adaptable while said footwear assembly is being processed according to said sequence of said processing steps.

[0226] Advantages of dynamic routing are that the first subset of one or more footwear manufacturing stations may be changed according to different parameters which may influence the efficiency of manufacture of footwear assemblies or the quality of manufactured footwear assemblies.

[0227] Dynamic routing indicates that the first subset of footwear manufacturing stations at which the footwear assembly is processed may be influenced for example by a characteristics evaluation associated with one or more footwear manufacturing stations. Based on this association, it may be determined that the first subset of one or more footwear manufacturing stations must be changed to ensure e.g. that quality standards of manufactured footwear assemblies are upheld or that manufacturing efficiency is prioritized.

[0228] In an embodiment of the invention, said routing of said footwear assembly is adapted according to a desired manufacturing order of said footwear assembly.

[0229] In an embodiment of the invention, at least one footwear manufacturing station of said first subset of said one or more footwear manufacturing stations is a redundant footwear manufacturing station.

[0230] At least one footwear manufacturing station of the first subset of footwear manufacturing stations performing the sequence of required processing steps upon a footwear assembly may be redundant in that it performs one or more processing steps that are identical to those performed by footwear manufacturing stations not part of the first subset. This redundancy allows for a footwear assembly to be processed using any of the redundant footwear manufacturing stations according to e.g. availability or performance.

[0231] Associating a characteristics evaluation of a footwear assembly with a redundant footwear manufacturing station therefore allows for the selection of a different footwear manufacturing station if the redundant footwear manufacturing station cannot deliver the desired characteristics of the footwear assembly and therefore must be adjusted, calibrated or repaired. Having redundant footwear manufacturing stations thus allows for increased flexibility in production such that the routing of footwear assemblies may be changed based on associating the characteristics evaluation with a redundant footwear manufacturing station.

[0232] A footwear manufacturing station may be defined as redundant even if only some of its processing steps are identical to that of other footwear manufacturing stations. Footwear assemblies which are processed according to those specific processing steps may thus be routed to any of the redundant footwear manufacturing stations for processing.

[0233] A characteristics evaluation associated with a redundant footwear manufacturing station may advantageously be used to calibrate other footwear manufacturing stations that perform identical processing steps. Calibration may for example be performed by adjusting the processing steps performed at other footwearmanufacturing stations to either match or deviate from the processing steps performed at the footwear manufacturing station associated with the characteristics evaluation.

[0234] In an embodiment of the invention, two or more footwear manufacturing stations of said first subset of said one or more footwear manufacturing stations are redundant footwear manufacturing stations.

[0235] In an embodiment of the invention, one or more of said first subset of said one or more footwear manufacturing stations comprise one or more human operators.

[0236] Some footwear manufacturing stations may involve human operators for processing steps that require a high degree of skill and precision. Such processing steps may for example include the lasting of an upper onto a last. The process of lasting an upper is complex and may benefit from highly experienced human operators. According to embodiments of the invention, it may be possible to compare the same processing step performed by a human and a robot in terms of the resulting characteristics of footwear assemblies i.e. the proposed method and system can be used to benchmark human operators against processing steps implemented automatically such as by using a robot.

[0237] In an embodiment of the invention, one or more of said one or more footwear manufacturing stations comprise a footwear identification representation reader configured for reading and updating said footwear identification representation associated with said footwear assembly.

[0238] One or more of the footwear manufacturing stations may advantageously read and update the footwear identification representation using a footwear identification representation reader such as e.g. an RFID reader. The footwear identification representation can thereby be updated and read as the footwear assembly is processed at the one or more footwear manufacturing stations. A footwear identification representation reader may enable the footwear manufacturing station or a central data processor to establish a link between the footwear assembly and the footwear identification representation.

[0239] The information contained in the footwear identification representation can be used to e.g. determine which processing step should be applied to the footwear assembly at the specific footwear manufacturing station. A footwear manufacturing station may for example be able to perform multiple different processing steps and may therefore require that the identity of the footwear assembly is determined such as via reading the footwear identification representation which provides the identity and status of the footwear assembly such that it may be processed according to the correct processing step in the sequence of processing steps that are to be performed by the footwear manufacturing stations.

[0240] In an embodiment of the invention, said footwear identification representation reader is a radio-frequency identification RFID reader.

[0241] An RFID tag is advantageous in that it can be integrated into the footwear in an unseeable manner, such that it can be utilized both during manufacturing of the footwear assembly but also by the consumer.

[0242] In an embodiment of the invention, said footwear identification representation reader is a bar code reader.

[0243] Within the context of the present invention a bar code may be any type of bar code such as a two-dimensional quick response QR code, a one-dimensional bar code, or a data matrix. A bar code offers advantages in that it may easily be attached to the footwear assembly i.e. in the form of a sticker or via a tag adhered to the footwear assembly. The bar code may also be impressed onto a part of the footwear assembly itself i.e. via stamping or spray application. The bar code may be read by a bar code reader such that information about the identity of a footwear assembly may be obtained.

[0244] In an embodiment of the invention, said footwear manufacturing station comprises one or more sensors.

[0245] In an embodiment of the invention, said characteristics evaluation comprises one or more characteristics measures.

[0246] One or more characteristics measures may advantageously be included in the characteristics evaluation such that a quantitative measure of one or more of the characteristics of the footwear assembly are obtained. Characteristics measures provide several advantages in that the characteristics of the footwear assembly may be graded i.e. according to the level of quality. The one or more characteristics measures may also be important criteria when associating the characteristics evaluation with the one or more footwear manufacturing stations.

[0247] Certain values or ranges of certain characteristics measures may optionally be used to trigger an alert or be used to implement an immediate stoppage of a particular footwear manufacturing station. Certain values of characteristics evaluations may also be used to adapt the routing of footwear assemblies i.e. such that footwear assemblies are subsequently routed away from or routed towards a particular footwear manufacturing station associated with a particular characteristics measure of a characteristics evaluation.

[0248] In an embodiment of the invention, said one or more characteristics measures relate to the presence or absence of parts of said footwear assembly.

[0249] Characteristics measures may indicate the presence or absence of certain parts of the footwear assembly. Such a characteristics measure may for example be in the form of an image feature which indicates that a certain part of the footwear assembly is present or absent. This provides the advantage that it becomes possible to identify whether certain parts of the footwear assembly are missing that should not be. The missing parts may be a footwear part of the footwear assembly such as e.g. a missing eyelet on an upper, a missing stitching on the leather part of an upper, or a missing edge of a sole part.

[0250] A characteristics measure may also indicate the presence of parts of the footwear assembly that should not be present. This may for example be a foreign object which should not be part of the footwear assembly such as e.g. a leather part that has not been cut correctly during a processing step that involves cutting. An advantage of identifying such undesired elements is that they may be removed. Additionally,associating the characteristics evaluation of a footwear assembly with one or more footwear manufacturing stations may be used to identify which footwear manufacturing stations cause undesired footwear parts or parts of the footwear assembly to be present when the imaging system provides image data of the footwear assembly.

[0251] In an embodiment of the invention, said one or more characteristics measures relate to the relative pose of two or more parts of said footwear assembly.

[0252] Characteristics measures which indicate the relative pose of two or more parts of the footwear assembly may also be obtained. This provides the advantage that a measure for the relative pose of different parts of the footwear assembly become available. Such characteristics measures may be used to associate the characteristics evaluation with footwear manufacturing stations where e.g. a processing step involving the joining of two or more footwear parts are performed such as e.g. the stitching together of two pieces of leather or the joining of an outsole to an upper via cementing or direct injection. The association of a characteristics evaluation comprising such characteristics measures with one or more footwear manufacturing stations therefore provides a way to indicate if the associated one or more footwear manufacturing stations do not perform the indicated processing step correctly such that e.g. the relative position or orientation of two or more parts of the footwear assembly are incorrect.

[0253] In an embodiment of the invention, said one or more of characteristics measures relate to surface characteristics of one or more parts of said footwear assembly.

[0254] A characteristics measure related to surface characteristics of a part of the footwear assembly allows for the characteristics evaluation to indicate characteristics related to the surface of different parts of the footwear assembly such as parts comprised of leather, polyurethane or other materials. Certain processing steps performed on a footwear assembly may involve a roughing step whereby for exampleleather is roughed to obtain particular surface characteristics of leather parts of the footwear assembly.

[0255] Other types of surface treatments such as spray applications may also be involved in certain processing steps. The characteristics of the surface of parts of the footwear assembly being quantified by a characteristics measure allows for the characteristics evaluation to be associated with the one or more footwear manufacturing stations which performed said processing steps involving modification of the surface characteristics of the footwear assembly.

[0256] In an embodiment of the invention, said characteristics evaluation comprises a mapping to a sequence of processing steps.

[0257] The characteristics evaluation may comprise a mapping between the quantified characteristics of a footwear assembly and the sequence of processing steps. This mapping identifies the link between a footwear assembly characteristic and corresponding one or more processing steps.

[0258] In an embodiment of the invention, said characteristics evaluation comprises an indication of defects of said footwear assembly.

[0259] The characteristics evaluation may advantageously include an indication of defects of the footwear assembly. The indication of defects may quantify the absence or presence of defects such as by enumeration or by the associated characteristics measures.

[0260] In an embodiment of the invention, said characteristics evaluation comprises a digital model of a footwear assembly.

[0261] The characteristics evaluation may advantageously comprise a digital footwear assembly model established on the basis of image data of a footwear assembly provided to the characteristics evaluation model. The digital model may e.g. be in the form of 3D model that establishes the shape and dimensions of the particular footwear assembly. This established model may in turn be compared or matchedagainst a reference model such as a reference model for a particular footwear design or size of footwear.

[0262] The digital model of a footwear assembly may also be used in simulation models such as those related to automatic stitching of footwear parts.

[0263] In an embodiment of the invention, said characteristics evaluation model implements an image processing model.

[0264] In an embodiment of the invention, said characteristics evaluation model implements a digital image processing pipeline.

[0265] In an embodiment of the invention, said digital image processing pipeline implements algorithms comprising one or more image processing steps selected from the list comprising: preprocessing, segmentation, postprocessing, visualization, or measurement.

[0266] An image processing model may be used to perform the processing of image data that are required by the characteristics evaluation model. The characteristics evaluation model may implement various image processing algorithms or pipelines implementing steps such as preprocessing of image data, segmentation of image data, visualization of image data, and the generating of an output characteristics evaluation to which one or more characteristics measures may be associated.

[0267] The image processing pipeline implemented by the characteristics evaluation model may employ any relevant image processing algorithm or any function that may be applied to image data such as scaling, unsharp masking, thresholding, segmentation, enhancement, cropping, complement, rotation, scaling, inversion, dilation, erosion, feature detection, object recognition, object detection, object tracking, or template matching and so on.

[0268] In an embodiment of the invention, said characteristics evaluation model is a machine learning model.

[0269] In an embodiment of the invention, said characteristics evaluation model is a trained classification model providing a classification output corresponding to said characteristics evaluation.

[0270] In an embodiment of the invention, said trained classification model is configured to provide a classification output by performing the steps of: receiving an input measurement at least based on said image data; classifying said characteristics of said footwear assembly into at least one of a plurality of predefined footwear assembly characteristics classes using said trained classification model to provide a classification output.

[0271] In an embodiment of the invention, said trained classification model is a neural network.

[0272] In an embodiment of the invention, said trained classification model is a convolutional neural network.

[0273] In an embodiment of the invention, said trained classification model comprises a feature extraction module.

[0274] In an embodiment of the invention, said trained classification model comprises: a feature extraction module configured to receive training input measurements and to generate a feature extraction output based on said training input measurements; and a classification module configured to classify footwear assembly characteristics classes based on the feature extraction output of the feature extraction module.

[0275] In an embodiment of the invention, said step of associating said characteristics evaluation with said second subset of said one or more footwear manufacturing stations based on said footwear identification representation is performed by said characteristics evaluation model.

[0276] In an embodiment of the invention, said footwear assembly characteristics classes correspond to at least one of said characteristics of said footwear assembly.

[0277] In an embodiment of the invention, said footwear assembly characteristics classes correspond to at least one of said second subset of said one or more footwear manufacturing stations.

[0278] In an embodiment of the invention, said trained classification model is a first trained classification model, and wherein a second trained classification model is configured to provide a second classification output by performing the steps of: receiving said classification output from said first trained classification model; receiving said footwear identification representation; classifying said second subset of said one or more footwear manufacturing stations into at least one of a plurality predefined footwear manufacturing station classes using said second trained classification model to provide a second classification output.

[0279] In an embodiment of the invention, said step of associating said characteristics evaluation with said second subset of said one or more footwear manufacturing stations comprises: providing a second trained classification model configured to classify predefined footwear manufacturing station classes; receiving one or more input measurement s) at least based on image data of a footwear assembly; classifying said second subset of said one or more footwear manufacturing stations into at least one of a plurality predefined footwear manufacturing station classes based on said one or more input measurement s) to provide a second classification output.

[0280] In an embodiment of the invention, said step of classifying said second subset of said one or more footwear manufacturing stations into at least one of a plurality of predefined footwear manufacturing station classes is further based on said footwear identification representation.

[0281] Classifying the second subset of said one or more footwear manufacturing stations based on the footwear identification representation may be understood as the footwear identification representation being provided as input to the second classification model. Further notice that the second classification model may be trained similarly to other classifications models described in this disclosure, e.g., classification models being trained to classify footwear assembly characteristics classes with theexception that the second classification model is trained based on labeled training data, wherein the label is one or more of the plurality of predefined footwear manufacturing station classes, while the input data is training input measurements.

[0282] The training input measurements may comprise one or more of the following non-limiting examples of training input measurements: image data, footwear identification representation, and classification output. Thereby, the second classification model may be trained to utilize these type of inputs or measurements to classify one or more predefined footwear manufacturing station classes. This is advantageous, in that this may enable automatic identification of which one or more manufacturing stations or combinations of manufacturing stations that may be associated with, e.g., footwear assembly characteristics of the manufactured footwear.

[0283] In an embodiment of the invention, said step of classifying said second subset of said one or more footwear manufacturing stations into at least one of a plurality of predefined footwear manufacturing station classes is further based on said classification output of said trained classification model.

[0284] In an embodiment of the invention, said second classification model is different from said trained classification model.

[0285] Advantageously, this may enable the second model to be specifically tailored to classify footwear manufacturing station classes, while the trained classification model may be specifically tailored to classify footwear assembly characteristics classes. Thereby, this may enable each of the two models to provide more accurate classification. This may be considered a form of an ensemble model that may both classify footwear assembly characteristics classes and footwear manufacturing station classes, with the advantages that ensemble models provide.

[0286] The models may be understood to be different, e.g., by being different types of machine learning models, by having different architectures if the implemented models are neural networks, or by having their hyperparameters tuned differently, or by utilizing different activation functions etc., to give a few non-limiting examples.

[0287] In an embodiment of the invention, said trained classification model and / or said second classification model comprises at least one pooling layer.

[0288] Advantageously, this may reduce the computational demands of the classification models.

[0289] In an embodiment of the invention, associating said characteristics evaluation with said second subset of said one or more footwear manufacturing stations is automatic.

[0290] The step of associating said characteristics evaluation with one or more footwear manufacturing stations based on the footwear identification representation may be automatic in that it is determined by a computer-implemented method instead of e.g. by a human operator. The automatic association between the characteristics evaluation and one or more footwear manufacturing stations may for example be performed by an automatic comparison performed by a data processor which compares the characteristics evaluation and the footwear identification representation and based on which an association between the characteristics evaluation and one or more footwear manufacturing stations may be established.

[0291] In an embodiment of the invention, associating said characteristics evaluation with said second subset of said one or more footwear manufacturing stations comprises associating said characteristics evaluation with said sequence of said processing steps.

[0292] The footwear identification representation may comprise an electronic record containing the sequence of processing steps according to which a corresponding footwear assembly has been processed. This sequence of processing steps and associated footwear manufacturing stations may be used when associating the characteristics evaluation with one or more footwear manufacturing stations. The association may be aided by linking each characteristics measure to one or more processing steps and mapping these processing steps onto the footwear identification representation from which associated footwear manufacturing stations can be identified.

[0293] In an embodiment of the invention, said method further comprises associating said characteristics evaluation with one or more materials of said footwear assembly.

[0294] The method of determining the characteristics of a footwear assembly may advantageously be configured to establish a relationship between the characteristics evaluation of the footwear assembly and its material composition. This may be done by relating one or more footwear parts of the footwear assembly to their constituting materials. In such a manner, it becomes possible to trace characteristics issues which may not be due to issues of manufacturing at any footwear manufacturing station, but instead are caused by an issue of the materials used in the footwear parts comprising the footwear assembly. In footwear production such issues could arise due to unacceptable characteristics of e.g. leather, rubber used as outsole material or polyurethane used for direct injection of outsoles.

[0295] In an embodiment of the invention, said method further comprises the step of providing an alert comprising said footwear identification representation and said second subset of said one or more footwear manufacturing stations associated with said characteristics evaluation.

[0296] A method of determining the characteristics of footwear assemblies according to the invention may in various embodiments provide an alert or warning associated with one or more footwear manufacturing stations or with one or more footwear identification representations. An alert or warning may be triggered according to the characteristics evaluation of a footwear assembly as provided by the characteristics evaluation model based on image data of the footwear assembly. The characteristics evaluation may indicate that a footwear assembly has a defect or that it has characteristics which do not conform to established quality standards of the footwear assembly and in turn an alert or warning may be generated.

[0297] The provision of an alert may have several advantages such as informing personnel at the footwear manufacturing line that manufactured footwear assemblies exhibit characteristics which violate standards for footwear assembly manufacturing.An alert or warning may also be provided in the form of an electronic signal which may also be transmitted to other electronic systems for storage.

[0298] The alert or warning may advantageously associate the characteristics evaluation of a footwear assembly with one or more footwear manufacturing stations thereby indicating which footwear manufacturing stations are causing the footwear assembly to exhibit unacceptable characteristics. The alert or warning may advantageously include the footwear identification representation of the footwear assembly for which the corresponding characteristics evaluation has triggered the alert or warning.

[0299] In an embodiment of the invention, said characteristics evaluation model executes said step of associating said characteristics evaluation with a second subset of one or more footwear manufacturing stations based on said footwear identification representation.

[0300] It may be advantageous that the characteristics evaluation model is further configured to perform the step of associating said characteristics evaluation with a second subset of footwear manufacturing stations. This may be especially advantageous if the footwear identification representation is stored on the same data processor that executes the characteristics evaluation model.

[0301] In an embodiment of the invention, one or more of said processing steps performed at said second subset of said one or more footwear manufacturing stations are adjusted based on said characteristics evaluation.

[0302] Processing steps that are performed at footwear manufacturing stations that have been associated with the characteristics evaluation may advantageously be adjusted. The characteristics evaluation may be used to determine an adjustment of one or more of the processing steps performed at the second subset of footwear manufacturing stations. Adjustment of a processing step within the context of the invention is understood as adjusting one or more parameters of the processing step that affect the resulting characteristics of the footwear assembly. When one or moreparameters of a processing step are adjusted, they may in turn change how a footwear assembly is processed according to that processing step.

[0303] Adjusting one or more parameters of a processing step may therefore result in a corresponding change in the characteristics of a footwear assembly and therefore in the corresponding characteristics evaluation. Adjustment of parameters of a processing step may therefore be advantageous when for example the characteristics evaluation indicates that certain characteristics measures do not conform to quality standards established for the footwear assembly.

[0304] Adjusting one or more parameters of a processing step of the second subset of one or more footwear manufacturing stations is advantageous when a plurality of footwear assemblies are to be manufactured at the second subset of one or more footwear manufacturing stations, because the parameter adjustment may effect an improvement in the quality of manufactured footwear assemblies which is consequently reflected in the corresponding characteristics evaluation which may quantify a desirable change in one or more characteristics measures.

[0305] In an embodiment of the invention, one or more of said processing steps performed at said second subset of said one or more footwear manufacturing stations are automatically adjusted.

[0306] Adjusting of the one or more processing steps of the second subset of one or more footwear manufacturing stations may advantageously be performed automatically such as by changing the inputs to a software program that controls the processing step. Adjusting of the one or more processing steps may accordingly be implemented by changing the parameters of a software program which controls the processing steps performed at a particular footwear manufacturing station.

[0307] In an embodiment of the invention, a conveyor transports said footwear assembly between said one or more footwear manufacturing stations for processing according to said sequence of said processing steps.

[0308] In an embodiment of the invention, a conveyor transports said footwear assembly between said one or more footwear manufacturing stations.

[0309] In an embodiment of the invention, a conveyor transports said footwear assembly on a footwear assembly carrier between said one or more footwear manufacturing stations.

[0310] In an embodiment of the invention, said a system controller is configured to route said footwear assembly between said first subset of said one or more footwear manufacturing stations.

[0311] In an embodiment of the invention, said system controller is further configured to track and update said footwear identification representation at said first subset of said one or more footwear manufacturing stations.

[0312] In an embodiment of the invention, said system controller is further configured to track and update said footwear identification representation while said footwear assembly is processed at said first subset of said one or more footwear manufacturing stations according to said sequence of said processing steps.

[0313] In an aspect, the invention relates to a footwear manufacturing line comprising: one or more footwear manufacturing stations configured to process a plurality of footwear assemblies; wherein a footwear assembly out of said plurality of footwear assemblies is processed according to a sequence of processing steps at a first subset of said one or more footwear manufacturing stations; wherein each of said plurality of footwear assemblies are associated with a respective footwear identification representation; an imaging system configured to provide image data of said footwear assembly; wherein said imaging system is communicatively coupled with a data processor executing a characteristics evaluation model to determine a characteristics evaluation; wherein said characteristics evaluation is associated with a second subset of said one or more footwear manufacturing stations on the basis of said footwear identification representation.

[0314] In an embodiment of the invention, said data processor is a server configured to execute said characteristics evaluation model.The drawings

[0315] Various embodiments of the invention will in the following be described with reference to the drawings where fig. 1 illustrates a footwear manufacturing line implemented to determine the characteristics of manufactured footwear assemblies, figs. 2a-2d illustrate how footwear assemblies are routed between footwear manufacturing stations to realize different sequences of processing steps, fig. 3 illustrates a footwear manufacturing line with multiple branching transportation paths providing routing of footwear assemblies between footwear manufacturing stations and imaging systems, fig. 4 illustrates a footwear manufacturing line with transportation paths providing a return transportation path on which an imaging system is placed, figs. 5a-5c illustrate how image data of a footwear assembly are processed to obtain a characteristics evaluation of the characteristics of the footwear assembly, figs. 6a-6e illustrate different embodiments of footwear manufacturing lines with an imaging system, fig. 7 illustrates the storing of a footwear identification representation according to an embodiment of the invention, fig. 8a illustrates a training method for training a classification model according to an embodiment of the invention, fig. 8b illustrates a classification method of characterizing a footwear assembly according to an embodiment of the invention, fig. 9a-9b illustrate schematic representations of a training system embodiment and a classification system embodiment, respectively, according to the invention,fig. 10 illustrates a schematic representation of a training module according to an embodiment of the invention, fig. 11-12 illustrate various aspects of a neural network classification model according to an embodiment of the invention, and fig. 13 a- 14 illustrate various aspects of a convolutional neural network classification model according to an embodiment of the invention.Detailed description

[0316] Fig. 1 illustrates an embodiment of the invention where a footwear manufacturing line FML implements a method of determining the characteristics of footwear assemblies.

[0317] The footwear manufacturing line FML is shown to have nine footwear manufacturing stations FMS designated footwear manufacturing station 1 FMS1, footwear manufacturing station 2 FMS2, footwear manufacturing station 3 FMS3, footwear manufacturing station 4 FMS4, footwear manufacturing station 5 FMS 5, footwear manufacturing station 6 FMS6, footwear manufacturing station 7 FMS7, footwear manufacturing station 8 FMS8, footwear manufacturing station 5A FMS5A, and footwear manufacturing station 5B FMS5B.

[0318] Each of the nine footwear manufacturing stations FMS are configured to process footwear assemblies FA according to one or more processing steps which are performed on the footwear assemblies FA at one or more of the footwear manufacturing stations FMS as the footwear assemblies FA are conveyed to the footwear manufacturing stations FMS using a conveyor CONV.

[0319] The footwear manufacturing stations FMS are illustrated as comprising equipment that performs the processing of the footwear assemblies FA according to one or more processing steps. This equipment may include footwear manufacturing robots FMR of which several different types are illustrated on fig. 1. A footwear manufacturing station FMS may also comprise equipment in the form of automatic sewing machines ASM used for the processing of footwear assemblies. Footwear manufacturing stations FMS may comprise other equipment or elements which are not illustrated on fig. 1 such as human operators or equipment for direct injection production. Fig. 1 illustrates some example embodiments of footwear manufacturing stations FMS.

[0320] Footwear assemblies FA are illustrated as being routed between the nine footwear manufacturing stations FMS for processing using a conveyor CONV. The direction of movement of the conveyor CONV is indicated by arrows. The indicateddirection illustrates how a footwear assembly FA may be conveyed from one footwear manufacturing station FMS to another such as from footwear manufacturing station 1 FMS1 to footwear manufacturing station 3 FMS3 and further on through the footwear manufacturing line FML.

[0321] The use of a conveyor CONV is merely an example of how footwear assemblies FA may be routed between footwear manufacturing stations FMS. Other embodiments may use for example rail systems or automatic guided vehicles to route the footwear assemblies FA.

[0322] The conveyor CONV illustrates that footwear assemblies FA may be routed to different footwear manufacturing stations FMS according to a sequence of processing steps that are to be performed for each footwear assembly FA. The embodiment on fig. 1 illustrates that a footwear assembly FA may enter the footwear manufacturing line FML on the left side of the fig. 1 and be conveyed along one of two branching paths to either footwear manufacturing station 2 FMS2 or footwear manufacturing station 1 FMS1.

[0323] Fig. 1 illustrates additional branching of the conveyor CONV between footwear manufacturing station 3 FMS3 and footwear manufacturing station 7 FMS7 where a footwear assembly FA may be routed to either footwear manufacturing station 5A FMS5A or footwear manufacturing station 5B FMS5B.

[0324] The embodiment of fig. 1 illustrates an imaging system IS which is configured to provide image data IMGD of a footwear assembly FA when it has been conveyed to the imaging system IS after having been processed according to a sequence of processing steps at a subset of one or more of the nine footwear manufacturing stations FMS. The imaging system IS is shown to provide image data IMGD as an input to a characteristics evaluation model CEM executing on a data processor DP.

[0325] The characteristics evaluation model CEM is shown as receiving an input of image data IMGD which is provided by the imaging system IS. The characteristics evaluation model CEM is illustrated as executing on a data processor DP, which maybe a hardware unit with at least digital storage and processing capabilities. The characteristics evaluation model CEM processes the input image data IMGD and provides as an output a characteristics evaluation CE.

[0326] A server SRV stores footwear identification representations FIR each corresponding to the footwear assemblies FA being processed at the footwear manufacturing line FML. Four footwear identification representations FIR are illustrated on fig. 1 and designated respectively footwear identification representation 1 FIR1, footwear identification representation 2 FIR2, footwear identification representation 3 FIR3, and footwear identification representation 4 FIR4. These correspond respectively to each of the four footwear assemblies FA designated respectively footwear assembly 1 FA1, footwear assembly 2 FA2, footwear assembly 3 FA3, and footwear assembly 4 FA4. Only four footwear identification representations FIR are shown for illustrative purposes, but there may of course be any number of footwear identification representations, each uniquely identifying a particular footwear assembly FA that is processed at the footwear manufacturing line FML.

[0327] The footwear identification representations FIR are illustrated as being stored in a footwear identification representation database FIRDB, which is implemented on a server SRV.

[0328] The server SRV takes as an input the characteristics evaluation CE provided by the characteristics evaluation model CEM. Fig. 1 illustrates an example where the provided characteristics evaluation CE is linked to footwear identification representation 1 FIR1 because this footwear identification representation FIR corresponds to footwear assembly 1 FA1, which is the footwear assembly FA for which image data IMGD has presently been provided because footwear assembly 1 FA1 is located at the imaging system IS.

[0329] Four arrows with dashed lines are illustrated on fig. 1 as originating at footwear identification representation 1 FIR1 and terminating respectively at footwearmanufacturing station 1 FMS1, footwear manufacturing station 3 FMS3, footwear manufacturing station 5A FMS5A, and footwear manufacturing station 7 FMS7.

[0330] The four arrows with dashed lines indicate the step of performing the associating ASSOC between the characteristics evaluation CE and one or more of the footwear manufacturing stations FMS at which the footwear assembly FA has been processed. In the embodiment of fig. 1, the characteristics evaluation CE has been determined by the characteristics evaluation model CEM based on image data IMGD of footwear assembly 1 FA1, and therefore the associating step is based on footwear identification representation 1 FIR1 and the characteristics evaluation CE.

[0331] The four arrows with dashed lines indicate that four footwear manufacturing stations FMS have been associated with the characteristics evaluation CE. In some embodiments of the invention, associating ASSOC may indicate that footwear assembly 1 FA1 was processed at those four footwear manufacturing stations FMS, i.e. footwear manufacturing station 1 FMS1, footwear manufacturing station 3 FMS3, footwear manufacturing station 5A FMS5A, and footwear manufacturing station 7. In other embodiments of the invention, the association indicates that one or more characteristics of the footwear assembly 1 FA1 as quantified by the characteristics evaluation CE are associated with the four footwear manufacturing stations FMS as indicated by the arrows with dashed lines linking footwear identification representation 1 FIR1 and the four footwear manufacturing stations FMS.

[0332] Fig. 1 illustrates four footwear assemblies FA located at various points on the footwear manufacturing line FML. The four footwear assemblies FA exemplify some of the different steps which are implemented for the manufacture of footwear assemblies FA and which permit the determination of a characteristics evaluation CE that can be associated with one or more of the footwear manufacturing stations FMS based on the footwear identification representation FIR.

[0333] Fig. 1 illustrates that footwear assembly 1 FA1 has undergone processing at several footwear manufacturing stations FMS and has presently been conveyed to the imaging system IS. Footwear assembly 1 FA1 may in an embodiment be assumed tohave been processed sequentially at footwear manufacturing station 1 FMS1, footwear manufacturing station 3 FMS3, footwear manufacturing station 5A FMS5A, and footwear manufacturing station 7 FMS7.

[0334] The subset of footwear manufacturing stations FMS at which footwear assembly 1 FA1 has been processed may be referred to as a first subset of footwear manufacturing stations FMS. The first subset in this example may include the four footwear manufacturing stations FMS identified in the preceding paragraph.

[0335] Each of the four footwear manufacturing stations FMS may have performed one or more processing steps on footwear assembly 1 FA1 according to a sequence of processing steps. This sequence of processing steps may in some embodiments have been defined before processing of footwear assembly 1 FA1 was initiated, or the sequence of processing steps may in some embodiments have been dynamically defined as footwear assembly 1 FA1 was conveyed from one footwear manufacturing station FMS to another.

[0336] The sequence of processing steps may in some embodiments be defined in the corresponding footwear identification representation FIR i.e. in the case of footwear assembly 1 FA1 the corresponding footwear identification representation 1 FIR1. Each of the four footwear manufacturing stations at which footwear assembly 1 FA1 was processed may have been configured to perform different processing steps according to the sequence of processing steps to be performed on footwear assembly 1 FA1.

[0337] In the present example, footwear manufacturing station 1 FMS1 may have been configured to perform a processing step wherein footwear manufacturing robot 1 FMRI picked up one or more footwear parts FP from a loading conveyor LCONV and placed them onto a surface of the footwear manufacturing station. Footwear manufacturing station 3 FMS3 may have been configured to subsequently perform another processing step on footwear assembly 1 FA1 such as a processing step involving footwear manufacturing robot 3 FMR3 cutting certain parts of footwear assembly 1 FA1 to obtain a desired shape. Subsequently, footwear assembly 1 FA1may have been conveyed to footwear manufacturing station 5A FMS5A where an automatic sewing machine 5A ASM5A may performed a processing step involving the application of stitches to footwear assembly 1 FA1. Finally, footwear assembly 1 FA1 may have been conveyed to footwear manufacturing station 7 FMS7 where a processing step such as lasting of a finished upper onto a last may have been performed.

[0338] The processing steps indicated in the preceding paragraphs are merely exemplary of the ways in which a footwear assembly FA may be processed. Footwear assembly 1 FA1 could have been processed according to different processing steps such as shaping, molding, or roughing. As is indicated herein, the different processing steps modify the footwear assembly FA such that it attains the characteristics that are required of it as an input to manufacturing a finished article of footwear.

[0339] Fig. 1 illustrates that footwear assembly 1 FA1 is positioned at the imaging system IS such that the imaging system IS can record image data of footwear assembly 1 FA1. The imaging system IS of fig. 1 is positioned such that it is at the end of the sequence of processing steps according to which footwear assembly 1 FA1 has been processed. I.e. the imaging system IS is shown to be located after footwear manufacturing station 7 FMS7 which is the last footwear manufacturing station FMS at which processing steps were performed on footwear assembly 1 FA1.

[0340] The imaging system IS of fig. 1 is therefore configured to provide an output of image data IMGD of footwear assemblies that have undergone all processing steps in their defined sequence of processing steps.

[0341] In alternative embodiments that are variations of the one illustrated on fig. 1, the imaging system IS may be located at different points on the footwear manufacturing line such as before footwear manufacturing station 1 FMS1 or between footwear manufacturing station 4 FMS4 and footwear manufacturing station 6 FMS6.

[0342] Placing the imaging system IS at other points on the footwear manufacturing line FML may allow for the image data IMGD provided by the imaging system IS to be used in quantifying other characteristics of the footwear assembly FA than wouldbe possible when the imaging system IS is placed at the end of the footwear manufacturing line FML.

[0343] Footwear assembly 1 FA1 is shown on fig. 1 to be in a position such that the imaging system IS can capture image data IMGD of the footwear assembly 1 FA1. The image data IMGD of footwear assembly 1 FA1 are provided to the characteristics evaluation model CEM which is implemented on a data processor DP.

[0344] The characteristics evaluation model CEM processes the image data IMGD and quantifies characteristics of footwear assembly 1 FA1 as an output in the form of a characteristics evaluation CE. The characteristics evaluation CE provides quantifiable measures of different characteristics of footwear assembly 1 FA1 such as the color, shape, absence or presence of parts, and possibly an indication of one or more defects.

[0345] The characteristics evaluation CE may also contain a mapping between the quantified characteristics and specific processing steps.

[0346] The characteristics evaluation CE is shown on fig. 1 as being provided to the server SRV on which the footwear identification representation database FIRDB is implemented. The characteristics evaluation CE is then matched against the footwear identification representation 1 FIR1 to which it corresponds. If footwear identification representation 1 FIR1 contains a record of the sequence of processing steps according to which footwear assembly 1 FA1 has been processed, then the characteristics evaluation may be readily matched against this record and associated with the footwear manufacturing stations which correspond to the particular processing steps.

[0347] As indicated earlier, it may be assumed that footwear assembly 1 FA1 was processed at footwear manufacturing station 1 FMS1, footwear manufacturing station 3 FMS3, footwear manufacturing station 5 A FMS5A, and footwear manufacturing station 7 FMS7. Therefore, the characteristics evaluation may be associated with one or more of these footwear manufacturing stations FMS indicating the relationship between the quantified characteristics and the footwear manufacturing stations at which the processing steps related to those characteristics were performed.

[0348] Fig. 1 indicates that the characteristics evaluation has been associated with each of the four footwear manufacturing stations at which footwear assembly 1 FA1 was processed. However, it may also be the case that fewer or more footwear manufacturing stations are identified as being associated with the characteristics evaluation CE.

[0349] As a second subset of footwear manufacturing stations have now been associated with the characteristics evaluation CE based on the footwear identification representation, a link between the processing steps, footwear manufacturing stations and characteristics of footwear assembly 1 FA1 has now been established. The information thus acquired may for example be used to make decisions about how to optimize the manufacture of subsequent footwear assemblies or which steps may be required to improve certain characteristics of manufactured footwear assemblies. It may also be necessary to take a particular footwear manufacturing station offline if it is found to cause manufactured footwear assemblies to exhibit defects as quantified by the characteristics evaluation CE.

[0350] The association between the characteristics evaluation and the second subset of associated footwear manufacturing stations can also be used to reroute footwear assemblies FA to other footwear manufacturing stations FMS, for example if a particular footwear manufacturing station FMS associated with one or more characteristics as quantified by the characteristics evaluation is found to cause the manufactured footwear assembly to not live up to quality standards.

[0351] Fig. 1 illustrates a footwear assembly 2 FA2 which has been routed along a different route from footwear assembly 1 FA1. When the characteristics evaluation CE determined on the basis of image data IMGD of footwear assembly 1 FA1 has been associated with one or more footwear manufacturing stations FMS, footwear assembly 1 FA1 is conveyed away from the imaging system IS along the conveyor path indicated on the right side of fig. 1. This allows for the subsequently manufactured footwear assembly 2 FA2 to be conveyed to the imaging system IS and for image data IMGD of this footwear assembly to be provided to the characteristics evaluation model CEM.

[0352] It must be noted that footwear assembly 2 FA2 may be completely different from footwear assembly 1 FA1 i.e. it may comprise different footwear parts FP, and it may have been subject to a different sequence of processing steps.

[0353] In the present example, it may be assumed that footwear assembly 2 FA2 has been processed sequentially at the first subset of footwear manufacturing stations FMS which includes footwear manufacturing station 2 FMS2, footwear manufacturing station 4 FMS4, footwear manufacturing station 6 FMS6, and footwear manufacturing station 8 FMS8.

[0354] A characteristics evaluation which is determined based on image data IMGD of footwear assembly 2 FA2 may quantify different characteristics to those of footwear assembly 1 FA1 reflecting the difference in processing between footwear assembly 2 FA2 and footwear assembly 1 FA1.

[0355] Fig. 1 illustrates a footwear assembly 3 FA3 which has not yet been processed according to all of the processing steps in the sequence of processing steps defined for footwear assembly 3 FA3. In the present example, it may be assumed that footwear assembly 3 FA3 has been processed according to a sequence of processing steps at footwear manufacturing station 1 FMS1, footwear manufacturing station 3 FMS3, and at footwear manufacturing station 5B FMS5B. Footwear assembly 3 FA3 is illustrated on fig. 1 as being conveyed between footwear manufacturing station 5B FMS5B and footwear manufacturing station 7 FMS7.

[0356] In the embodiment illustrated on fig. 1, it may be assumed that footwear manufacturing station 5A FMS5A and footwear manufacturing station 5B FMS5B perform the same processing steps. Each of the two footwear manufacturing stations comprise an automatic sewing machine which may be used to perform identical one or more processing steps involving stitching. The footwear manufacturing stations 5 A and 5B are therefore redundant footwear manufacturing stations in that they perform the same processing steps as each other. Footwear assemblies FA may therefore be routed between the footwear manufacturing stations 5A and 5B based on acharacteristics evaluation CE associated with either of the footwear manufacturing stations 5 A or 5B.

[0357] In an example with reference to fig. 1, it may be the case that the characteristics evaluation CE obtained for footwear assembly 1 FA1 has been associated with footwear manufacturing station 5A FMS5A and indicates that footwear manufacturing station 5A FMS5A is the cause of a defect on footwear assembly 1 FA1. In this example, it may be advantageous to effect a rerouting of footwear assembly 3 FA3 from footwear manufacturing station 5A to footwear manufacturing station 5B FMS5B such that footwear assembly 3 FA3 may be manufactured without the defect as quantified by the characteristics evaluation corresponding to footwear assembly 1 FA1.

[0358] Fig. 1 further illustrates a footwear assembly 4 FA4 which is located at footwear manufacturing station 4 FMS4 and is currently being processed by a footwear manufacturing robot 4 FMR4. In the present example, it may be assumed that footwear assembly 4 FA4 has already been processed at footwear manufacturing station 2 FMS2 and is yet to be processed at footwear manufacturing station 6 FMS6 and footwear manufacturing station 8 FMS8. Footwear manufacturing station 6 FMS6 is illustrated as comprising both a footwear manufacturing robot 6 FMR6 and an automatic sewing machine 6 ASM6. Footwear manufacturing station 6 FMS6 is consequently able to perform multiple different processing steps to a footwear assembly FA which may involve one or more processing steps performed by the footwear manufacturing robot 6 FMR6 and one or more different processing steps performed by the automatic sewing machine 6 ASM6.

[0359] Fig. 1 illustrates four footwear identification representations FIR being stored in a footwear identification representation database FIRDB on a server SRV. As footwear assemblies FA are being conveyed from one footwear manufacturing station FMS to another, the corresponding footwear identification representations FIR may be updated such that they reflect the current state of the footwear assemblies FA.

[0360] As an example, as footwear assembly 1 FA1 is processed at a first subset of the nine footwear manufacturing stations FMS illustrated on fig. 1, the corresponding footwear identification representation 1 FIR1 may be sequentially updated to reflect e.g. which processing steps have been performed on the footwear assembly FA, and which footwear manufacturing stations FMS the footwear assembly has been processed at.

[0361] Tracking the status and identity of each of the footwear assemblies FA illustrated on fig. 1 may be achieved in various ways. In the embodiment on fig. 1, several footwear identification readers FIRD are illustrated. These may be placed at any point on the footwear manufacturing line FML such as at or between footwear manufacturing stations FMS. The footwear identification reader FIRD reads a code from the footwear assembly FA using a technology such as a bar code or radiofrequency identity. The code attached to the footwear assembly FA may correspond to a footwear identification representation FIR whereby the link between the physical location of the footwear assembly FA and its corresponding footwear identification representation FIR has been established. The information obtained from the code may be transmitted to the server SRV and be used to update the corresponding footwear identification representation FIR.

[0362] Fig. 1 illustrates only a single footwear identification reader FIRD but other embodiments may employ a plurality of footwear identification readers FIRD placed at different points on the footwear manufacturing line FML.

[0363] Other embodiments may employ a footwear manufacturing line controller which centrally keeps track of the position and status of each of the footwear assemblies FA and continuously updates them thus obviating the need for footwear identification readers FIRD.

[0364] A footwear assembly carrier such as a last or a tray on which the footwear assembly is placed may also be used to track and update the footwear identification representation such that the need for footwear identification readers FIRD is obviated.

[0365] While fig. 1 illustrates a footwear manufacturing line FML that may be interpreted as being substantially inline with some branching, a variation of the embodiment of fig. 1 can as well be implemented without the use a conveyor CONV or with a more dynamic routing system wherein for example the footwear manufacturing stations FMS are connected to the rest of the footwear manufacturing line FML via additional transportation paths.

[0366] In an embodiment with reference to fig. 1, the characteristics evaluation model CEM may implement at least a template matching algorithm by which image data IMGD supplied by the imaging system IS are processed. The template matching algorithm compares pixel values of the input image data IMGD with a reference template according to which footwear characteristics may be quantified i.e. as similarity values. A characteristics evaluation model CEM implementing a template matching algorithm provides a characteristics evaluation that quantifies, i.e. according to similarity values, characteristics of the footwear assembly.

[0367] Figs. 2a, 2b, 2c, and 2d illustrate schematically how footwear assemblies FA may be processed according to different sequences SEQ of processing steps PS (not shown) at different footwear manufacturing stations FMS and how some footwear manufacturing stations FMS are redundant in that they perform processing steps that are identical to those of other footwear manufacturing stations FMS. Footwear assemblies FA are thus routed based on for example their footwear identification representation and each footwear assembly may therefore arrive at the imaging system IS after having visited its own unique first subset of footwear manufacturing stations.

[0368] Fig. 2a illustrates schematically how a footwear assembly is routed between footwear manufacturing stations FMS. Eleven footwear manufacturing stations FMS are illustrated, and each footwear manufacturing station FMS is given an identifier in the form of a number and a letter such that the twelve footwear manufacturing stations may be identified as 1A, IB, 1C, ID, 2, 3A, 3B, 3C, 3D, 4A, 4B respectively. The number indicates unique processing steps, whereas the letter uniquely identifies each of the redundant footwear manufacturing stations FMS that perform those one or more particular processing steps.

[0369] Footwear manufacturing stations 1A, IB, 1C, and ID thus all perform the same processing steps on footwear assemblies FA that are processed at any one of those four footwear manufacturing stations. Footwear manufacturing stations 1 A, IB, 1C, and ID are therefore redundant footwear manufacturing stations in that they each perform the one or more processing steps designated 1.

[0370] Footwear manufacturing station 2 is the only footwear manufacturing station FMS that performs processing steps 2. Footwear manufacturing station 2 is therefore not a redundant footwear manufacturing station FMS because there are no other footwear manufacturing stations FMS that can perform processing step 2. Any footwear assembly FA that must be processed according to processing step 2 therefore has to be processed at footwear manufacturing station 2.

[0371] Four footwear manufacturing stations identified as 3A, 3B, 3C, and 3D each perform processing steps 3 and are consequently also each redundant footwear manufacturing stations FMS.

[0372] Fig. 2a illustrates a particular routing of a footwear assembly FA through the footwear manufacturing stations FMS. The footwear manufacturing stations FMS at which the footwear assembly FA is processed are indicated with arrows going from one footwear manufacturing station FMS to the next.

[0373] The footwear assembly FA is thus conveyed first to footwear manufacturing station 1A where it is processed according to processing steps 1. Subsequently, the footwear assembly FA is transported to footwear manufacturing station 2 where it is processed according to processing step 2. The footwear assembly FA is then routed to footwear manufacturing station 3A which is one of four redundant footwear manufacturing stations 3A, 3B, 3C, and 3D. The footwear assembly is then processed according to processing step 3. A last processing step is performed at footwear manufacturing station 4A to which the footwear assembly FA is transported as indicated by the arrow from footwear manufacturing station 3A to footwear manufacturing station 4A.

[0374] The footwear assembly FA illustrated on fig. 2a is thus processed according to a sequence of processing steps in the order of processing step 1, processing step 2, processing step 3, and processing step 4. The footwear assembly FA is, after processing, conveyed to an imaging system IS which provides image data IMGD (not shown) for quantifying the characteristics of the footwear assembly FA.

[0375] Fig. 2a illustrates that some footwear manufacturing stations may not be redundant i.e. footwear manufacturing station 2 in the case of fig. 2a. In some cases, having redundant footwear manufacturing stations may not be technically feasible or desirable due to factors such as available space on the factory floor or the general setup of the footwear manufacturing line. Footwear manufacturing station 2 may for example be an automated station with a multiple of stations i.e. a round table or carrousel type footwear production machine where processing steps such as direct injection production may be performed.

[0376] Fig. 2b illustrates a footwear manufacturing line FML which is identical to that of fig. 2a i.e. the same footwear manufacturing stations 1A, IB, 1C, ID, 2, 3A, 3B, 3C, 3D, 4A, and 4B performing the same processing steps 1, 2, 3, and 4 are shown. In this example, a different footwear assembly FA is shown being routed between the footwear manufacturing stations FMS. The footwear assembly FA is initially transported to footwear manufacturing station 1 A where it is processed according to processing steps 1. Then it is transported to footwear manufacturing station 2 for processing according to processing steps 2 and onto footwear manufacturing station 3B for processing according to processing steps 3. Finally, the footwear assembly FA is processed at footwear manufacturing station 4A according to processing step 4 before it is routed to the imaging system IS where the process of evaluation of its characteristics and association to one or more footwear manufacturing stations FMS may be initiated.

[0377] The sequence of processing steps performed on the footwear assemblies FA illustrated on fig. 2a and 2b are the same, i.e. the sequence of processing steps is given by processing steps 1 followed by processing steps 2 followed by processing steps 3 followed by processing steps 4. An important difference between fig. 2a and 2b is thatfig. 2b indicates that the processing steps 3 are performed at footwear manufacturing station 3B instead of 3A. The routing of the footwear assembly therefore takes advantage of the presence of redundant footwear manufacturing stations FMS that are available for performing processing steps 3.

[0378] The footwear assemblies FA illustrated on figs. 2a and 2b may be of the same type i.e. both footwear assemblies FA may constitute an input to manufacturing the same footwear design or size. The benefit of having redundant footwear manufacturing stations is that further processing of the image data IMGD (not shown) obtained of the footwear assembly FA at the imaging system IS makes it possible to make decisions about using a particular redundant footwear manufacturing station FMS on the basis of a characteristics evaluation determined on the basis of the image data IMGD (not shown).

[0379] A characteristics evaluation determined on the basis of image data IMGD (not shown) obtained from imaging system of fig. 2a may for example indicate that certain characteristics of the footwear assembly do not live up to established quality standards. Associating the characteristics evaluation with one or more of the footwear manufacturing stations 1A, 2, 3A, and 4A based on the corresponding footwear identification representation may aid in making decisions about rerouting footwear assemblies FA that are to be produced subsequently.

[0380] Fig. 2b may therefore illustrate a situation where footwear assemblies FA have been rerouted as a response to a characteristics evaluation determined based on image data obtained from the footwear assembly FA illustrated on fig. 2a. In the present example, the processing steps 3 performed at footwear manufacturing station 3 A may have caused characteristics in the footwear assembly illustrated on fig. 2a to not live up to quality standards, and these unacceptable characteristics have been identified from the characteristics evaluation output by the characteristics evaluation model. The indicated unsatisfactory characteristics have then been associated with footwear manufacturing station 3A by utilizing the corresponding footwear identification representation. To avoid the same problem of quality issues in subsequently manufactured footwear assemblies, the subsequent footwear assemblyFA of fig. 2b has been rerouted from footwear manufacturing station 3A to 3B whereby issues related to certain characteristics of the manufactured footwear assemblies FA may be remedied.

[0381] Footwear manufacturing station 3A may for example perform a processing step PS (not shown) whereby two leather pieces comprising the footwear assembly FA are stitched together. But this processing step PS (not shown) may be performed in an unsatisfactory manner e.g. the stitching is not performed in the correct location, the stitching is staggered, or stitches are skipped. Footwear manufacturing station 3B may perform the same processing step but without any issues. A rerouting of subsequent footwear manufacturing stations based on identifying this problem may therefore be advantageous as illustrated on fig. 2b.

[0382] Fig. 2c illustrates a footwear manufacturing line FML which is identical to that of figs. 2a and 2b i.e. the footwear manufacturing line FML comprises the same footwear manufacturing stations 1A, IB, 1C, ID, 2, 3A, 3B, 3C, 3D, 4A, and 4B performing individually the processing steps 1, 2, 3, or 4.

[0383] Fig. 2c illustrates that different sequences of processing steps are possible for different footwear assemblies FA processed at the same footwear manufacturing line FML by simply rerouting the footwear assemblies FA according to the sequence SEQ of processing steps as indicated in e.g. the corresponding footwear identification representation FIR (not shown).

[0384] With reference to fig. 2c, the footwear assembly FA is first transported to footwear manufacturing station 1 A where it is processed according to processing steps 1. The footwear assembly FA is then routed to footwear manufacturing station 3B where it is processed according to processing steps 3. The footwear assembly FA thus skips footwear manufacturing station 2 and is therefore not processed according to processing steps 2. Finally, the footwear assembly FA is transported to footwear manufacturing station 4A where it is processed according to processing steps 4 after which it is finally transported to the imaging system IS where the process of determining the characteristics of the footwear assembly FA may be initiated.

[0385] The sequence SEQ of processing steps performed on the footwear assembly FA illustrated on fig. 2c is therefore process steps 1 followed by process steps 3 followed by process steps 4. The requirements of the footwear assembly FA illustrated on fig. 2c may be such that processing according to processing step 2 is not required such that the footwear assembly FA is not routed to footwear manufacturing station 2 for processing according to processing steps 2.

[0386] The footwear assembly FA illustrated on fig. 2c may be different to the footwear assembly FA illustrated on either fig. 2a or 2b, wherefore the required processing steps are different. Process steps 2 may for example comprise a spray application of a particular graphic design onto a part of an upper. The spray application processing steps may only be required for some types of footwear assemblies i.e. it is not essential to obtain a finished article of footwear and thus only relevant for particular footwear designs. It may therefore be skipped in certain instances such as illustrated on fig. 2c.

[0387] Fig. 2d illustrates a footwear manufacturing line FML with the same number of footwear manufacturing stations FMS as on figs. 2a, 2b, and 2c. However, the processing steps performed by the footwear manufacturing stations illustrated on fig. 2d may not necessarily be the same as those depicted on figs. 2a, 2b, and 2c.

[0388] Fig. 2d illustrates the manufacture of a footwear assembly FA according to a sequence of processing steps that is different from the sequences SEQ of processing steps that are illustrated on figs. 2a, 2b, and 2c.

[0389] With reference to fig. 2d, the footwear assembly FA is first transported to footwear manufacturing station 3D where it is processed according to processing steps 3. The footwear assembly FA is then routed to footwear manufacturing station 1C where it is processed according to processing steps 1. Subsequent processing takes place first at footwear manufacturing station 2 according to processing steps 2 and finally at footwear manufacturing station 4A according to processing steps 4.

[0390] In an example, the processing steps 1, 2, 3, and 4 performed by the footwear manufacturing stations FMS of fig. 2d may be identical to the processing steps 1, 2, 3,and 4 performed by footwear manufacturing stations FMS of figs. 2a, 2b, and 2c. In such an example, fig. 2d illustrates that the order of processing steps may be different for different types of footwear assemblies that are processed on the same footwear manufacturing line FML. The different sequence SEQ of processing steps is simply obtained by changing the routing of the footwear assembly FA and thus the footwear manufacturing stations FMS at which the footwear assembly FA is processed. Accordingly, the footwear assembly FA of fig. 2d is processed according to a sequence SEQ of processing steps that is processing steps 3 followed by processing steps 1 followed by processing steps 2 followed by processing steps 4 after which the footwear assembly FA is transported to the imaging system IS.

[0391] In an example, the processing steps 1, 2, 3, and 4 performed by footwear manufacturing stations FMS of fig. 2d may be different from the processing steps 1, 2, 3, and 4 illustrated on figs. 2a, 2b, and 2c. In such an example, fig. 2d illustrates a footwear manufacturing line where a footwear assembly FA is manufactured according to a sequence of processing steps where the processing steps may be different from those performed by the footwear manufacturing stations FMS illustrated on figs. 2a, 2b, and 2c.

[0392] Fig. 3 illustrates an embodiment of the invention. The illustrated embodiment shows a footwear manufacturing line FML having multiple transportation paths TP, footwear manufacturing stations FMS, and imaging systems IS. The left side of fig. 3 indicates that a transportation path TP is branched into two transportation paths Bl and B2 which are then respectively branched into three further transportation paths given by BIA, BIB, BIC and B2A, B2B, B2C respectively.

[0393] Imaging systems IS are placed on multiple transportation paths at different points on the footwear manufacturing line FML. A footwear assembly FA may be routed through the footwear manufacturing line FML to any of the different footwear manufacturing systems FMS which are located on the different transportation paths TP.

[0394] The footwear manufacturing line FML comprises three imaging systems IS. An imaging system IS is placed at the end of transportation path Bl such that any footwear assembly FA which is conveyed onto either of the transportation paths BIA, BIB, or BIC may be further transported to the imaging system IS such that image data IMGD (not shown) can be provided and the characteristics of the manufactured footwear assemblies FA may be evaluated and associated with one or more of the footwear manufacturing stations FMS at which the footwear assembly FA was processed.

[0395] A different imaging system IS is located on transportation path B2B. This imaging system IS makes it possible to provide image data of footwear assemblies FA that are transported from transportation path B2 onto transportation path B2B. The imaging system IS is located after a footwear manufacturing station FMS on transportation path B2B such that a footwear assembly FA on transportation path B2B may be processed at this footwear manufacturing station FMS before it is transported to the imaging system IS, where image data IMGD (not shown) of the footwear assembly FA may be provided for subsequent processing.

[0396] Fig. 4 illustrates an embodiment of the invention. The illustrated embodiment shows a footwear manufacturing line FML with multiple transportation paths TP, footwear manufacturing stations FMS and imaging systems IS. The transportation paths TP are branched such that footwear assemblies FA may be routed onto different transportation paths and onto different footwear manufacturing stations FMS to realize different sequences of processing steps that may be required for manufacturing different types of footwear assemblies FA.

[0397] Fig. 4 illustrates a return transportation path RTP whereby footwear assemblies FA may be transported in a loop and return to footwear manufacturing stations FMS at which they have been processed.

[0398] The different imaging systems IS may be used to provide image data of different types of footwear assemblies FA or footwear assemblies that have beenprocessed at different footwear manufacturing stations FMS and / or according to different sequences of processing steps.

[0399] An imaging system IS is illustrated as being placed on a bypass path BPP by means of which a footwear assembly FA may be sent to bypass the footwear manufacturing stations FMS. This makes it possible to provide image data IMGD (not shown) of footwear assemblies FA before they are processed at any of the footwear manufacturing stations FMS of the footwear manufacturing line FML.

[0400] Fig. 5a illustrates a footwear assembly FA upon which different characteristics C have been indicated. The characteristics C are indicated as being related to different parts of the footwear assembly such as the upper or the outsole.

[0401] Characteristic Cl may in one embodiment relate to the color of the footwear assembly. Characteristic Cl may in another embodiment relate to the surface characteristics of the upper such as the roughing or the location of a graphic on the upper.

[0402] Characteristic C2 relates to the outsole and may in different embodiments be defined as relating to some dimension of the outsole such as its height, width, or location with respect to the lower part of the upper of the footwear.

[0403] Characteristics C3, C4, and C5 are all characteristics which in various embodiments may be characterized as defects. Characteristic C3 may in various embodiments indicate scuff marks on the toe part of the outer material of the upper part of the footwear assembly FA.

[0404] Characteristic C4 may indicate a defect related to the outsole. In some embodiments, characteristic C4 may indicate that the dimensions of the outsole deviate in some way from the desired dimensions in a particular part of the outsole.

[0405] Characteristic C5 indicates a defect related to the outer material of the upper in a particular area. In a particular embodiment, characteristic C5 may a hole in a part of the upper, specifically in the outer material of the upper.

[0406] Fig. 5b illustrates an embodiment of the invention wherein image data IMGD generated by an imaging system IS of the footwear assembly illustrated on fig. 5a have been generated. The image data IS are illustrated as a two-dimensional digital image in the present embodiment but could be any other form of image data IMGD of a footwear assembly FA.

[0407] Several characteristics of the footwear assembly are visible on the image data IMGD. These characteristics may simply be identified visually by for example an operator on the footwear manufacturing line FML. However, they may also be formalized and identified automatically by processing the image data IMGD in a characteristics evaluation model. The characteristics evaluation model may implement any form of image processing algorithm for the processing of digital images.

[0408] Fig. 5c illustrates an embodiment of the invention wherein the image data IMGD of fig. 5b have been processed. In the present embodiment, the characteristics have now been quantified and represented as characteristics measures CM which represent corresponding characteristics of the footwear assembly FA.

[0409] Characteristics measure CM1 may for example quantify the color of the footwear according to a digital representation of color or it may quantify a roughness measure of the outer material of the upper. Characteristics measure CM1 may thus in some embodiments correspond to characteristic 1 CM1 of fig. 5a.

[0410] Characteristics measure CM2 quantifies some aspect related to the outsole such as its width, height, or thickness. In some embodiments, characteristics measure CM2 equivalent to a digital quantification of characteristic 1 CM1 as illustrated on fig. 5a.

[0411] Characteristics measure 3 CM3 indicates the presence of a defect on the footwear assembly FA. Characteristics measure 3 CM3 may in some embodiments classify particular pixels of the image data IMGD illustrated on fig. 5b as being scuff marks or imperfections on the outer material of the upper at the toe end of the upper. In some embodiments, characteristics measure 3 CM3 is therefore a quantification of characteristic 3 C3 as indicated on the footwear assembly illustrated on fig. 5a.

[0412] Characteristics measure 4 CM4 illustrated on fig. 5c is a quantification of a defect related to the outsole of the footwear assembly illustrated on fig. 5a. Characteristics measure 4 CM may for example quantify the shape of the outsole in a particular which indicates a defect i.e. when the height, thickness, or width of the outsole is not as desired. In some embodiments, characteristics measure CM4 is therefore a quantification of characteristic C4 which has been indicated on the footwear assembly FA illustrated on fig. 5a.

[0413] Characteristics measure 5 CM5 illustrated on fig. 5c is a quantification of a defect related to outer material of the upper of the footwear assembly illustrated on fig. 5a with corresponding image data IMGD indicated on fig. 5b. Characteristics measure 5 CM5 may in some embodiments quantify the location of a hole in the outer material of the upper. In some embodiments, characteristics measure 5 CM5 is therefore a quantification of characteristic C5 which has been indicated on the footwear assembly FA illustrated on fig. 5a.

[0414] Fig. 6a illustrates an embodiment of the invention wherein a plurality of footwear assemblies FA are manufactured sequentially on a footwear manufacturing line FML. Four footwear assemblies FA are illustrated. Two footwear assemblies FA have not yet been processed at the footwear manufacturing station FMS according to one or more processing steps whereas two footwear assemblies FA have already been processed at the footwear manufacturing station and are thus positioned downstream from it in the direction of the arrows.

[0415] The illustrated footwear manufacturing station FMS is merely shown schematic, but it may of course comprise some of the parts as illustrated on fig. 1 e.g. a footwear manufacturing robot FMS or an automatic sewing machine ASM.

[0416] A conveyor CONV is shown to move the footwear assemblies from left to right on fig. 6a, i.e. the footwear assemblies FA are initially conveyed to the footwear manufacturing station at which they are processed and subsequently they are conveyed to the imaging system IS where image data are generated and provided as an input to the characteristics evaluation model CEM.

[0417] A footwear assembly FA is shown as currently being at the imaging system IS. The footwear assembly is positioned in the proximity of the imaging system IS, which is therefore able to provide image data IS of it to establish a characteristics evaluation CE and subsequent association with one or more footwear manufacturing stations FMS of which the footwear manufacturing station FMS illustrated on fig. 6a may be one.

[0418] The footwear assembly FA illustrated on the right side of fig. 6a has been conveyed past both the footwear manufacturing station FMS and the imaging system IS, and it may be conveyed further along the footwear manufacturing line FML to other footwear manufacturing stations FMS not illustrated on fig. 6a.

[0419] Fig. 6b illustrates a footwear manufacturing line FML seen from a bird’s eye view wherein four footwear manufacturing stations 1, 2, 3, and 4 FMS1, FMS2, FMS3, FMS4 are illustrated.

[0420] Footwear assemblies FA are conveyed from one footwear manufacturing station FMS to another, i.e. from footwear manufacturing station 1 FMS1 to footwear manufacturing station 2 FMS2 to footwear manufacturing station FMS3 and on to footwear manufacturing station 4 FMS4 by the use of a conveyor CONV. The imaging system IS is therefore shown to be located after a plurality of footwear manufacturing stations FMS such that footwear assemblies that are conveyed to the imaging system IS may have already been processed at one or more of the footwear manufacturing stations 1, 2, 3, and / or 4 FMS1, FMS2, FMS3, FMS4.

[0421] The footwear assemblies FA may at each of the footwear manufacturing stations FMS illustrated on fig. 6b be processed according to one or more processing steps. Placing the imaging system IS after these processing steps means that the imaging system IS will be able to provide image data IMGD of the footwear assembly FA from which one or more characteristics of the footwear assembly FA are quantifiable in a characteristics evaluation CE (not shown) obtained via processing of the image data IMGD by a characteristics evaluation model CEM (not shown).

[0422] Some characteristics of footwear assemblies FA may only be quantifiable after the footwear assembly FA has been processed according to specific processing steps at one or more footwear manufacturing stations FMS such as at one or more of the four footwear manufacturing stations FMS illustrated on fig. 6b.

[0423] Fig. 6c illustrates a footwear manufacturing line FML from a bird’s eye view wherein there are four footwear manufacturing stations 1, 2, 3, and 4 FMS1, FMS2, FMS3, FMS4 and wherein the imaging system IS has been placed after footwear manufacturing station 2 FMS2 but before footwear manufacturing station 3 FMS3 such that footwear assemblies FA which have been processed at footwear manufacturing station 1 FA1 and / or footwear manufacturing station 2 FA2 may then be transported to the imaging system where image data IMGD of the footwear assembly FA may be obtained.

[0424] The imaging system IS can be placed between footwear manufacturing stations FMS on the footwear manufacturing line FML as illustrated on fig. 6c if it is required to quantify certain characteristics of the footwear assembly FA. Some characteristics may only be quantifiable based on image data IMGD when the footwear assembly has been processed according to a specific number of processing steps, i.e. if the footwear assembly FA is subsequently processed before image data IMGD are acquired, then it may not be possible to quantify relevant characteristics in a characteristics evaluation CE (not shown).

[0425] Different processing steps may make the quantification of certain characteristics of the footwear assembly FA impossible. A processing step whereby an outsole is adhered to an upper may for example obscure the view of the insole as seen from the outside of the upper and image data IMGD generated after the outsole has been adhered will therefore not be usable for quantifying characteristics related to e.g. the strobel stitching between the insole and the lower part of the outer material of an upper. Certain embodiments such as the one illustrated on fig. 6c are therefore advantageous in that the imaging system IS is placed such that it captures image data IMGD (not shown) at a specific point in the sequence SEQ of processing steps (not shown) according to which the footwear assembly FA is to be processed.

[0426] Fig. 6d shows an embodiment wherein a footwear manufacturing line FML is shown to have multiple branching paths and a loop which connects both the footwear manufacturing stations 1, 2, 3, and 4 FMS1, FMS2, FMS3, FMS4 and the imaging system IS. The imaging system IS is illustrated as being located between footwear manufacturing station 3 FMS3 and footwear manufacturing station 4 FMS4.

[0427] The footwear assemblies FA illustrated on fig. 6d are indicated as being able to be routed between different footwear manufacturing stations such that the sequence of processing steps according to which each footwear assembly is processed may be subject to change.

[0428] A footwear assembly FA may for example according to an embodiment be processed according to one or more processing steps at footwear manufacturing station 1 FMS1 and then be processed at footwear manufacturing station 2 FMS2 subsequent to which it is processed at footwear manufacturing station 4 FMS4. The footwear assembly may then be transported to the imaging system IS such that image data IMGD can be provided. The imaging system IS of fig. 6d is therefore able to provide image data IMGD of footwear assemblies FA which have been processed according to different processing steps at different footwear manufacturing stations FMS.

[0429] An imaging system IS according to fig. 6d may thus be used to provide image data IMGD from which different unique characteristics of footwear assemblies FA can be identified. The imaging system IS may also be used to provide image data IMGD of footwear assemblies FA of different types.

[0430] Fig. 6e illustrates an embodiment in which a footwear manufacturing line FML is shown to comprise six footwear manufacturing stations FMS identified respectively as footwear manufacturing station 1, 2, 3, 4, 5, and 6. Four footwear assemblies FA are shown as being transported between the different footwear manufacturing stations FMS on automated guided vehicles AGV such that a footwear assembly after being processed according to one or more processing steps at a footwear manufacturing station may be transported to any other footwear manufacturing station for processing.

[0431] The imaging system IS of fig. 6e is used to provide image data IMGD (not shown) of any of the footwear assemblies FA that are being processed on the footwear manufacturing line FML, i.e. after processing of a footwear assembly FA at one or more of the six footwear manufacturing stations FMS, the footwear assembly FA may be transported directly to the imaging system IS such that image data IMGD can be provided which is then used as an input to a characteristics evaluation model CEM (not shown).

[0432] The footwear manufacturing line FML of fig. 6e further illustrates that a dynamic footwear manufacturing line FML wherein footwear manufacturing stations FMS are separated and wherein routing of footwear assemblies may have a high degree of variance, the imaging system IS is uniquely suited to provide image data IMGD (not shown) based on which characteristics of the footwear assemblies FA may be evaluated.

[0433] Fig. 7 illustrates a footwear identification representation database FIRDB which may be implemented on a data processor i.e. as illustrated on fig. 1. The footwear identification representation database FIRDB stores and updates footwear identification representations FIR that each identify a particular footwear assembly. Four different footwear identification representations are shown, and they each correspond to a particular footwear assembly. The footwear identification representation FIR may contain data fields which identify each of the footwear manufacturing stations at which a footwear assembly has been processed or at which it must be processed.

[0434] The present embodiment illustrates that a footwear identification representation FIR contains a record of three footwear assemblies identified as footwear manufacturing station 1 FMS1, footwear manufacturing station 2 FMS2, and footwear manufacturing station 3 FMS3. The footwear identification representation indicates that processing step 1 PSI and processing step 2 PS2 are performed at footwear manufacturing station 1, processing step 3 PS3 and processing step 4 PS4 are performed on footwear manufacturing station 2 FMS2, processing step 5 PS5, processing step 6 PS6, and processing step 7 PS7 are performed at footwearmanufacturing station 3 FMS3. The footwear identification representation therefore contains a record of the sequence of processing steps according to which the footwear assembly has been processed and additionally identifies the footwear manufacturing stations at which these processing steps were performed.

[0435] The embodiment illustrated on fig. 7 indicates that the processing steps PS form a sequence SEQ which defines the order of consecutive processing steps according to which the footwear assembly is processed. The processing steps PS are here represented in a digital format e.g. as entries in a footwear identification representation database FIRDB. The digital representation of processing steps may define merely the name and identity of a processing step, but it may also encode how the processing step is to be performed by defining e.g. a tool path or the path of the robot arm of a footwear manufacturing robot.

[0436] Fig. 8a illustrates training method steps TMS1-TMS5 of a training method for training a classification model according to an embodiment of the invention. The classification model is trained to enable the classification model to determine characteristics of a footwear assembly FA and provide as an output a characteristics evaluation CE. Hence, the classification model may be considered an example of a characteristics evaluation model CEM. The method is a computer implemented method in the sense that at least the particular step of training the classification model is computer implemented.

[0437] In a first training method step TMS1 training input measurements are received. The training input measurements are at least based on image data IMGD of training footwear assemblies, wherein a subset of the training footwear assemblies comprises one or more defect(s). Training footwear assemblies FA may, for example, be understood as footwear assemblies FA from which the training input measurements are acquired. Hence, the training footwear assemblies are in principle footwear assemblies FA.

[0438] In a further training method step TMS2, labelled training input measurements are generated by individually labelling the received training input measurements. Thelabelling is performed in accordance with a plurality of predefined footwear assembly characteristics classes. Hence, individual training input measurements are labelled with at least one of each footwear assembly characteristics classes to produce labelled training input measurements.

[0439] The one or more footwear assembly characteristics classes of each label are predefined at least in the sense that the footwear assembly characteristics class is not merely determined by the classification model but instead they are determined prior to the training of the classification model. The predefined footwear assembly characteristics classes associated with a particular training input measurement represent characteristics of the footwear assembly, from which the training input measurement was acquired.

[0440] In an additional training method step TMS3, a training data set is established based on the labelled training input measurements.

[0441] In a next training method step TMS4, a classification model is provided.

[0442] In a final training step TMS5 of this exemplified embodiment of the invention, the provided classification model is trained on the basis of the training data set to provide a trained classification model.

[0443] Advantageously, by training the classification model on the basis of the training data set with labelled training input data, relations between the training input data and the labels associated with each training input data may be modelled. Thereby, upon receiving an input measurement, e.g., including image data IMGD, the trained classification model may be applied to classify the input measurement into at least one footwear assembly characteristics class.

[0444] Depending on the implementation of the invention, the training method steps described in relation to fig. 8a may be performed according to different orders. For example, training method step TMS4 of providing the classification model may be performed as a first step of the training method or alternatively in between the step ofgenerating labelled training input measurements TMS2 and the training method step TMS3 of establishing training data set.

[0445] The training method step TMS2 of generating labelled training input measurements may be performed manually by one or more individuals that label the received training input measurements with one or more corresponding known footwear assembly characteristics classes.

[0446] Optionally, labelling the training input measurements may be performed by first classifying the training input measurements with a trained supervised classification model and / or with a trained unsupervised classification model and / or with a semi-supervised classification model, to obtain a temporary label based on one or more of the trained classification models, and then manually validating the obtained temporary label to obtain a validated label, which is then used to label the corresponding received training input measurement. In case the obtained temporary label is deemed incorrect in the manual validation, the person validating the temporary label may change the temporary label to the correct label that corresponds to the actual received training input measurement.

[0447] Optionally, training input measurements may be acquired from training footwear assemblies wherein the training footwear assemblies comprise a tag, such as, e.g., a visual label. The tag comprises information about the training footwear assembly characteristics class of the training footwear. Advantageously, a tag reading system may read the tag and thereby label the training input measurements of the training footwear assembly with the correct footwear assembly characteristics class based on the tag. The tag may, e.g., be a visual tag, e.g., comprising numerals, however it may also be a bar code, QR code, or an RFID tag. An imaging system IS comprising a camera may then interpret the tag to obtain the footwear assembly characteristics class, which is used for labelling the training input measurements of the footwear assembly.

[0448] Fig. 8b illustrates classification method steps CMS1-CMS3 of a classification method of characterizing a footwear assembly according to an embodiment of the invention. The method is a computer implemented classification method.

[0449] In a first classification method step CMS1, a trained classification model is provided. The provided classification model has been trained based on a training data set comprising labelled training input measurements, which are essentially individual training input measurements that have been labelled with at least one of each footwear assembly characteristics classes. The training input measurements are at least based on image data IMGD of training footwear assemblies that is characterized by being associable with at least one predefined footwear assembly characteristics class of a plurality of footwear assembly characteristics classes. The training of the provided classification model may, e.g., be performed using the training method for training a classification model described in relation to fig. 8a.

[0450] In a next classification method step CMS2 an input measurement is received. The input measurement is based on at least image data of a footwear assembly. Input measurements may preferably be acquired using an imaging system, for example as described with reference to figs. 1-4 and 6a-6e. Examples of image data that may be used as input measurements for the classification method step CMS2 are provided in fig. 5b.

[0451] In a further classification method step CMS3, the characteristics of the footwear assembly associated with the input measurement is classified into at least one of the predefined footwear assembly characteristics classes based on the input measurement using the trained classification model. A classification output is produced from the model as a result of the classification, and the classification output is associated with at least one footwear assembly characteristics classes of the plurality of the footwear assembly characteristics classes.

[0452] Optionally, a footwear assembly may be classified into an unknown class denoting that the footwear assembly has one or more unknown footwear assembly characteristics. Classifying a footwear assembly as unknown may optionally be basedon a certainty parameter associated with the classification model, and a certainty parameter threshold.

[0453] As an example, a footwear assembly classified as having an unknown footwear assembly characteristic may, e.g., advantageously, be a footwear assembly that is different to the particular footwear assembly characteristics classes that the trained classification model has been trained to recognize and thereby classify. Advantageously, this has the effect of minimizing misclassifications of such unrecognizable footwear assembly characteristics into one or more of the other footwear assembly characteristics classes.

[0454] Optionally, input measurements may be determined as unsuitable for classification, in which case the classification method may not be performed. An input measurement may be determined as unsuitable for classification for various reasons. E.g., a bad acquisition may result from incorrect use of the imaging system during acquisition of image data.

[0455] Optionally, the classification model may be retrained based on input measurements. In this case, the label associated with the input measurement is the class determined by the trained classification model. Advantageously, before applying input measurements and the associated class (label) for retraining the classification model, the validity of the associated class (label) may be evaluated. E.g., an input measurement may be validated and used for retraining of the classification model if the certainty of the class as provided by the classification model is sufficiently high.

[0456] Alternatively, the validity may be evaluated based on a human assessment, to ensure that the classification provided by the classification model was correct. Other ways of ensuring the validity of the provided class of the input measurement may be utilized to validate the provided class associated with the input measurement prior to using the input measurement and associated class as a training input measurement and label (a training example).

[0457] Fig. 9a illustrates a schematic representation of a training system TS according to an embodiment of the invention. Advantageously, the training system may be used to perform the training method described in relation to fig. 8a.

[0458] The training system TS is configured to train a classification model CM, such that the classification model CM becomes capable of characterizing a footwear assembly, by means of footwear assembly characteristics classes. The training is based on training input measurements TIM and footwear assembly characteristics classes FACC. The training system TS comprises a training input measurement receiver TIMR, a training input measurement labeller TIML, a training dataset generator TDSG and a training module TM.

[0459] The training input measurement receiver TIMR receives training input measurements TIM, which are based on at least image data IMGD of training footwear assemblies. The training input measurements TIM are passed on to the training input measurement labeller TIML, which also receives a plurality of predefined footwear assembly characteristics classes FACC. The training input measurements TIM are then labelled with at least one of the predefined footwear assembly characteristic classes FACC, utilizing the training input measurement labeller TIML. The label thereby associates at least image data of a footwear assembly, with at least one predefined footwear assembly characteristics class FACC, via the labelling of the training input measurement. See examples from an embodiment on figs. 5a-5c, which may be considered representing labelled training input measurements LTIM, as it contains image data IMGD usable as training input measurements TIM and contains a footwear assembly characteristics class FACC.

[0460] The training dataset generator TDSG collects the labelled training input measurements LTIM from the training input measurement labeller TIML and generates a training data set TDS on the basis of the labelled training input measurements LTIM. The training dataset TDS is then received by the training module TM along with the classification model CM, and the training module TM then trains the classification model CM based on the received training input dataset. The output from the training of the classification model CM is a trained classification model TCM.

[0461] The trained classification model TCM may then be used as a characteristics evaluation model CEM such as in the embodiment illustrated on fig. 1.

[0462] Advantageously, the trained classification model TCM may be provided to a classification system, in which the trained classification model TCM may be utilized for classification of footwear assembly characteristics based on input measurements. The classification of footwear assembly characteristics as performed by the trained classification model TCM therefore constitutes a characteristics evaluation i.e. as illustrated on fig. 1.

[0463] Fig. 9b illustrates a schematic representation of a classification system CS according to an embodiment of the invention. The classification system CS is an example of a characteristics evaluation model CEM as illustrated on fig. 1.

[0464] Advantageously, the classification system CS may be used to perform the classification method described in relation to fig. 8b. The classification system CS is configured to characterize a footwear assembly. More specifically, the classification system is configured to classify footwear assembly characteristics according to predefined footwear assembly characteristics classes using an input measurement that is based on at least image data IMGD of the footwear assembly.

[0465] The image data IMGD may, e.g., be acquired using the imaging system IS, according to embodiments of the invention such as illustrated on figs. 1-4 and 6a-6e. The classification system CS comprises an input measurement receiver IMR, a classifier CLA, and a trained classification model TCM, which is trained at least based on a training data set, e.g., as previously described in relation to fig. 8a and fig. 9a.

[0466] The input measurement receiver IMR receives an input measurement IM, which is based on at least image data IMGD of a footwear assembly. The input measurement is passed on to the classifier CLA from the input measurement receiver IMR. The classifier CLA then utilizes the trained classification model TCM to classify the footwear assembly into at least one of the predefined footwear assembly characteristics classes, based on the input measurement IM and using the trainedclassification model TCM. A classification output CO is produced from the classifier CLA as a result of the classification.

[0467] The classification output CO is an example of an implementation of a characteristics evaluation CE as illustrated on fig. 1. The classification output CO is associated with at least one footwear assembly characteristics classes of the plurality of footwear assembly characteristics classes.

[0468] In this exemplified embodiment of the invention, the classification output CO is the particular footwear assembly characteristics class classified by the classifier CLA based on the trained classification model TCM and the input measurement IM. Thereby, in this embodiment, the classification output CO is the footwear assembly characteristic class of the footwear assembly from which the input measurement IM was obtained.

[0469] In a first exemplified embodiment of the invention, the classification system CS is configured to classify defect type which may relate to different aspects of the footwear assembly such as color, surface characteristics, size or shape of parts of the footwear assembly, or absence of parts of the footwear assembly. Hence, as an example, the input measurement IM could be image data IMGD of a footwear assembly comprising an outsole edge defect i.e. as illustrated on fig. 5a-5b.

[0470] The input measurement is received by the classification system CS via the input measurement receiver IMR, which passes the input measurement to the classifier CLA, which provides the input measurement IM to the trained classification model TCM, which classifies the input measurement as an outsole edge defect and outputs the class as a classification output CO. This particular trained classification model TCM has been trained based on a training data set comprising at least training input measurements of training footwear assemblies comprising an outsole edge defect, and wherein these training measurements have been labelled with the class outsole edge defect.

[0471] The training method and the training system illustrated in fig. 8a and fig. 9a may be implemented to train the classification model, and the classification methodand classification system illustrated in fig. 8b and fig. 9b may be implemented to classify footwear assembly characteristics classes using the trained classification model. The classification model may be trained based on various different footwear assembly characteristics classes to provide a trained classification model that may be capable of classifying various different footwear assembly characteristics classes. As mentioned, the footwear assembly characteristics classes may comprise various different types of defects of footwear assemblies, and hence, the classification model can be trained by the disclosed training system and / or by using the disclosed training method, to classify, e.g., various different classes related to defects of footwear assemblies based on the disclosed classification method and / or classification system.

[0472] In a first example, labelled training input measurements are established by labelling the training input measurements according to footwear assembly defect class, which is selected at least among the footwear assembly defect classes abrasion, detachment and absence related to respectively abrasions on a footwear assembly, detachment of parts of the footwear assembly or absence of parts of the footwear assembly.

[0473] A training dataset is then established based on the labelled training input measurements, and the classification model is trained using this particular training dataset, to provide a trained classification model. Since the model was trained based on the footwear assembly defect class comprising at least the classes abrasion, detachment and absence, the trained classification model is able to classify input measurements of footwear assemblies into these particular classes.

[0474] In a second example, the labelled training input measurements are established by labelling the training input measurements according to the one or more defect causing footwear manufacturing stations, i.e. by accordingly defining a class for each footwear manufacturing station at which the footwear assembly has been processed. This may be done in addition to labelling input data according to a footwear defect class. By performing the training method to train the classification model based on these particular labelled training input measurements, the trained classification model is able to classify input measurements according to the defect causing footwearmanufacturing stations utilized in the training method and according to the footwear defects utilized in the training method.

[0475] In a third example, the training of the classification model is performed based on labelled training input measurements established by labelling the training input measurements according to defect severity classes. Various defect severity classes could be applied. Nonlimiting examples of defect severity classes may, e.g., comprise: no defect, low defect, medium defect, high defect, severe defect. Thereby, advantageously, the trained classification model is able to classify input measurements according to the defect severity classes, utilized in the training method.

[0476] Optionally, the classification model may comprise an unknown class, to enable the trained classification model to classify an input measurement into the unknown class. Advantageously, the unknown class enables classification into an unknown class when the input measurement represents, e.g., a type of footwear assembly characteristic that is unknown to the trained classification model. Optionally, the unknown class may be utilized when the probability of the input measurement belonging to a certain footwear assembly defect class is below a certainty threshold, thereby minimizing the false positive rate of the classification model.

[0477] Fig. 10 illustrates a schematic representation of a training module TM according to an embodiment of the invention. The training module TM may, for example, be implemented as part of the embodied training system illustrated in fig. 9a. Advantageously, the training module TM may be used to train a classification model CM and hence, the training module TM may provide a trained classification model TCM based on the classification model, training input measurements and associated training input labels. Advantageously, the trained classification model TCM may, e.g., be used with embodiments of the classification system (not shown) of the invention and with embodiments of the classification method (not shown) of the invention, to classify footwear assembly characteristics classes based on input measurements comprising at least image data. The operations performed by the training module TM may be computer implemented.

[0478] The training module TM comprises an error calculation module ECM, a classification model optimizer CMO, and a provided classification model CM comprising classification model parameters CMP.

[0479] In a first training iteration, the provided classification model CM receives training input measurements TIM from a training dataset TDS comprising labelled training input measurements. The classification model then classifies the training input measurements into one or more footwear assembly characteristics classes, and thereby provides a training classification output TCO comprising a classified footwear assembly characteristics class for each training input measurement. The error calculation module ECM receives the training classification outputs TCO and the training input labels TIL, each of which are associated with the corresponding training input measurements TIM from which the training classification output is generated. The error calculation module ECM compares the training classification output TCO with the training input labels TIL, to determine a training error TE. The training error TE represents a degree of classification wrongness, e.g. represented by a representation of a difference between the training classification outputs TCO and the associated training input labels TIL. The classification model optimizer CMO then adjusts the classification model parameters CMP to generate updated classification model parameters UCMP.

[0480] The classification model CM is initiated with an initial set of classification model parameters CMP, e.g. according to a model initialization strategy. The model initialization strategy may comprise various ways of selecting the initial classification model parameters, depending on the implementation of the invention. Nonlimiting examples of model initiation strategies may, e.g., comprise, sampling the initial set of classification model parameters CMP, e.g., from a statistical distribution such as e.g. from a Gaussian distribution, such as from a uniform distribution, such as from a truncated normal distribution. The classification model parameters CMP may, for example, be randomly selected (sampled) from the statistical distribution, or alternatively, be selected according to other selection strategies. E.g., the classification model parameters may in an alternative embodiment be initialized with equalparameter values across all the model parameters that are adjusted during training, e.g. with zeros, ones or other values. Alternatively, and optionally, the model initialization strategy may comprise transfer learning, in which, the classification model parameters CMP are adopted from a pre-training of the classification model, wherein the pretraining of the classification model is based on different training data. Advantageously, this may improve the performance of the trained classification model.

[0481] Following the first training iteration; in a next training iteration, the classification model classifies the training input measurements TIM based on the updated classification model parameters UCMP, to produce a new training classification output TCO. The new training classification output TCO is received by the error calculation module ECM, which calculates a new training error based on the new training classification output and the training input labels TIL. The classification model optimizer CMO then provides updated classification model parameters UCMP. The training continues with further training iterations, wherein each training iteration provides updated classification model parameters.

[0482] Advantageously, the error calculation module ECM and the classification model optimizer CMO may cooperate to provide iterative training of the classification model CM, such that the classification model parameters are adjusted to minimize training error. This may be achieved in various ways according to the invention, such as, e.g., based on various types of optimization methods, e.g. including iterative optimization algorithms configured to update the classification model parameters such that the training error are minimized. The optimization algorithms may include calculating training error based on a cost function, wherein the cost function is dependent on the classification model parameters. Thus, determining a minimum of such cost function provides updated classification model parameters associated with a minimized training error (minimized cost) across the whole training dataset or across a batch of the training dataset. Nonlimiting examples of iterative optimization algorithms that may be implemented to provide updated classification model parameters UCMP may, e.g., include gradient descent types of algorithms, e.g. stochastic gradient descent, and other types of gradient descent algorithms. Alternativeembodiments of the invention may also utilize different types of cost functions. The choice of cost function may depend on the implementation of the invention. Further nonlimiting examples of cost functions include least squares mean, multi-class cross entropy loss, and Kullback Leibler Divergence loss.

[0483] The optimization algorithm may comprise a learning rate parameter that specifies the step size for each iteration. More specifically, in each training iteration, the classification model parameters are updated stepwise, such that the cost function are minimized step by step towards a minima of the cost function. The learning rate specifies the size of each such step towards a minima of the cost function taken during an iteration of the optimization algorithm. Selecting a large learning rate may yield faster convergence of the model, meaning that determination of the updated classification model parameters UCMP that provides a minimal training error (minimal cost) is determined fast. However, a large learning rate may result in overshoot of the minima. Selecting a smaller learning rate results in a slower descent towards the minima of the cost function, however, the chance of converging towards a global minimum of the cost function rather than towards a larger local minima of the cost function, is improved. The learning rate may be predetermined by a user, and / or the learning rate may be varied across training iterations. A user may experiment with different learning rates and select the learning rate that represents a good compromise between fast convergence, and an acceptable training error (cost).

[0484] Optionally, the optimization algorithm may comprise an adaptive learning rate. Non-limiting examples of optimization algorithms with an adaptive and / or varying learning rate include, e.g., root means squared propagation, which may be considered an extension of gradient descent and the AdaGrad version of gradient descent. Root mean squared propagation uses a decaying average of partial gradients in the adaptation of the step size for each parameter. Advantageously, the use of a decaying moving average allows the algorithm to focus on the most recently observed partial gradients seen during the progress of the search.

[0485] Optionally, a training termination condition may specify when to terminate training of the classification model. The training termination condition may be basedon various conditions, including, e.g., a predetermined number of training iterations, a predetermined training error, and a measure of the change in training error between one or more training iterations. The training termination condition may vary according to different implementations of the invention, and may further comprise combinations of one or more of, e.g., the mentioned training termination conditions and / or of other training termination conditions.

[0486] Optionally, the training data set TDS may be grouped into one or more training data batches that each comprise a subset of training data. The classification model may then be trained on each training data batch. This form of training may also sometimes be referred to as batch learning. The training may be performed iteratively, such that the classification model parameters from a previous iteration are utilized to initiate the next training iteration where training is performed with a next training data batch. Advantageously, this may improve the training error TE and thereby it may improve the performance of the trained classification model. The training may also be performed without initiating the model with the classification model parameters determined form a previous training based on a previous training data batch.

[0487] Optionally, the training of the classification model may be performed multiple times based on the same training dataset. Advantageously, this may improve the model performance. The number of passes of the entire training dataset that the classification model CM should complete during training of the model may be specified with the term ‘epoch’. Hence, one epoch indicates that the classification model CM has been trained based on one pass of the training dataset, two epochs indicates that the classification model has been trained based on two passes of the training dataset, etc. The optimal number of epochs may be determined in various ways depending on the implementation of the invention, including, e.g., based on the early stopping method. The number of epochs may, e.g., also be determined manually by continuing the training until the performance of the classification model does not improve further, as manually evaluated based on different model performance metrics. Model performance metrics may, e.g. advantageously be based on one or more of the following metrics: confusion matrix, type I error, type II error, accuracy, recall,precision and Fl -score, specificity, ROC (Receiver Operating Characteristics curve) curve, ROC curve AUC (area under the curve) score, PR score.

[0488] Optionally, the training termination condition may comprise a predetermined number of epochs.

[0489] Optionally the number of epoch may be based on a change in one or more model performance metrics. E.g., the training termination condition may specify a threshold value for difference in one or more model performance metrics between epochs. When that threshold (training termination condition) is exceeded and / or reached, the training is terminated.

[0490] Optionally, the model initiation strategy may include transfer learning, wherein the classification model parameters are initiated based on a pre-training using a pre-training training dataset. More specifically, the classification model parameters are adopted from a pre-training, wherein the classification model has been trained using pre-training training data. Advantageously, transfer learning enables the classification model to utilize knowledge gained from training on the pre-training training dataset to classify footwear assembly characteristics classes. A nonlimiting example of a pre-training dataset may, e.g., be the ImageNet dataset, which comprises a large number of image-label pairs comprising an image of an object and an associated label that defines the class of the object of the image.

[0491] Optionally, the error calculation model and the training model parameter optimizer may be comprised by one module comprised by the training module. E.g., the classification model optimizer CMO may comprise the error calculation module, in some embodiments of the invention or the error calculation module may comprise the classification model optimizer.

[0492] Optionally, the training may be based on various types of backpropagation. This may, e.g., be advantageous when utilizing a neural network type classification model, including, e.g., convolutional neural network classification models, multilayer perceptron models, but also other types of neural network models.

[0493] It should be understood that in case a trained classification model performs worse based on testing compared to previous trained classification models, the model with the best performance may, advantageously, be utilized. Furthermore, training of a classification model may be based on a previous model that has already been trained. In fact transfer learning based on previous trained models may be applied, starting with any of the previously trained classification models, and not only the most recently trained model. Also, as the training data set grows, the classification model may be trained from scratch, without any prior knowledge from previous models. Multiple classification models may be trained and evaluated, and the trained classification model that provides the best performance may be selected and applied for classification of input measurements. E.g., applied for classification by implementing this trained classification model in the footwear manufacturing line, according to embodiments of the invention, and / or by implementing the classification model in a cloud computing environment communicating with the footwear manufacturing line including with the data processor of the footwear manufacturing line, according to an embodiment of the invention. When implementing the trained classification model in a cloud computing environment, the classification model may be made accessible to various footwear manufacturing lines or imaging systems across different geographical locations or across different imaging systems at the same site, which is advantageous.

[0494] Fig. 11 illustrates a schematic representation of a classification model CM, according to an embodiment of the invention. More specifically, the illustrated classification model CM is an example of a neural network classification model. The neural network classification model may be trained using a training method such as the training method illustrated in fig. 8a, and using the training systems of the invention, including, for example, the training system illustrated in fig. 9a. The classification model CM represents a specific implementation of a characteristics evaluation model CEM as illustrated in a system context on fig. 1.

[0495] When trained, the neural network classification model may be considered an example of a trained classification model according to the invention, which may beused in classification systems according to the invention, including, for example, the classification system illustrated in fig. 9b. Hence, when trained, the trained classification model may perform a classification method according to the invention, including the method described in relation to fig. 8b. The embodied neural network classification model consists of sets of neurons arranged in layers and wherein each neuron of a layer is connected with each neuron of the next layer, and with each neuron of the previous layer. In that sense, the neural network may be considered a fully connected neural network. Other embodiments of the classification model may include a neural network classification model that is not a fully connected neural network.

[0496] The neural network classification model illustrated in fig. 11 comprises an input layer IL comprising a number of neurons INl-INn, a number of hidden layers HL, and wherein each hidden layer comprises a number of hidden neurons HN11 - HNnn. The network further comprises a number of output neurons ONI - ONn, and a set of weights Wl, W2, wherein each connection between neurons in the hidden layer(s) and between a hidden layer and the output layer OL is associated with at least one weight. The individual weights of the sets of weights W1,W2 is illustrated as lines connecting the neurons. In addition to the weights, each layer except for the input layer may comprise a bias (not shown). The input layer IL is configured to receive input measurements IMl-IMn, for example image data or a representation of image data as e.g., image data as illustrated in fig. 5b, and further configured to pass these input measurements to the next layer, which is the first hidden layer HL. The hidden layers provides input to the neurons ONI -ONn of the output layer OL, which in turn outputs data (e.g. a value) for each footwear assembly characteristics classes FACCl-FACCn. A classification output CO is then determined based on the footwear assembly characteristic classes FACCl-FACCn.

[0497] Note that the classification output CO is an example of an implementation of a characteristics evaluation as illustrated in a system context on fig. 1.

[0498] The neural network classification model may be trained based on a training dataset and based on a training algorithm such as, e.g. backpropagation including variations of training algorithms based on different types of backpropagation. Noticethat other training algorithms may also be used, and that the skilled person would be able to select such different training algorithm if this would be beneficial. Nevertheless, alternatives to backpropagation based training algorithms may be less efficient. When the neural network classification model is trained, a trained classification model is obtained in the form of a trained neural network classifier.

[0499] The trained neural network classifier may then be utilized to classify footwear assembly characteristics classes. In a classification scenario, the input layer of the trained neural network classification model receives input measurements represented as IMl-IMn. The received input measurements are then passed to each neuron HN11- HNln of a first hidden layer HL. The input measurements IMl-IMn may, e.g., be pixel values of an image of a footwear assembly (image data), or it may be features extracted from image data of a footwear assembly, e.g., obtained in foregoing preprocessing steps. Each neuron of the first hidden layer of the hidden layers (HL) outputs a response based on the received input measurement, the weights W 1 and a bias (now shown). The response is received by each neuron HN21-HNnn of the second hidden layer of the hidden layers HL, which each in turn outputs a response based on the received response from the first hidden layer, based on the individual weight associated with each neuron, and based on a second bias (now shown). The response of each neuron in the last hidden layer is received by each neuron in the output layer OL. Each neuron in the output layer then outputs data (value) for each footwear assembly characteristics class FACCl-FACCn based on the response received from the last hidden layer and based on an output activation function.

[0500] In this example, the output activation function is a SoftMax function, which outputs a relative probability for each footwear assembly characteristics class FACCl- FACCn. In principle, many other types of activation functions could be utilized as the output activation function.

[0501] In an optional step, a classification output CO may be determined based on the output of the output activation function. The classification output CO may e.g. be determined as the largest footwear assembly characteristics class value of all the output footwear assembly characteristics class values FACCl-FACCn. When a SoftMaxactivation function is utilized as output activation function, the largest footwear assembly characteristics class value would correspond to the footwear assembly characteristics class having the largest relative probability given the input measurements. This class would then be determined as the classification output.

[0502] The neural network classifier could be trained to classify different class groups including e.g. footwear assembly defect class, defect causing footwear manufacturing stations, defect severity class and other class groups mentioned elsewhere in this disclosure. As an example, the neural network classification model could be trained based on training data associated with the footwear assembly defect class. Hence, the training data utilized to train such neural network classifier would comprise training input measurements that are based on image data of a footwear assembly, and each of the training input measurements would be labeled with one or more specific defects from one or more footwear assembly defect classes.

[0503] Fig. 12 illustrates a schematic representation of a neuron of a neural network classification model, according to an embodiment of the invention. The neuron could, e.g., be a neuron of a neural network classifier such as that illustrated in fig 25.

[0504] In principle, the illustrated neuron could be an example of both a neuron of a hidden layer as well as an example of a neuron of an output layer of a neural network. The main difference between these two mentioned types of neurons being a difference in activation function AF. In this example the neuron is a neuron of a hidden layer, such as the hidden layer neuron HN21 of the neural network classifier illustrated in fig. H.

[0505] In this particular example, the hidden layer neuron HN21 calculates a weighted sum of the output from three hidden layer neurons of an upstream hidden layer HN11, HN12, HN13, using the weights W2. A bias is added to the weighted sum. Each layer comprises one bias parameter. Notice that in this example, W2 represents individual weights associated with each output from the hidden neurons HN11, HN12, HN13. The weighted sum with the bias added is then fed to an activation function AF, which calculates an output based on the received weighted sum plus the bias. Theactivation function can thereby be said to determine the output of the neuron. Some activation functions may e.g. be chosen to enable the neural network classifier to learn complex relationships such as, e.g., non-linear relations between an input measurement such as image data, and the footwear assembly characteristics class associated with that input measurement.

[0506] In an advantageous embodiment of the invention, each hidden layer neuron and each output neuron of the embodied neural network classification model comprises an activation function. However, in other embodiments of the invention not all neurons may comprise an activation function. Non-limiting examples of activations functions of hidden layer neurons of a neural network classification model that may be utilized to characterize footwear assembly characteristics includes, e.g., the sigmoid function, tanh function, exponential linear units, self-exponential linear units, the ReLU function (rectified linear unit), leaky ReLU function, parametric ReLU function, self-gated activation function, among others.

[0507] Advantageously, ReLU type functions have a derivative function and allow for backpropagation, while simultaneously making it computationally efficient. It further enables the neural network classification model to learn nonlinear relations. Further advantageously, since with ReLU functions only a certain number of neurons are activated, meaning that the output of the neuron is non-zero, the ReLU function is far more computationally efficient, e.g., when compared to e.g. the sigmoid and tanh functions. Furthermore, ReLU function accelerates the convergence of gradient descent towards the global minimum of the loss function due to its linear, nonsaturating property.

[0508] In some situations, the ReLU function may results in dead neurons, e.g., neurons that output only zero values, thereby diminishing the flexibility and / or complexity of the neural network classifier. In this case, the leaky ReLu function, which has a small slope in the negative area of the function, may be applied instead to alleviate this problem. Alternatively, if the leaky ReLU function fails to alleviate the problem of dead neurons, the parametric ReLU function may be applied instead. Theparametric ReLU function comprises a slope parameter that may be learned during backpropagation.

[0509] In implementations of the invention utilizing one or more deep neural networks, e.g., networks deeper than four layers, the self-gated activation function may advantageously be applied.

[0510] The activation function of the output neurons of a neural network classification model according to an embodiment of the invention may comprise an activation function different to the activation functions applied in the hidden layers of the network. Examples of activation functions of the output layer comprise e.g. the sigmoid function, and the SoftMax function.

[0511] Fig. 13a illustrates a schematic representation of a classification model, according to an embodiment of the invention. More specifically, the illustrated classification model is an example of a convolutional neural network classification model CNNCM. The convolutional neural network classification model may be trained using a training method such as the training method illustrated in fig. 8a, and using the training systems of the invention, including, for example, the training system illustrated in fig. 9a. When trained, the convolutional neural network classification model CNNCM may be considered an example of a trained classification model according to the invention, which may be used in classification systems according to the invention, including, for example, the classification system illustrated in fig. 9b. Hence, when trained, the trained classification model may perform a classification method according to the invention, including the method described in relation to fig. 8b. In short, the convolutional neural network classification model CNNCM may advantageously be used to classify footwear assembly characteristics of a footwear assembly, where the footwear assembly characteristics are classified as footwear assembly characteristics classes.

[0512] The convolutional neural network classification model CNNCM may be considered a specific implementation of the characteristics evaluation model CEM i.e.as illustrated on fig. 1. The classification output CO may equivalently be considered as a specific example of a characteristics evaluation CE i.e. as illustrated on fig. 1.

[0513] The convolutional neural network classification model CNNCM comprises a feature extraction module FEM followed by a neural network model NNM. The neural network model NNM classifies footwear assembly characteristics classes and generates a classification output based on a feature extraction output FEO received from the feature extraction module FEM. The feature extraction module FEM is configured to receive a matrix as input measurement IM, and to extract features of the matrix to generate the feature extraction output FEO. The matrix may, e.g., represent image data. The image data may, e.g., comprise one or more images of a footwear assembly, including, e.g., images of the footwear assembly that illustrate different parts of the footwear assembly. The image data may, e.g., be acquired using the image system described in relation to various embodiments of the invention. The image data may be obtained based on various different types of image acquisition methods, as described elsewhere in this disclosure, including e.g. 2D and 3D imaging, color imaging etc. The input measurements may be image data, however, the input data may also be pre-processed image data, which have been pre-processed in different ways, e.g., smoothed, filtered, converted, normalized, resampled, reduced in size using principal component analysis or singular value decomposition or other data reduction methods.

[0514] Note that the input measurements may be a multidimensional matrix, and it may further represent more than one image of a footwear assembly. Advantageously, by supplying an input measurement that represents multiple images of a footwear assembly representing different parts of the footwear assembly, the convolutional neural network may exploit dependencies between pixels in these multiple images. This may improve the accuracy of the classification model. Furthermore, the input measurement may sometimes include multiple images of the same footwear assembly obtained at different times. Furthermore, in principle the input measurement may also include video data of a footwear assembly.

[0515] The feature extraction module FEM comprises at least one convolutional layer CL, and optionally a following pooling layer PL. The convolutional layer comprises at least one kernel. Typically, the feature extraction module may perform better when using multiple kernels in each layer, since this enables the feature extraction model to learn more features of the input measurements. For similar reasons, the classification model also typically comprises more than one convolutional layer, since this enables the model to learn more complex features of the input measurements, and hence, enable the model to perform better. Each kernel of a layer is individually convolved with the input measurement IM, to extract features from the input measurement IM. If the model comprises more than one convolutional layer, the output of the first convolutional layer is received by the next convolutional layer, and so forth.

[0516] Again optionally, the output of a convolutional layer is typically received by a pooling layer PL, as mentioned above. So, typically, a convolutional layer CL comprises a plurality of kernels to enable extraction of a plurality of features, to enable better performance of convolutional neural network classification model CNNCM. The kernel is a small matrix with a size that is less than the size of the input measurement. The size of the kernel may vary depending on the particular implementation of the embodiment of the invention, and may, e.g., be based on empirical testing of the model with different filter sizes. The filter is moved across the height and width of the input measurements and the dot product of the kernel and the image are computed at every spatial position of the filter. The length by which the kernel slides across the input measurement IM is the stride length. Different stride lengths may be tested to determine the stride length that provide the optimal performance of the convolutional neural network classification model CNNCM.

[0517] The stride length may be part of what is sometimes referred to as the hyper parameters of the model. The hyperparameters may be tuned based on evaluation of the model performance, e.g. based on empirical testing of the model, wherein the hyper parameters are varied for each training and test iteration, and ultimately, the best performing model of the trained and tested models are chosen as the trained classification model. The actual coefficients of the kernels are determined based onI l l training of the network, and the training of the network is performed following the previously described methodology, and thereby the training is based on a training dataset with labeled training input measurements. Optionally, when the convolutional layer comprises multiple kernels, the output of each kernel may be stacked, and an activation function may be applied to the stack of kernel outputs.

[0518] Advantageously, by applying activation functions, the classification model may learn complex non-linear features of the input measurements, as previously discussed in relation to the neural network classification model described in relation to fig. 11. Various types of activation functions including, e.g., ReLU functions and / or tanh functions may be used, however, other activation functions may also be utilized, as discussed elsewhere in this disclosure.

[0519] Following a convolutional layer, the pooling layer PL may optionally be arranged to receive the output of the convolutional layer CL. The pooling layer reduces the size of the feature extraction output (sometimes referred to as feature maps) outputted by the convolutional layers and thereby may speed up the computation of training and footwear assembly characteristics classification of the classification model. Nevertheless, embodiments of the invention are not limited to using max pooling, and so, e.g., average pooling and other types of pooling may also be utilized, depending on the particular implementation of the invention. In short, using max pooling; from each patch of a feature map (the output of a convolution with a kernel of the convolutional layer CL), the maximum value is selected to create a feature map with a reduced size. In average pooling; from each patch of a feature map (the output of a kernel of a convolutional layer CL), the average value is selected to create a reduced feature map. The size of the map output by the pooling layer depends on the size of the patch applied in the pooling layer and may further depend on the stride length of the applied patch.

[0520] To reduce computational resources required to perform a footwear assembly characteristics classification using the convolutional neural network classification model and the computational resources required to train the classification model, the patch size and stride length may be determined such that the output of the pooling layerreduces the size of the feature map received from the convolutional layers substantially. On the other hand, reducing the feature maps may result in loss of information and degraded performance of the classification model.

[0521] The optimal hyper parameters, e.g. the stride length and patch size (sometimes referred to as kernel) of the one or more pooling layers may be selected by training the model using different hyper parameters and then comparing each of the trained model, and then selecting the classification model with the best classification performance, or alternatively, selecting the model with the best compromise between classification performance and required computer resources.

[0522] The feature extraction output of the feature extraction module is the result of the input measurement being convolved with kernels of the convolutional layer and with the kernels of the pooling layers of the feature extraction module. The feature extraction output is flattened to a vector. Then this vector is fed to a classification module CLM, which in this exemplified embodiment comprises a neural network model, e.g., a multilayer perceptron model MLPM, the output of which is the classification output. The classification module may in other embodiments of the invention comprise other types of classifiers.

[0523] Fig. 13b illustrates a schematic representation of a neural network model. In this exemplified embodiment of the invention, the neural network model is a fully connected neural network model (sometimes referred to as a multilayer perceptron model MLPM). Nevertheless, other embodiments of the invention may utilize neural network models that are not fully connected neural network models. The multilayer perceptron model MLPM of this embodiment may, for example, be utilized in the classification module CLM of the convolutional neural network classification model CNNCM illustrated in fig. 13a, for producing a classification output CO comprising one or more footwear assembly characteristics classes, based on a feature extraction output received from a feature extraction module FEM.

[0524] In this exemplified embodiment, the multilayer perceptron model MLPM comprises a flattening layer FL, which flattens the feature extraction output into avector, an input layer IL, a hidden layer HL, and an output layer OL, which are all fully connected. After the multilayer perceptron model MLPM, a classification output determiner COD is arranged to receive the output of the output layer OL, and based on this output, determine a classification output CO. The classification determiner COD is an optional feature, which may be omitted in embodiments of the invention, when it is preferred to obtain the output of the output layer. E.g. when it is preferred to obtain a relative probability of an input measurement belong to each of the footwear assembly characteristics classes. This may, e.g., be achieved when the output layer comprises a SoftMax activation function.

[0525] The flattening layer flattens the received feature extraction output FEO to a vector and then feeds each value of the vector to each neuron in the input layer of the multilayer perceptron model MLPM. Weights, bias, and activation functions of the layers are then applied as previously described in relation to the description of the neural network model illustrated in fig. 11, and in relation to the description of a neuron of a neural network illustrated in fig. 12.

[0526] The output of the output layer is the relative probability for each footwear assembly characteristics class, including for example, footwear defect types such as abrasion, detachment, or absence. The output of the output layer is received by the classification output determiner COD. The classification output determiner COD then selects a classification output comprising a footwear assembly characteristics class. In this example, the classification output is selected by choosing the footwear assembly characteristics class with the highest probability. The classification output determiner COD may alternatively be configured to bypass the output of the output layer.

[0527] Notice that the training of the convolutional multilayer perceptron model MLPM is performed for the full model, including both the multilayer perceptron model multilayer perceptron model MLPM and the feature extraction module comprising the convolutional layer(s) and the pooling layer(s). Thus, when training the convolutional neural network classification model CNNCM, the training data comprises training examples which consists of training input measurement and an associated known labelcomprising the actual footwear assembly characteristics class of that training input measurement.

[0528] In an advantageous embodiment of the multilayer perceptron model MLPM, the model comprises more than one hidden layer. This advantageously has the effect that the model is capable of learning more complex relations between training input measurements and associated labels, and thus it may elevate the performance of the multilayer perceptron model MLPM.

[0529] In this exemplified embodiment, the multilayer perceptron model MLPM is illustrated with four neurons in the hidden layer HL. However, as illustrated by the dots shown between the bottom two neurons of the hidden layer HL, the multilayer perceptron model MLPM may comprise additional neurons. Including more neurons may advantageously increase model performance. Increasing the number of neurons in the network may, however, diminish computational speed. Empirical testing of different model architectures enables the user to select a model with a good ratio between performance and computational speed.

[0530] Fig. 14 illustrates an example of a classification model CM implemented as a convolutional neural network classification model, according to an embodiment of the invention. The particular convolutional neural network classification model illustrated in fig. 14 may sometimes be referred to as VGG16. VGG16 can be considered an example of the convolutional neural network classification model described in relation to fig. 13a. As such, VGG16 requires training input measurements and input measurements to be received in a matrix format as previously described in relation to the generic convolutional neural network classification model illustrated in fig. 13a. Thus, the training input measurements and input measurements applied for use with VGG16 may, e.g., image data of a footwear assembly, for example image data as illustrated in fig. 5b.

[0531] The VGG16 convolutional neural network classification model may be trained using a training method such as the training method illustrated in fig. 8a, and using the training systems of the invention, including, for example, the training systemillustrated in fig. 9a. When trained, the VGG16 convolutional neural network classification model may be considered an example of a trained classification model according to the invention, which may be used in classification systems according to the invention, including, for example, the classification system illustrated in fig. 9b. Hence, when trained, the trained classification model may perform a classification method according to the invention, including the method described in relation to fig. 8b. In short, the VGG16 convolutional neural network classification model may advantageously be used to classify footwear assembly characteristics classes using input measurements at least based on image data of the footwear assembly.

[0532] The VGG16 classification model comprises 16 layers with weights that may be determined through training of the classification model. These layers include 13 convolutional layers followed sequentially by a multilayer perceptron model comprising three dense layers DL including the output layer OL. The padding of the convolutional layers is such that the spatial resolution is preserved after convolution (also referred to as same padding). The convolution stride length is set to one pixel. In addition to the convolutional layers, VGG16 comprises five max-pooling layers, which follow some of the convolutional layers. Max-pooling is performed over a 2- by-2 pixel window, with a stride of two. The ReLU activation function is used for each of the convolutional layers and each of the dense layers

[0533] In a sequential order from the first convolutional layer to the last output layer of the multilayer perceptron model, the VGG16 model comprises five convolution blocks CB1-CB5.

[0534] The first convolution block CB1 comprises two convolution layers CL11, CL12 of 64 channels (channel refers to the number of filters in a layer) with a kernel size of 3-by-3 and same padding, and a max-pooling layer PL1 of 2-by-2 pool size and a stride of two. The first layer CL11 of the first convolution block CB1 receives input measurements. The input measurements are required to be in a matrix format.

[0535] The first convolution block CB1 is connected to the second convolution block CB2, which comprises two convolutional layers CL21, CL22, each having 128channels of 3-by-3 kernels and same padding, and a max-pooling layer PL2 of 2-by-2 and a stride of two.

[0536] The second convolution block CB2 is connected to the third convolution block CB3, which comprises three convolutional layers CL31, CL32, CL33, each having 256 channels of 3-by-3 kernels and same padding, and a max-pooling layer PL3 of 2-by-2 and a stride of two.

[0537] The third convolution block CB3 is connected to the fourth convolution block CB4, which comprises three convolutional layers CL41, CL42, CL43, each having 512 channels of 3-by-3 kernels and same padding, and a max-pooling layer PL4 of 2-by-2 and a stride of two.

[0538] The fourth convolution block CB4 is connected to the fifth convolution block CB5, which comprises three convolutional layers CL51, CL52, CL53, each having 512 channels of 3-by-3 kernels and same padding, and a max-pooling layer PL5 of 2-by-2 and a stride of two.

[0539] The fifth convolution block CB5 is connected to the first dense layer of the multilayer perceptron model MLPM via a flattening layer (not shown), which is configured to flatten the output of the fifth convolutional block CB5, meaning that the output of the fifth convolutional block is transformed into a vector. The vector is then received by a first dense layer DL of the multilayer perceptron model MLPM, which comprises 256 neurons.

[0540] The first dense layer DL1 is connected to the second dense layer DL2, which comprises 128 neurons that is fully connected to the first dense layer DLL The output of the second dense layer is connected to the output layer OL, which comprises a number of neurons corresponding to the number of footwear assembly characteristics classes that the model should be able to predict. The neurons of the output layer are fully connected to the neurons of the second dense layer DL2. The output layer outputs a classification output CO, which is the relative probability for each of the footwear assembly characteristics classes given the input measurement IM. Each of the neurons of the output layer OL represents a footwear assembly characteristics class. Therelative probability of the footwear assembly characteristics classes is calculated by the soft-max activation function, utilized by the output layer OL.

[0541] Being a convolutional neural network type classification model, the VGG16 classification model requires the input measurements to be in matrix format.

[0542] Optionally, classification models of the invention may be trained based on categorical cross-entropy loss function. Further optionally, the RMSprop may be utilized to control learning rate. Also, optionally, class weights may be applied to advantageously handle imbalance.

[0543] Optionally, transfer learning may be applied to the VGG16 classification model. Transfer learning may be based on, e.g., the ImageNet dataset, but may also be based on other datasets.

[0544] In addition to the classification models described in relation to the figures, other supervised classification models may advantageously be utilized in different embodiments of the invention. The classification models may receive input measurements in the form of raw input measurements or in the form of input measurements that have been preprocessed in various ways. Furthermore, the classification models may not necessarily be a type of neural network. Indeed, other types of classification models may also provide solid performance, e.g., when training data is limited.

[0545] Non-limiting examples of such other types of classification models that may advantageously be used in embodiments of the invention includes, e.g., various types of decision trees, support vector machine models, probability based models, Bayesian statistical models, logistic models, elastic net models, gradient boosting models including, e.g., XGBoost.

[0546] Notice that these models may be trained using the training system of the invention. Furthermore, e.g., elastic net may optionally be used to handle regularization and to reduce the weight of “useless” features. Also, optionally principle component analysis (PCA) may be utilized to group features based on variance andthereby reduce the size of the input to the classification model. Thereby, advantageously reducing the computational requirements of the model and improving the computational speed of the classification of footwear assembly characteristics classes.

[0547] The performance of the classification models of the various embodiments of the invention may be evaluated based on different model performance metrics. Model performance metrics may, e.g., advantageously be based on or more of the following metrics: confusion matrix, type I error, type II error, accuracy, recall, precision and Fl -score, specificity, ROC (Receiver Operating Characteristics curve) curve, AUC (area under the curve) score, PR score, etc. The model performance may advantageously in some embodiments of the invention be automatically evaluated by a test module, which may calculate one or more of the model performance metrics.

[0548] Hyper parameters of the classification models may be tuned (sometimes also referred to as adjusted) in different ways. Optionally, the hyper parameters of the classification models may, e.g., be adjusted based on Grid and / or random search for optimal parameters, and / or based on cross validation. Other types of methods for adjusting hyper parameters may also be applied, according to different embodiments of the invention.

[0549] Optionally neural network based classification models may advantageously utilize one or more normalization layers, such as e.g., dropout layer. Advantageously this has the effect of regularizing the classification model, and thereby, it may improve the performance of the classification model, e.g. by avoiding overfitting to e.g. the training data.

[0550] In an example implementation, the VGG16 classification model can be trained and tested in regards to classifying footwear assembly characteristics classes based on image data acquired using an imaging system. In this particular example, the specific footwear assembly characteristics classes include the footwear assembly defect classes of abrasion, detachment and absence.

[0551] Using the imaging system, image data comprising digital color images of footwear assemblies comprising a known defect were acquired. Each of the acquired image data were evaluated and some were discarded. The remaining image data were used for training and testing of the classification model.

[0552] In a next step, the individual image data of different footwear assemblies were pre-processed according to resizing, cropping, normalization and standardization.

[0553] The pre-processed image data were then split such that 60% of the digital color images were used to generate a training dataset, and 40% of the digital color images were used to generate a test dataset. Each digital color image was then labelled according to the known defect associated with the particular footwear assembly i.e. the broad footwear assembly characteristics classes of abrasion, detachment and absence thereby resulting in labelled image data, wherein the portion of these utilized for the training data set was denoted as labelled training input measurements.

[0554] The VGG16 model was adopted as classification model. In this example, transfer learning was applied in the sense that the VGG16 model was pretrained based on the publicly available ImageNet dataset prior to being trained on the training data set comprising the examples of labelled training input measurements. The pretrained VGG16 was then trained based on the training data set, using backpropagation and the categorical cross entropy loss function. Root mean square propagation (RMSprop) was used as optimizer and class weights were used to handle imbalance.

[0555] The performance of the trained VGG16 classification model was then evaluated based on the test data set, wherein the VGG16 model was applied to classify the labelled test measurements of the test data set, and wherein the classification was compared with the correct label associated with each labelled test measurement. The performance of the VGG16 classification model was evaluated based on a confusion matrix (see table 1 below), and further based on the performance metrics: recall, precision and Fl (see table 2 below).

[0556] Table 1 below shows how a confusion matrix which may be generated based on the example test data set. The confusion matrix shows the classification of the testmeasurements in relation to the actual true label of the test measurement i.e. the diagonal of the matrix shows the percentage of correctly classified images of a particular class whereas the off-diagonal elements indicate test samples that were misclassified.Table 1 - Confusion matrix

[0557] Table 2 below shows examples of how the performance of the VGG16 classification model can be quantified based on the test data set. The applied performance metrics (recall, precision and Fl) are well accepted performance metrics widely applied to measure performance of classification models. The Macro avg parameter of Table 2 is the average score for the metrics across all three classes. The three performance metrics can be calculated as follows:Recall = TruePositives / (TruePositives + FalseNegatives) Precision = TruePositives / (TruePositives + FalsePositives) Fl = (2 * Precision * Recall) / (Precision + Recall)Table 2

[0558] Note that the present example has been presented as a multi-class classification problem for simplicity. In practice, two or more types of defects may ofcourse be present which would necessitate a reformulation of the problem as a multilabel classification problem.

[0559] In a second multi-label classification example, a neural network as described above with reference to figs. 25-26 is trained and tested for classifying the same footwear assembly characteristics classes as in the preceding example. In addition, the neural network is trained to classify each example of image data according to footwear manufacturing station classes. The classes therefore comprise two class groups pertaining respectively to footwear assembly characteristics and footwear manufacturing stations. The footwear identification representation associated with each footwear assembly is also fed to the model such that the one or more footwear manufacturing stations at which each assembly has been processed are known to the model.

[0560] The digital color images used for training set and test set were labelled according to the known defect associated with each particular footwear assembly, and with the identity of the known one or more footwear manufacturing stations at which the defect was caused. The portion of labelled image data utilized for the training data set was then denoted as labelled training input measurements.

[0561] The trained classification model is consequently able to provide a prediction regarding not only the type of footwear assembly defect but also regarding which one or more footwear manufacturing stations were likely to have caused the particular defect.

[0562] The trained classification model of the second example therefore executes both the step of determining a characteristics evaluation based on image data and associating the characteristics evaluation with one or more footwear manufacturing stations.

[0563] In a third example, a first classification model may be trained according to the first example, i.e. for classifying footwear assembly defects. The classification output obtained from such a model may then be used as an input to a second model which is trained based on this classification output and the footwear identificationrepresentations associated with each of the different footwear assemblies for which image data have been provided. The input to the second classification model is labelled according to footwear manufacturing station classes and used to train the second classification model to classify each of the input images according to the one or more footwear manufacturing stations that caused the defect.

[0564] List of reference signs:ASM Automatic sewing machineASSOC AssociatingC CharacteristicsCE Characteristics evaluationCEM Characteristics evaluation modelCL Convolutional layerCLA ClassifierCLM Classification moduleCM Classification modelCMO Classification model optimizerCMP Classification model parametersCNNCM Convolutional neural network classification modelCO Classification outputCOD Classification output determinerCONV ConveyorCS Classification systemDL Dense layerDP Data processorECM Error calculation moduleFA Footwear assemblyFACCl-n Footwear assembly characteristics classesFEM Feature extraction moduleFEO Feature extraction outputFIR Footwear identification representationFIRD Footwear identification readerFIRDB Footwear identification representation databaseFL Flattening layerFML Footwear manufacturing lineFMR Footwear manufacturing robotFMS Footwear manufacturing stationFMW Footwear manufacturing workerFP Footwear partHL Hidden layerHNl l-HNnn Hidden neuronIL Input layerIM Input measurementIMGD Image dataIMR Input measurement receiverINl-INm NeuronIS Imaging systemLCONV Loading conveyorLTIM Labelled training input measurementMLPM Multilayer perceptron modelNNM Neural network modelNO Neuronal outputONl-ONn Output neuronPS Processing stepsPL Pooling layerRTP Return transportation pathSEQ SequenceSRV ServerTCM Trained classification modelTOO Training classification outputTDS Training datasetTDSG Training dataset generatorTE Training errorTIL Training input labelsTIM Training input measurementTIML Training input measurement labelerTM Training moduleTP Transportation pathTS Training systemUCMP Updated classification model parametersWl-Wn WeightCMS 1 -CM3 Classification method step TMS1-TMS4 Training method step

Claims

1. Claims1. A method of determining the characteristics of a footwear assembly (FA), comprising:- processing said footwear assembly (FA) at a first subset of one or more footwear manufacturing stations (FMS) according to a sequence (SEQ) of processing steps (PS);- establishing and storing a footwear identification representation (FIR) associated with said footwear assembly (FA);- receiving from an imaging system (IS) an output of image data (IMGD) of said footwear assembly (FA);- determining a characteristics evaluation (CE) quantifying said characteristics of said footwear assembly (FA) by analyzing said image data (IMGD) on the basis of a characteristics evaluation model (CEM);- associating (ASSOC) said characteristics evaluation (CE) with a second subset of said one or more footwear manufacturing stations (FMS) based on said footwear identification representation (FIR).

2. A method according to claim 1, wherein said footwear assembly (FA) comprises one or more footwear parts (FP).

3. A method according to any of the preceding claims, wherein said footwear assembly (FA) comprises an identification marking corresponding to said footwear identification representation (FIR), and wherein said identification marking is attached to said footwear assembly (FA).

4. A method according to any of the preceding claims, wherein said footwear assembly (FA) comprises footwear.

5. A method according to any of the preceding claims, wherein said footwear assembly (FA) comprises footwear mounted on a last.

6. A method according to any of the preceding claims, wherein said footwear assembly (FA) comprises an upper mounted on a last.

7. A method according to any of the preceding claims, wherein said footwear assembly (FA) comprises a sole or sole part.

8. A method according to any of the preceding claims, wherein said footwear assembly (FA) comprises an upper or upper parts.

9. A method according to claim 8, wherein said upper or upper parts is made from an upper material, wherein said upper material is one or more of the upper materials selected from the list comprising: cotton, polyester, nylon, propylene, lycra and / or wool.

10. A method according to any of the preceding claims, wherein said footwear assembly (FA) comprises a midsole or midsole parts.

11. A method according to claim 10, wherein said midsole or midsole parts are made from a midsole material, wherein said midsole material is one or more of the midsole materials selected from the list comprising: thermoplastic rubber, polyurethane, and / or ethylene vinyl acetate.

12. A method according to any of the preceding claims, wherein said footwear assembly (FA) comprises an outsole or outsole parts.

13. A method according to claim 12, wherein said outsole or outsole parts are made from an outsole material, wherein said outsole material is one or more from the outsole materials selected from the list comprising: thermoplastic rubber, polyvinyl chloride, ethylene vinyl acetate, thermoplastic polyurethane, or thermoplastic elastomer.

14. A method according to any of the preceding claims, wherein said footwear assembly (FA) comprises an insole or insole parts.

15. A method according to any of the preceding claims, wherein said footwear assembly (FA) comprises joined footwear parts (FP).

16. A method according to any of the preceding claims, wherein said footwear assembly (FA) comprises loosely gathered footwear parts.

17. A method according to any of the preceding claims, wherein said footwear assembly (FA) undergoes at least one state change from a first type of footwear assembly (FA) to a second type of footwear assembly (FA) as it is processed according to said sequence (SEQ) of said processing steps (PS) at said first subset of said one or more footwear manufacturing stations (FMS).

18. A method according to any of the preceding claims, wherein said footwear assembly (FA) is transported between said one or more footwear manufacturing stations (FMS) by use of a footwear assembly carrier.

19. A method according to claim 18, wherein said method further comprises the step of providing image data (IMGD) of said footwear assembly carrier.

20. A method according to any of the claims 18-19, wherein said footwear assembly carrier is a last.

21. A method according to any of the claims 18-20, wherein said footwear assembly carrier is a mold.

22. A method according to claim 17, wherein said type of footwear assembly (FA) includes one or more selected from the list comprising upper, upper parts, midsole, midsole parts, outsole, outsole parts, insole, insole parts, joined footwear parts, loosely gathered footwear parts, last or mold.

23. A method according to any of the preceding claims, wherein said footwear assembly (FA) corresponds to one out of a plurality of footwear sizes.

24. A method according to any of the preceding claims, wherein said footwear assembly (FA) corresponds to one out of a plurality of footwear designs.

25. A method according to any of the preceding claims, wherein said sequence (SEQ) of said processing steps (PS) is predetermined for said footwear assembly (FA).

26. A method according to any of the preceding claims, wherein said sequence (SEQ) of said processing steps (PS) is uniquely determined for said footwear assembly (FA).

27. A method according to any of the preceding claims, wherein one or more of said processing steps (PS) of said sequence (SEQ) of processing steps (PS) are different.

28. A method according to any of the preceding claims, wherein said footwear identification representation (FIR) comprises a representation of said first subset of said one or more footwear manufacturing stations (FMS) at which said footwear assembly (FA) has been processed according to said sequence (SEQ) of said processing steps (PS).

29. A method according to any of the preceding claims, wherein said footwear identification representation (FIR) comprises a representation of said sequence (SEQ) of said processing steps (PS) according to which said footwear assembly (FA) has been processed.

30. A method according to any of the preceding claims, wherein said footwear identification representation (FIR) comprises a timestamp indicating the time at which one or more of said processing steps (PS) in said sequence (SEQ) occurred.

31. A method according to any of the preceding claims, wherein said footwear assembly (FA) is automatically identified by said footwear identification representation (FIR) at one or more of said first subset of said one or more footwear manufacturing stations (FMS).

32. A method according to any of the preceding claims, wherein said footwear identification representation (FIR) is automatically updating as said footwear assembly (FA) is processed at said first subset of said one or more footwear manufacturing stations (FMS) by reading a footwear identification code using a footwear identification reader (FIRD), and wherein said footwear identification code is attached to said footwear assembly (FA).

33. A method according to any of the preceding claims, wherein said footwear identification representation (FIR) is automatically updating as said footwear assembly(FA) is processed at said first subset of said one or more footwear manufacturing stations (FMS) by reading a footwear identification code using a footwear identification reader (FIRD), and wherein said footwear identification code is attached to a footwear assembly carrier (FAC) to which said footwear assembly (FA) is connected.

34. A method according to any of the preceding claims, wherein establishing and storing said footwear identification representation (FIR) is performed on a server (SRV).

35. A method according to any of the preceding claims, wherein said image data (IMGD) are of said footwear assembly (FA) at the end of said sequence (SEQ) of processing steps (PS).

36. A method according to any of the preceding claims, wherein said imaging system (IS) is located such that it provides said image data (IMGD) of said footwear assembly (FA) that has been processed according to a final processing step in said sequence (SEQ) of said processing steps (PS).

37. A method according to any of the preceding claims, wherein said image data (IMGD) are of said footwear assembly (FA) at the beginning of said sequence (SEQ) of said processing steps (PS).

38. A method according to any of the preceding claims, wherein said imaging system (IS) is placed such that it provides said image data (IMGD) of said footwear assembly (FA) after said footwear assembly (FA) has been processed at one or more of said first subset of said one or more footwear manufacturing stations (FMS).

39. A method according to any of the preceding claims, wherein said imaging system (IS) is distributed across two or more of said one or more footwear manufacturing stations (FMS).

40. A method according to any of the preceding claims, wherein said imaging system (IS) is integrated with a third subset of said one or more footwear manufacturing stations (FMS).

41. A method according to claim 40, wherein said imaging system (IS) provides said image data (IMGD) of said footwear assembly (FA) after said footwear assembly (FA) has been processed according to one or more of said processing steps (PS) at said third subset of said one or more footwear manufacturing stations (FMS) with which said imaging system (IS) is integrated.

42. A method according to claim 40, wherein said imaging system (IS) provides said image data (IMGD) of said footwear assembly (FA) before said footwear assembly (FA) has been processed according to one or more of said processing steps (PS) at said third subset of said one or more footwear manufacturing stations (FMS) with which said imaging system (IS) is integrated.

43. A method according to any of the preceding claims, wherein said imaging system (IS) is configured to provide said image data (IMGD) of said footwear assembly (FA) before said footwear assembly (FA) has been processed at said first subset of said one or more footwear manufacturing stations (FMS).

44. A method according to any of the preceding claims, wherein said imaging system (IS) is configured to provide said image data (IMGD) of said footwear assembly (FA) according to different subsets of said one or more footwear manufacturing stations (FMS) and different sequences of processing steps.

45. A method according to any of the preceding claims, wherein said imaging system (IS) is one of a plurality of imaging systems (IS) each configured to provide image data (IMGD) of said footwear assembly (FA).

46. A method according to any of the preceding claims, wherein said imaging system (IS) is configured to identify said footwear assembly (FA) by linking said footwear assembly (FA) with said footwear identification representation (FIR).

47. A method according to any of the preceding claims, wherein said imaging system (IS) is configured to perform one or more image processing steps as part of an image processing pipeline before providing said image data (IMGD).

48. A method according to any of the preceding claims, wherein said imaging system (IS) is a digital imaging system.

49. A method according to any of the preceding claims, wherein said imaging system (IS) comprises a camera.

50. A method according to claim 49, wherein said camera comprises an adjustable camera pose.

51. A method according to any of the preceding claims, wherein said imaging system (IS) comprises a lighting system.

52. A method according to any of the preceding claims, wherein said imaging system (IS) comprises optical filters.

53. A method according to any of the preceding claims, wherein said imaging system (IS) comprises optics.

54. A method according to any of the preceding claims, wherein said imaging system (IS) comprises a processor.

55. A method according to claim 54, wherein said processor is one or more of the processor types selected from the list comprising: central processing unit CPU, graphics processing unit GPU, application-specific integrated circuit ASIC, digital signal processor DSP, field-programmable gate array FPGA, or microcontroller.

56. A method according to any of the preceding claims, wherein said imaging system (IS) comprises a manipulator configured to adjust the pose of said footwear assembly (FA).

57. A method according to any of the preceding claims, wherein said imaging system (IS) comprises a box configured to provide consistent illumination of said footwear assembly (FA).

58. A method according to any of the preceding claims, wherein said imaging system (IS) comprises a communication module.

59. A method according to any of the preceding claims, wherein said imaging system (IS) comprises one or more sensors configured to provide an output of sensor data.

60. A method according to claim 59, wherein said one or more sensors are one or more of the types of sensors selected from the list comprising: temperature sensors, humidity sensors, elastic displacement sensors, contact thickness gauge, vibration sensors, capacitive sensors, or inductive sensors.

61. A method according to any of the preceding claims, wherein said image data (IMGD) comprises one or more of the image types selected from the list comprising: 2D images, video sequences, 3D images, 3D point clouds, 2D laser scans.

62. A method according to any of the preceding claims, wherein said image data(IMGD) comprise metadata.

63. A method according to any of the preceding claims, wherein said image data(IMGD) comprise digital color images.

64. A method according to any of the preceding claims, wherein said image data(IMGD) comprise digital grayscale images.

65. A method according to any of the preceding claims, wherein said image data(IMGD) comprises digital binary images.

66. A method according to any of the preceding claims, wherein said first subset of said one or more footwear manufacturing stations (FMS) each perform one or more of said processing steps (PS) from said sequence (SEQ) of processing steps (PS) on said footwear assembly (FA).

67. A method according to any of the preceding claims, wherein said second subset of said one or more footwear manufacturing stations (FMS) is a subset of said first subset of said one or more footwear manufacturing stations (FMS).

68. A method according to any of the preceding claims, wherein said footwear assembly (FA) is transported between said first subset of said one or more footwearmanufacturing stations (FMS) according to a routing based on said footwear identification representation (FIR).

69. A method according to claim 68, wherein said routing is predetermined and defines said first subset of said one or more footwear manufacturing stations (FMS) at which said footwear assembly (FA) is processed according to said sequence (SEQ) of said processing steps (PS).

70. A method according to claim 68, wherein said routing is dynamic such that said first subset of said one or more footwear manufacturing stations (FMS) is adaptable while said footwear assembly (FA) is being processed according to said sequence (SEQ) of said processing steps (PS).

71. A method according to claim 68, wherein said routing of said footwear assembly (FA) is adapted according to a desired manufacturing order of said footwear assembly (FA).

72. A method according to any of the preceding claims, wherein at least one footwear manufacturing station (FMS) of said first subset of said one or more footwear manufacturing stations (FMS) is a redundant footwear manufacturing station (FMS).

73. A method according to any of the preceding claims, wherein two or more footwear manufacturing stations (FMS) of said first subset of said one or more footwear manufacturing stations (FMS) are redundant footwear manufacturing stations (FMS).

74. A method according to any of the preceding claims, wherein one or more of said first subset of said one or more footwear manufacturing stations (FMS) comprise one or more human operators.

75. A method according to any of the preceding claims, wherein one or more of said one or more footwear manufacturing stations comprise a footwear identification representation reader (FIRD) configured for reading and updating said footwear identification representation (FIR) associated with said footwear assembly (FA).

76. A method according to claim 75, wherein said footwear identification representation reader (FIRD) is a radio-frequency identification RFID reader.

77. A method according to any of the claims 75-76, wherein said footwear identification representation reader (FIRD) is a bar code reader.

78. A method according to any of the preceding claims, wherein said footwear manufacturing station (FMS) comprises one or more sensors.

79. A method according to any of the preceding claims, wherein said characteristics evaluation (CE) comprises one or more characteristics measures.

80. A method according to any of the preceding claims, wherein said one or more characteristics measures relate to the presence or absence of parts of said footwear assembly (FA).

81. A method according to any of the claims 79-80, wherein said one or more characteristics measures relate to the relative pose of two or more parts of said footwear assembly (FA).

82. A method according to any of the claims 79-80, wherein said one or more of characteristics measures relate to surface characteristics of one or more parts of said footwear assembly.

83. A method according to any of the preceding claims, wherein said characteristics evaluation (CE) comprises a mapping to a sequence (SEQ) of processing steps (PS).

84. A method according to any of the preceding claims, wherein said characteristics evaluation (CE) comprises an indication of defects of said footwear assembly (FA).

85. A method according to any of the preceding claims, wherein said characteristics evaluation (CE) comprises a digital model of a footwear assembly.

86. A method according to any of the preceding claims, wherein said characteristics evaluation model (CEM) implements an image processing model.

87. A method according to any of the preceding claims, wherein said characteristics evaluation model (CEM) implements a digital image processing pipeline.

88. A method according to claim 87, wherein said digital image processing pipeline implements algorithms comprising one or more image processing steps selected from the list comprising: preprocessing, segmentation, postprocessing, visualization, or measurement.

89. A method according to any of the preceding claims, wherein said characteristics evaluation model (CEM) is a machine learning model.

90. A method according to any of the preceding claims, wherein said characteristics evaluation model (CEM) is a trained classification model (TCM) providing a classification output (CO) corresponding to said characteristics evaluation (CE).

91. A method according to claim 90, wherein said trained classification model (TCM) is configured to provide a classification output (CO) by performing the steps of:- receiving an input measurement (IM) at least based on said image data (IMGD);- classifying said characteristics of said footwear assembly (FA) into at least one of a plurality of predefined footwear assembly characteristics classes (FACC) using said trained classification model (TCM) to provide a classification output (CO).

92. A method according to any of the claims 90-91, wherein said trained classification model (TCM) is a neural network.

93. A method according to any of the claims 90-91, wherein said trained classification model (TCM) is a convolutional neural network (CNNCM).

94. A method according to any of the claims 90-93, wherein said trained classification model (TCM) comprises a feature extraction module (FEM).

95. A method according to any of the claims 90-94, wherein said trained classification model (TCM) comprises:- a feature extraction module (FEM) configured to receive training input measurements (TIM) and to generate a feature extraction output (FEO) based on said training input measurements (TIM); and- a classification module (CLM) configured to classify footwear assembly characteristics classes (FACC) based on the feature extraction output (FEO) of the feature extraction module (FEM).

96. A method according to any of the preceding claims, wherein said step of associating (ASSOC) said characteristics evaluation (CE) with said second subset of said one or more footwear manufacturing stations (FMS) based on said footwear identification representation (FIR) is performed by said characteristics evaluation model (CEM).

97. A method according to any of the claims 91-95, wherein said footwear assembly characteristics classes (FACC) correspond to at least one of said characteristics of said footwear assembly (FA).

98. A method according to any of the claims 91-95, wherein said footwear assembly characteristics classes (FACC) correspond to at least one of said second subset of said one or more footwear manufacturing stations (FMS).

99. A method according to any of the claims 90-95, wherein said trained classification model (TCM) is a first trained classification model, and wherein a second trained classification model is configured to provide a second classification output by performing the steps of:- receiving said classification output (CO) from said first trained classification model;- receiving said footwear identification representation (FIR);- classifying said second subset of said one or more footwear manufacturing stations into at least one of a plurality predefined footwear manufacturing station classes using said second trained classification model to provide a second classification output.

100. A method according to any of the claims 90-99, wherein said step of associating (ASSOC) said characteristics evaluation (CE) with said second subset of said one or more footwear manufacturing stations (FMS) comprises: providing a second trained classification model configured to classify predefined footwear manufacturing station classes; receiving one or more input measurements at least based on image data of a footwear assembly; classifying said second subset of said one or more footwear manufacturing stations into at least one of a plurality predefined footwear manufacturing station classes based on said one or more input measurements to provide a second classification output.

101. A method according to claim 100, wherein said step of classifying said second subset of said one or more footwear manufacturing stations into at least one of a plurality of predefined footwear manufacturing station classes is further based on said footwear identification representation (FIR).

102. A method according to any of the claims 100-101, wherein said step of classifying said second subset of said one or more footwear manufacturing stations into at least one of a plurality of predefined footwear manufacturing station classes is further based on said classification output of said trained classification model.

103. A method according to any of the claims 100-102, wherein said second trained classification model is different from said trained classification model.

104. A method according to any of the claims 100-103, wherein said trained classification model and / or said second trained classification model comprises at least one pooling layer.

105. A method according to any of the preceding claims, wherein associating (ASSOC) said characteristics evaluation (CE) with said second subset of said one or more footwear manufacturing stations (FMS) is automatic.

106. A method according to any of the preceding claims, wherein associating (ASSOC) said characteristics evaluation (CE) with said second subset of said one or more footwear manufacturing stations (FMS) comprises associating (ASSOC) said characteristics evaluation (CE) with said sequence (SEQ) of said processing steps (PS).

107. A method according to any of the preceding claims, wherein said method further comprises associating (ASSOC) said characteristics evaluation (CE) with one or more materials of said footwear assembly (FA).

108. A method according to any of the preceding claims, wherein said method further comprises the step of providing an alert comprising said footwear identification representation (FIR) and said second subset of said one or more footwear manufacturing stations (FMS) associated with said characteristics evaluation (CE).

109. A method according to any of the preceding claims, wherein said characteristics evaluation model (CEM) executes said step of associating (ASSOC) said characteristics evaluation (CE) with a second subset of one or more footwear manufacturing stations (FMS) based on said footwear identification representation (FIR).

110. A method according to any of the preceding claims, wherein one or more of said processing steps (PS) performed at said second subset of said one or more footwear manufacturing stations (FMS) are adjusted based on said characteristics evaluation (CE).

111. A method according to any of the preceding claims, wherein one or more of said processing steps (PS) performed at said second subset of said one or more footwear manufacturing stations (FMS) are automatically adjusted.

112. A method according to any of the preceding claims, wherein a conveyor (CONV) transports said footwear assembly (FA) between said one or more footwear manufacturing stations (FMS) for processing according to said sequence (SEQ) of said processing steps (PS).

113. A method according to any of the preceding claims, wherein a conveyor (CONV) transports said footwear assembly (FA) between said one or more footwear manufacturing stations (FMS).

114. A method according to any of the preceding claims, wherein a conveyor (CONV) transports said footwear assembly (FA) on a footwear assembly carrier between said one or more footwear manufacturing stations (FMS).

115. A method according to any of the preceding claims, wherein a system controller is configured to route said footwear assembly (FA) between said first subset of said one or more footwear manufacturing stations (FMS).

116. A method according to claim 115, wherein said system controller is further configured to track and update said footwear identification representation (FIR) at said first subset of said one or more footwear manufacturing stations (FMS).

117. A method according to any of the claims 115-116, wherein said system controller is further configured to track and update said footwear identification representation (FIR) while said footwear assembly (FA) is processed at said first subset of said one or more footwear manufacturing stations (FMS) according to said sequence (SEQ) of said processing steps (PS).

118. A footwear manufacturing line (FML) comprising: one or more footwear manufacturing stations (FMS) configured to process a plurality of footwear assemblies; wherein a footwear assembly (FA) out of said plurality of footwear assemblies is processed according to a sequence (SEQ) of processing steps (PS) at a first subset of said one or more footwear manufacturing stations (FMS); wherein each of said plurality of footwear assemblies are associated with a respective footwear identification representation (FIR); an imaging system (IS) configured to provide image data (IMGD) of said footwear assembly (FA);wherein said imaging system (IS) is communicatively coupled with a data processor (DP) executing a characteristics evaluation model (CEM) to determine a characteristics evaluation (CE); wherein said characteristics evaluation (CE) is associated with a second subset of said one or more footwear manufacturing stations (FMS) on the basis of said footwear identification representation (FIR).

119. A footwear manufacturing line according to claim 118, wherein said footwear manufacturing line (FML) is operated according to any of the claims 1-117.

120. A footwear manufacturing line according to any of the claims 118-119, wherein said data processor (DP) is a server (SRV) configured to execute said characteristics evaluation model (CEM).

Images