Computer architecture for generating footwear digital assets

Abstract

A computing machine accesses an input image of footwear, the input image being a pixel or voxel-based image. Based on the input image, the computing machine uses a machine learning engine to generate a 3D (three-dimensional) CAD (computer-aided design) digital asset including a NURBS (non-uniform rational basis spline) model of the footwear. Generating the 3D CAD digital asset includes: identifying visual components of the footwear using a trained classification engine; and iteratively fitting splines for the visual components of the footwear using a trained reinforcement learning engine until the splines are within a predetermined average distance from the visual components. The 3D CAD digital asset includes the splines. The machine learning engine includes a trained classification engine and a trained reinforcement learning engine. The computing machine provides an output representing the 3D CAD digital asset.

Description

Technical Field

[0001] Embodiments relate to computer architecture. Some embodiments relate to machine learning. Some embodiments relate to using machine learning to generate digital assets representing footwear.

[0002] This application claims the benefit of priority of U.S. Provisional Patent Application No. 63 / 089,435, filed Oct. 8, 2020, the entire content of which is incorporated herein by reference.

Background Art

[0003] The design of footwear is often complex. A footwear item can include a plurality of different components, each with a different shape and position. A digital representation of footwear may be preferred.

Summary of the Invention

[0004] The detailed description and drawings below fully illustrate specific embodiments and are presented so that those skilled in the art can implement them. Other embodiments can incorporate structural, logical, electrical, process, and other changes. Portions and features of some embodiments can be included in or substituted for those of other embodiments. The embodiments described in the claims cover any equivalents of those claims.

[0005] As described above, for example, the design of apparel such as footwear is often complex. An apparel item can include a plurality of different components, each with a different shape and position. Therefore, generating a 3D computer-aided design (CAD) digital model for a hand-designed 2D or 3D apparel model can be a complex process. Such a digital model can be used, for example, to generate instructions during the complex process of manufacturing footwear. As illustrated above, a digital representation of apparel may be preferred.

[0006] Some embodiments relate to generating an apparel CAD file for manufacturing apparel (e.g., from an image, model, or other visual data). According to some embodiments, a computing machine accesses an input image of apparel such as footwear. The input image is a pixel or voxel-based image. The computing machine generates a 3D CAD digital asset comprising a non-uniform rational basis spline (NURBS) model for the apparel, based on the input image and using one or more machine learning engines implemented on the computing machine. Generating the 3D CAD digital asset includes identifying visual components of the apparel using a trained classification engine. Generating the 3D CAD digital asset also includes iteratively fitting a spline to the visual components of the apparel using a trained reinforcement learning engine until the spline stays within a predetermined average distance from the visual components. The 3D CAD digital asset includes the spline. The machine learning engines include a trained classification engine and a trained reinforcement learning engine. The computing machine provides an output representing the 3D CAD digital asset.

[0007] As used herein, a NURBS model can include a mathematical representation of a 2D or 3D object, which can be a standard shape (e.g., a cone) or a freeform shape (e.g., a vehicle).

[0008] A spline can include a curve pattern used to guide the formation of large objects such as ship hulls. A B-spline (where B is short for basis) is based on four (or another number) of local functions or control points existing outside the curve. Non-uniform (NU) can refer to the ability to stretch or shrink a section of a defined shape (between any two points) in relation to other sections of the overall shape. Rational (R) can represent the ability to assign more weight to some points of a shape than others when considering the relationship of each position to another object. (This can be called a 4D characteristic.)

[0009] A spline can be defined using control points and a mathematical function based on the control points. The graphical representation form of a spline can be calculated mathematically based on the control points. The control points determine the shape of the curve corresponding to the mathematical function.

[0010] Aspects of the present invention can be implemented as part of a computer system. The computer system can be a single physical machine or, in the case of a cloud computing distributed model, can be distributed among multiple physical machines, for example, by roles or functions or process threads. In various embodiments, aspects of the present invention can be configured to be run within virtual machines, and these virtual machines are executed on one or more physical machines. Those skilled in the art will understand that the features of the present invention can be embodied by various different suitable machine implementations.

[0011] The system includes various engines, each of which is constructed, programmed, configured, or otherwise adapted to execute a function or set of functions. As used herein, the term engine means a tangible device, component, or combination of components, which may be implemented by using hardware such as, for example, ASICs or FPGAs, or may be implemented by a combination of hardware and software such as, for example, a processor-based computing platform and program instructions that cause the computing platform to implement a specific function by turning it into a dedicated-purpose device. The engine may also be implemented as a combination of these two, where certain functions are realized by hardware only and other functions may be realized by a combination of hardware and software.

[0012] By way of example, software can exist on a tangible machine-readable storage medium in executable form or in non-executable form. Software existing in non-executable form can be compiled, converted, or otherwise transformed into executable form prior to or during execution. By way of example, when executed by the underlying hardware of an engine, the software causes the hardware to perform a specified operation. Thus, the engine is physically constructed, specifically configured (e.g., hard-wired), or temporarily configured (e.g., programmed) to operate in a specified manner or to perform all or part of any operation described in connection with that engine.

[0013] To consider an example where an engine is temporarily configured, each engine can be instantiated at different times. For example, if an engine comprises a general-purpose hardware processor core configured using software, the general-purpose hardware processor core can be configured as each different engine at different times. Thus, for example, software can cause a particular engine to be configured in an instance of the hardware processor core at one time, and a different engine to be configured in an instance at a different time.

[0014] In a particular implementation example, at least a part of the engine, and in some cases all of it, can be executed on the processors of one or more computers that execute an operating system, system programs, and application programs. In this case, when appropriate, the engine can also be implemented using multitasking, multithreading, distributed (e.g., cluster, P2P, cloud, etc.) processing, or other such techniques. Thus, each engine can be realized with various appropriate configurations and, generally, does not need to be limited to any particular embodiment exemplified in this specification without special notice.

[0015] In addition, it should be noted that an engine can itself be composed of one or more sub-engines, each of which can itself be regarded as an engine. Also, in the embodiments described in the present disclosure, each of the various engines corresponds to a defined function. However, it should also be noted that in other assumed embodiments, each function can be distributed among one or more engines. Similarly, in other assumed embodiments, a plurality of defined functions can be implemented by a single engine responsible for these functions, and in some cases, can be shared with other functions or distributed to a set of engines in a manner different from the examples specifically exemplified in this specification.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

DETAILED DESCRIPTION OF THE INVENTION

[0017] FIG. 1 shows the training and use of a machine learning program according to some exemplary embodiments. In some exemplary embodiments, a machine learning program (MLP), also referred to as a machine learning algorithm or tool, is utilized to perform operations associated with machine learning tasks such as image recognition and machine translation.

[0018] Machine learning is a field of study that enables a computer to learn without explicit programming. Machine learning is concerned with the study and construction of algorithms, also referred to as tools, which can learn from existing data and make predictions about new data. Such machine learning tools function by constructing a model from exemplary training data 112, which is done to make data-driven predictions or decisions referred to as output or evaluation 120. Although exemplary embodiments are presented in the context of a few machine learning tools, the principles presented herein can also be applied to other machine learning tools.

[0019] In some embodiments, different machine learning tools can be used. For example, logistic regression (LR), naive Bayes, random forest (RF), neural network (NN), matrix factorization, and support vector machine (SVM) tools can be used for job posting classification and scoring.

[0020] Two common problems in machine learning are classification problems and regression problems. A classification problem, also referred to as a categorization problem, attempts to classify an item into one of a plurality of category values (e.g., whether an object is an apple or an orange). A regression algorithm attempts to quantify several items (e.g., by providing a numerical value that is a real number). Machine learning algorithms utilize training data 112 to determine correlations among identified features 102 that affect an outcome.

[0021] Machine learning algorithms utilize features 102 for data analysis to generate an evaluation 120. A feature 102 is a measurable individual property of an observed event. The concept of "feature" is related to explanatory variables used in statistical techniques such as linear regression. Selecting features that have information, discriminative power, and independence is crucial for the effective operation of an MLP in pattern recognition, classification, and regression. Features can be of different types, such as numerical features, character strings, graphs, etc.

[0022] In one exemplary embodiment, features 102 can be of different types and can include, for example, one or more of the following: words 103 of a message, concepts 104 of a message, communication history 105, past user behavior 106, subject 107 of a message, other message attributes 108, sender 109, and user data 110.

[0023] The machine learning algorithm utilizes training data 112 to determine correlations among the identified features 102 that affect the result or evaluation 120. In some exemplary embodiments, the training data 112 includes labeled data, which is known data about one or more identified features 102 and one or more results, and can include, for example: detection of communication patterns, detection of message meaning, generation of message summaries, detection of action items within a message, detection of urgency within a message, detection of a user's relationship to the sender, calculation of score attributes, calculation of message scores, and the like.

[0024] With the training data 112 and the identified features 102, the machine learning tool is trained at operation 114. The machine learning tool evaluates the values of the features 102 according to their correlations with the training data 112. As a result of the training, a trained machine learning program 116 is produced.

[0025] When the machine learning program 116 is used for evaluation, new data 118 is provided as input to the trained machine learning program 116, and the machine learning program 116 generates an evaluation 120 as output. For example, when checking a message for action items, the machine learning program determines whether there is a request for an action within the message, utilizing the message content and message metadata.

[0026] In machine learning techniques, a model is trained to accurately make predictions about the data input into the model (e.g., what the matter is that the user has spoken in a given utterance; whether a noun is a thing, a place, or an object; what the weather will be like the next day, etc.). During the learning phase, the model is developed against an input training dataset, and the model is optimized to correctly predict the output for a given input. Generally, the learning phase can be supervised, semi-supervised, or unsupervised, indicating a decreasing level of providing the "correct" output corresponding to the training input. In the supervised learning phase, all outputs are provided to the model, and the model is instructed to formulate general rules or algorithms for mapping the input to the output. In contrast, in the unsupervised learning phase, the desired output is not provided to the input, and the model is allowed to formulate its own rules so that it can discover the relationships within the training dataset. In the semi-supervised learning phase, an incompletely labeled training set is provided, and some of the outputs for the training dataset are known and some are unknown.

[0027] The model can be run on the training dataset for multiple epochs (e.g., iterations), during which the training dataset can be repeatedly fed into the model to refine the results. For example, in the supervised learning phase, a model that predicts the output for a given input set is constructed and evaluated over multiple epochs, and the output specified as corresponding to the given input for the maximum number of inputs regarding the training dataset is provided more stably. In another example, in the unsupervised learning phase, a model that clusters the dataset into n groups is constructed and evaluated for the model over multiple epochs, and the evaluation is made on how consistently the model assigns a given input to a given group and how reliably the model brings about n desired clusters over each epoch.

[0028] Once an epoch is executed, the model is evaluated and the values of those variables are adjusted in an iterative manner to attempt to better refine the model. In various ways, the evaluations can be biased towards false negatives, biased towards false positives, or evenly biased in relation to the overall accuracy of the model. Depending on the machine learning technique used, the values can be adjusted according to several methods. For example, in genetic or evolutionary algorithms, the values for the model to be used in subsequent epochs are constructed using the values for the model that is most productive in predicting the desired output, which can include random variations / mutations and additional data points can be provided. A person skilled in the art should be familiar with several other machine learning algorithms applicable to the present disclosure, which can include linear regression, random forest, decision tree learning, neural networks, deep neural networks, etc.

[0029] Each model tunes a rule or algorithm over multiple epochs, doing so by varying the values of one or more variables that affect the input to more faithfully map to a desired result. However, the training dataset can be rich in variation and is desirably quite large, so perfect accuracy and precision may not be achievable. Thus, the number of epochs that make up the learning phase can be set as a given number of trials or as a fixed time / computing constraint, or can be terminated before reaching that number / constraint if the accuracy of a given model is sufficiently high or low or has plateaued with respect to accuracy. For example, if the training phase is designed to run for n epochs to yield a model with at least 95% accuracy and such a model is yielded before the nth epoch, the learning phase can be terminated early and the model that meets the end-goal accuracy threshold can be used. Similarly, if a given model is inaccurate enough to meet a randomness threshold (e.g., if the model yields only about 55% accuracy in determining a positive / negative output for a given input), the learning phase for that model can be terminated early, although other models within the learning phase can continue training. Similarly, if a given model yields similar accuracy or results that fluctuate over multiple epochs (when performance has plateaued), the learning phase for the given model can be terminated before reaching the number of epochs / computing constraint.

[0030] Once the learning phase is complete, the model is finalized. In some exemplary embodiments, the finalized model is evaluated against test criteria. In a first example, a test data set including outputs known for its inputs is fed into the finalized model to determine the model's accuracy regarding the handling of data that it has not yet been trained on. In a second example, the model may be evaluated after finalization using a false positive rate or a false negative rate. In a third example, a model that results in the clearest boundary for the data cluster is selected using the delineation between data clusters.

[0031] FIG. 2 is a diagram showing an exemplary neural network 204 according to some embodiments. As shown, the artificial neural network 204 receives source domain data 202 as input. The input passes through a plurality of layers 206 to reach the output. Each layer 206 includes a plurality of neurons 208. The neurons 208 receive inputs from the previous layer and also apply weights to the values received from those neurons, thereby generating a neuron output. The neuron outputs from the final layer 206 are combined to generate the output of the artificial neural network 204.

[0032] As shown at the bottom of FIG. 2, the input is the vector x. The input passes through a plurality of layers 206, and weights W1, W2, …, W i are applied to the input to each layer, and f 1 (x), f 2 (x), …, f i-1 (x) are obtained, and this is done until the output f(x) is calculated.

[0033] In some exemplary embodiments, an artificial neural network 204 (e.g., a deep learning, deep convolutional, or recurrent neural network) comprises a series of neurons 208 arranged as a network such as, for example, a long short-term memory (LSTM) network. Neurons 208 are architectural elements used in data processing and artificial intelligence, particularly machine learning, and determine when to "remember" and when to "forget" the values stored in their memory based on the weights of the inputs provided to a given neuron 208. Each of the neurons 208 used herein is configured to receive a predetermined number of inputs from other neurons 208 within the artificial neural network 204 and provide relational and sub-relational outputs regarding the content of the frame being analyzed. The individual neurons 208 can be arranged in a chain and / or arranged in a tree structure to provide various configurations of the neural network and can provide interaction and relational learning modeling regarding how each frame within an utterance relates to one another.

[0034] For example, an LSTM node serving as a neuron includes several gates for handling an input vector (e.g., a phoneme from an utterance), a memory cell, and an output vector (e.g., a contextual representation). The input gate and the output gate control the information flowing into and out of the memory cell, respectively, while the forget gate optionally removes information from the memory cell based on the input from a preceding linked cell within the artificial neural network. The weights and bias vectors for the various gates are adjusted during the training phase, and once the training phase is completed, those weights and biases are finalized for normal operation. Those skilled in the art will appreciate that neurons and neural networks can be constructed programmatically (e.g., via software instructions) or via dedicated hardware, and individual neurons can be linked to form an artificial neural network.

[0035] A neural network utilizes features for data analysis to generate an evaluation (e.g., units-of-speech recognition). Features are measurable individual properties of the observed events. The concept of "features" is related to the explanatory variables used in statistical methods such as linear regression. Furthermore, deep features represent the outputs of the nodes within the hidden layers of a deep neural network.

[0036] A neural network, sometimes also referred to as an artificial neural network, is a computing system / device based on the study of the biological neural network of an animal's brain. Such a system / device improves its performance progressively, which is also referred to as learning, and typically performs tasks without task-specific programming. For example, in image recognition, a neural network can be trained to identify an image containing an object, which can be done by analyzing images as examples tagged with the name of the object. Once it has learned about the object and the name, it can use the analysis results to identify the object in an untagged image. An artificial neural network is based on a collection of connected units called neurons, and each connection between neurons, called a synapse, can transmit a unidirectional signal with an activation strength that varies according to the strength of the connection. The receiving neuron can typically activate and propagate a signal to the downstream neurons connected to itself based on whether the received signal, which is potentially a combination of signals from a number of transmitting neurons, is of sufficient strength, where the strength is a parameter.

[0037] A deep neural network (DNN) is a stacked neural network composed of multiple layers. The layers are composed of nodes, which are areas where calculations are performed, and are roughly based on neurons in the human brain, which emit when faced with sufficient stimuli. A node combines an input from data with a set of coefficients or weights that amplify or attenuate that input, thereby giving importance to the input with respect to the task the algorithm is trying to learn. The product of these input-weights is summed, and that sum is passed through what is called the activation function of the node, which determines whether and to what extent the signal further progresses into the network with respect to affecting the final result. A DNN uses a cascade of multiple layers of non-linear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. High-level features are derived from low-level features, forming a hierarchical representation. The layer following the input layer can be a convolutional layer that results in a feature map that is the filtering result for the input and is used by the next convolutional layer.

[0038] In the training of a DNN architecture, regression, which is configured as a set of statistical processes for estimating the relationships between variables, can involve minimizing a cost function. The cost function can be implemented as a function that returns a numerical value representing how well the artificial neural network functioned when mapping training examples to the correct output. During training, when the value of the cost function is not within a predetermined range, backpropagation is used based on known training images, where backpropagation is a common method for training an artificial neural network used in conjunction with an optimization method such as the stochastic gradient descent (SGD) method.

[0039] The use of backpropagation may include propagation and weight updates. When an input is presented to an artificial neural network, it is propagated forward through the artificial neural network layer by layer until it reaches the output layer. Then, the output of the artificial neural network is compared with the desired output using a cost function, and an error value is calculated for each node in the output layer. The error value is backpropagated, starting from the output, until each node has an associated error value, which roughly represents its contribution to the original output. In backpropagation, these error values can be used to calculate the gradient of the cost function with respect to the weights in the artificial neural network. The calculated gradient is fed into a selected optimization method to update the weights and minimize the cost function.

[0040] Figure 3 is a schematic diagram showing the training of an image recognition machine learning program according to some embodiments. The machine learning program can be implemented on one or more computing devices. A training set is shown in block 302, which includes a plurality of classes 304. Each class 304 includes a plurality of images 306 associated with the class. Each class 304 may correspond to the type of object in the image 306 (e.g., numbers from 0 to 9, male or female, dog or cat, etc.). In one example, the machine learning program is trained to recognize images of US presidents, and each class corresponds to a respective president (e.g., one class corresponds to Barack Obama, one class corresponds to George W. Bush, etc.). In block 308, the machine learning program is trained using, for example, a deep neural network. The trained classifier generated by the training in block 308 recognizes the image 312 in block 310, and the image is recognized in block 314. For example, if the image 312 is a photo of Bill Clinton, the classifier recognizes the image as corresponding to Bill Clinton in block 314.

[0041] Figure 3 shows the training of a classifier according to some exemplary embodiments. The machine learning algorithm is designed to recognize faces, and the training set 302 includes data that maps samples to a class 304 (e.g., the class includes all images of purses). A class can also be referred to as a label. Although the embodiments presented in this application are presented in the context of object recognition, the same principles can be applied to train a machine learning program for recognizing any type of item.

[0042] The training set 302 includes a plurality of images 306 (e.g., image 306) for each class 304, and each image is associated with one of the categories (e.g., class) to be recognized. The machine learning program is trained with the training data (308) to generate a classifier 310 operable to recognize images. In some embodiments, the machine learning program is a DNN.

[0043] When the input image 312 is to be recognized, the classifier 310 analyzes the input image 312 to identify the class (e.g., class 314) corresponding to the input image 312.

[0044] Figure 4 shows the feature extraction process and classifier training according to some exemplary embodiments. The training of the classifier can be divided into a feature extraction layer 402 and a classifier layer 414. Each image is sequentially analyzed by a plurality of layers 406 - 413 within the feature extraction layer 402.

[0045] With the development of DNN, the main concern in face recognition has become learning a good-quality face feature space, that is, the faces of the same person are placed close to each other, and the faces of different people are placed far from each other. For example, the authentication task associated with the LFW (Labeled Faces in the Wild) dataset is often used for face verification.

[0046] Many face identification tasks (e.g., those associated with the MegaFace and LFW datasets) are based on similarity comparisons between images within a gallery set and a query set, which is essentially the KNN (K-nearest-neighborhood) method in person identity estimation. Ideally, the face feature extractor is good (the inter-class distance is always greater than the intra-class distance), and the KNN method is sufficient for person identity estimation.

[0047] Feature extraction is a process for reducing the amount of resources required to describe a large dataset. When analyzing complex data, one of the main problems stems from the large number of variables involved. In an analysis with multiple variables, a large amount of memory and computing power are generally required, and this can cause the classification algorithm to overfit to the training samples and show poor generalization for new samples. Feature extraction is a general term for describing a method of constructing a combination of variables that bypasses the problems related to the large dataset mentioned above, while still describing the data with sufficient accuracy in relation to the desired purpose.

[0048] In some exemplary embodiments, feature extraction begins with an initial set of measurement data, and derived values (features) that are intended to be informative and non-redundant are constructed, which aids subsequent learning and generalization steps. Additionally, feature extraction is related to dimensionality reduction, involving reducing a large vector (possibly with extremely sparse data in some cases) to a smaller vector that contains the same or a similar amount of information.

[0049] Determining a subset of the initial features is called feature selection. The selected features are expected to contain information relevant from the input data, so that this degenerate representation can be used to perform the desired task instead of the complete initial data. In a DNN, a stack of layers is utilized, and each layer undertakes a function. For example, a layer can undertake convolution, non-linear transformation, average calculation, etc. Eventually, this DNN yields an output by a classifier 414. In FIG. 4, the data is transmitted from left to right, and features are extracted. The purpose of training an artificial neural network is to find the parameters of all layers sufficient for the desired task.

[0050] As shown in FIG. 4, a filter with a "stride of 4" is applied in layer 406, and max pooling is applied in layers 407 - 413. The stride controls how the filter convolves around the input volume. A "stride of 4" means that the filter convolves around the input volume in units of 4 at a time. Max pooling refers to downsampling by selecting the maximum value within each max pooling region.

[0051] In some exemplary embodiments, the structure of each layer is predefined. For example, a convolutional layer can include small convolutional kernels and their respective convolutional parameters, and a sum layer can calculate the sum or weighted sum of two pixels of the input image. Training aids in defining the weight coefficients in the sum.

[0052] As one measure to improve the performance of DNN, identifying a newer structure for the feature extraction layer can be mentioned. As another measure, improving the manner in which parameters are identified in different layers for achieving a desired task can be mentioned. The problem is that for a typical neural network, there can be millions of parameters to be optimized. Optimizing all of these parameters from the beginning can take hours or days or weeks, depending on the amount of computing resources available and the amount of data in the training set.

[0053] FIG. 5 shows a circuit block diagram of a computing machine 500 according to some embodiments. In some embodiments, the components of the computing machine 500 can store or be integrated with other components shown in the circuit block diagram of FIG. 5. For example, the computing machine 500 can be located within a processor 502, and can also be referred to hereinafter as a “processing circuit”. The processing circuit can include, for example, processing hardware such as one or more CPUs and one or more GPUs. In alternative embodiments, the computing machine 500 can operate as a stand-alone device or be connected (e.g., networked) to other computers. In a networked deployment, the computing machine 500 can operate in the role of a server, a client, or both, in a server-client network environment. By way of example, the computing machine 500 can operate in the role of a peer machine in a P2P (or other distributed) network environment. In this document, the terms P2P, device-to-device (D2D), and sidelink can be used interchangeably. The computing machine 500 can be a dedicated computer, a personal computer (PC), a tablet PC, a personal digital assistant (PDA), a mobile phone, a smartphone, a web appliance, a network router, a switch, a bridge, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by that machine.

[0054] Examples of the present disclosure may include or operate based on logic or some components, modules, or mechanisms. Modules and components can be physical entities (e.g., hardware) that can perform a specified operation and can be configured or arranged in a certain manner. By way of example, a circuit can be arranged as a module in a specified manner (e.g., internally or in relation to external entities such as other circuits). By way of example, all or part of one or more computer systems / devices (e.g., stand-alone, client, or server computer systems) or one or more hardware processors can be configured as a module that operates to perform a specified operation by firmware or software (e.g., instructions, portions of an application, or an application). By way of example, software can reside on a machine-readable medium. By way of example, when executed by the underlying hardware of a module, the software causes the hardware to perform a specified operation.

[0055] Accordingly, the terms “module” (and “component”) include tangible entities, where the entities are physically constructed, specifically configured (e.g., hardwired), or temporarily (e.g., ephemerally) configured (e.g., programmed) to operate in a specified manner or to perform all or part of any of the operations described in this disclosure. Considering an example where a module is temporarily configured, each module need not be instantiated at any given point in time. For instance, if a module comprises a general-purpose hardware processor configured using software, the general-purpose hardware processor can be configured as each different module at different points in time. Thus, for example, software can cause a particular module to be configured with respect to the hardware processor at one instance of time and a different module to be configured at a different instance of time.

[0056] The computing machine 500 may include a hardware processor 502 (e.g., a CPU, GPU, hardware processor core, or any combination thereof), a main memory 504, and a static memory 506, all or some of which may communicate with each other via an interlink (e.g., a bus) 508. Although not shown, the main memory 504 may include any or all of removable storage, non-removable storage, volatile memory, or non-volatile memory. The computing machine 500 may further include a video display unit 510 (or other display unit), an alphanumeric input device 512 (e.g., a keyboard), and a user interface (UI) navigation device 514 (e.g., a mouse). By way of example, the display unit 510, the input device 512, and the UI navigation device 514 may be a touch screen display. The computing machine 500 may additionally include a storage device (e.g., a drive unit) 516, a signal generating device 518 (e.g., a speaker), a network interface device 520, and one or more sensors 521 such as a GPS sensor, a compass, an accelerometer, and other sensors. The computing machine 500 may include an output controller 528 such as a serial (e.g., USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection for communicating with or controlling one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0057] The drive unit 516 (e.g., a storage device) may include a machine-readable medium 522 storing one or more sets of data structures or instructions 524 (e.g., software) that embody or are utilized by any one or more of the techniques or functions of the present disclosure. The instructions 524, when executed by the computing machine 500, may be located, wholly or at least partially, within the main memory 504, or within the static memory 506, or within the hardware processor 502. By way of example, one or any combination of the hardware processor 502, the main memory 504, the static memory 506, or the storage device 516 may constitute a machine-readable medium.

[0058] The machine-readable medium 522 is shown as a single medium, but the term "machine-readable medium" may include a single medium or a plurality of media (e.g., a centralized or distributed database, and / or associated caches and servers) configured to store one or more instructions 524.

[0059] The term "machine-readable medium" may include any medium that can store, encode, or carry instructions for execution by the computing machine 500, cause the computing machine 500 to perform any one or more of the techniques of the present disclosure, or be used by or be associated with such instructions, and may include any medium that can store, encode, or carry the data structures used by such instructions. Non-limiting examples of machine-readable media may include solid-state memory, and optical and magnetic media. Specific examples of machine-readable media may include: non-volatile memory such as semiconductor memory devices (e.g., EPROM, EEPROM, etc.) and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; RAM; and CD-ROM and DVD-ROM disks. In some examples, the machine-readable medium may include a non-transitory machine-readable medium. In some examples, the machine-readable medium may include a machine-readable medium that is not a transitory propagated signal.

[0060] Command 524 can be further transmitted and received on communication network 526 using a transmission medium via network interface device 520 by leveraging any one of several transfer protocols (e.g., Frame Relay, Internet Protocol (IP), TCP, UDP, HTTP, etc.). Exemplary communication networks can include, among others: LAN, WAN, packet data networks (e.g., the Internet), cellular phone networks (e.g., cellular networks), plain old telephone (POTS) networks, and wireless data networks (e.g., the IEEE 802.11 standard family known as Wi-Fi (registered trademark), the IEEE 802.16 standard family known as WiMax), the IEEE 802.15.4 standard family, the LTE standard family, the UMTS standard family, P2P networks, etc. By way of example, network interface device 520 can include one or more physical jacks (e.g., Ethernet (registered trademark), coaxial, or telephone jacks) or one or more antennas for connecting to communication network 526.

[0061] Some embodiments are directed to computerized processes for using machine learning techniques to convert an image input (e.g., a design sketch, a photograph, or a 3D solid model) of a footwear (or other apparel) item into a 3D CAD digital asset (e.g., NURBS). Some embodiments are directed to processes that receive an input and generate a robust digital output that can be used for design inspiration, manufacturing, marketing, digital asset generation (e.g., for use in AR (augmented reality) / VR (virtual reality) and gaming environments, etc.). Some embodiments are disclosed in the context of footwear. However, the techniques disclosed herein are not limited to footwear and can be extended to any other apparel (e.g., jeans, leather jackets, skirts, etc.) or non-apparel items (e.g., tables, chairs, houses, bicycles, baseball mitts, hockey sticks, baseballs, basketballs, etc.).

[0062] At a high level, an image is provided to a machine learning engine (e.g., the one described in conjunction with FIGS. 1-4) for processing. The machine learning engine provides a NURBS model. The NURBS model can be used to provide an output (e.g., a 3D model used as a manufacturing instruction). Iterative refinement and adjustment can be made to the 3D model until some metric matches the original design intent.

[0063] As one exemplary problem, designers of footwear (and other apparel) have been creating their designs by sketching on paper or making molds without the use of 3D CAD systems. Creating and building 3D CAD designs is tedious and time-consuming and does not necessarily utilize the artistic talents of most footwear designers. Also, this does not fully utilize the skill set of 3D CAD engineers, and in other cases, manual iterative refinement and adjustment can be automatically processed with the advancement of machine learning systems. The effort and cost to convert or transform a design (e.g., 2D line art, a photo of a sketch, a sketch in a tablet, a photo, a mesh object, or a sculpture (e.g., a scan of an object)) into a rich 3D parametric “digital asset” can be prohibitive and time-consuming.

[0064] The 3D parametric representation of a design object (e.g., NURBS) has numerous advantages with respect to its manufacturability. Also, a process for making quicker and consistent changes can be used in conjunction with this. A rich 3D parametric asset can serve as a platform for cross-organizational digital collaboration. The 3D parametric asset can function as a building block for future use cases in AR or VR.

[0065] A process has been developed for implementing the process of converting NURBS to a mesh. In some cases, NURBS can contain significantly more information than the mesh format. This is because NURBS is essentially a mathematical model of geometry. Therefore, this NURBS information can be easily converted to a mesh representation (or other required digital representation for an object). In some cases, a computing machine can manipulate a template with NURBS. The manipulated template can include layers with information such as structural data. The manipulated template can contain more information than a cloud of points in a mesh or the like.

[0066] Some alternative approaches to creating 3D assets from a design can leverage the skill set of 3D CAD engineers. The CAD engineer begins by first reproducing or "copying" the initial design into a 3D CAD system. Most of the time-consuming and tedious work lies in iteratively refining and adjusting the 3D copy until some metric aligns with and satisfies the intent of the initial design (see below for metrics). An example of a metric could be a 3D orthographic projection onto a 2D plane to show faithfulness to the original 2D version.

[0067] High parameter curves can have a large number of shape specifying points (e.g., 10 or more, 20 or more, etc.). In contrast, splines can include low parameter curves and incorporate a specific relatively small number of control points enhanced by mathematical relationships, making splines more easily manipulable and less resource intensive. In one of the approaches described below, the input image is converted to a spline and the control points of the spline are manipulated to satisfy the target template range (e.g., based on constraints such as manufacturing constraints or footwear specific constraints) until a metric reaches a threshold.

[0068] Some aspects include pipeline processing for footwear design, which automates the rapid conversion of a design input (e.g., a sketch, photograph, or 3D solid model) to a 3D CAD digital asset (e.g., NURBS) through the use of machine learning. The pipeline includes multiple stages and services and can be orchestrated via a web application that provides an end-user experience. The stages within the pipeline can include: preprocessing, part identification, 3D NURBS construction, and save / convert.

[0069] In the preprocessing stage, the computing machine adapts the design input to each 2D orthographic view or projection. In the part identification stage, the computing machine performs a "part identification" procedure or semantic segmentation, which is done using machine learning techniques for identifying and defining various components of the footwear. These defined components are labeled and represented by curves indicating their respective contours.

[0070] In the 3D NURBS construction stage, a set of labeled curves is received as input. The 3D CAD digital asset is created from the set of labeled curves. The construction of the 3D asset is done within a 3D CAD system, and a method involving reinforcement learning is used. The backend service is trained and learns how to "copy" or evolve from the orthographic projection of the input into an equivalent curve for the template footwear baseline spline within the 3D CAD system. Successful construction is guided and determined through iterative comparison and adjustment of the 3D created asset with its orthographic input. Other constraints such as manufacturing tolerances and design category rules are applied to the iterative refinement as needed.

[0071] In the save / convert stage, the fully parameterized 3D asset is saved, flattened, unfolded, or converted into an appropriate format for downstream applications such as 3D and machine printing, visual rendering, and AR / VR.

[0072] FIG. 6 is a diagram showing a data flow pipeline 600 for generating digital assets representing footwear according to some embodiments. The pipeline 600 is implemented by one or more computing machines 500. Although the pipeline 600 is described in the context of footwear in the present disclosure, the pipeline 600 can be used with any other apparel or non-apparel item. Although the data flow pipeline 600 is particularly described in the context of footwear, it should be noted that the data flow pipeline 600 can also be implemented for other objects and articles (including but not limited to apparel and supplies). The data flow pipeline is described here at a high level. Some examples of the data flow pipeline are described in more detail below with reference to FIGS. 7-13.

[0073] As shown, inputs (e.g., line drawing sketch 602, photograph 604, and / or 3D model 606) associated with an article of footwear (or other apparel, supplies, etc.) are provided to preprocessing engine 610. Concept generator 608 also provides data to preprocessing engine 610. Concept generator 608 may implement a GAN (generative adversarial network) machine learning engine or any other engine that can automatically (or alternatively manually) synthesize new footwear sketches and designs. The GAN machine learning engine can also complement, complete, or enhance unavailable or missing views from available views. Preprocessing engine 610 outputs an orthographic projection 612, which is used by part identification engine 614 to identify parts of the footwear article. The identified parts are used by 3D model construction engine 620 to construct a 3D model for the footwear article corresponding to the footwear article. Computational design 616 is provided to 3D component catalog 618, which provides input to 3D model construction engine 620 and also performs functional design optimization and computational evaluation for the footwear to predict performance in the real world. Category rules 622 (e.g., related to categories of footwear such as basketball, soccer, running, lifestyle, etc.) and manufacturing rules 626 are provided to constraint application engine 624, which provides constraints to be applied by 3D model construction engine 620 and postprocessing engine 628. The 3D model constructed by 3D model construction engine 620 is provided to postprocessing engine 628. Postprocessing engine 628 adds other features and properties to the 3D asset, such as grading of the 3D asset for different sizes, adult - youth - child configurations, or nuances optimized for gender fit, comfort, performance, or functionality. Postprocessing engine 628 can also provide other dimensions and values, which can be used to make the 3D asset into manufacturable parts or components.The outputs of the material catalog 630 and the post-processing engine 628 are provided to the material application engine 632. The material application engine 632 identifies the material and provides a representation of the identified material to the color application engine 636. The color concept generator 634 generates a color concept for the color application engine 636. The applied color is provided for verification 640 based on the simulation 638. In the affirmative, the result of the verification 640 is used to generate 3D NURBS 642, 2D patterns 644, and machine code 646 (e.g., for 3D printing or manufacturing). In the affirmative, the result of the verification 640 is used to generate a 3D mesh 648 and a realistic model 650. The pipeline 600 of FIG. 6 is "multi-directional" and multi-faceted, allowing appropriate inputs to be initiated at any stage or step of the pipeline having any relevance, and for the downstream outputs for the 3D assets to be combined and constructed, or for the upstream 3D assets to be decomposed into their minimum or nominal article configurations acceptable at any pipeline stage.

[0074] Computational design 616 can include a computational engine that performs functional design optimization and computational evaluation for footwear (or other apparel, articles, etc.) to predict performance in the real world. A 3D component catalog 618 can include a collection of components of available footwear (or other apparel, articles, etc.) generated or archived either antecedently or contemporaneously, which can be used for 3D model construction or computational design. Category rules 622 can include a knowledge base (e.g., represented as a list or collection) about rules from various categories (basketball, soccer, running, lifestyle, etc.) applied to the development of footwear (or other apparel, articles, etc.) (e.g., a warning that using red with white fabric can cause bleeding). Manufacturing rules 626 can include a knowledge base (e.g., represented as a list or collection) about factory and manufacturing rules applied to the development of footwear. For example, some undercuts can be given specific angles and thicknesses to ensure moldability. Some etch cuts must be larger than X due to factory / machine performance or tolerance. The stitch spacing and stitch width may not be closer than a certain amount. Stitch order or layering rules can be included within manufacturing rules 626.

[0075] A post - processing engine 628 can add other properties to 3D assets, for example, its grading for different sizes, adult - youth - child composition, or nuances optimized for gender fit, comfort, performance, or functionality, and other dimensions and values are also added to make it a manufacturable part or component. A material catalog 630 can include a GAN engine (or other machine - learning engine) that generates, suggests, or recommends materials and material combinations for various footwear articles. This can be done holistically.

[0076] The color concept generator 634 may include a GAN engine (or other machine learning engine) that generates, suggests, and recommends single colors or color combinations for articles of various footwear (or other apparel, supplies, etc.). This can be done holistically. Verification 640 may include an engine for providing feedback or a final check to the user regarding the input applied so far before being finally processed at a later stage of the pipeline 600. Verification 640 can also stage the costs or listings regarding material analysis. The 3D mesh 648 may include a 3D CAD object made of polygon meshes stitched together.

[0077] Due to its being rendered quickly, the 3D mesh 648 can be used as a format mainly selected in the animation industry. The 3D mesh 648 may include a 3D "wireframe" of the object. This may be similar to a point cloud, but the 3D mesh 648 can be formed by connected polygons instead of points.

[0078] In some cases, the 3D mesh 648 is different from the 3D NURBS 642. The 3D mesh curve includes a polygon pattern meshed or joined to form a structure or drawing. Unlike NURBS curves, the 3D mesh curve may lack rich parameters such as the control points of a spline (e.g., the spline is a NURBS curve) that can adjust the curvature of the curve.

[0079] For example, as shown in FIG. 7, the first stage includes a pre - processing 610 that generates an orthographic projection 612 for an initial artistic design representation. At this stage, an initial design artwork (e.g., a line drawing sketch 602, a photograph 604, and / or a 3D model 606) is obtained and decomposed into an orthographic projection 612. The orthographic projection 612 may include 2D orthographic view (e.g., top, left, right, back, front, bottom). If these views are already available along with the 2D line drawing input, some of these steps during the first stage may be omitted. If some views are missing, for these, they can be artistically generated by the designer or obtained using a GAN machine - learning engine trained to synthesize other unavailable “missing” 2D orthographic views from available ones. For example, if only the left - side view of a footwear is available, the GAN engine may generate views corresponding to the top view, the front view, the back view, the bottom view, and the right - side view. The GAN engine can continuously create and generate other views until it matches the designer's intention. For the 3D model 606 or a sculptural design, the orthographic projection 612 may be a perspective view of the model. The GAN machine - learning engine may implement some of the concepts described in FIGS. 1 - 4.

[0080] FIG. 7 is a diagram showing a data - flow pipeline 700 for pre - processing (e.g., by a pre - processing engine 610) according to some embodiments. As shown, a line drawing or sketch 702 of a footwear, a photograph 704 of the footwear, and / or a 3D model 706 of the footwear are provided to a pre - processing engine 708 (which may correspond to the pre - processing engine 610). The output of the pre - processing engine 708 is an orthographic view 710 of the footwear.

[0081] Returning to FIG. 6, (as shown in FIG. 8, for example,) the second stage is the identification of smart - automated parts. At this stage, the orthographic views within the orthographic projection 612 are used to apply a machine - learning identification process (part - identification engine 614) to segments and define the various components that make up the initial design object (or the footwear).

[0082] FIG. 8 is a diagram showing a data flow pipeline 800 for part identification according to some embodiments. As shown, a part identification engine 804 (implementing, for example, a smart automated part identification process corresponding to part identification engine 614) identifies parts 804 of the footwear. As illustrated, the topline, foxing, logo, midsole, and vamp are identified.

[0083] Returning to FIG. 6, during the training of part identification engine 614, a machine learning program under the semantic segmentation category is trained to recognize and classify design parts. For the example of footwear, the aforementioned engine is trained to classify various parts of a footwear item (e.g., midsole, outsole, vamp, foxing, logo, race guard, etc.). In one applicable approach here, multiple line drawing variations for components are generated using 2D transformations such as rotation and scaling. For example, for a logo, rotation, enlargement, reduction, translation, or any combination of these operations is performed. This enhanced dataset is used to train a machine learning classification engine to identify and classify component parts. The same transformation is applied to each part of the footwear to enable the trained engine to better identify the part.

[0084] In the classification / semantic segmentation phase for the smart part with the trained machine learning engine described above, the classification method is used to identify rectangular subsections of the original 2D line drawing that contain the object of interest. For example, the trained machine learning engine draws boxes around logos, midsoles, etc. This is recursively applied as needed to hierarchically sub-classify the curves / segments that make up the components. For the example of the midsole (see Figure 9), this can be used to further decompose the 2D line drawing curves of the component into sub-components such as the byte line, the grounding part, the heel lake, the heel spring, the toe spring, and the toe counter. For the logo (see Figure 10), this is applied to identify the upper part of the logo with respect to the logo bottom segment.

[0085] Figure 9 is a diagram showing boundary boxes for identifying components of the midsole of a footwear 900 according to some embodiments. As shown, the midsole of the footwear 900 includes boundary boxes for a heel counter 902, a byte line 904, a toe counter 906, a heel lake 908, a grounding part 910, and a toe spring 912.

[0086] Figure 10 is a diagram showing boundary boxes for identifying components of a logo of a footwear 1000 according to some embodiments. As shown, the logo of the footwear 1000 includes boundary boxes for an upper part of the logo 1002 and a lower part of the logo 1004.

[0087] Figures 9-10 are described as applying the boundary box concept to specific visual components of footwear, but it should be noted that the boundary box concept is applicable to any visual component of footwear and is not limited to those listed here. Further, the techniques described herein are not limited to footwear and can be extended to any other apparel (e.g., jeans) or any other apparel component (e.g., pockets of jeans, belt loops, zippers, and buttons). The techniques can be further extended to non-apparel items (e.g., dining tables) or components of non-apparel items (e.g., table legs and table tops of a dining table).

[0088] One challenge in machine learning in this field is search space reduction. In the present disclosure, the challenge of search space reduction is addressed by first classifying a first level of boundary boxes and then deepening the search and refining the identification within each boundary box of the first level. For example, it is easier to find the midsole of a footwear than to identify the heel counter, byte line, toe counter, etc.

[0089] To return to FIG. 6, the computing machine discriminates curves that are part of the classified components, along with trace generation and post - processing 628, and excludes curves that are not. The exclusion process can be useful because 2D curves of other classified segments (e.g., the intersections of byte lines and vias) can be displayed or captured within the same rectangular bounding box classifier. If the classifier generates a bounding box that includes the 2D curve of interest (e.g., a byte line) and possibly other curves (e.g., vias and phoxing) in some cases, a set of points on the curve segment of interest is identified (e.g., edge points (the "start" and "end" points)). These points are the "guide posts" or "guide points" used to regenerate the curve from the original 2D line drawing (see FIG. 11). Once these "guide points" are defined, the computing machine identifies the mathematical relationships described above, doing so by retracing the set of pixels within the original 2D line drawing, which is done by finding the "shortest path" between these points that make up the curve, and this can be referred to as "trace - generation". For curves with a width or size at the multi - pixel level, an "averaging" technique can be applied to "thin" the curve, for example, for a width of 3 pixels, adopting the middle pixel. The contour of each curve that makes up the classified portion is later labeled. This explicit labeling of components and curve tracing can be useful for ensuring downstream reproducibility of each component.

[0090] FIG. 11 is FIG. 1100 showing an example of footwear components 1102 and 1104 marked with bounding boxes, according to some embodiments.

[0091] To return to FIG. 6, generating NURBS from the input image / drawing and conforming to specific manufacturing constraints of the footwear is done by the 3D model construction engine 620. The 3D model construction engine 620 can utilize a reinforcement learning engine. The reinforcement learning engine can be implemented using the techniques described in conjunction with FIGS. 1-4. In the 3D model construction engine 620, the manufacturing constraints become part of the code of the reinforcement learning engine in the form of "boundary conditions" (using AI (Artificial Intelligence) / ML (Machine Learning) and heuristics). 3D NURBS can be rendered when these rules and constraints are satisfied, and thus the reinforcement learning engine iterates until those conditions are satisfied.

[0092] The part identification engine 614 extracts the part of interest (e.g., a logo) so that it can be imported into the 3D model construction engine 620. Once the classifier has identified the part of interest and created a bounding box, it can have points that make up the contour of the part (e.g., the contour of the logo). The part identification engine 614 can redraw the contour of the part using pixels available on the original image and that match the points identified by the classifier. In some embodiments, pixel data points present in the original image can be utilized to augment or supplement the points identified during the classification / segmentation stage.

[0093] The 3D model construction engine 620 can utilize reinforcement learning to perform continuous and iterative refinement and adjustment of 3D-to-2D pattern engineering (see FIG. 12). The 3D model construction engine 620 creates a 3D parametric NURBS asset from an initial design rendition (e.g., a sketch, photograph, 3D mold, etc. created by an artist).

[0094] FIG. 12 is a diagram showing a data flow pipeline 1200 for constructing a 3D asset according to some embodiments. As shown, the partial identification result 1202 is provided to the 3D CAD system 1204. The 3D CAD system 1204 communicates iteratively with a server 1206 that is running a generative learning engine or an RL engine to construct a 3D asset.

[0095] FIG. 13 is a diagram showing an example of details of a 3D model 1300 of footwear according to some embodiments. As shown, the 3D model 1300 includes views 1302, 1304, 1306, 1308 of the footwear from different observation positions or observation angles.

[0096] After the "smart automated part identification" by the part identification engine 614 (e.g., using bounding boxes, guide points, and / or curves) identifies and labels various curve components of the 2D line drawing footwear design, these curves are imported into the 3D model construction engine 620. Then, these curves will take the form of a list of three-dimensional (e.g., XYZ) coordinates, direction information, scaling factors, labels, etc.

[0097] The 3D model construction engine 620 advances the generative learning engine or the RL engine to interact iteratively with the 3D CAD system to construct an initial 3D model, or to refine and adjust the 3D initial 3D model of the baseline footwear template, and evolve and "morph" it into the original design intent through some predefined success criteria (e.g., the footwear manufacturing constraints of the constraint application engine 624). This iterative process moves the points on the initial curves within the 3D CAD system until the new curves match the original design. The final evolution of the 3D model or the template curves is rendered into its own 3D shape through a code sequence (e.g., a 3D CAD software definition file) regarding the construction of the footwear item. Some examples of the operations involved are detailed below.

[0098] In the operation of the 3D model construction engine 620, the computing machine imports the target curve generated by "intelligent automated part identification" into the 3D CAD engine by the part identification engine 614. The computing machine uses the 3D CAD engine to rescale, relocate, realign, and reorient the target curve, and wraps the footwear profile around the digital space using the generative learning or RL engine. And also using the generative learning or RL engine, the computing machine iteratively adjusts the splines in the baseline footwear template, and does this by moving the control points until the splines match those target curves. To ensure that the constraints are not only coded as a set of heuristics but are learned and summarized in an automated manner, the 3D footwear construction / rendering operation is made part of the overall feedback loop of the generative learning or RL engine. The generative learning or RL engine learns the art of stably aligning the template splines with the target curves with the constraints. Once the 3D parametric footwear object is constructed, it is then decomposed and unfolded (e.g., flattened into something that can be cut from sheet material) to obtain its component pieces. Some parts are decorated and / or designed in a generative manner while some parts, such as the upper of the footwear, are flattened into 2D panels that are manufacturable patterns. The 2D panels can also be used as starting templates, and RL adjustments are iteratively made to them so that when they are wrapped onto the digital footwear, their orthographic projections match the target 2D curves.

[0099] In some cases, when it can be generated on-the-fly via a generative machine learning system, the use of a shoe template may become unnecessary. In some cases, there is a template target being used, which can potentially be generated using machine learning techniques (e.g., the starting point can be generated by a GAN).

[0100] In some embodiments, these created 3D assets provide a standard and consistent basis or foundation with respect to adjacent workstreams. These 3D designs are fed back to the 3D model construction engine 620 where their performance is simulated, establishing a complete feedback loop and resulting in a better overall design cycle or process. For example, a computing machine can perform physical simulations (e.g., traction, friction, etc.) on virtual 3D objects, and more attributes can also be provided based on the physical simulations. Traction can include performance as a constraint in addition to fidelity. Optimization can be based on cost, performance, materials used, etc. Multivariable optimization is also possible.

[0101] In the post-processing engine 628, additional "rules and constraints" can be applied to the final 3D assets to fill in the details. Manufacturing constraints such as minimum etching intervals and tolerances can be applied. Footwear stitch intervals, stitch widths, and stitch order / layering can also be applied. Different rules can be adopted for different footwear categories. Examples of footwear categories include basketball footwear, soccer footwear, golf footwear, and running footwear. The process can be done in parallel with the construction of the 3D model by the 3D model construction engine 620 and as a post-processing step after the construction of the 3D model. The process assigns parametric parameters, dimensions, and values to the 3D assets, thereby turning them into manufacturable parts of the components.

[0102] The material application engine 632 and the color application engine 636 apply and assign material attributes to components of the 3D asset. Examples of these material attributes include color or hue, the type of upper material (e.g., leather, synthetic, etc.), the composition of the midsole foam part (e.g., EVA (ethylene vinyl acetate), PU (polyurethane), etc.). The assignment and application of color / hue can be done in a heuristic manner and can be based on the marketing needs grasped or the desires of the designer. Alternatively, it can be done generatively, and a machine learning GAN system can be used, which can allow the system to explore adjacent areas and unexplored possibilities.

[0103] The computing machine can generate a list of materials and manufacturable parts. The computing machine manages and edits materials and procedures to build the 3D asset into the final physical product. These steps enable better planning and cost analysis. Also, these steps provide an immediate feedback loop to the previous stages and can trigger exploratory actions regarding alternative materials or other constraints to be met, such as cost targets. These steps can result in a "recipe" that includes raw materials and procedures, by which shoes can be transformed from 3D digital form to physical form.

[0104] The computing machine provides realistic and video renderings, thereby providing a means of visual display for the design. Attributes such as shadows, reflections, light sources, sound, etc. can be added and appended to the 3D digital asset, making them visually appealing and realistic to laypeople and consumers.

[0105] The advantages of some embodiments may include one or more of the following: (i) improved productivity of designers and developers due to reduced manual operations and processing; (ii) the applicability of pipeline techniques (additions, extensions, and / or enhancements can be made at any time for some stages); (iii) error tolerance, more consistent output, and predictable deviation resulting from full automation; (iv) the ability to perform rapid cost analysis and high-speed development reviews; (v) shortening of the development cycle and time-to-market timeframe; (vi) the ability to provide a fast, worldwide-scale, and always-on remote collaboration platform for designers, developers, cost analysts, marketing personnel, etc.; (vii) the provision of a standard and consistent basis or foundation for adjacent workstreams by 3D assets (e.g., computational design, high-resolution rendering, etc.); (viii) the preparation of a foundation for emerging downstream applications such as AR and VR by 3D assets; and (ix) the allowance of a contactless development process by minimizing physical contact.

[0106] In some embodiments, the computing machine imports the target curves from the semantic segmentation service into the 3D CAD engine. The computing machine rescales, relocates, realigns, and reorients all these target curves, doing so using some heuristics, thereby enabling the downstream 3D CAD engine code to build footwear items from them. In some embodiments, instead of modifying the splines in the baseline footwear template, these new curves (and splines) are recreated. For each target curve, the computing machine creates a new straight-line type spline that starts and ends at the endpoints of each target curve with 20 (or some other number) of control points. The computing machine performs another transformation operation of reorientation and rescaling to “standard scale” for each of these target curve / straight-line type spline pairs. The computing machine applies reinforcement learning to the straight-line type spline and adjusts the 20 control points until it matches the target curve. The computing machine transforms the finalized spline back to the original scale and orientation so that the 3D CAD engine code can be applied. Reduction of control points is sometimes done using mathematical functions.

[0107] Regarding the above reinforcement learning operation, in some aspects, “matching” is defined and quantified as the root sum square (RSS) of the differences between the points of the target curve and the points of the spline approaching 0, as in Equation 1. An exemplary function of RSS is used as the reward function in the reinforcement learning algorithm.

Number

[0108] According to some embodiments, the computing machine imports a target curve from a semantic segmentation service into a 3D CAD engine. The computing machine re-scales, relocates, realigns, and re-orients the target curve, doing so using some heuristics, and converts them into 3D splines, thereby enabling downstream 3D CAD engine code to build footwear items from them.

[0109] In some embodiments, all or some of the heuristics used are incorporated into a reinforcement learning process (e.g., based on constraints and subject matter expertise). In some embodiments, the techniques used within the reinforcement learning engine are applied to perform these iterative densification and adjustment to automate the construction of a 3D parametric version of any design input. Specifically, this can be used to create 3D parametric NURBS from 2D line drawings.

[0110] According to some implementation examples, a computing machine imports a target curve from a semantic segmentation service into a 3D CAD engine. The computing machine rescales, relocates, realigns, and reorients all these target curves and uses reinforcement learning (RL) to wrap the shoe profile around a digital last. Also, the computing machine uses RL to iteratively adjust the splines in a baseline footwear template by moving control points until the splines match those target curves. In some embodiments, to ensure that constraints and subject matter expertise are not only coded as a set of heuristics but are learned and consolidated in an automated manner, the 3D footwear construction / rendering steps in the 3D CAD engine code are made part of the entire RL feedback loop. The RL engine can learn what makes the 3D CAD engine code valid or invalid in this context and how to align the template splines to the target curves. Once a 3D parametric footwear object is constructed, it can be disassembled or unfolded to obtain component fragments. Some parts are decorated or designed using generative techniques while some parts, such as the upper of a shoe, can be flattened into 2D panels that are manufacturable patterns. In some cases, the 2D panel can also be used as the starting template and, when it is wrapped around a digital footwear item last, RL adjustments are iteratively made to it so that its orthographic projection matches the target 2D line-art curve.

[0111] Figure 14 is a flow diagram illustrating an exemplary method 1400 for generating a digital asset representing a footwear (alternatively, other objects such as apparel or articles) according to some embodiments. Method 1400 can be implemented using one or more computing machines such as computing machine 500. A reverse process can also be implemented that starts with the digital asset and goes upstream to create an image based on footwear pixels or voxels.

[0112] In step 1410, the computing machine accesses an input image of a footwear (alternatively, other objects such as apparel or articles). The input image is a pixel or voxel-based image.

[0113] In step 1420, the computing machine generates a 3D CAD digital asset comprising a NURBS model for the footwear (alternatively, other objects such as apparel or articles) based on the input image and using a machine learning engine implemented on the computing machine. Step 1420 can include steps 1422 and 1424.

[0114] In step 1422, the computing machine identifies visual components of the footwear (alternatively, other objects such as apparel or articles) using a trained classification engine. In step 1424, the computing machine iteratively adapts a spline for the visual components of the footwear using a trained reinforcement learning engine until the spline stays within a predetermined average distance from the visual components. The 3D CAD digital asset includes the spline. The machine learning engine includes the trained classification engine and the trained reinforcement learning engine.

[0115] In some embodiments, the classification engine is trained using an image representing a visual component of a footwear and a training data set including a transformation of the visual component of the footwear. The transformation may include one or more of scaling, rotation, and translation. The trained classification engine may generate a bounding box for the visual component. The bounding box may include at least a first set of pixels shown as being associated with the visual component and at least a second set of pixels shown as not being associated with the visual component. The trained classification engine may include a first sub - engine that identifies a first portion of the visual component and a second sub - engine that identifies a second portion of the visual component. The second portion of the visual component may be a sub - component of the first portion of the visual component. The second sub - engine may act on the first portion of the visual component identified by the first sub - engine.

[0116] Iteratively conforming to the spline may include ensuring that the spline meets manufacturing constraint criteria (or any other criteria that can be measured or compared in relation to metrics). The trained reinforcement learning engine may operate based on at least one manufacturing tolerance constraint and / or at least one design category rule constraint. The visual component may include at least one of a midsole, an outsole, a vamp, a foxing, a race guard, a heel counter, a heel rake, a bite line, a ground contact portion, a toe spring, and a toe counter.

[0117] In step 1430, the computing machine provides an output representing the 3D CAD digital asset. In some embodiments, the computing machine, at one or more computing machines, iteratively densifies the 3D CAD digital asset based on metrics provided by an end user until one or more metrics are within a predetermined range. The computing machine provides an output representing the densified 3D CAD digital asset. The iterative refinement may include: representing the 3D CAD digital asset as a plurality of splines, and iteratively manipulating one or more control points of the plurality of splines based on the predetermined range. In some cases, the output representing the 3D CAD digital asset is provided to a downstream engine. The downstream engine may include one or more of: a 3D printing engine, a visual rendering engine, and an AR / VR engine. The computing machine may control the manufacture of the footwear represented by the 3D CAD digital asset at one or more manufacturing machines based on the 3D CAD digital asset.

[0118] In some embodiments, the input to method 1400 is a pixel or voxel-based file, and the output is a NURBS mathematical representation of the curves and surfaces of the modeled footwear item. The output can be used to generate manufacturing instructions for the footwear and can be provided to a manufacturing machine.

[0119] In some embodiments, a spline comprises control points and a mathematical function based on the control points, and iteratively conforming to the spline involves applying a transformation to the control points of the spline based on visual components.

[0120] In some embodiments, prior to operation 1410, the computing machine preprocesses the input image to convert the input image into one or more 2D orthographic views. The preprocessing may include one or more 2D transformations.

[0121] In some embodiments, the trained classification engine comprises a GAN engine trained to synthesize unavailable 2D views based on pre-processed 2D orthographic views. Visual components are identified based on the pre-processed 2D orthographic views and the synthesized 2D views.

[0122] Some embodiments are described in relation to images, models, and digital assets representing footwear. However, the techniques disclosed herein are not limited to footwear and can be used for other items that can be modeled in 2D or 3D (e.g., apparel, supplies, objects, etc.). Examples of other items for which embodiments of this technology can be used in addition to or instead of footwear include: sports supplies, vehicle tires, children's toys, mobile phone cases, desktop ornaments, festival ornaments, candlesticks, etc. This technology can be useful for any item suitable for 3D printing.

[0123] Some embodiments are described as numbered examples (e.g., Example 1, 2, 3, etc.). These are presented as examples only and do not limit the technology disclosed herein.

[0124] Example 1 is a method comprising: accessing, on one or more computing machines, an input image of a footwear item, wherein the input image is a pixel or voxel-based image; generating, based on the input image and using a machine learning engine implemented on the one or more computing machines, a 3D CAD (computer-aided design) digital asset comprising a NURBS (non-uniform rational basis spline) model of the footwear item, the step of generating the 3D CAD digital asset comprising: identifying visual components of the footwear item using a trained classification engine; and iteratively adapting a spline, for the visual components of the footwear item, using a trained reinforcement learning engine until the spline lies within a predetermined average distance from the visual components, wherein the 3D CAD digital asset comprises the spline and the machine learning engine comprises the trained classification engine and the trained reinforcement learning engine; and providing an output representing the 3D CAD digital asset.

[0125] In Example 2, the subject matter of Example 1 includes: the spline comprises control points and a mathematical function based on the control points, and iteratively adapting the spline involves applying a transformation to the control points of the spline based on the visual components.

[0126] In Example 3, the subject matter of Examples 1-2 includes: the classification engine is trained using a training dataset comprising an image representing a visual component of a footwear item and a transformation for the representation format of the visual component of the footwear item, the transformation being one or more of scaling, rotation, and translation.

[0127] In Example 4, the subject matter of Examples 1-3 includes: iteratively adapting the spline includes ensuring that the spline meets manufacturing constraint criteria.

[0128] In Example 5, the subject matter of Examples 1 to 4 is: on the one or more computing machines, based on metrics provided by an end user, iteratively refining a 3D CAD digital asset until the one or more metrics are within a predetermined range, and providing an output representing the refined 3D CAD digital asset.

[0129] In Example 6, the subject matter of Example 5 includes: the step of iteratively refining the 3D CAD digital asset includes: on the one or more computing machines, representing the 3D CAD digital asset as a plurality of splines, and iteratively manipulating one or more control points of the plurality of splines based on the predetermined range.

[0130] In Example 7, the subject matter of Examples 1 to 6 includes: the step of generating the 3D CAD digital asset further includes: preprocessing the input image to convert the input image into one or more two-dimensional orthographic views.

[0131] In Example 8, the subject matter of Example 7 includes: the preprocessing involves one or more two-dimensional transformations.

[0132] In Example 9, the subject matter of Examples 7 to 8 includes: the trained classification engine includes a GAN (Generative Adversarial Network) engine trained to synthesize unavailable two-dimensional views based on the preprocessed two-dimensional orthographic views, and the visual components are identified based on the preprocessed two-dimensional orthographic views and the synthesized two-dimensional views.

[0133] In Example 10, the subject matter of Examples 1 to 9 includes: the trained reinforcement learning engine operates based on at least one manufacturing tolerance constraint and at least one design category rule constraint.

[0134] In Example 11, the subject matter of Examples 1 to 10 includes the following: The visual component may include at least one of a midsole, an outsole, a vamp, a foxing, a race guard, a heel counter, a heel rake, a bite line, a grounding portion, a toe spring, and a toe counter.

[0135] In Example 12, the subject matter of Examples 1 to 11 includes the following: The trained classification engine generates a bounding box for the visual component, and the bounding box includes at least a first set of pixels shown as being associated with the visual component and at least a second set of pixels shown as not being associated with the visual component.

[0136] In Example 13, the subject matter of Examples 1 to 12 includes the following: The trained classification engine includes a first sub - engine for identifying a first part of the visual component and a second sub - engine for identifying a second part of the visual component, where the second part of the visual component is a sub - component of the first part of the visual component, and the second sub - engine acts on the first part of the visual component identified by the first sub - engine.

[0137] In Example 14, the subject matter of Examples 1 to 13 includes the following: The output representing the 3D CAD digital asset is provided to a downstream engine, and the downstream engine may include one or more of a 3D printing engine, a visual rendering engine, and an AR / VR engine.

[0138] In Example 15, the subject matter of Examples 1 to 14 includes the following: Manufacturing the footwear represented by the 3D CAD digital asset based on the 3D CAD digital asset and using the one or more computing machines, including the step of controlling manufacturing on one or more manufacturing machines.

[0139] Example 16 is a method comprising: accessing, by one or more computing machines, an input image of an apparel, wherein the input image is a pixel- or voxel-based image; generating, based on the input image and using a machine learning engine implemented on the one or more computing machines, a 3D CAD (computer-aided design) digital asset including a NURBS (non-uniform rational basis spline) model of the apparel, wherein the step of generating the 3D CAD digital asset comprises: identifying, using a trained classification engine, visual components of the apparel; and iteratively adapting, for the visual components of the apparel, a spline using a trained reinforcement learning engine until the spline falls within a predetermined average distance from the visual components, wherein the 3D CAD digital asset includes the spline and the machine learning engine comprises the trained classification engine and the trained reinforcement learning engine; and providing an output representing the 3D CAD digital asset.

[0140] Example 17 is a method comprising: accessing an input image of an article on one or more computing machines, wherein the input image is a pixel or voxel-based image; generating, based on the input image and using a machine learning engine implemented on the one or more computing machines, a 3D CAD (computer-aided design) digital asset including a NURBS (non-uniform rational basis spline) model of the article, the step of generating the 3D CAD digital asset comprising: identifying visual components of the article using a trained classification engine; and iteratively adapting, for the visual components of the article, a spline using a trained reinforcement learning engine until the spline lies within a predetermined average distance from the visual components, wherein the 3D CAD digital asset includes the spline and the machine learning engine comprises the trained classification engine and the trained reinforcement learning engine; and providing an output representing the 3D CAD digital asset.

[0141] Example 18 is a method comprising: accessing an input image of an object on one or more computing machines, wherein the input image is a pixel- or voxel-based image; generating, based on the input image and using a machine learning engine implemented on the one or more computing machines, a 3D CAD (computer-aided design) digital asset including a NURBS (non-uniform rational basis spline) model of the object, wherein the step of generating the 3D CAD digital asset comprises: identifying visual components of the object using a trained classification engine; and iteratively adapting, for the visual components of the object, a spline using a trained reinforcement learning engine until the spline lies within a predetermined average distance from the visual components, wherein the 3D CAD digital asset includes the spline and the machine learning engine comprises the trained classification engine and the trained reinforcement learning engine; and providing an output representing the 3D CAD digital asset.

[0142] Example 19 is at least one machine-readable medium including instructions that, when executed by a processing circuit, cause the processing circuit to perform the operations of any of Examples 1-18.

[0143] Example 20 is an apparatus comprising means for performing any of Examples 1-18.

[0144] Example 21 is a system for performing any of Examples 1-18.

[0145] Example 22 is a method for performing any of Examples 1-18.

[0146] Although the embodiments have been described by way of examples of specific embodiments, it is apparent that various modifications and changes can be made to these embodiments without departing from the broad spirit and scope of the present disclosure. Therefore, the specification and drawings should be regarded as illustrative rather than restrictive. The accompanying drawings, which form a part of this specification, are illustrative and not restrictive, and show specific embodiments in which the subject matter can be practiced. The embodiments shown are described in sufficient detail to enable those skilled in the art to practice the teachings of the disclosure. Other embodiments can be utilized and derived therefrom, and structural and logical substitutions and changes can be made without departing from the scope of the present disclosure. Therefore, this detailed description is not to be taken in a limiting sense, and the scope of the various embodiments is defined only by the appended claims, which extend to the full scope of equivalents given by those claims.

[0147] It should be noted that, although specific embodiments have been illustrated and described herein, any configuration intended to achieve the same purpose can be substituted for the specific embodiments illustrated. The present disclosure is intended to cover any and all adaptations or variations of the various embodiments. Combinations of the above-described embodiments, as well as other embodiments not specifically described herein, will be apparent to those skilled in the art in view of the above description.

[0148] In this manuscript, as is common in patent documents, the "indefinite article" can be used with the intention of including one or more, independently of other examples or usages of "at least one" or "one or more". In this book, unless otherwise specified, "or" is used to indicate non-exclusivity such that "A or B" includes "A but not B", "B but not A", and "A and B". In this manuscript, the terms "comprising" and "herein" are used as equivalents of the plain words "including" and "here", respectively. Also, in the claims described below, "comprising" and "including" are open-ended, i.e., a system, article, composition, formulation, or process that includes elements in addition to those listed after them in the claim is still considered to be within the scope of that claim. Further, in the claims described below, terms such as "first", "second", "third", etc. are used merely for identification purposes and are not intended to impose numerical requirements on the subject matter.

[0149] The abstract of the disclosure is provided to comply with 37 C.F.R. §1.72(b), which requires an abstract that enables a reader to quickly grasp the nature of the technical disclosure. The abstract is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the patent claims. Also, in the foregoing detailed description, it should be noted that for purposes of streamlining the present disclosure, various features are combined in a single embodiment. This method of disclosure should not be construed as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as reflected in the claims below, the inventive subject matter is claimed in terms of fewer features than all of the features of a single disclosed embodiment. Accordingly, the claims described below are incorporated into the detailed description with each claim standing on its own as a separate embodiment.

Claims

Claim 1 A method comprising: accessing, by one or more computing machines, an input image of footwear, wherein the input image is a pixel or voxel-based image; generating, based on the input image and using a machine learning engine implemented on the one or more computing machines, a 3D (three-dimensional) CAD (computer-aided design) digital asset including a NURBS (non-uniform rational basis spline) model of the footwear, wherein the step of generating the 3D CAD digital asset includes: identifying visual components of the footwear using a trained classification engine; iteratively adapting, for the visual components of the footwear, a spline using a trained reinforcement learning engine until the spline lies within a predetermined average distance from the visual components, wherein the 3D CAD digital asset includes the spline and the machine learning engine comprises the trained classification engine and the trained reinforcement learning engine; providing an output representing the 3D CAD digital asset. Claim 2 The method of claim 1, wherein the spline comprises control points and a mathematical function based on the control points, and iteratively adapting the spline comprises applying a transformation to the control points of the spline based on the visual components. Claim 3 The method of claim 1, wherein the classification engine is trained using a training dataset comprising an image representing a visual component of footwear and a transformation for the representation of the visual component of footwear, the transformation being one or more of scaling, rotation, and translation. Claim 4 The method of claim 1, wherein iteratively adapting the spline includes ensuring that the spline satisfies manufacturing constraints. Claim 5 The method of claim 1, further comprising In the one or more computing machines, iteratively refining the 3D CAD digital asset until one or more metrics are within a predetermined range based on metrics provided by an end user; providing an output representing the refined 3D CAD digital asset. A method comprising: **Claim 6** The method according to claim 5, wherein the step of iteratively refining the 3D CAD digital asset comprises: representing, in the one or more computing machines, the 3D CAD digital asset as a plurality of splines; iteratively manipulating one or more control points of the plurality of splines based on the predetermined range. A method comprising: **Claim 7** The method according to claim 1, wherein the step of generating the 3D CAD digital asset further comprises: preprocessing the input image to convert the input image into one or more 2D (two-dimensional) orthographic views. A method comprising: **Claim 8** The method according to claim 7, wherein the preprocessing involves one or more 2D transformations. **Claim 9** The method according to claim 7, wherein the trained classification engine comprises a GAN (Generative Adversarial Network) engine trained to synthesize unavailable 2D views based on the preprocessed 2D orthographic views, and the visual components are identified based on the preprocessed 2D orthographic views and the synthesized 2D views. **Claim 10** The method according to claim 1, wherein the trained reinforcement learning engine operates based on at least one manufacturing tolerance constraint and at least one design category rule constraint. **Claim 11** The method according to claim 1, wherein the visual components comprise at least one of a midsole, an outsole, a vamp, a foxing, a race guard, a heel counter, a heel rake, a bite line, a grounding portion, a toe spring, and a toe counter. **Claim 12** The method according to claim 1, wherein the trained classification engine generates a bounding box for the visual component, and the bounding box comprises at least a first set of pixels indicated as being associated with the visual component and at least a second set of pixels indicated as not being associated with the visual component.

13. The method according to claim 1, wherein the trained classification engine comprises a first sub - engine for identifying a first part of the visual component and a second sub - engine for identifying a second part of the visual component, the second part of the visual component being a sub - component of the first part of the visual component, and the second sub - engine acting on the first part of the visual component identified by the first sub - engine.

14. The method according to claim 1, wherein the output representing the 3D CAD digital asset is provided to a downstream engine, and the downstream engine may include one or more of: a 3D printing engine, a visual rendering engine, and an AR / VR engine.

15. The method according to claim 1, further comprising manufacturing the footwear represented by the 3D CAD digital asset based on the 3D CAD digital asset and using the one or more computing machines, the method including the step of controlling the manufacturing on one or more manufacturing machines.

16. A non - transitory machine - readable medium storing instructions, which when executed by one or more computing machines, cause the one or more computing machines to: access an input image of footwear at the one or more computing machines, the input image being a pixel - or voxel - based image. Based on the input image and using a machine learning engine implemented on the one or more computing machines, a step of generating a 3D (three-dimensional) CAD (computer-aided design) digital asset including a NURBS (non-uniform rational basis spline) model of the footwear, wherein the step of generating the 3D CAD digital asset comprises: Identifying visual components of the footwear using a trained classification engine; For the visual components of the footwear, a step of iteratively adapting a spline using a trained reinforcement learning engine until the spline falls within a predetermined average distance from the visual components, wherein the 3D CAD digital asset includes the spline, and the machine learning engine comprises the trained classification engine and the trained reinforcement learning engine; A non-transitory machine-readable medium that causes an operation including providing an output representing the 3D CAD digital asset.

17. The machine-readable medium according to claim 16, wherein the spline comprises control points and a mathematical function based on the control points, and iteratively adapting the spline involves applying a transformation to the control points of the spline based on the visual components.

18. The machine-readable medium according to claim 16, wherein the classification engine is trained using a training dataset comprising an image representing a visual component of a footwear and a transformation for a representation format of the visual component of the footwear, and the transformation is one or more of scaling, rotation, and translation.

19. A system comprising: A processing circuit; A memory storing instructions that, when executed by the processing circuit, cause the processing circuit to: Access an input image of a footwear at the processing circuit, wherein the input image is a pixel- or voxel-based image. Generating a 3D (three-dimensional) CAD (computer-aided design) digital asset including the NURBS (non-uniform rational basis spline) model of the footwear based on the input image and using a machine learning engine, wherein the step of generating the 3D CAD digital asset comprises: Identifying the visual components of the footwear using a trained classification engine; For the visual components of the footwear, iteratively adapting a spline using a trained reinforcement learning engine until the spline lies within a predetermined average distance from the visual components, wherein the 3D CAD digital asset includes the spline and the machine learning engine comprises the trained classification engine and the trained reinforcement learning engine; Providing an output representing the 3D CAD digital asset; and a system comprising a memory for performing the operations including the steps.

20. The system according to claim 19, wherein the spline comprises control points and a mathematical function based on the control points, and iteratively adapting the spline involves applying a transformation to the control points of the spline based on the visual components.

Images