Learning a unified embedding

Abstract

A computer-implemented method for generating a unified machine learning model using a neural network on a data processing device is described. The method includes the data processing device determining a corresponding learning target for each of a plurality of object vertices. The data processing device determines the corresponding learning target based on two or more embedding outputs of the neural network. The method also includes the data processing device training the neural network to identify data associated with each of the plurality of object vertices. The data processing device trains the neural network using the corresponding learning target and based on a first loss function. The data processing device generates a unified machine learning model using the trained neural network, wherein the model is configured to identify specific data items associated with each of the plurality of object vertices.

Description

Technical Field

[0001] This specification relates to training a unified neural network model. Background Art

[0002] A neural network is a machine learning model that uses one or more layers of operations to generate an output, such as a classification, for a received input. Some neural networks include one or more hidden layers in addition to the output layer. The output of each hidden layer is used as the input of the next layer in the network (i.e., the next hidden layer or output layer of the network). Some or all layers of the network generate outputs from the received input according to the current values ​​of the corresponding parameter sets.

[0003] Some neural networks include one or more convolutional neural network layers. Each convolutional neural network layer has a set of associated kernels. Each kernel includes values ​​established by a neural network model created by a user. In some embodiments, the kernel recognizes a specific image contour, shape, or color. The kernel can be represented as a matrix structure of weight inputs. Each convolutional layer can also process a set of activation inputs. The set of activation inputs can also be represented as a matrix structure. Summary of the invention

[0004] This specification describes systems and methods for generating a unified machine learning model using a neural network on a data processing device. According to the described technology, the data processing device determines a learning target for each object vertex in a set of object vertices. The data processing device can determine each learning target based on two or more embedding outputs of the neural network. Each embedding output can be generated by an independent dedicated model that is separately trained using a triplet loss function. Each dedicated model is configured to identify data associated with a specific object vertex.

[0005] When the data processing device trains the neural network to identify data associated with each object vertex in the group, a unified machine learning model is generated. The data processing device trains the neural network based on the L2 loss function and using the corresponding learning objectives of the dedicated model. The data processing device uses the trained neural network to generate the unified machine learning model. The unified model can be configured to identify specific electronic data items of the object representation including items in each of the object vertices.

[0006] One aspect of the subject matter described in this specification can be embodied in a computer-implemented method for generating a unified machine learning computational model using a neural network on a data processing device. The method includes: determining, by the data processing device and for the neural network, a corresponding learning objective for each of a plurality of object vertices, wherein each object vertex defines a different category of objects belonging to the vertex; training the neural network by the data processing device and based on a first loss function to identify data associated with each of the plurality of object vertices, wherein the neural network is trained using the corresponding learning objective; and generating, by the data processing device and using the neural network trained based on the first loss function, a unified machine learning model, the unified machine learning model being configured to identify items included in the data associated with each of the plurality of object vertices.

[0007] These and other embodiments may each optionally include one or more of the following features. For example, in some embodiments, determining a corresponding learning objective for a neural network further includes: training at least one other neural network by a data processing device and based on a second loss function to identify data associated with each of a plurality of object vertices; in response to the training, generating two or more embedding outputs by the data processing device, wherein each embedding output indicates a specific learning objective and includes a vector of parameters corresponding to data associated with a specific object vertex; and generating a corresponding machine learning model by the data processing device and using at least one other neural network trained based on the second loss function, each machine learning model being configured to use a specific embedding output.

[0008] In some embodiments, determining the corresponding learning objectives for the neural network further comprises: providing the corresponding learning objectives generated from the corresponding independent models for training the neural network. In some embodiments, each of the plurality of object vertices corresponds to a particular category of items, and the data associated with each of the plurality of object vertices comprises image data of items in the particular category of items. In some embodiments, the particular category is an apparel category, and the items in the particular category comprise at least one of the following: a handbag, a shoe, a dress, a trousers, or a coat; and wherein the image data indicates an image of at least one of the following: a particular handbag, a particular shoe, a particular dress, a particular trousers, or a particular coat.

[0009] In some embodiments, each of the respective machine learning models is configured to identify data associated with a particular object vertex within a first accuracy; and the unified machine learning model is configured to identify data associated with each of the plurality of object vertices within a second accuracy that exceeds the first accuracy. In some embodiments, determining a respective learning objective for each of the plurality of object vertices includes: analyzing two or more embedding outputs, each embedding output corresponding to a particular object vertex in the plurality of object vertices; and determining a respective learning objective for each of the plurality of object vertices based on the analysis.

[0010] In some embodiments, the first loss function is an L2 loss function, and generating a unified machine learning model includes: generating a specific unified machine learning model that minimizes a computational output associated with the L2 loss function. In some embodiments, the neural network includes multiple neural network layers that receive multiple layer inputs, and wherein training the neural network based on the first loss function includes: performing batch normalization to normalize the layer inputs to a specific neural network layer; and in response to performing batch normalization, minimizing covariate shift. In some embodiments, the second loss function is a triplet loss function, and generating a corresponding machine learning model includes: generating a specific machine learning model based on the association between the anchor image, the positive image, and the negative image.

[0011] Other embodiments of this and other aspects include corresponding systems, devices and computer programs, which are configured to perform the actions of the method encoded on a computer storage device. The computing system of one or more computers or circuits can be configured so that the system performs these actions in operation through software, firmware, hardware or a combination thereof installed on the system. One or more computer programs can be configured so by having instructions that, when executed by a data processing device, cause the device to perform these actions.

[0012] The subject matter described in this specification can be implemented in certain embodiments to achieve one or more of the following advantages. Object recognition is receiving increasing attention in vision research. In this context, the teachings described include a process for using a neural network to generate a unified machine learning model using an L2 loss function, where the unified model can be used to recognize or recognize various objects (e.g., dresses, handbags, shoes) from multiple object vertices.

[0013] For example, given image data depicting a representation of a piece of clothing, a unified model generated according to the described teachings can be used to locate or retrieve the same or similar items. In some instances, the appearance of the items may change with lighting, viewpoint, occlusion, and background conditions. Different object vertices may also have different characteristics, such that an image from a dress vertex may experience more deformation than an image from a shoe vertex. Therefore, due to these differences, independent models are trained to recognize the items in each object vertex.

[0014] However, independent specialized models require a large amount of resources for model storage and increased computational requirements to support the deployment of multiple models. These resource burdens may become more severe when multiple models are used on mobile platforms. Therefore, a unified model for object recognition across different clothing vertices can reduce processor utilization and provide increased computational efficiency of the example object recognition system. In addition, object retrieval functions can be efficiently performed by combining multiple specialized models into a single unified model using a smaller computational footprint. This leads to technical improvements in the field of model generation and use technology.

[0015] The details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other potential features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A neural network architecture for generating a machine learning model based on a first loss function is shown.

[0017] Figure 2 A neural network architecture for generating a machine learning model based on a second loss function is shown.

[0018] Figure 3 An example graphical representation of embedded data associated with different object vertices is shown.

[0019] Figure 4 is an example flow chart of the process of generating a unified machine learning model for multiple object vertices based on a specific loss function.

[0020] Figure 5 A diagram of a graphical representation of a corresponding embedding model including object vertices corresponding to a particular apparel category is shown.

[0021] Figure 6 A diagram is shown having computational functionality for obtaining image data for training one or more machine learning models.

[0022] The same reference numbers and names in different drawings indicate the same elements. DETAILED DESCRIPTION

[0023] A machine learning system can be trained using a deep neural network to discern specific categories of items based on learned inferences. The deep neural network can generate inferences based on an analysis of input data received by the machine learning system. The trained machine learning system can produce or generate one or more specialized models that use a specific set of learned inferences for identification or discrimination of specific categories of items.

[0024] For example, in the context of clothing recognition, a specialized model may be trained to recognize items associated with a particular category or object vertical, such as a dress, pants, or handbag in image data to which the model is applied. In alternative embodiments, the categories or object verticals may correspond to various items or objects, such as cars, animals, human individuals, and various physical objects, such as represented in image data. In some embodiments, the object verticals may correspond to audio signal data.

[0025] In general, specialized models may significantly outperform general models, e.g., models trained to recognize items associated with a wide range of object vertices. Therefore, item recognition models generated using deep neural networks are typically trained independently for different object vertices. Object vertices define different categories of objects belonging to that vertex, e.g., for apparel, object vertices may be item categories of hats, shoes, shirts, jackets, etc. However, an item recognition system that includes a set of specialized models for recognizing items for different object vertices may be expensive to deploy and may not be sufficiently scalable.

[0026] In this context, the subject matter described in this specification includes systems and methods for generating a unified machine learning model using a neural network on a data processing device. The unified embedding model can be generated using a deep neural network that utilizes a learning objective indicated by an embedding output generated by a corresponding dedicated model. An example machine learning system can use a neural network (or a deep neural network) to learn reasoning for identifying various items across multiple object vertices (e.g., item categories corresponding to various clothing categories).

[0027] For example, a machine learning system including a neural network can access a corresponding learning target for each object vertex in a set of object vertices. The system can determine the corresponding learning target based on two or more embedding outputs of the neural network. Each embedding output can be generated by a corresponding dedicated model that is individually trained using a triplet loss function and recognizes data (e.g., an image of a luxury handbag) associated with a specific object vertex (e.g., a handbag).

[0028] The data processing device of the system trains a neural network to identify data associated with each vertex in the set of object vertices. The neural network can be trained using a corresponding learning objective of the dedicated model and based on an L2 loss function. The data processing device uses the trained neural network to generate a unified machine learning model that is configured to identify specific data items (e.g., branded sneakers, luxury wallets, luxury shirts, etc.) associated with each object vertex in the set (e.g., shoes, handbags, tops / shirts, etc.).

[0029] Figure 1 A neural network system architecture 100 ("system 100") for generating an example machine learning model based on a first loss function is shown. Generating the machine learning model may include the system 100 performing neural network computations associated with an inference workload. Specifically, computations of the inference workload may include processing neural network inputs (e.g., input activations) through layers of the neural network. Each layer of the neural network may include a set of parameters (e.g., weights), and processing the inputs through the neural network layers may include computing a dot product using the input activations and the parameters as operands of the computation.

[0030] System 100 generally includes an example neural network indicated by neural network architecture 102. The neural network of architecture 102 may include a base network 103, a pooling layer 104, a first set of connected layers 106, a second set of connected layers 108, and an embedding output 110. Base network 103 may include a subset of the neural network layers of architecture 102.

[0031] For example, the deep neural network may include a base network 103 including a plurality of convolutional layers. These convolutional layers may be used to perform complex calculations for computer-based recognition of various items included in various image data. In some embodiments, the base network 103 may be inception v2, inception v3, inception v4, or another related neural network structure. Although described in the context of image data, the processes of the present specification may be applied to the detection or recognition of audio signal data.

[0032] The architecture 102 may include various additional neural network layers that perform various functions associated with inference computations for training machine learning models. For example, the pooling layer 104 may be an average pooling layer or a maximum pooling layer that performs functions associated with pooling output activations for a downsampling operation. The downsampling operation may reduce the size of an output data set by modifying certain spatial dimensions associated with an input data set.

[0033] The set of connected layers 106, 108 may be a corresponding set of fully connected layers, which include fully connected artificial neurons with all activations in the previous layer. The embedding output 110 may correspond to one or more output feature sets including vectors of floating point numbers / parameters of a given output dimension (64-d, 256-d, etc.). As described in more detail below, the embedding output 110 is formed, produced, or generated when the example neural network of the system 100 is trained to perform certain computational functions for object / item discrimination or recognition.

[0034] The system 100 may include one or more processors and other related circuit components that form one or more neural networks. In general, the methods and processes described in this specification may be implemented using various processor architectures, such as a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or other related processor architectures.

[0035] System 100 may include multiple computers, computing servers, and other computing devices, each including a processor and a memory storing computing logic or software instructions executable by the processor. In some embodiments, system 100 includes one or more processors, memories, and data storage devices, which together form one or more architectures 102. The processors of architecture 102 process instructions executed by system 100, including instructions stored in memory or on storage devices. Execution of the stored instructions may cause the machine learning processes described herein to be performed.

[0036] Reference again Figure 1 , the system 100 is configured to perform various computational operations associated with the machine learning process. For example, the system 100 performs learning operations 112 and 114 and various other operations associated with training a neural network to generate one or more application-specific machine learning models. In some embodiments, the system 100 executes programming code or software instructions to perform computations associated with the learning operations 112 and 114. As described in more detail below, the learning operations 112 and 114 are performed by the system 100 to train the corresponding application-specific learning model based on the triple loss function indicated by the computational logic 116.

[0037] The learning operation 112 includes the system 100 generating model training data using the neural network of the architecture 102. The model training data may correspond to the embedding outputs generated by the system 100 when the system 100 is trained to generate a specific application-specific model. In some embodiments, the system 100 generates multiple different application-specific models and generates separate sets of embedding outputs, wherein a particular set of embedding outputs corresponds to a particular application-specific model.

[0038] For example, in the context of clothing recognition or clothing retrieval, separate dedicated models can be generated to recognize and retrieve clothing items of different clothing categories (e.g., dresses, tops, handbags, etc.). In some embodiments, a separate sub-network can be used to learn an embedding model for recognizing images from various websites or other user-generated images (e.g., "street" or "real life" digital images captured using a mobile device / smartphone).

[0039] During model training, the subnetworks for each vertex (such as dresses, handbags, glasses, and pants) can be fine-tuned independently. For these subnetworks, the results of model training can enable the machine learning system to generate up to eleven independent dedicated models, each independent dedicated model corresponding to one of the eleven vertices. As used herein, a "vertical" or "object vertex" corresponds to an object or item category. For clothing recognition, an object vertex can be a clothing item category, such as a dress, handbag, glasses, pants, etc. As discussed in more detail below, using independent models for object recognition of items in a particular category / vertex (e.g., clothing items) can produce substantially accurate item recognition results.

[0040] In some implementations, system 100 uses image data of apparel items, each of which is associated with a different apparel category, to train a neural network of architecture 102. For example, system 100 can train a neural network using image data of multiple different types of handbags. System 100 can then generate a specialized model that recognizes or discriminates a particular type of handbag based on the embedding output generated in response to training the neural network.

[0041] As described above, a particular set of embedding outputs may correspond to a particular application-specific model. For example, in apparel recognition, a first set of embedding outputs may correspond to neural network training data used to generate learned inferences for a first model for recognizing a particular shirt / blouse or blouse (e.g., see operation 114). Similarly, a second set of embedding outputs may correspond to neural network training data used to generate learned inferences for a second model for recognizing a particular pair of jeans / pants / skirts or bottoms.

[0042] Each set of embedding outputs includes embedding feature vectors that can be extracted and used for object or term retrieval. As described in more detail below, the sets of extracted embedding feature vectors can correspond to corresponding learning objectives, and the embedding outputs of the trained neural network model can include these embedding feature vectors.

[0043] One or more learning objectives may be used to train a machine learning system (e.g., system 100) to generate a specific type of specialized computational model. Figure 2 As discussed above, multiple different learning objectives can be used to train at least one unified machine learning model that discriminates terms associated with multiple different vertices or categories.

[0044] Reference again Figure 1 , the system 100 performs a learning operation 114 to generate a dedicated learning model based on the model training data determined in the learning operation 112. In some instances, determining or generating the model training data corresponds to an example process of "learning" a separate embedding model. In some embodiments, for model training and feature vector extraction, the system 100 uses a two-stage approach when training (e.g., the first stage) and when extracting embedding feature vectors for object retrieval (e.g., the second stage).

[0045] In the context of retrieving clothing from image data, the first level can include localizing and classifying an item of clothing of the image data. In some instances, classifying an item of clothing of the image data includes system 100 determining an object category label for the item of clothing of the image data. For example, system 100 can use an example object detector to analyze the image data. System 100 can then use the analysis data to detect an object or item of clothing of the image data that includes object attributes associated with a handbag. Based on the analysis and detection, system 100 can then determine that the object category label of the item of clothing is a "handbag" category label.

[0046] In some embodiments, system 100 includes an object detection architecture that is a single-shot multi-box (SSD) detector for base network 103, which is an initial V2 base network. In other embodiments, system 100 can be configured to use or include various other object detection architectures and base network combinations. SSD can be an example computing module of system 100 that executes program code to perform one or more object detection functions.

[0047] For example, the SSD detector module can provide a bounding box of an object bounding the image data. The SSD can also provide a clothing category label indicating whether the bounded object is a handbag, an item of eyeglasses, or a dress. In some embodiments, the object pixels of the image data can be cropped, and then various features can be extracted on the cropped image using a specific embedding model of the system 100. The sub-process steps associated with the first stage in the two-stage process can be used to train a dedicated embedding model based on various image data.

[0048] In response to determining the object category label at the first level, the system 100 can proceed to the second level and train a dedicated embedding model to calculate similarity features for object retrieval. For example, the system 100 can perform embedding model training using a triplet loss function indicated by the computation logic 116. More specifically, the system 100 uses a triplet ranking loss to learn feature embeddings for each individual item / object vertex or category.

[0049] For example, a triplet includes an anchor image, a positive image, and a negative image. During triplet learning, the system 100 seeks to generate an embedding such that the positive image is close to the anchor image in the feature space of the neural network, while the negative image is pushed away from the anchor image. The embedding learned from triplet training is used to calculate image similarity.

[0050] For example, let is a triple, where Denote the anchor image, positive image, and negative image, respectively. The learning objective of the system 100 is to minimize the computational output associated with the following loss function as shown in equation (1),

[0051] (1)

[0052] where: i) α is the enforced margin between positive and negative pairs, ii) f(I) is the feature embedding of image I, and iii) D(f x ,f y ) are two feature embeddings f x and f y The distance between.

[0053] For embodiments involving training a separate model for a particular apparel object vertex, the positive image is of the same product as the anchor image (e.g., a Chanel handbag), and the negative image is of another product, but in the same apparel vertex (e.g., a luxury handbag). In some embodiments, the system 100 executes computational logic for a semi-hard negative mining function for obtaining negative image data. For example, the system 100 can access online / web-based resources and use semi-hard negative mining to identify strong negative object images. The system 100 can then use these object images to enhance or improve the effectiveness of training a particular dedicated model.

[0054] As described above, model training based on the triplet loss function of logic 116 produces training data such as embedding outputs including feature vectors. The set of extracted embedding feature vectors can correspond to corresponding learning objectives. Therefore, at learning operation 118, system 100 can determine corresponding learning objectives based on the triplet loss model training data. System 100 can then use two or more learning objectives to train a machine learning system (e.g., Figure 2 system 200) to generate at least one unified computing model, as described below.

[0055] Figure 2 A neural network system architecture 200 ("system 200") for generating an example machine learning model based on a second loss function (e.g., an L2 loss function) is shown. As shown, system 200 includes substantially the same features as system 100 described above. However, system 200 includes an L2 normalization layer 212, which will be described in more detail below. In some embodiments, system 200 is a subsystem of system 100 and can be configured to perform various computational functions of system 100 described above.

[0056] System 200 is configured to learn or generate a unified embedding model trained based on learned inferences. These learned inferences enable object discrimination for various item groupings, where each grouping corresponds to a different object vertex or category (e.g., clothing category). System 200 learns one or more unified models by combining training data generated when system 100 is used to train corresponding dedicated models as described above. In some related model learning / training scenarios, when triplet loss is used to train a unified model, combining training data from independent models to generate a unified model can result in performance degradation.

[0057] For example, using the triplet loss function of logic 116 to train a unified model based on combined training data (e.g., embedding output 112) of different object vertices may result in a significant decrease in discrimination accuracy compared to the accuracy of models trained for each individual vertex. These decreases may occur as more object categories (or vertices) are accumulated into a single unified model. However, as described in more detail below, when embeddings from multiple dedicated models are combined to generate a single unified model, reducing training difficulty and training complexity can result in substantial improvements in performance and discrimination accuracy.

[0058] According to the described teachings, the system 200 can generate a unified embedding model that achieves equivalent performance and discrimination accuracy when compared to separate dedicated models. In addition, the unified model can have the same or even less model complexity than a single separate dedicated model. Therefore, the present specification describes processes and methods for alleviating or reducing the difficulty in training model embeddings for multiple vertices that enable the generation of unified model improvements.

[0059] For example, independent dedicated models may be first trained to achieve a desired threshold accuracy level for distinguishing objects included in the image data. As described above, the independent models may be trained using the system 100 and based on the triplet loss function of the computation logic 116. The embedding output of each independently trained model is then used as a learning target to train an example unified model.

[0060] In some implementations, a particular dedicated model may have an example accuracy metric of 0.66, where the model accurately identified a particular handbag 66.1% (66.1) of the time. A unified model generated according to the described teachings may achieve accurate object recognition results that exceed the accuracy metric (e.g., 66.1) of the object recognition results of the particular dedicated model. For example, in the context of apparel recognition, a unified model generated according to the described teachings may have an accuracy metric of 0.723, or an object retrieval or recognition accuracy of 72.3% accuracy for the handbag apparel category.

[0061] In some embodiments, accurately identifying / recognizing an object using the unified model includes determining a category of the object (e.g., "handbag"), determining an owner or designer of the object (e.g., "Chanel" or "Gucci"), and / or determining a type / style of handbag (e.g., "Chanel 2.55 Classic Flap Bag"). In some instances, identifying an object via the unified model can include model retrieval (e.g., object retrieval) of associated image data including a graphical representation of the object.

[0062] Reference again Figure 2The system 200 is configured to generate a unified model that can perform multiple tasks for accurate object recognition and retrieval that are performed by separate models in existing systems, but with lower accuracy. In addition, the described teachings include methods and processes for improving the output of simulated separate model embeddings (e.g., learning objectives) by using an L2 loss function.

[0063] For example, training a unified model based on a triplet loss and by combining training data for two different object vertices (e.g., a handbag and a shoe) can generate a unified model that performs object discrimination of the items in those vertices with reasonable accuracy. However, using a triplet loss when combining training data for three or more different object vertices may result in a unified model that performs with substantially poor object discrimination accuracy. The poor accuracy is caused by the difficulties and complex computational challenges that arise when training a unified model based on a triplet loss function for several different vertices.

[0064] To alleviate this training difficulty, this specification proposes a learning scheme that uses the embedding output from a dedicated model as a learning target, so that L2 loss can be used instead of triple loss. The use of the L2 loss function alleviates the training difficulty of generating a unified model and provides a more efficient use of the feature space of the neural network. The end result is a unified model that can achieve the same (or higher) retrieval accuracy as many independent dedicated models while having the model complexity of a single dedicated model.

[0065] For example, system 200 uses the respective learning objectives of the independent models to learn a unified learning model such that the embeddings generated from the unified model are the same as (or very close to) the embeddings of the independent dedicated models generated by system 100. In some embodiments, system 200 determines the respective learning objectives for each object vertex in a set of object vertices using the neural network of architecture 102. Each of the respective learning objectives can be based on a particular embedding output of the neural network.

[0066] Figure 2 The computational operations used to generate a unified machine learning model are shown. For example, in Figure 2 At learning operation 214, system 200 accesses learning targets corresponding to feature embeddings of corresponding specialized models. At learning operation 216, system 200 generates unified model training data corresponding to feature embeddings of objects for detecting various vertices. The feature embeddings are based on neural network inference calculations that occur during unified model learning.

[0067] For example, let Each V i is the set of vertices whose data can be combined to train the embedding model. Let is a collection of embedding models, where each Mi is the vertex set V i The learning model. is a set of N training images. If I j The vertex ∈V s , s=1...K, then the corresponding model M s To generate image I j Embedding features of . Let f sj Represented as image I j From M s Generated feature embeddings.

[0068] At learning operation 220, system 200 generates a unified machine learning model configured to recognize specific items included in the example image data. The image data can be associated with each object vertex in the group, and the unified model is generated using a neural network trained based on a specific loss function (e.g., L2 loss). For example, system 200 is configured to learn model U so that the features generated from model U are the same as the features generated from the independent dedicated model generated by system 100.

[0069] Specifically, let f uj Denotes the feature embedding generated from model U. The learning objective of system 200 is to determine a model U that can minimize the computational output associated with the following loss function as shown in equation (2).

[0070] (2)

[0071] Referring to equation (2), the characteristic f uj Calculated from model U, and feature f sj It can be calculated from the different dedicated models described above. The above model learning description uses the L2 loss function indicated by the calculation logic 218 and equation (2) above, rather than the triplet loss function indicated by the calculation logic 116 and equation (1) above.

[0072] In some embodiments, the system 200 is configured to generate a unified model having an output dimension 107 of 256-d. In contrast, the individual specialized models generated at the learning operation 114 can have an output dimension 107 that is greater than or substantially greater than the 256-d output dimension of the unified model, such as 4096-d. As described above, the use of the L2 loss provides a less complex and less difficult training process than the triplet loss.

[0073] In addition, the use of the L2 loss function provides less complexity and difficulty in the application of learning techniques such as batch normalization. For example, with batch normalization, neural network layer inputs can be normalized to allow for higher learning rates. For the example image classification model, batch normalization can be used to achieve a desired threshold accuracy metric (e.g., 0.60 or higher) using fewer training steps when compared to learning techniques used with other loss functions.

[0074] In some embodiments, covariate shift can be minimized in response to the system 200 performing a batch normalization function applied via the L2 normalization layer 212. For example, a deep neural network can include multiple layers in a sequence. Training a deep neural network is often complicated by the fact that the distribution of inputs to each layer may change during model training, for example, as parameters of previous layers in the sequence change.

[0075] This change can slow down the training of models using deep neural networks, requiring slower learning rates and careful parameter initialization. Parameter changes that adversely affect training speed can be described as covariate shift within the neural network. However, by using L2 loss, a batch normalization process for normalizing layer inputs can be performed to address or minimize the adverse effects on training speed caused by covariate shift.

[0076] Furthermore, learning methods that generate a unified model using L2 loss allow for the use of an increased amount of unlabeled data relative to triple loss. For example, a triple loss learning method used to train an apparel recognition model may require product identification (e.g., "Chanel 2.55 Classic Flap Bag") to generate embedding data for training triples. However, model training and learning methods using L2 loss require only vertex labels, which can be automatically generated by the example localization / classification model. Therefore, the use of L2 loss can reduce processor utilization through the aforementioned calculations used to determine product identification and increase system bandwidth for additional calculations.

[0077] Alternatively, the described teachings also include methods for selecting combinations of vertex data to produce a specific model (e.g., a unified model or other related combined model form) that can be used to reduce the number of dedicated models. This specific combined model can be successfully learned and can have a comparable object recognition accuracy that is similar to, equal to, or greater than the recognition accuracy of each independent dedicated model.

[0078] For example, selective or "intelligent" vertex combination can be used to determine a combined model (e.g., an example unified model). In some embodiments, system 200 can include computational logic to determine which embedded data from different vertices of a dedicated model can be combined to produce an example combined model. Specifically, starting with a first vertex, system 200 can incrementally add embedded data from other vertices. As embedded data is added, system 200 can perform a sample term discrimination task to determine whether a model learned from the combined data results in an observed decrease in accuracy.

[0079] In some instances, the system 200 can steadily add embedding data from other vertices until a drop in accuracy is observed. In other instances, the system 200 determines a specific combination of vertices for multiple dedicated models, where each dedicated model is used for item discrimination across a subset of vertices. In addition, the system 200 can determine a specific combination of vertices for multiple dedicated models while also maintaining a threshold level of accuracy. The system 200 can then use the feature embeddings of the dedicated models corresponding to the specific vertices in the subset and generate a combined model based on the feature embeddings.

[0080] Figure 3 A graphical representation of embedding data 300 of different object vertices in the feature space of an example neural network is shown. The graphical representation generally indicates that a unified model based on the described teaching training (e.g., learning) can provide more efficient and extensive use of the feature space of the neural network. For example, the described learning method using L2 loss can effectively train the unified model by utilizing pre-established feature maps (e.g., learning objectives) learned for independent dedicated models.

[0081] The embedding data 300 includes a t-distributed stochastic neighbor embedding (t-SNE) visualization generated from feature embeddings of an independent dedicated model. Specifically, the embedding data 300 includes two thousand images from each vertex 302, 304, and 306, where the data is projected down into a 2D space for visualization.

[0082] Figure 3 indicates the feature embedding f sj In feature space, it is separated across vertices 302, 304, 306. In other words, (from model M s ) for each vertex f sj The embedding model of uses only a portion of the dimensional space (e.g., 64-d), so a unified model can be learned to combine the embedding outputs of each clothing vertex (e.g., 8 vertices in total) included in the embedding of data 300.

[0083] Figure 4 is an example flow chart of a process for generating a unified machine learning model for multiple object vertices based on a specific loss function. Process 400 corresponds to an improved process for generating a unified machine learning model, wherein the generated model has an item discrimination accuracy measure that is at least equal to the accuracy measures of two or more different dedicated models. Process 400 can be implemented using the above-mentioned system 100 or 200, wherein system 100 can perform all the described functions associated with subsystem 200.

[0084] Process 400 includes block 402, in which system 100 determines a corresponding learning objective for each object vertex in a set of object vertices. In some implementations, a neural network of system 100 determines the corresponding learning objective based on two or more embedding outputs of the neural network.

[0085] For example, an object vertex may be an apparel category, such as a dress, a shoe, or a handbag. Furthermore, each vertex may correspond to an embedding output produced when a particular model is trained to recognize or identify the apparel or clothing item in that vertex. Example apparel items may include a cocktail dress, a basketball sneaker, or a brand name monogram handbag.

[0086] At block 404 of process 400, system 100 trains a neural network based on a first loss function (e.g., L2 loss) to identify data associated with each object vertex in the group. In some embodiments, the neural network is trained using a respective learning objective determined for each object vertex. For example, given an image file or image data, system 100 may train a neural network to at least: i) identify a dress item in the image based on an analysis of pixel data of the image; ii) identify a shoe item in the image based on an analysis of pixel data of the image; or iii) identify a handbag item in the image based on an analysis of pixel data of the image.

[0087] At block 406, the system 100 generates a unified machine learning model configured to identify items included in the data associated with each object vertex in the set of vertices. For example, the data processing device of the system 100 can use a neural network trained based on the first loss function to generate a unified machine learning model that performs one or more of the object recognition functions described herein.

[0088] In some examples, determining the corresponding learning objective includes: i) training a neural network to identify data associated with each of the object vertices, wherein the neural network is trained based on a second loss function; and ii) generating at least two embedding outputs, wherein each embedding output indicates a specific learning objective of the corresponding learning objective. In addition to indicating the specific learning objective, each embedding output may include a vector of floating point numbers (or parameters) that generally correspond to attributes of the image data associated with the specific object vertex.

[0089] In some embodiments, the system 100 generates corresponding machine learning models, wherein each of the models is generated using a neural network trained based on a second loss function (e.g., a triplet loss) that is different from the first loss function. In addition, each of the models can use a vector of floating point numbers output by a particular embedding to identify a garment or clothing item for a particular object vertex. In some instances, generating the embedding occurs in response to training the neural network.

[0090] As briefly referenced above Figure 2 As discussed, in some embodiments, generating a unified machine learning model can include combining training data associated with different clothing vertices. To calibrate object recognition and retrieval performance (e.g., to determine a learning objective), an embedding for each vertex is first learned using a triplet loss. The purpose of vertex combination can be to use a smaller number of separate dedicated models without any observed decrease in retrieval accuracy.

[0091]

[0092] Table 1: Comparison of retrieval accuracy (percentage) for different vertex combinations

[0093] Table 1 shows examples of retrieval accuracy metrics for (1) individual models, (2) dress-top joint or combined models, and (3) dress-top-jacket joint models on the “dress”, “top”, and “jacket” vertices. Compared to the individual models, the dress-top joint model performs very similar or slightly better on “dress” and “top”, however, the dress-top joint model performs poorly in terms of retrieval accuracy for clothing items in the “jacket” vertex category.

[0094] Furthermore, the dress-top-jacket joint model results in significant accuracy drops on all three vertices. The accuracy data in Table 1 indicate that some vertices can be combined to achieve better accuracy than individually trained models, but only up to a point, after which the difficulty of model training with the triplet loss function results in a drop in accuracy (as described above).

[0095] The example process of the system 100, 200 may include combining different vertices of the training data. Specifically, nine clothing vertices may be combined into four groups, wherein a combined model is trained for each group. In some instances, more or less than nine clothing vertices may be combined into a specific number of groups based on user needs.

[0096]

[0097]

[0098] Table 2: Vertex combination groups

[0099] The four groups are shown in Table 2, and can have performance retrieval accuracy comparable to a model trained individually for each group. In some embodiments, "clean triplets" are used to fine-tune each of the four models, where the clean triplets are obtained from the "image search" triplets (described below). For example, the system 200 can be configured to use clean data to fine-tune model performance to achieve an effective improvement in retrieval accuracy for each of the four models.

[0100] Typically, by combining the training data, four embedding models can be obtained for nine different clothing vertices. Based on the above teachings, a unified model can be trained or learned for all nine vertices and then generated. In some embodiments, the generated model is deployed for operational use by multiple users. For example, the unified model can receive image data sent by a user, where the user seeks to obtain recognition data about a specific object or clothing item included in the image.

[0101] In some embodiments, the unified model is configured to receive image data of an image, identify or discern an item of apparel in the image, and determine identification information about the item of apparel in the image. In some instances, the unified model is configured to retrieve related images of the item of apparel, and provide identification information about the item of apparel and / or related images of the item of apparel for output to a user via a mobile device.

[0102] Figure 5 A diagram 500 is shown that includes a graphical representation of a corresponding embedding model of object vertices corresponding to a particular apparel category. In addition, the depiction of diagram 500 may correspond to a process that may be performed by system 100, 200 for extracting one or more feature embeddings. As described above, in the context of apparel recognition, an example two-stage method may be used to extract feature embeddings associated with image data of an item or apparel object.

[0103] As shown in diagram 500, for a given image / image data 502, at block 504, clothing items may first be detected and located in the image. At block 506, an embedding (e.g., a vector of floating point numbers) is then obtained from the cropped image to represent the clothing item. The system 100, 200 uses the embedding to compare similarity images for item / object retrieval. In some embodiments, the embedding obtained from the cropped image may correspond to a learning target for the clothing vertex to which the clothing item belongs.

[0104] As indicated by corresponding arrows 507, each embedding model 508, 510, 512, and 514 corresponds to an object retrieval function for identifying and retrieving certain items of apparel. An item of apparel may correspond to an item depicted in a particular cropped portion of the image data. In contrast, arrow 515 indicates that unified embedding model 516 corresponds to an object retrieval function for identifying and retrieving an item of apparel for an object depicted in each cropped portion of the image data in block 506.

[0105] Block 506 includes corresponding cropped images, each cropped image including a representation of a clothing or apparel item of the object vertex category of the embedding model. For example, the first cropped image data depicting a handbag corresponds to the embedding model 510 for identifying and retrieving image data of a handbag. Similarly, the second cropped image data depicting pants corresponds to the embedding model 512 for identifying and retrieving image data of pants. However, each cropped image data at block 506 corresponds to a unified embedding model 516 for identifying and retrieving image data of various types of apparel items.

[0106] Figure 6 An example diagram 600 is shown including computing functions for obtaining image data for training one or more machine learning models. Diagram 600 may correspond to computing functions that may be performed by one or more computing modules of the above-described system 100 or 200.

[0107] At logic block 602, training data related to images is first collected from search queries. The search queries may be accessed from an example search system that receives and stores a large volume of image search queries. For example, system 100 may be configured to access a query data storage device, such as a private query repository that stores thousands (e.g., 200,000) of user queries submitted using Google Image Search. The search query may include a specified product or apparel item name.

[0108] At block 604, the system 100 executes parsing logic for the example query text parser to obtain clothing category labels for each text query obtained from the query repository. The text query can be associated with nine different vertices: i) dresses, ii) tops, iii) footwear, iv) handbags, v) glasses, vi) coats, vii) skirts, viii) shorts, and ix) pants. At block 606, the system 100 selects a specific number of top-rated images (e.g., 30 images) for each search query, where the images are rated based on image pixel quality and the degree to which the query text data accurately describes the clothing item of the image.

[0109] The image data of the highest rated images can be used to form the above-mentioned "image search triplets", where the triplets include positive images, negative images, and anchor images. The system 100 can identify at least a subset of the triplets (e.g., 20,000 triplets per object vertex) for system or user rating and verification of the correctness of each image in the triplet. In some embodiments, the rating and image verification includes determining whether the anchor image and the positive image in the triplet are from the same product / vertex category. The subset of triplet images that are rated and verified as correct can be used to form a second set of triplets, referred to herein as "clean triplets".

[0110] At logic block 608, prior to generating the unified model, the base network 103 may be initialized from a model pre-trained using one or more types of image data (e.g., ImageNet data). In some embodiments, when generating the unified embedding model, the unified embedding model may be learned using the same training data as the triple feature learning used for the two or more specialized models. For example, generating the unified embedding learning model may require only vertex label data, which may be obtained via a locator / classifier as described above. Thus, the unified embedding learning may be generated using the same training images as those generated during the triple embedding learning.

[0111] Embodiments of the subject matter and operations described in this specification may be implemented in digital electronic circuitry, or in computer software, firmware, or hardware including the structures disclosed in this specification and their structural equivalents, or in a combination of one or more of them. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on computer storage media for execution by, or to control the operation of, data processing apparatus.

[0112] Alternatively or additionally, the program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device for execution by a data processing device. A computer storage medium may be a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them or included therein. In addition, although a computer storage medium is not a propagation signal, a computer storage medium may be a source or destination of computer program instructions encoded in an artificially generated propagation signal. A computer storage medium may also be one or more independent physical components or media (e.g., multiple CDs, disks or other storage devices) or included therein.

[0113] The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0114] The term "data processing apparatus" covers all kinds of apparatus, devices and machines for processing data, including, for example, a programmable processor, a computer, a system on a chip, or a plurality or combination of the above. The apparatus may include a dedicated logic circuit, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit). In addition to hardware, the apparatus may also include code that creates an operating environment for the computer program in question, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of them. The apparatus and the operating environment may implement a variety of different computing model infrastructures, such as network services, distributed computing, and grid computing infrastructures.

[0115] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store portions of one or more modules, subroutines, or code). A computer program may be deployed to execute on multiple computers or one computer located at one site or distributed across multiple sites and interconnected by a communications network.

[0116] The processes and logic flows described in this specification can be performed by one or more programmable processors running one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuits, and the apparatus can also be implemented as special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).

[0117] Processors suitable for running a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, a processor will receive instructions and data from a read-only memory or a random access memory, or both. The essential elements of a computer are a processor for performing actions in accordance with instructions and one or more memory devices for storing instructions and data.

[0118] Typically, the computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or be operably coupled to receive data from one or more mass storage devices or transfer data to it or both. However, the computer need not have such a device. In addition, the computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (such as a universal serial bus (USB) flash drive), just to name a few examples.

[0119] Devices suitable for storing computer program instructions and data include all forms of nonvolatile memory, media and storage devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0120] To provide interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computer having a display device (e.g., a CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer.

[0121] Other types of devices may also be used to provide interaction with a user; for example, feedback provided to a user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from a user may be received in any form, including sound, voice, or tactile input. In addition, a computer may interact with a user by sending documents to and receiving documents from a device used by a user; for example, by sending a web page to a web browser on a user device of a user in response to a request received from the web browser.

[0122] Embodiments of the subject matter described in this specification may be implemented in a computing system that includes a back-end component, such as a data server, or includes a middleware component, such as an application server, or includes a front-end component, such as a user computer with a graphical user interface or a web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (Local Area Networks, "LANs") and wide area networks (Wide Area Networks, "WANs"), interconnected networks (e.g., the Internet), and peer-to-peer networks (e.g., self-organizing peer-to-peer networks).

[0123] The computing system may include a user and a server. The user and the server are generally remote from each other and usually interact through a communication network. The relationship between the user and the server is generated by a computer program that runs on a corresponding computer and has a user-server relationship with each other. In some embodiments, the server sends data (e.g., an HTML page) to the user device (e.g., for the purpose of displaying data to the user interacting with the user device and receiving user input from the user interacting with the user device). Data generated at the user device (e.g., the result of user interaction) can be received from the user device at the server.

[0124] Although this specification contains many specific implementation details, these should not be construed as limitations on any invention or the scope of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment.

[0125] Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. In addition, although features may be described above as functioning in certain combinations, or even initially claimed as such, in some cases one or more features in the combination may be deleted from the claimed combination, and the claimed combination may be directed to a subcombination or variation of the subcombination.

[0126] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that the operations be performed in the particular order shown or in a continuous order, or that all of the operations shown be performed to obtain the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated into a single software product or packaged into multiple software products.

[0127] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing may be advantageous.

Claims

1. A computer-implemented method for generating a unified machine learning computational model using a neural network on a data processing device, the method comprising: Determining, by a data processing device and for a neural network, a corresponding learning objective for each of a plurality of object vertices, wherein each object vertex defines a different category of an object belonging to the vertex, and wherein each learning objective is based on two or more embedding outputs of at least one other neural network; wherein determining the corresponding learning objective for the neural network further comprises: training, by the data processing device and based on a second loss function, a plurality of specialized models, wherein each of the plurality of specialized models is trained to recognize a different object vertex; and generating, by a data processing device, one or more embedding outputs using each of the plurality of specialized models, wherein a corresponding learning objective is determined based on the embedding outputs; training, by a data processing device and based on a first loss function, a neural network to identify data associated with each of the plurality of object vertices, wherein the neural network is trained using a respective learning objective; and generating, by a data processing device and using a neural network trained based on a first loss function, a unified machine learning model configured to identify an item included in the data associated with each of the plurality of object vertices, The first loss function is an L2 loss function, and generating the unified machine learning model includes: generating a specific unified machine learning model that minimizes a computational output associated with the L2 loss function, Therein, multiple dedicated models are first trained based on a second loss function to achieve a desired threshold accuracy level for distinguishing objects included in image data, and then the embedding output of each of the trained dedicated models is used as a learning target to train a unified machine learning model.

2. The method according to claim 1, wherein: Determining a corresponding learning goal for the neural network also includes: training, by the data processing device and based on a second loss function, the at least one other neural network to identify data associated with each of the plurality of object vertices; generating, by the data processing device, the two or more embedding outputs in response to the training, wherein each embedding output indicates a particular learning objective and includes a vector of parameters corresponding to data associated with a particular object vertex; and A corresponding machine learning model is generated by the data processing device and using the at least one other neural network trained based on the second loss function, each machine learning model is configured to use a specific embedding output.

3. The method according to claim 2, wherein: Determining a corresponding learning goal for the neural network also includes: Providing corresponding learning objectives generated from corresponding independent models for training the neural network.

4. The method according to claim 2, wherein: Each of the plurality of object vertices corresponds to a particular category of items, and the data associated with each of the plurality of object vertices includes image data of an item in the particular category of items.

5. The method according to claim 4, wherein: The specific category is an apparel category, and the items of the specific category include at least one of: a handbag, a shoe, a dress, a trousers, or a coat; and The image data indicates an image of at least one of: a specific handbag, a specific pair of shoes, a specific dress, a specific pair of pants, or a specific coat.

6. The method according to claim 5, wherein: Each of the respective machine learning models is configured to identify data associated with a particular object vertex within a first degree of accuracy; and The unified machine learning model is configured to identify data associated with each of the plurality of object vertices within a second accuracy that exceeds the first accuracy.

7. The method according to claim 2, wherein: Determining a corresponding learning objective for each of the plurality of object vertices includes: analyzing the two or more embedding outputs, each embedding output corresponding to a particular object vertex among the plurality of object vertices; and Based on the analysis, a corresponding learning objective is determined for each of the plurality of object vertices.

8. The method according to claim 2, wherein: The neural network includes a plurality of neural network layers receiving a plurality of layer inputs, and wherein training the neural network based on the first loss function includes: Performing batch normalization to normalize layer inputs to a specific neural network layer; and In response to performing batch normalization, covariate shift is minimized.

9. The method according to claim 2, wherein: The second loss function is a triplet loss function, and generating the corresponding machine learning model includes: A specific machine learning model is generated based on the associations among the anchor image, the positive image, and the negative image.

10. The method according to claim 1, wherein: The data associated with each of the plurality of object vertices includes image data.

11. A system for generating a unified machine learning computational model using a neural network, the system comprising: A data processing apparatus for a neural network, the data processing apparatus comprising one or more processing devices; as well as One or more non-transitory machine-readable storage devices for storing instructions executable by the one or more processing devices to cause performance of operations including a method according to any preceding claim.

12. One or more non-transitory machine-readable storage devices storing instructions executable by one or more processing devices to cause execution of operations including the method according to any one of claims 1 to 10.

13. A computer-implemented method for generating a unified machine learning computational model using a neural network on a data processing device, the method comprising: Determining, by a data processing device and for a neural network, a corresponding learning objective for each of a plurality of object vertices, wherein each object vertex defines a different category of an object belonging to the vertex, and wherein each learning objective is based on two or more embedding outputs of at least one other neural network; wherein determining the corresponding learning objective for the neural network further comprises: training, by the data processing device and based on a second loss function, a plurality of specialized models, wherein each of the plurality of specialized models is trained to recognize a different object vertex; and generating, by a data processing device, one or more embedding outputs using each of the plurality of specialized models, wherein a corresponding learning objective is determined based on the embedding outputs; training, by a data processing device and based on a first loss function, a neural network to identify data associated with each of the plurality of object vertices, wherein the neural network is trained using a respective learning objective; and generating, by a data processing device and using a neural network trained based on a first loss function, a unified machine learning model configured to identify an item included in the data associated with each of the plurality of object vertices, The first loss function is an L2 loss function, and generating the unified machine learning model includes: generating a specific unified machine learning model that minimizes a computational output associated with the L2 loss function, Therein, multiple dedicated models are first trained based on a second loss function to achieve a desired threshold accuracy level for distinguishing objects included in image data, and then the embedding output of each of the trained dedicated models is used as a learning target to train a unified machine learning model.

14. The method according to claim 13, wherein: Determining a corresponding learning goal for the neural network also includes: training, by the data processing device and based on a second loss function, the at least one other neural network to identify data associated with each of the plurality of object vertices; generating, by the data processing device, the two or more embedding outputs in response to the training, wherein each embedding output indicates a particular learning objective and includes a vector of parameters corresponding to data associated with a particular object vertex; and A corresponding machine learning model is generated by the data processing device and using the at least one other neural network trained based on the second loss function, each machine learning model is configured to use a specific embedding output.

15. The method according to claim 14, wherein: Determining a corresponding learning goal for the neural network also includes: Providing corresponding learning objectives generated from corresponding independent models for training the neural network.

16. The method according to claim 14, wherein: Each of the plurality of object vertices corresponds to a particular category of items, and the data associated with each of the plurality of object vertices includes image data of an item in the particular category of items.

17. The method according to claim 16, wherein: The specific category is an apparel category, and the items of the specific category include at least one of: a handbag, a shoe, a dress, a trousers, or a coat; and The image data indicates an image of at least one of: a specific handbag, a specific pair of shoes, a specific dress, a specific pair of pants, or a specific coat.

18. The method of claim 17, wherein: Each of the respective machine learning models is configured to identify data associated with a particular object vertex within a first accuracy; as well as The unified machine learning model is configured to identify data associated with each of the plurality of object vertices within a second accuracy that exceeds the first accuracy.

19. The method according to claim 14, wherein: Determining a corresponding learning objective for each of the plurality of object vertices includes: analyzing the two or more embedding outputs, each embedding output corresponding to a particular object vertex among the plurality of object vertices; and Based on the analysis, a corresponding learning objective is determined for each of the plurality of object vertices.

20. The method according to claim 14, wherein: The neural network includes a plurality of neural network layers receiving a plurality of layer inputs, and wherein training the neural network based on the first loss function includes: Performing batch normalization to normalize layer inputs to a specific neural network layer; and In response to performing batch normalization, covariate shift is minimized.

21. The method according to claim 14, wherein: The second loss function is a triplet loss function, and generating the corresponding machine learning model includes: A specific machine learning model is generated based on the associations among the anchor image, the positive image, and the negative image.

22. A system for generating a unified machine learning computational model using a neural network, the system comprising: A data processing apparatus for a neural network, the data processing apparatus comprising one or more processing devices; as well as One or more non-transitory machine-readable storage devices storing instructions executable by the one or more processing devices to cause operations to be performed, the operations comprising: Determining, by a data processing device and for a neural network, a corresponding learning objective for each of a plurality of object vertices, wherein each object vertex defines a different category of an object belonging to the vertex, and wherein each learning objective is based on two or more embedding outputs of at least one other neural network; wherein determining the corresponding learning objective for the neural network further comprises: training, by the data processing device and based on a second loss function, a plurality of specialized models, wherein each of the plurality of specialized models is trained to recognize a different object vertex; and generating, by a data processing device, one or more embedding outputs using each of the plurality of specialized models, wherein a corresponding learning objective is determined based on the embedding outputs; training, by a data processing device and based on a first loss function, a neural network to identify data associated with each of the plurality of object vertices, wherein the neural network is trained using a respective learning objective; and generating, by a data processing device and using a neural network trained based on a first loss function, a unified machine learning model configured to identify an item included in the data associated with each of the plurality of object vertices, The first loss function is an L2 loss function, and generating the unified machine learning model includes: generating a specific unified machine learning model that minimizes a computational output associated with the L2 loss function, Therein, multiple dedicated models are first trained based on a second loss function to achieve a desired threshold accuracy level for distinguishing objects included in image data, and then the embedding output of each of the trained dedicated models is used as a learning target to train a unified machine learning model.

23. The system of claim 22, wherein: Determining a corresponding learning goal for the neural network also includes: training, by the data processing device and based on a second loss function, the at least one other neural network to identify data associated with each of the plurality of object vertices; generating, by the data processing device, the two or more embedding outputs in response to the training, wherein each embedding output indicates a particular learning objective and includes a vector of parameters corresponding to data associated with a particular object vertex; and A corresponding machine learning model is generated by the data processing device and using the at least one other neural network trained based on the second loss function, each machine learning model is configured to use a specific embedding output.

24. The system of claim 23, wherein: Each of the plurality of object vertices corresponds to a particular category of items, and the data associated with each of the plurality of object vertices includes image data of an item in the particular category of items.

25. The system of claim 24, wherein: The specific category is an apparel category, and the items of the specific category include at least one of: a handbag, a shoe, a dress, a trousers, or a coat; and The image data indicates an image of at least one of: a specific handbag, a specific pair of shoes, a specific dress, a specific pair of pants, or a specific coat.

26. The system of claim 25, wherein: Each of the respective machine learning models is configured to identify data associated with a particular object vertex within a first degree of accuracy; and The unified machine learning model is configured to identify data associated with each of the plurality of object vertices within a second accuracy that exceeds the first accuracy.

27. The system of claim 23, wherein: Determining a corresponding learning objective for each of the plurality of object vertices includes: analyzing the two or more embedding outputs, each embedding output corresponding to a particular object vertex among the plurality of object vertices; and Based on the analysis, a corresponding learning objective is determined for each of the plurality of object vertices.

28. The system of claim 23, wherein: The neural network includes a plurality of neural network layers receiving a plurality of layer inputs, and wherein training the neural network based on the first loss function includes: Performing batch normalization to normalize layer inputs to a specific neural network layer; and In response to performing batch normalization, covariate shift is minimized.

29. The system of claim 23, wherein: The second loss function is a triplet loss function, and generating the corresponding machine learning model includes: A specific machine learning model is generated based on the associations among the anchor image, the positive image, and the negative image.

30. One or more non-transitory machine-readable storage devices storing instructions executable by one or more processing devices to cause operations to be performed, the operations comprising: Determining, by a data processing device and for a neural network, a corresponding learning objective for each of a plurality of object vertices, wherein each object vertex defines a different category of an object belonging to the vertex, and wherein each learning objective is based on two or more embedding outputs of at least one other neural network; wherein determining the corresponding learning objective for the neural network further comprises: training, by the data processing device and based on a second loss function, a plurality of specialized models, wherein each of the plurality of specialized models is trained to recognize a different object vertex; and generating, by a data processing device, one or more embedding outputs using each of the plurality of specialized models, wherein a corresponding learning objective is determined based on the embedding outputs; training, by a data processing device and based on a first loss function, a neural network to identify data associated with each of the plurality of object vertices, wherein the neural network is trained using a respective learning objective; and generating, by a data processing device and using a neural network trained based on a first loss function, a unified machine learning model configured to identify an item included in the data associated with each of the plurality of object vertices, The first loss function is an L2 loss function, and generating the unified machine learning model includes: generating a specific unified machine learning model that minimizes a computational output associated with the L2 loss function, Therein, multiple dedicated models are first trained based on a second loss function to achieve a desired threshold accuracy level for distinguishing objects included in image data, and then the embedding output of each of the trained dedicated models is used as a learning target to train a unified machine learning model.

Images