A clothing compatibility prediction method based on multi-layer masked Transformer

Abstract

The present invention relates to a clothing compatibility prediction method based on a multi-layer masked Transformer. The method comprises the following steps: collecting a clothing dataset; constructing an MLMT network model, using multi-layer masks to display the stylistic features of individual items based on hierarchical comparison relationships; fusing the overall features of the outfits through a Transformer encoder; and determining the clothing compatibility score using a multi-layer perceptron. By calculating correlations using a multi-layer masked model and extracting global features through Transformer encoding, the method can achieve more accurate compatibility prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of clothing compatibility prediction, and in particular to a clothing compatibility prediction method based on a multi-layer masked Transformer. Background Art

[0002] Clothing recommendation is a key application area in computer vision, where clothing compatibility plays a crucial role. Different clothing combinations create distinct visual perceptions, and a well-matched outfit can enhance a user's appearance. Early compatibility research focused on learning pairwise representations between items or end-to-end representations. However, this did not consider the complex relationships between multiple items. Specifically, each item forms a complex network of relationships between low-level features such as color and texture, and higher-level features such as style and pattern. In recent years, sequence-based compatibility prediction has been employed. However, the fixed serialization of items is not rational, as there is no fixed order for items, and clothing compatibility is not solely about the relationship between adjacent items. Therefore, predicting fashion compatibility remains a significant challenge for fashion recommendation.

[0003] Based on the hierarchical structure of convolutional neural networks, low-level and high-level features of clothing can be obtained separately. Low-level features have higher resolution and contain more positional and detailed information. However, they are less semantic and more noisy. High-level features have stronger semantic information, but lower resolution and poorer ability to perceive details. For clothing images, low-level features representing clothing color and material come from the lower layers of the convolutional neural network, while high-level features representing clothing category and style come from the higher layers of the network. Compatibility learning is subjective, so internal correlations between clothing items can be found based on the user's previous purchase history, expert recommendations, and various clothing characteristics.

[0004] Patent application number CN202210296195.3 proposes a hypergraph-based clothing compatibility prediction method. By introducing a hypergraph, it effectively reflects the complex relationship between suits and clothing items. At the same time, the hypergraph is converted into a simple graph to obtain a simple graph-enhanced network model. The graph node representation in the simple graph structure is iteratively updated using a message propagation mechanism, and the final representation of the graph nodes is obtained through a gated loop. The model's predictive ability is enhanced through an attention mechanism, and the final representation of the graph nodes is used to calculate the compatibility score of the suit. However, the patent has the problem that, due to the complex relationship between clothing items, it is impossible to accurately judge compatibility and cannot determine the correlation between two covered items. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to address the above shortcomings and provide a clothing compatibility prediction method based on a multi-layer masked Transformer, which aims to solve the problems of complex relationships between clothing, the inability to accurately judge compatibility, and the inability to judge the correlation between two masked items.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A clothing compatibility prediction method based on a multi-layer masked Transformer includes the following steps:

[0008] Step S1, collecting clothing dataset;

[0009] Step S2: Input the clothing image and description text data of the individual clothing items in the clothing dataset into the visual semantic embedding module, establish the association between the clothing image and the description text, and determine whether the clothing image and the description text match. If they match, the data is input into the multi-layer mask module in step S3; if not, the data returns to the visual semantic embedding module to continue matching;

[0010] Step S3: Input all items in the clothing dataset whose clothing images and description texts match into the multi-layer mask module to calculate the correlation between the items;

[0011] Step S4: input multiple clothing sets in the clothing dataset into the clothing compatibility prediction model to train the model, thereby obtaining a trained clothing compatibility prediction model;

[0012] Step S5: Use the trained clothing compatibility prediction model to predict compatibility.

[0013] Furthermore, the method of determining whether the image and text match by the visual semantic embedding module in step S2 includes the following steps S2A to S2C:

[0014] Step S2A, extracting clothing image features of individual items in the clothing dataset through the Resnet-50 convolutional module;

[0015] Step S2B: extract the descriptive text features of all items in the clothing dataset through the one-hot encoding module;

[0016] Step S2C: input the clothing image features and the description text features into the same vector space, establish the correlation between the clothing image features and the description text features, and determine whether the clothing image and the description text match based on the distance between the clothing image features and the description text features in the vector space.

[0017] Furthermore, the step S2C includes the following steps S2C-1 to S2C-3:

[0018] Step S2C-1: I represents an image of a single item of clothing, and T represents the description text describing the single item. The calculation formula for the set of clothing images and description texts is as follows:

[0019]

[0020] Among them, S is a collection of clothing images and description texts, I is a clothing image of a single item, T is the description text of the single item, (I n , T n ) is directly opposite, (I n , T m≠n ) is a negative pair, n is the number, and N is the total number;

[0021] Step S2C-2, set is the clothing image feature vector based on clothing image I, is the description text feature vector of the description text T, where and θ φ The model parameters representing the corresponding mapping of clothing images and description texts are mapped to the same dimensional space through linear projection. The calculation formula is as follows:

[0022]

[0023]

[0024] Among them, I is a clothing image of a single item, T is the description text of the single item, f and g are the feature spaces of clothing image and description text respectively, (I n , T n ) is directly opposite, (I n , T m≠n ) is a negative pair, n is the number, W f and The model parameters representing the corresponding mapping of clothing images, W g and θ φ Model parameters representing the corresponding mapping of description text;

[0025] Step S2C-3: defining a similarity function S(I, T) and a preset range value to determine whether the clothing image and the description text match; if the similarity function S(I, T) is within the preset range value, the clothing image and the description text match; if the similarity function S(I, T) is not within the preset range, the clothing image and the description text do not match;

[0026] Regulation and If they are of the same dimension, the similarity function in the embedding space is defined as the inner product representation, and its calculation formula is as follows:

[0027]

[0028] Among them, I is a clothing image of a single item, T is the description text of the single item, f and g are the feature spaces of clothing image and description text respectively, W f and The model parameters representing the corresponding mapping of clothing images, W g and θ φ Model parameters representing the corresponding mapping of description text.

[0029] Furthermore, the method for calculating the correlation between items by the multi-layer mask module in step S3 includes the following steps S3A to S3C:

[0030] Step S3A: Input any clothing set O and randomly cover the i-th item to obtain the context representation of this item, thereby obtaining the vector F a , and then randomly cover the jth item to obtain the context representation of this item, thereby obtaining the vector F b ;

[0031] Step S3B: Extract the global features of a and b through the Transformer encoder. The calculation formula is as follows:

[0032] G a =Encoder(F a ×Mask i )

[0033] G b =Encoder(F b ×Mask j )

[0034] Among them, G a represents a global feature, G b represents the global feature of b;

[0035] Step S3C, by calculating the correlation matrix R(G a , G b ) to judge the correlation between two items. The larger the R, the higher the correlation. The correlation R of the i-th item relative to the j-th item is calculated as follows:

[0036] R=G a ×G b

[0037] Among them, G a represents a global feature, G b Represents the global feature of b.

[0038] Furthermore, the step S4 includes the following steps:

[0039] Step S4A: input a set of clothing sets containing n items;

[0040] Step S4B: Let the set of features at the same level be Y = {y1, y2, ..., y n}, where y i is the eigenvector of the i-th item (i=1, 2…n). The correlation between any two items is expressed in matrix form, and its calculation formula is as follows:

[0041]

[0042] Among them, M is the comparison matrix, m ij represents y i and y j The correlation between (i = 1, 2...n) (j = 1, 2...n);

[0043] Step S4C: When comparing features at u different levels, there are u different comparison matrices (M 1 , M 2 ,...,M u}, get all the comparison relations H, the calculation formula is as follows:

[0044]

[0045] Among them, u represents the number of levels, and H represents the comparison relationship;

[0046] Step S4D: Calculate the compatibility score S using a three-layer perceptron and an activation function. The calculation formula is as follows:

[0047] S=ReLU(W2ReLU(W1ReLU[M 1 ;M 2 ;...;M u ]+b)+k))

[0048] Where S represents the compatibility score, ReLU represents the activation function, M is the comparison matrix, b represents the parameters of the second layer perceptron, k represents the parameters of the third layer perceptron, W2 and W1 represent the model parameters of the corresponding mapping between clothing images and description texts;

[0049] Step S4E: Use the Sigmoid function to map the output compatibility score S to between 0 and 1. The calculation formula is as follows:

[0050]

[0051] Binary cross entropy is used as the loss function, and its calculation formula is as follows:

[0052] l clf =y×logσ(S)+(1-y)×log(1-σ(S))

[0053] Among them, l clf Represents the loss value of binary cross entropy, Sigmoid(x) represents the activation function on the output unit, x represents the input clothing image and description text, and y represents the binary label 0 or 1;

[0054] Step S4F, by minimizing the loss value l clf The clothing compatibility prediction model is trained to obtain a trained clothing compatibility prediction model.

[0055] After adopting the above technical solution, the present invention has the following advantages compared with the prior art:

[0056] (1) The present invention proposes a Transformer encoder to extract the overall features of the entire clothing set, making the feature extraction more comprehensive and thus obtaining a more accurate compatibility judgment;

[0057] (2) Considering the complex relationship between clothing items, a hierarchical network is proposed to extract global features of all items in a suit at different levels, thus improving the accuracy of compatibility judgment;

[0058] (3) A new comparison method is introduced. By comparing the global features after covering any single item in turn with the global features after covering another single item, the correlation between the two covered items can be determined.

[0059] The present invention is described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a schematic diagram of the process of the present invention;

[0061] Figure 2 To search for the results of the fill-in-the-blank task of the present invention;

[0062] Figure 3 The MAP of each category of products is lacking under different methods of the present invention. DETAILED DESCRIPTION

[0063] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0064] Example 1:

[0065] like Figure 1 The clothing compatibility prediction method based on a multi-layer masked Transformer shown in FIG. 1 includes the following steps:

[0066] Step S1, collecting clothing dataset;

[0067] The present invention uses the Polyvore-T dataset, which is built on the Polyvore dataset and contains 12,000 expert-selected outfits. Each outfit has five categories to mark the information of each item, namely tops, bottoms, shoes, bags, and accessories. Each item has a corresponding image, text description, and category label (such as jeans, skirts, sports, a total of 381 categories). Items that do not contain any of the labels will be directly deleted, and the remaining items will be marked according to their categories. For missing items, the average value of other items is calculated as filler to ensure that each outfit has 5 items.

[0068] Step S2: Construct the MLMT network model. The MLMT model consists of three parts: visual semantic embedding module, multi-layer mask module, and clothing compatibility prediction module. Figure 1 The process of the entire network is shown. The specific processing process of each module is as follows:

[0069] In step S2A, a visual semantic embedding module is constructed to learn the common representation between the image and text corresponding to a single item, map the clothing image and text description into the same space, establish the correlation between the two, and judge whether there is a match between the image features and text features based on the distance in the vector space. If the match is successful, it will reach the multi-layer mask module; if the match is unsuccessful, it will return to the visual semantic embedding module to continue matching.

[0070] Let I represent an image of a clothing item and T represent a sentence describing the clothing item. The set of clothing images and descriptions is calculated as follows:

[0071]

[0072] Among them, S represents a set of clothing images and descriptions, I represents an image of a clothing item, and T represents a sentence describing the clothing item. (I n , T n ) is directly opposite, (I n , T m≠n ) is a negative pair, n is the number, and N is the total number.

[0073] set up is a feature-based representation computed from image I, is the semantic representation of the text T in the embedding space, where and θ φ The model parameters representing the corresponding mapping between image and text are mapped to the space of the same dimension by linear projection. The calculation formula is as follows:

[0074]

[0075]

[0076] Among them, I represents an image of a clothing item, T represents a sentence describing the clothing item, (I n , T n ) is a positive pair, f and g are image and text feature spaces respectively, (I n , T m≠n ) is a negative pair, n is the number, W f , W g , θ represents the model parameters of the corresponding mapping between image and text.

[0077] Define a similarity function S(I, T) to determine whether the image text pair matches, and specify and If they are of the same dimension, the similarity function in the embedding space is defined as the inner product representation, and its calculation formula is as follows:

[0078]

[0079] Among them, I represents an image of a clothing item, T represents a sentence describing the clothing item, f and g are image and text feature spaces respectively, and W f ,Wx, θ φ Model parameters representing the corresponding mapping between images and text.

[0080] Step S2B: Build a multi-layer mask module based on the visual semantic embedding module obtained above. Then, use the multi-layer mask model to perform hierarchical comparisons on the complex relationships between products, drawing on the idea of BERT's masked language model to handle the correlation between products.

[0081] Step S2B-1: Input an image of set O into the BERT model, randomly mask the i-th item, and obtain the contextual representation of this item, thereby obtaining the vector F a ;

[0082] Step S2B-2, then randomly mask the jth item at the same time to obtain the context representation of this item, thereby obtaining the vector F b , and then extract the global features of a and b through the encoder to get G a and G b ; G a =Encoder(F a ×Mask i ), G b=Encoder(F b ×Mask j );

[0083] Step S2B-3, by calculating the correlation matrix R(G a , G b ) to determine the correlation between the two. The larger R is, the greater the influence of j on item i is, and the higher the correlation is. The correlation R of each item is calculated as follows:

[0084] R=G a ×G b

[0085] Among them, G a and G b Represents the global features of a and b.

[0086] Step S2C constructs a clothing compatibility prediction module. Compatibility prediction involves inputting a set into the model and determining whether the set is compatible by comparing the correlations between the individual items. This can also be considered a feature fusion that compares each item with other items in terms of low-level features (color, material, texture) and high-level features (style). Compatibility prediction can be used for both fill-in-the-blank and diagnostic tasks. In the fill-in-the-blank task, a series of candidate items are provided, and one item needs to be selected from these candidate items to fill the blank. For example, if a user wants to choose a pair of bottoms that matches their clothes and shoes, they can make the correct choice in a relatively short time.

[0087] Diagnose incompatible items, which can automatically diagnose and correct outfits. This task can find incompatible items within an outfit (tops, bottoms, shoes, bags, accessories) and select the one with the highest score among the candidate items, which is the most compatible item.

[0088] Given a set of n items, let their color features, material features, texture features, or style features be represented as a set Y = {y1, y2, ..., y n}, where y i is the eigenvector of the i-th item. The pairwise similarity after masking any item can be expressed in matrix form, and its calculation formula is as follows:

[0089]

[0090] Among them, M is the comparison matrix, m ij represents y i and y j similarity between

[0091] For u different levels of comparison, there are u different comparison matrices {M1 , M 2 ,...,M u}, get all the comparison relations H, the calculation formula is as follows:

[0092]

[0093] Among them, u represents different items, and H represents the comparative relationship;

[0094] All vectors in the comparison matrix will be passed through a 3-layer perceptron and activation function to calculate the compatibility score S, which is calculated as follows:

[0095] S=ReLU(W2ReLU(W1ReLU[M 1 ;M 2 ;...;M u ]+b)+k))

[0096] Where S represents the compatibility score, ReLU represents the activation function, M is the comparison matrix, b represents the parameters of the second layer perceptron, k represents the parameters of the third layer perceptron, and W2 and W1 represent the model parameters of the corresponding mapping of images and texts.

[0097] When predicting and training the clothing compatibility prediction method based on the multi-layer mask Transformer, the Sigmoid function is used for prediction and training. The calculation formula is as follows:

[0098]

[0099] The output score is mapped to a range between 0 and 1, i.e., the compatibility probability, and is predicted using the clothing compatibility prediction module. Binary cross entropy is used as the loss function, which is calculated as follows:

[0100] l clf =y×logσ(S)+(1-y)×log(1-σ(S))

[0101] Among them, l clf Represents the loss value of binary cross entropy, Sigmoid(x) represents the activation function on the output unit, x represents the input image and text, and y represents the binary label 0 or 1.

[0102] Experimental process of the present invention:

[0103] All experiments were trained on a Tesla V100 GPU (16GB) using the PyTorch 1.9.0 framework in a CentOS 7 and Cuda 10.2 environment. Both image features and text features were 1000-dimensional feature vectors. The batch size was set to 16, the momentum of stochastic gradient descent (SGD) was 0.9, the learning rate was 0.5e-2, the multiplier for learning rate decay was 0.5, binary cross entropy was used as the loss function, and the number of iterations was 100.

[0104] The area under the ROC curve (AUC) is selected as shown in formula (1), where the accuracy (ACC) is calculated by formula (2), and the harmonic mean (F1) of precision and recall is calculated by formula (3) to evaluate the model performance.

[0105]

[0106] Among them, a and b represent the number of positive and negative samples respectively, and I is the indicator function.

[0107]

[0108]

[0109] Among them, TP is the number of correct positive example predictions; FP is the number of incorrect negative example predictions; TN is the number of correct negative example predictions; FN is the number of incorrect positive example predictions; N is the total number of samples.

[0110] like Figure 2 Figure 2 shows the query results for a fill-in-the-blank task. Given an incomplete outfit and four candidate items, the algorithm calculates the compatibility score of each candidate item with the outfit and sorts them from high to low. Comparing the outputs of the MLMT model and the MCN model shows that the proposed method has a certain advantage in accuracy.

[0111] We extracted 6 different types of items from the data set and conducted experiments under 5 different algorithms. In the absence of different types of items, the retrieval accuracy is as follows: Figure 3 As shown in Figure 2. Since different types of clothing have different influences on overall compatibility, the accuracy fluctuates, but the average compatibility prediction accuracy of the proposed method is still higher than that of other methods. Figure 3 The MAP results of missing items in each category under different methods are shown.

[0112] Method AUC ACC F1-Score Self-Attention 63.71％ 55.94％ 94.53％ GGNN 72.10％ 65.22％ 94.02％ Contact 81.17％ 74.92％ 98.35％ MCN 91.60％ 78.26％ 98.35％ MLMT(Ours) 91.52％ 79.60％ 97.89％

[0113] The experimental results compared with the above methods and several other baselines are shown in the table. It can be seen from the table that MLMT outperforms MCN and improves the ACC and F1 indicators by 1.34% and 6.1% respectively. The AUC indicator is also closest to the expected value of the entire test set, proving that the layered mask method can improve the accuracy of clothing compatibility prediction.

[0114] We also disabled different layers and the Transformer encoder, as shown in the table below, to see if the missing layer had any impact on the final prediction performance. As can be seen from the table, accuracy improved with increasing layers. However, accuracy decreased without the Transformer encoder, confirming its effectiveness in extracting features.

[0115] In summary, calculating correlations through a multi-layer mask model and extracting global features through Transformer encoding can make compatibility prediction more accurate.

[0116] Method AUC ACC F1-Score Layer 1 87.26％ 75.25％ 79.24％ Layer 1+2 88.06％ 74.41％ 78.13％ Layer 1+2+3 89.65％ 79.77％ 82.28％ Layer 1+2+3+4 90.69％ 80.07％ 83.12％ MLM 90.00％ 79.52％ 75.08％ MLMT 91.52％ 79.60％ 82.50％

[0117] The foregoing is an example of the best mode of carrying out the present invention. Any portion not described in detail herein is common knowledge within the skill of one of ordinary skill in the art. The scope of protection of the present invention is determined by the claims. Any equivalent transformation based on the technical teachings of the present invention is also within the scope of protection of the present invention.

Claims

1. A clothing compatibility prediction method based on multi-layer masked Transformer, characterized in that: The following steps are involved: Step S1, collecting clothing data sets; Step S2: Input the clothing image and description text data of the individual clothing items in the clothing dataset into the visual semantic embedding module, establish the association between the clothing image and the description text, and determine whether the clothing image and the description text match. If they match, the data is input into the multi-layer mask module in step S3; if not, the data returns to the visual semantic embedding module to continue matching; Step S3: Input all items in the clothing dataset whose clothing images and description texts match into the multi-layer mask module to calculate the correlation between the items; The method for calculating the correlation between items by the multi-layer mask module in step S3 includes the following steps S3A to S3C: Step S3A: Input any clothing set O and randomly cover the i-th item to obtain the context representation of this item, thereby obtaining the vector F a , and then randomly cover the jth item to obtain the context representation of this item, thereby obtaining the vector F b ; Step S3B: Extract the global features of a and b through the Transformer encoder. The calculation formula is as follows: G a =Encoder(F a ×Mask i ) G b =Encoder(F b ×Mask j ) Among them, G a represents a global feature, G b represents the global feature of b; Step S3C, by calculating the correlation matrix R(G a ,G b ) to judge the correlation between two items. The larger the R, the higher the correlation. The correlation R of the i-th item relative to the j-th item is calculated as follows: R=G a ×G b Among them, G a represents a global feature, G b Represents b global feature; Step S4, multiple clothing sets in the clothing data set are input into the clothing compatibility prediction model to train the model to obtain a trained clothing compatibility prediction model; Step S5: Use the trained clothing compatibility prediction model to predict compatibility.

2. The clothing compatibility prediction method based on multi-layer masked Transformer according to claim 1 is characterized in that: The method for determining whether the image and text match by the visual semantic embedding module in step S2 includes the following steps S2A to S2C: Step S2A, extracting clothing image features of individual items in the clothing dataset through the Resnet-50 convolutional module; Step S2B: extract the descriptive text features of all items in the clothing dataset through the one-hot encoding module; Step S2C: input the clothing image features and the description text features into the same vector space, establish the correlation between the clothing image features and the description text features, and determine whether the clothing image and the description text match based on the distance between the clothing image features and the description text features in the vector space.

3. The clothing compatibility prediction method based on multi-layer masked Transformer according to claim 2 is characterized in that: The step S2C includes the following steps S2C-1 to S2C-3: Step S2C-1: I represents an image of a single item of clothing, and T represents the description text describing the single item. The calculation formula for the set of clothing images and description texts is as follows: Among them, S is a collection of clothing images and description texts, I is a clothing image of a single item, T is the description text of the single item, (I n ,T n ) is directly opposite, (I n ,T m≠n ) is a negative pair, n is the number, and N is the total number; Step S2C-2, set is the clothing image feature vector based on clothing image I, φ(T;θ φ ) is the description text feature vector of the description text T, where and θ φ The model parameters representing the corresponding mapping of clothing images and description texts are mapped to the same dimensional space through linear projection. The calculation formula is as follows: Among them, I is a clothing image of a single item, T is the description text of the single item, f and g are the feature spaces of clothing image and description text respectively, (I n ,T n ) is directly opposite, (I n ,T m≠n ) is a negative pair, n is the number, W f and The model parameters representing the corresponding mapping of clothing images, W g and θ φ Model parameters representing the corresponding mapping of description text; Step S2C-3: defining a similarity function S(I, T) and a preset range value to determine whether the clothing image and the description text match; if the similarity function S(I, T) is within the preset range value, the clothing image and the description text match; if the similarity function S(I, T) is not within the preset range, the clothing image and the description text do not match; Regulation and If they are of the same dimension, the similarity function in the embedding space is defined as the inner product representation, and its calculation formula is as follows: Among them, I is a clothing image of a single item, T is the description text of the single item, f and g are the feature spaces of clothing image and description text respectively, W f and The model parameters representing the corresponding mapping of clothing images, W g and θ φ Model parameters representing the corresponding mapping of description text.

4. The clothing compatibility prediction method based on multi-layer masked Transformer according to claim 1 is characterized in that Step S4 The following steps are included Step S4A: input a set of clothing sets containing n items; Step S4B: Let the set of features at the same level be Y = {y1, y2, ..., y n }, where y i is the eigenvector of the i-th item (i=1, 2...n). The correlation between any two items is expressed in matrix form, and its calculation formula is as follows: Among them, M is the comparison matrix, m ij represents y i and y j Correlation between (i=1, 2···n) (j = 1, 2···n); Step S4C: When comparing features at u different levels, there are u different comparison matrices {M 1 ,M 2 ,...,M u }, get all the comparison relations H, the calculation formula is as follows: Among them, u represents the number of levels, and H represents the comparison relationship; Step S4D: Calculate the compatibility score S using a three-layer perceptron and an activation function. The calculation formula is as follows: S=ReLU(W2ReLU(W1ReLU[M 1 ;M 2 ;…;M u ]+b)+k)) Where S represents the compatibility score, ReLU represents the activation function, M is the comparison matrix, b represents the parameters of the second layer perceptron, k represents the parameters of the third layer perceptron, W2 and W1 represent the model parameters of the corresponding mapping between clothing images and description texts; Step S4E: Use the Sigmoid function to map the output compatibility score S to between 0 and 1. The calculation formula is as follows: Binary cross entropy is used as the loss function, and its calculation formula is as follows: l clf =y×logσ(S)+(1-y)×log(1-σ(S)) Among them, l clf Represents the loss value of binary cross entropy, Sigmoid(x) represents the activation function on the output unit, x represents the input clothing image and description text, and y represents the binary label 0 or 1; Step S4F, by minimizing the loss value l clf The clothing compatibility prediction model is trained to obtain a trained clothing compatibility prediction model.

Images