Multi-modal personalized clothing recommendation method and system based on single-item compatibility

Abstract

The invention provides a multi-modal personalized clothing recommendation method and system based on single-item compatibility. The method comprises the steps of 1, performing modeling representation on clothing and a user by using a multi-head self-attention mechanism; 2, by means of existing single item matching in historical data and in combination with position codes of different types of single items, modeling of the matching degree between the user and the different types of single items is completed; and step 3, based on the interaction history of the user and a plurality of single items, using the user-single item interaction diagram to carry out collaborative position coding so as to enhance the representation of user behavior characteristics, and combining the position codes with the modal characteristics of the interest group corresponding to the user to complete the modeling of the user preference degree so as to improve the user preference degree. And diversified recommendation is carried out in combination with collaborative information between the user and the single item. According to the invention, learning of single item features is introduced, so that a more personalized and flexible recommendation strategy is provided, and diversified requirements of the user on clothing matching are met.

Description

Technical Field

[0001] The present invention relates to the technical field of clothing recommendation, and in particular to a multi-modal personalized clothing recommendation method and system based on item compatibility. Background Art

[0002] With the rapid development of the fashion industry, the clothing market is becoming more personalized and diversified, and consumers' demand for clothing recommendation systems is increasing. Traditional recommendation methods are mostly based on complete sets of clothing, ignoring the matching relationship between items and the personalized needs of users. Therefore, an innovative recommendation strategy is urgently needed to achieve more accurate and personalized clothing recommendations by comprehensively considering the matching degree between users and items and the correlation between items, so as to better meet the diverse needs of modern consumers.

[0003] However, existing recommendation systems face many challenges in the feature mining stage, especially in the implementation of diversified recommendations. First, the design of many systems only supports a single recommendation mode and cannot flexibly adapt to the diverse needs of users. Specifically, recommendation systems can often only recommend sets of clothing or single items, and cannot simultaneously support input of user-recommended sets, user-recommended items, and combinations of users and items to recommend other types of items. This limitation limits the accuracy and personalization of recommendation results.

[0004] In addition, ensuring the recommendation effect is also a key challenge. In diversified recommendations, how to ensure that the system can still provide high-quality recommendation results while supporting multiple input modes is an important consideration in the design. Summary of the invention

[0005] In view of the defects in the prior art, the object of the present invention is to provide a multi-modal personalized clothing recommendation method and system based on item compatibility.

[0006] The multimodal personalized clothing recommendation method based on item compatibility provided by the present invention includes:

[0007] Step 1: Use a multi-head self-attention mechanism to model and represent clothing and users respectively. Different types of items in a clothing set are split and feature models are established separately. For each type of item, a modality-aware encoder is used to independently model various modalities to obtain multimodal fusion features of the item.

[0008] Step 2: Use the existing product combinations in historical data and combine them with the position codes of different types of products to complete the modeling of the matching degree between users and different types of products;

[0009] Step 3: Based on the interaction history between users and multiple items, the user-item interaction graph is used for collaborative position encoding to enhance the representation of user behavior characteristics. The position encoding is combined with the modal characteristics of the user's corresponding interest group to complete the modeling of user preferences. Diversified recommendations are made based on the collaborative information between users and items.

[0010] Preferably, the step 1 comprises:

[0011] Given a set of clothing items, it is represented as a set of modality-aware features In the formula, represents the original feature representation of the i-th item in the clothing in mode m, d m represents the feature dimension of a single item on the m mode; n is the number of items in the clothing;

[0012] The text and visual modality features of each item are represented as: and where t i and v i are the text and visual samples of a single product, respectively; Bert(·) and ResNet(·) are the language representation function and the deep residual network function, respectively;

[0013] Calculate the self-attention output of clothing modality-aware features with multi-head attention, the expression is:

[0014] H m(l) =MultiHeadAtt(H m(l-1) ,H m(1-1) ,H m(l-1) )

[0015]

[0016] Among them, MultiHeadAtt(·) is the multi-head attention function, H m(l) is the output of the lth layer in modality m; and is a trainable parameter in the modality m, LayerNorm(·) represents the layer normalization operation; stacking L self-attention blocks, we get the enhanced representation H of the input set m(L) , and aggregate them into an overall representation of the suit using a fully connected layer:

[0017]

[0018] Among them, LeakyReLU(·) is a nonlinear activation function, is the embedding representation of set o after the L-th multi-head self-attention block, and are trainable parameters in modality m.

[0019] Preferably, step 2 comprises:

[0020] Using the historical data given in the set of single product matching information history interaction to build a graph G = {(i1,..i n )|,i n ∈I n}, where the feature vectors of all items i in a set act as ID embeddings in the matching behavior space, where the edges connecting all different types of items represent the interactive information in the historical data;

[0021] Using the GCN algorithm, information is propagated and local position features are extracted from the graph as collaborative position encoding signals, expressed as:

[0022]

[0023] Among them, the collaborative behavior characteristics represents the l-hop neighbors of clothing item i, set N i and N u Represents the neighbor nodes of set i∈I and user u∈U in the user-item interaction graph G;

[0024] Multiple multi-head attention modules are used to model the matching degree of individual items. The previously obtained suit modality features and collaborative position encoding are integrated layer by layer. For each input clothing combination, all interactions generated by the individual items are used as input and initialized using residual network units. The single product matching preference model is as follows:

[0025]

[0026] Among them, Attention(·) is the attention mechanism function, softmax(·) is the activation function; Q, K, V are the query vector, key vector and value vector respectively; T is the transposition symbol; d k is the dimension of the key-value vector; is the state vector of the residual network of item i in layer l; w3 and b3 are trainable parameters; represents the collaborative position code of item i, represents the overall representation of the n historical items that user u has interacted with in modality m, that is,

[0027] After L multi-head attention modules, we get:

[0028]

[0029] Among them, among them, Represents the matching characteristics of product k in modality m; w6, w7, b6, and b7 are all trainable parameters; calculate the matching degree of other types of different products i with product k in each modality m:

[0030]

[0031] in, Indicates the matching characteristics of candidate items in other categories;

[0032] The scores for all modalities are summed to get the final predicted preference score:

[0033]

[0034] Where M represents the total number of all modes.

[0035] Preferably, the step 3 comprises:

[0036] Build a graph G = {(u,i)|u∈U,i∈I} based on the historical interactions between users and items. Both user u and item i act as ID embeddings in the user behavior space, where the edge connecting user u and item i indicates that there has been a historical interaction between them.

[0037] Using GCN, user features are extracted as follows:

[0038]

[0039] Among them, the collaborative behavior characteristics represents the l-hop neighbors of user u, set N i and N u Represents the neighbor nodes of product i∈I and user u∈U in the user-product interaction graph G;

[0040] Multiple multi-head attention modules are used to model user preferences. The previously obtained set modal features and collaborative position encodings are integrated layer by layer. All historical items interacted with user u are taken as input to build a user preference model. The transformer layer is applied to the user features to obtain the preference representation of user u and obtain the final predicted preference score:

[0041]

[0042] in, Represents the matching score between user u and item i in modality m.

[0043] Preferably, the area under the ROC curve AUC and the normalized discounted cumulative gain NDCG are used as evaluation indicators to evaluate the accuracy of the personalized package recommendation task; AUC is used to measure the probability that the model's predicted preference score for a positive sample exceeds the predicted preference score for a negative sample for a given user's selected product; NDCG is used to evaluate the position of the positive sample in the positive ranking list.

[0044] The multimodal personalized clothing recommendation system based on item compatibility provided by the present invention includes:

[0045] Module M1: Use a multi-head self-attention mechanism to model and represent clothing and users respectively. Different types of items in a clothing set are split and feature models are established for each type of item. For each type of item, a modality-aware encoder is used to independently model various modalities to obtain multimodal fusion features of the item.

[0046] Module M2: Use the existing product combinations in historical data and combine the position codes of different types of products to complete the modeling of the matching degree between users and different types of products;

[0047] Module M3: Based on the interaction history between users and multiple items, the user-item interaction graph is used for collaborative position encoding to enhance the representation of user behavior characteristics. The position encoding is combined with the modal characteristics of the user's corresponding interest group to complete the modeling of user preferences, and diversified recommendations are made based on the collaborative information between users and items.

[0048] Preferably, the module M1 comprises:

[0049] Given a set of clothing items, it is represented as a set of modality-aware features In the formula, represents the original feature representation of the i-th item in the clothing in mode m, d m represents the feature dimension of a single item on the m mode; n is the number of items in the clothing;

[0050] The text and visual modality features of each item are represented as: and where t i and v i are the text and visual samples of a single product, respectively; Bert(·) and ResNet(·) are the language representation function and the deep residual network function, respectively;

[0051] Calculate the self-attention output of clothing modality-aware features with multi-head attention, the expression is:

[0052] H m(l) =MultiHeadAtt(Hm(l-1) ,H m(l-1) ,H m(l-1) )

[0053]

[0054] Among them, MultiHeadAtt(·) is the multi-head attention function, H m(l) is the output of the lth layer in modality m; and is a trainable parameter in the modality m, LayerNorm(·) represents the layer normalization operation; stacking L self-attention blocks, we get the enhanced representation H of the input set m(L) , and aggregate them into an overall representation of the suit using a fully connected layer:

[0055]

[0056] Among them, LeakyReLU(·) is a nonlinear activation function, is the embedding representation of set o after the L-th multi-head self-attention block, and are trainable parameters in modality m.

[0057] Preferably, the module M2 comprises:

[0058] Using the historical data given in the set of single product matching information history interaction to build a graph G = {(i1,..i n )|,i n ∈I n}, where the feature vectors of all items i in a set act as ID embeddings in the matching behavior space, where the edges connecting all different types of items represent the interactive information in the historical data;

[0059] Using the GCN algorithm, information is propagated and local position features are extracted from the graph as collaborative position encoding signals, expressed as:

[0060]

[0061] Among them, the collaborative behavior characteristics represents the l-hop neighbors of clothing item i, set N i and N u Represents the neighbor nodes of set i∈I and user u∈U in the user-item interaction graph G;

[0062] Multiple multi-head attention modules are used to model the matching degree of individual items. The previously obtained suit modality features and collaborative position encoding are integrated layer by layer. For each input clothing combination, all interactions generated by the individual items are used as input and initialized using residual network units. The single product matching preference model is as follows:

[0063]

[0064]

[0065]

[0066]

[0067] Among them, Attention(·) is the attention mechanism function, softmax(·) is the activation function; Q, K, V are the query vector, key vector and value vector respectively; T is the transposition symbol; d k is the dimension of the key-value vector; is the state vector of the residual network of item i in layer l; w3 and b3 are trainable parameters; represents the collaborative position code of item i, represents the overall representation of the n historical items that user u has interacted with in modality m, that is,

[0068] After L multi-head attention modules, we get:

[0069]

[0070]

[0071] Among them, among them, Represents the matching characteristics of product k in modality m; w6, w7, b6, and b7 are all trainable parameters; calculate the matching degree of other types of different products i with product k in each modality m:

[0072]

[0073] in, Indicates the matching characteristics of candidate items in other categories;

[0074] The scores for all modalities are summed to get the final predicted preference score:

[0075]

[0076] Where M represents the total number of all modes.

[0077] Preferably, the module M3 comprises:

[0078] Build a graph G = {(u,i)|u∈U,i∈I} based on the historical interactions between users and items. Both user u and item i act as ID embeddings in the user behavior space, where the edge connecting user u and item i indicates that there has been a historical interaction between them.

[0079] Using GCN, user features are extracted as follows:

[0080]

[0081] Among them, the collaborative behavior characteristics represents the l-hop neighbors of user u, set N i and N u Represents the neighbor nodes of product i∈I and user u∈U in the user-product interaction graph G;

[0082] Multiple multi-head attention modules are used to model user preferences. The previously obtained set modal features and collaborative position encodings are integrated layer by layer. All historical items interacted with user u are taken as input to build a user preference model. The transformer layer is applied to the user features to obtain the preference representation of user u and obtain the final predicted preference score:

[0083]

[0084] in, Represents the matching score between user u and item i in modality m.

[0085] Preferably, the area under the ROC curve AUC and the normalized discounted cumulative gain NDCG are used as evaluation indicators to evaluate the accuracy of the personalized package recommendation task; AUC is used to measure the probability that the model's predicted preference score for a positive sample exceeds the predicted preference score for a negative sample for a given user's selected product; NDCG is used to evaluate the position of the positive sample in the positive ranking list.

[0086] Compared with the prior art, the present invention has the following beneficial effects:

[0087] The present invention focuses on personalized clothing recommendations that take into account user preferences and item compatibility. Personalized clothing recommendations not only require modeling of clothing items, but also require exploring user preferences at the clothing level and the matching degree between clothing items of different categories. Through CSTRecN, personalized clothing recommendations that take into account both user preferences and item compatibility are realized for the first time. By integrating the information of clothing model characteristics with user behavior characteristics, the diversification and accuracy of personalized recommendations are achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0089] Figure 1 Schematic diagram of the process recommended for personalized clothing;

[0090] Figure 2 A schematic diagram of a CSTRecN framework for personalized clothing recommendation according to the present invention;

[0091] Figure 3a Performance when recommended for a set, Figure 3b Performance when recommended for a single product. DETAILED DESCRIPTION

[0092] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0093] Example

[0094] The present invention provides a multimodal personalized clothing recommendation CSTRecN based on item compatibility, which consists of three basic modules: Figure 2 shown.

[0095] i) Single product feature modeling module: Modeling is performed using the features of different modes of clothing items;

[0096] ii) Product pair matching degree coding module: using collaborative position coding, it aims to discover the association between different types of products in the same set;

[0097] iii) User preference encoding module: It utilizes user historical interaction records and aims to integrate clothing features and user behavior characteristics.

[0098] like Figure 1 The present invention also provides a personalized clothing recommendation method, the specific steps are as follows:

[0099] 1. Single product feature modeling module

[0100] In order to ensure the effectiveness of the model, this method considers modeling the features of multiple modes of clothing items when modeling clothing items. This method uses the self-attention mechanism to build a clothing embedding model under different modes. Given a set of clothing items, it is represented as a set of modality-aware features. In the formula, represents the original feature representation of the i-th item in clothing, dm Represents the feature dimension of a single product in m modes (e.g., textual, visual). The textual and visual modality features of each single product come from different pre-trained models, namely: and The self-attention output of clothing modality-aware features with multi-head attention is calculated as follows:

[0101] H m(l) =MultiHeadAtt(H m(l-1) ,H m(l-1) ,H m(1-1) )

[0102]

[0103] in, and is a trainable parameter in the modality m, and LayerNorm(·) represents the layer normalization operation. Stacking L self-attention blocks, we get an enhanced representation H of the input set m(L) , and aggregate them into an overall representation of the suit using a fully connected layer:

[0104]

[0105] Among them, LeakyReLU(·) is a nonlinear activation function, is the embedding representation of the suit o after the L-th multi-head self-attention block. So far, the overall representation of the suit in different model spaces has been obtained by using transformer blocks for each modality separately.

[0106] 2. Product pair matching degree encoding module

[0107] The item pair matching degree encoding module aims to quantify the matching degree between different types of items. Based on the item feature module and the user preference module, after interacting the user feature vector with the feature vector of each item to calculate the matching score of each item, this module also considers the correlation between items and constructs a multi-dimensional matching degree encoding by integrating the feature information of the items to ensure the accuracy and personalization of the recommendation results.

[0108] Specifically, we use the historical interaction of the item matching information in the given set matching in the dataset to construct a graph G = {(i1, ..i n )|,i n ∈I n}. The feature vectors of all item nodes i in a set act as ID embeddings in the matching behavior space, where the edges connecting all different types of items represent the interactive information in the dataset. Using the idea of ​​collaborative filtering in the recommendation system, items with higher matching degrees have closer features in the vector space, and other items near similar items may also observe similar features. Using the GCN algorithm, information is propagated and local position features are extracted from the graph as collaborative position encoding signals:

[0109]

[0110] Collaborative behavior characteristics The l-hop neighbors of clothing item i are set N i and N u Represents the neighbor nodes of suit i∈I and user u∈U in the user-suite interaction graph G.

[0111] At the same time, multiple multi-head attention modules are used to model the matching degree of single product pairs, and the previously obtained suit modality features and collaborative position encoding are integrated layer by layer. For each input clothing combination, all interactions generated by the single products in it are used as input, and the residual network unit is used to initialize The single product matching preference model is as follows:

[0112]

[0113]

[0114] Among them, Attention(·) is the attention mechanism function, softmax(·) is the activation function; Q, K, V are the query vector, key vector and value vector respectively; T is the transposition symbol; d k is the dimension of the key-value vector; is the state vector of the residual network of item i in layer l; w3 and b3 are trainable parameters; represents the collaborative position code of item i, represents the overall representation of the n historical items that user u has interacted with, that is,

[0115] After L multi-head attention modules, we get:

[0116]

[0117] in, Represents the matching characteristics of product k in modality m; w6, w7, b6, and b7 are all trainable parameters; calculate the matching degree of other types of different products i with product k in each modality m:

[0118]

[0119] in, Indicates the matching characteristics of candidate items in other categories;

[0120] The scores for all modalities are summed to get the final predicted preference score:

[0121]

[0122] Where M represents the total number of all modes.

[0123] 3. User Preference Encoding Module

[0124] After modeling the multimodal features of clothing items, the user's preferences are encoded using historical clothing that the user has interacted with. In current methods in the field of clothing recommendation, user behavior characteristics (i.e., user-clothing interaction data) are almost always used to model the preference of the whole suit. However, these methods may cause the model to ignore the preference relationship between users and items, affecting the flexibility of recommendations. Therefore, this method splits the suit when modeling, establishes a preference model for the user and each type of item separately, and uses a transformer-based network to capture the collaborative information in user behavior.

[0125] Specifically, similar to building a single product matching module, we first construct a graph G = {(u,i)|u∈U,i∈I} based on the user's historical interaction with the single product. Both the user node u and the single product node i act as ID embeddings in the user behavior space, where the edge connecting user u and single product i indicates that they have had historical interactions. Based on the concept of collaborative filtering in the recommendation system, users with similar preferences may show proximity in the behavior space, and similar features may be observed in clothing near similar users. Using GCN, the user features are extracted as follows:

[0126]

[0127] Among them, the collaborative behavior characteristics represents the l-hop neighbors of user u, set N i and N u Represents the neighbor nodes of product i∈I and user u∈U in the user-product interaction graph G.

[0128] Multiple multi-head attention modules are used to model user preferences. The previously obtained set modal features and collaborative position encodings are integrated layer by layer. All historical items interacted with user u are taken as input to build a user preference model. The transformer layer is applied to the user features to obtain the preference representation of user u and obtain the final predicted preference score:

[0129]

[0130] Represents the matching score between user u and item i in modality m.

[0131] 4. Dataset Experimental Results and Analysis

[0132] In this example, experimental verification is performed on the polyvore-u dataset and the performance is compared with other existing open source models. In order to evaluate the accuracy of the personalized package recommendation task, the area under the ROC curve (AUC) and the normalized discounted cumulative gain (NDCG) are used as evaluation indicators. In essence, AUC measures the model's predicted preference score for positive samples over the predicted preference score for negative samples for a given user's selected product. NDCG evaluates the position of the positive sample in the positive ranking list.

[0133] FIG3 shows the performance comparison of different open source models on two indicators. The CSTRecN proposed in the present invention has no obvious disadvantage compared with other models when performing package recommendation, and is always better than other open source models when performing single product recommendation, which proves the superiority of this method.

[0134] Those skilled in the art know that, in addition to implementing the system, device and its various modules provided by the present invention in a purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, the system, device and its various modules provided by the present invention can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing the method and structures within the hardware component.

[0135] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A multimodal personalized clothing recommendation method based on item compatibility, characterized in that: include: Step 1: Use a multi-head self-attention mechanism to model and represent clothing and users respectively. Different types of items in a clothing set are split and feature models are established separately. For each type of item, a modality-aware encoder is used to independently model various modalities to obtain multimodal fusion features of the item. Step 2: Use the existing product combinations in historical data and combine them with the position codes of different types of products to complete the modeling of the matching degree between users and different types of products; Step 3: Based on the interaction history between users and multiple items, the user-item interaction graph is used for collaborative position encoding to enhance the representation of user behavior characteristics. The position encoding is combined with the modal characteristics of the user's corresponding interest group to complete the modeling of user preferences. Diversified recommendations are made based on the collaborative information between users and items.

2. The multimodal personalized clothing recommendation method based on item compatibility according to claim 1, characterized in that: The step 1 comprises: Given a set of clothing items, it is represented as a set of modality-aware features In the formula, represents the original feature representation of the i-th item in the clothing in mode m, d m represents the feature dimension of a single item on the m mode; n is the number of items in the clothing; The text and visual modality features of each item are represented as: and where t i and v i are the text and visual samples of a single product, respectively; Bert(·) and ResNet(·) are the language representation function and the deep residual network function, respectively; Calculate the self-attention output of clothing modality-aware features with multi-head attention, the expression is: H m(l) =MultiHeadAtt(H m(l-1) ,H m(l-1) ,H m(l-1) ) Among them, MultiHeadAtt(·) is the multi-head attention function, H m(l) is the output of the lth layer in modality m; and is a trainable parameter in the modality m, LayerNorm(·) represents the layer normalization operation; stacking L self-attention blocks, we get the enhanced representation H of the input set m(L) , and aggregate them into an overall representation of the suit using a fully connected layer: Among them, LeakyReLU(·) is a nonlinear activation function, is the embedding representation of set o after the L-th multi-head self-attention block, and are trainable parameters in modality m.

3. The multimodal personalized clothing recommendation method based on item compatibility according to claim 2, characterized in that: The step 2 comprises: Using the historical data given in the set of single product matching information history interaction to build a graph G = {(i1,..i n )|,i n ∈I n }, where the feature vectors of all items i in a set act as ID embeddings in the matching behavior space, where the edges connecting all different types of items represent the interactive information in the historical data; Using the GCN algorithm, information is propagated and local position features are extracted from the graph as collaborative position encoding signals, expressed as: Among them, the collaborative behavior characteristics represents the l-hop neighbors of clothing item i, set N i and N u Represents the neighbor nodes of set i∈I and user u∈U in the user-item interaction graph G; Multiple multi-head attention modules are used to model the matching degree of individual items. The previously obtained suit modality features and collaborative position encoding are integrated layer by layer. For each input clothing combination, all interactions generated by the individual items are used as input and initialized using residual network units. The single product matching preference model is as follows: Among them, Attention(·) is the attention mechanism function, softmax(·) is the activation function; Q, K, V are the query vector, key vector and value vector respectively; T is the transposition symbol; d k is the dimension of the key-value vector; is the state vector of the residual network of item i in layer l; w3 and b3 are trainable parameters; represents the collaborative position code of item i, represents the overall representation of the n historical items that user u has interacted with in modality m, that is, After L multi-head attention modules, we get: Among them, among them, Represents the matching characteristics of product k in modality m; w6, w7, b6, and b7 are all trainable parameters; calculate the matching degree of other types of different products i with product k in each modality m: in, Indicates the matching characteristics of candidate items in other categories; The scores for all modalities are summed to get the final predicted preference score: Where M represents the total number of all modes.

4. The multimodal personalized clothing recommendation method based on item compatibility according to claim 3, characterized in that: The step 3 comprises: Build a graph G = {(u,i)|u∈U,i∈I} based on the historical interactions between users and items. Both user u and item i act as ID embeddings in the user behavior space, where the edge connecting user u and item i indicates that there has been a historical interaction between them. Using GCN, user features are extracted as follows: Among them, the collaborative behavior characteristics represents the l-hop neighbors of user u, set N i and N u Represents the neighbor nodes of product i∈I and user u∈U in the user-product interaction graph G; Multiple multi-head attention modules are used to model user preferences. The previously obtained set modal features and collaborative position encodings are integrated layer by layer. All historical items interacted with user u are taken as input to build a user preference model. The transformer layer is applied to the user features to obtain the preference representation of user u and obtain the final predicted preference score: in, Represents the matching score between user u and item i in modality m.

5. The multimodal personalized clothing recommendation method based on item compatibility according to claim 1, characterized in that: The area under the ROC curve (AUC) and the normalized discounted cumulative gain (NDCG) are used as evaluation indicators to evaluate the accuracy of the personalized package recommendation task. AUC is used to measure the probability that, for a given user-selected product, the model's predicted preference score for the positive sample exceeds the predicted preference score for the negative sample. NDCG is used to evaluate the position of the positive sample in the positive ranking list.

6. A multimodal personalized clothing recommendation system based on item compatibility, characterized in that: include: Module M1: Use a multi-head self-attention mechanism to model and represent clothing and users respectively. Different types of items in a clothing set are split and feature models are established for each type of item. For each type of item, a modality-aware encoder is used to independently model various modalities to obtain multimodal fusion features of the item. Module M2: Use the existing product combinations in historical data and combine them with the position codes of different types of products to complete the modeling of the matching degree between users and different types of products; Module M3: Based on the interaction history between users and multiple items, the user-item interaction graph is used for collaborative position encoding to enhance the representation of user behavior characteristics. The position encoding is combined with the modal characteristics of the user's corresponding interest group to complete the modeling of user preferences, and diversified recommendations are made based on the collaborative information between users and items.

7. The multimodal personalized clothing recommendation system based on item compatibility according to claim 6, characterized in that: The module M1 comprises: Given a set of clothing items, it is represented as a set of modality-aware features In the formula, represents the original feature representation of the i-th item in the clothing in mode m, d m represents the feature dimension of a single item on the m mode; n is the number of items in the clothing; The text and visual modality features of each item are represented as: and where t i and v i are the text and visual samples of a single product, respectively; Bert(·) and ResNet(·) are the language representation function and the deep residual network function, respectively; Calculate the self-attention output of clothing modality-aware features with multi-head attention, the expression is: H m(l) =MultiHeadAtt(H m(l-1) ,H m(l-1) ,H m(1-1) ) Among them, MultiHeadAtt(·) is the multi-head attention function, H m(l) is the output of the lth layer in modality m; and is a trainable parameter in the modality m, LayerNorm(·) represents the layer normalization operation; stacking L self-attention blocks, we get the enhanced representation H of the input set m(L) , and aggregate them into an overall representation of the suit using a fully connected layer: Among them, LeakyReLU(·) is a nonlinear activation function, is the embedding representation of set o after the L-th multi-head self-attention block, and are trainable parameters in modality m.

8. The multimodal personalized clothing recommendation system based on item compatibility according to claim 7, characterized in that: The module M2 comprises: Using the historical data given in the set of single product matching information history interaction to build a graph G = {(i1,..i n )|,i n ∈I n }, where the feature vectors of all items i in a set act as ID embeddings in the matching behavior space, where the edges connecting all different types of items represent the interactive information in the historical data; Using the GCN algorithm, information is propagated and local position features are extracted from the graph as collaborative position encoding signals, expressed as: Among them, the collaborative behavior characteristics represents the l-hop neighbors of clothing item i, set N i and N u Represents the neighbor nodes of set i∈I and user u∈U in the user-item interaction graph G; Multiple multi-head attention modules are used to model the matching degree of individual items. The previously obtained suit modality features and collaborative position encoding are integrated layer by layer. For each input clothing combination, all interactions generated by the individual items are used as input and initialized using residual network units. The single product matching preference model is as follows: Among them, Attention(·) is the attention mechanism function, softmax(·) is the activation function; Q, K, V are the query vector, key vector and value vector respectively; T is the transposition symbol; d k is the dimension of the key-value vector; is the state vector of the residual network of item i in layer l; w3 and b3 are trainable parameters; represents the collaborative position code of item i, represents the overall representation of the n historical items that user u has interacted with in modality m, that is, After L multi-head attention modules, we get: Among them, among them, Represents the matching characteristics of product k in modality m; w6, w7, b6, and b7 are all trainable parameters; calculate the matching degree of other types of different products i with product k in each modality m: in, Indicates the matching characteristics of candidate items in other categories; The scores for all modalities are summed to get the final predicted preference score: Where M represents the total number of all modes.

9. The multimodal personalized clothing recommendation system based on item compatibility according to claim 8, characterized in that: The module M3 comprises: Build a graph G = {(u,i)|u∈U,i∈I} based on the historical interactions between users and items. Both user u and item i act as ID embeddings in the user behavior space, where the edge connecting user u and item i indicates that there has been a historical interaction between them. Using GCN, user features are extracted as follows: Among them, the collaborative behavior characteristics represents the l-hop neighbors of user u, set N i and N u Represents the neighbor nodes of product i∈I and user u∈U in the user-product interaction graph G; Multiple multi-head attention modules are used to model user preferences. The previously obtained set modal features and collaborative position encodings are integrated layer by layer. All historical items interacted with user u are taken as input to build a user preference model. The transformer layer is applied to the user features to obtain the preference representation of user u and obtain the final predicted preference score: in, Represents the matching score between user u and item i in modality m.

10. The multimodal personalized clothing recommendation system based on item compatibility according to claim 6, characterized in that: The area under the ROC curve (AUC) and the normalized discounted cumulative gain (NDCG) are used as evaluation indicators to evaluate the accuracy of the personalized package recommendation task. AUC is used to measure the probability that, for a given user-selected product, the model's predicted preference score for the positive sample exceeds the predicted preference score for the negative sample. NDCG is used to evaluate the position of the positive sample in the positive ranking list.

Images