Multi-modal personalized clothing recommendation method and system based on single product compatibility

Abstract

The application provides a multi-modal personalized clothing recommendation method and system based on single product compatibility, comprising the following steps: step 1: using a multi-head self-attention mechanism to model and represent clothes and users respectively; step 2: using the existing single product matching in the historical data, combining the position encoding of different types of single products, modeling the matching degree between the user and different types of single products; step 3: based on the interaction history of the user and multiple single products, using user-single product interaction graph for collaborative position encoding, thereby enhancing the representation of user behavior characteristics, combining the position encoding with the modal features of the user's corresponding interest group, modeling the user preference, and combining the collaborative information between the user and the single product for diversified recommendation. The application introduces the learning of single product features, provides a more personalized and flexible recommendation strategy to meet the diversified needs of users for clothing matching.

Description

Technical Field

[0001] The present invention relates to the technical field of clothing recommendation, and in particular to a multimodal personalized clothing recommendation method and system based on item compatibility. Background Art

[0002] With the rapid development of the fashion industry, the clothing market is becoming increasingly personalized and diversified, and consumer demand for clothing recommendation systems is constantly increasing. Traditional recommendation methods often focus on complete sets of clothing, neglecting the matching relationships between individual items and the individual needs of users. Therefore, an innovative recommendation strategy is urgently needed. By comprehensively considering the matching degree between users and individual items and the correlation between individual items, it can achieve more accurate and personalized clothing recommendations to better meet the diverse needs of modern consumers.

[0003] However, existing recommendation systems face numerous challenges during the feature mining phase, particularly in achieving diverse recommendations. First, many systems are designed to support only a single recommendation model, failing to flexibly adapt to diverse user needs. Specifically, recommendation systems often only recommend clothing sets or individual items, failing to simultaneously support inputting user-recommended sets, user-recommended items, or combinations of user and item recommendations to recommend other types of items. This limitation limits the accuracy and personalization of recommendation results.

[0004] Furthermore, ensuring recommendation effectiveness is a key challenge. In diverse recommendation systems, ensuring that the system can provide high-quality recommendations while supporting multiple input modes is an important design consideration. Summary of the Invention

[0005] In view of the defects in the prior art, the purpose of the present invention is to provide a multimodal personalized clothing recommendation method and system based on item compatibility.

[0006] The multimodal personalized clothing recommendation method based on item compatibility provided by the present invention includes:

[0007] Step 1: Use a multi-head self-attention mechanism to model and represent clothing and users separately. Different types of items in the clothing set are separated and feature models are established for each type. For each type of item, a modality-aware encoder is used to independently model each modality to obtain multimodal fusion features of the item.

[0008] Step 2: Utilize the existing product combinations in historical data and combine them with the position codes of different types of products to model the matching between users and different types of products.

[0009] Step 3: Based on the user's interaction history with multiple items, the user-item interaction graph is used for collaborative position encoding to enhance the representation of user behavior characteristics. The position encoding is combined with the modal characteristics of the user's corresponding interest group to complete the modeling of user preferences. Diversified recommendations are made based on the collaborative information between users and items.

[0010] Preferably, the step 1 comprises:

[0011] Given a set of clothing items, it is represented as a set of modality-aware features Where, represents the original feature representation of the i-th item in the clothing in mode m, d m represents the feature dimension of a single item on the m-mode; n is the number of items in the clothing;

[0012] The text and visual modality features of each item are expressed as: and where t i and v i are text and visual samples of a single item respectively; Bert(·) and ResNet(·) are language representation function and deep residual network function respectively;

[0013] Calculate the self-attention output of the clothing modality perception feature with multi-head attention, the expression is:

[0014] H m(l) =MultiHeadAtt(H m(l-1) ,H m(1-1) ,H m(l-1) )

[0015]

[0016] Among them, MultiHeadAtt(·) is the multi-head attention function, H m(l) is the output of the lth layer in modality m; and is a trainable parameter in the modality m, LayerNorm(·) represents the layer normalization operation; stacking L self-attention blocks, we get the enhanced representation H of the input set m(L) and aggregate them into the overall representation of the suit using a fully connected layer:

[0017]

[0018] Among them, LeakyReLU(·) is a nonlinear activation function, is the embedding representation of set o after the L-th multi-head self-attention block, and are trainable parameters in modality m.

[0019] Preferably, the step 2 comprises:

[0020] Using the historical data to construct the graph G = {(i1, ..i n )|,i n ∈I n}, where the feature vectors of all items i in a set serve as ID embeddings in the matching behavior space, where the edges connecting all different types of items represent the interaction information in the historical data;

[0021] Using the GCN algorithm, information is propagated and local position features are extracted from the graph as collaborative position encoding signals, which are expressed as:

[0022]

[0023] Among them, the collaborative behavior characteristics represents the l-hop neighbors of clothing item i, set N i and N u Represents the neighbor nodes of set i∈I and user u∈U in the user-item interaction graph G;

[0024] Multiple multi-head attention modules are used to model the matching degree of individual items. The previously obtained suit modality features and collaborative position encoding are integrated layer by layer. For each input clothing combination, all interactions generated by the individual items are used as input and initialized using residual network units. The single product matching preference model is as follows:

[0025]

[0026] Among them, Attention(·) is the attention mechanism function, softmax(·) is the activation function; Q, K, V are the query vector, key vector and value vector respectively; T is the transpose symbol; d k is the dimension of the key-value vector; is the state vector of the residual network of item i in layer l; w3 and b3 are trainable parameters; represents the collaborative position code of item i, represents the overall representation of n historical items that user u has interacted with in modality m, that is,

[0027] After L multi-head attention modules, we get:

[0028]

[0029] Among them, among them, Represents the matching characteristics of item k in modality m; w6, w7, b6, and b7 are all trainable parameters. Calculate the matching degree of other types of items i with item k in each modality m:

[0030]

[0031] in, Indicates the matching characteristics of candidate items in other categories;

[0032] The scores for all modalities are summed to get the final predicted preference score:

[0033]

[0034] Where M represents the total number of all modes.

[0035] Preferably, the step 3 includes:

[0036] Based on the historical interactions between users and items, we construct a graph G = {(u,i)|u∈U,i∈I}. User u and item i both serve as ID embeddings in the user behavior space, where the edge connecting user u and item i indicates the historical interactions between them.

[0037] Using GCN, user features are extracted as follows:

[0038]

[0039] Among them, the collaborative behavior characteristics represents the l-hop neighbors of user u, set N i and N u Represents the neighbor nodes of item i∈I and user u∈U in the user-item interaction graph G;

[0040] We use multiple multi-head attention modules to model user preferences, integrate the previously obtained set modality features with the collaborative position encoding layer by layer, take all historical items that user u interacts with as input, build a user preference model, apply the transformer layer to the user features, obtain the preference representation of user u, and obtain the final predicted preference score:

[0041]

[0042] in, Represents the matching score between user u and item i in modality m.

[0043] Preferably, the area under the ROC curve (AUC) and the normalized discounted cumulative gain (NDCG) are used as evaluation indicators to assess the accuracy of the personalized package recommendation task. AUC is used to measure the probability that, for a given user-selected item, the model's predicted preference score for the positive sample exceeds the predicted preference score for the negative sample. NDCG is used to evaluate the position of the positive sample in the positive ranking list.

[0044] The multimodal personalized clothing recommendation system based on item compatibility provided by the present invention includes:

[0045] Module M1: Uses a multi-head self-attention mechanism to model and represent clothing and users separately. Different types of items in a clothing set are separated and feature models are established for each. For each type of item, a modality-aware encoder is used to independently model each modality to obtain multimodal fusion features of the item.

[0046] Module M2: Utilizes existing product combinations in historical data and combines them with the position codes of different types of products to model the matching between users and different types of products.

[0047] Module M3: Based on the interaction history between users and multiple items, collaborative position encoding is performed using the user-item interaction graph to enhance the representation of user behavior characteristics. The position encoding is combined with the modal characteristics of the user's corresponding interest group to complete the modeling of user preferences. Diversified recommendations are made based on the collaborative information between users and items.

[0048] Preferably, the module M1 includes:

[0049] Given a set of clothing items, it is represented as a set of modality-aware features Where, represents the original feature representation of the i-th item in the clothing in mode m, d m represents the feature dimension of a single item on the m-mode; n is the number of items in the clothing;

[0050] The text and visual modality features of each item are expressed as: and where t i and v i are text and visual samples of a single item respectively; Bert(·) and ResNet(·) are language representation function and deep residual network function respectively;

[0051] Calculate the self-attention output of the clothing modality perception feature with multi-head attention, the expression is:

[0052] H m(l) =MultiHeadAtt(Hm(l-1) ,H m(l-1) ,H m(l-1) )

[0053]

[0054] Among them, MultiHeadAtt(·) is the multi-head attention function, H m(l) is the output of the lth layer in modality m; and is a trainable parameter in the modality m, LayerNorm(·) represents the layer normalization operation; stacking L self-attention blocks, we get the enhanced representation H of the input set m(L) and aggregate them into the overall representation of the suit using a fully connected layer:

[0055]

[0056] Among them, LeakyReLU(·) is a nonlinear activation function, is the embedding representation of set o after the L-th multi-head self-attention block, and are trainable parameters in modality m.

[0057] Preferably, the module M2 includes:

[0058] Using the historical data to construct the graph G = {(i1, ..i n )|,i n ∈I n}, where the feature vectors of all items i in a set serve as ID embeddings in the matching behavior space, where the edges connecting all different types of items represent the interaction information in the historical data;

[0059] Using the GCN algorithm, information is propagated and local position features are extracted from the graph as collaborative position encoding signals, which are expressed as:

[0060]

[0061] Among them, the collaborative behavior characteristics represents the l-hop neighbors of clothing item i, set N i and N u Represents the neighbor nodes of set i∈I and user u∈U in the user-item interaction graph G;

[0062] Multiple multi-head attention modules are used to model the matching degree of individual items. The previously obtained suit modality features and collaborative position encoding are integrated layer by layer. For each input clothing combination, all interactions generated by the individual items are used as input and initialized using residual network units. The single product matching preference model is as follows:

[0063]

[0064]

[0065]

[0066]

[0067] Among them, Attention(·) is the attention mechanism function, softmax(·) is the activation function; Q, K, V are the query vector, key vector and value vector respectively; T is the transpose symbol; d k is the dimension of the key-value vector; is the state vector of the residual network of item i in layer l; w3 and b3 are trainable parameters; represents the collaborative position code of item i, represents the overall representation of n historical items that user u has interacted with in modality m, that is,

[0068] After L multi-head attention modules, we get:

[0069]

[0070]

[0071] Among them, among them, Represents the matching characteristics of item k in modality m; w6, w7, b6, and b7 are all trainable parameters. Calculate the matching degree of other types of items i with item k in each modality m:

[0072]

[0073] in, Indicates the matching characteristics of candidate items in other categories;

[0074] The scores for all modalities are summed to get the final predicted preference score:

[0075]

[0076] Where M represents the total number of all modes.

[0077] Preferably, the module M3 includes:

[0078] Based on the historical interactions between users and items, we construct a graph G = {(u,i)|u∈U,i∈I}. User u and item i both serve as ID embeddings in the user behavior space, where the edge connecting user u and item i indicates the historical interactions between them.

[0079] Using GCN, user features are extracted as follows:

[0080]

[0081] Among them, the collaborative behavior characteristics represents the l-hop neighbors of user u, set N i and N u Represents the neighbor nodes of item i∈I and user u∈U in the user-item interaction graph G;

[0082] We use multiple multi-head attention modules to model user preferences, integrate the previously obtained set modality features with the collaborative position encoding layer by layer, take all historical items that user u interacts with as input, build a user preference model, apply the transformer layer to the user features, obtain the preference representation of user u, and obtain the final predicted preference score:

[0083]

[0084] in, Represents the matching score between user u and item i in modality m.

[0085] Preferably, the area under the ROC curve (AUC) and the normalized discounted cumulative gain (NDCG) are used as evaluation indicators to assess the accuracy of the personalized package recommendation task. AUC is used to measure the probability that, for a given user-selected item, the model's predicted preference score for the positive sample exceeds the predicted preference score for the negative sample. NDCG is used to evaluate the position of the positive sample in the positive ranking list.

[0086] Compared with the prior art, the present invention has the following beneficial effects:

[0087] This paper focuses on personalized clothing recommendations that take into account user preferences and item compatibility. Personalized clothing recommendations not only require modeling of clothing items, but also require exploring user preferences at the clothing level and the matching degree between clothing items of different categories. Through CSTRecN, personalized clothing recommendations that simultaneously consider user preferences and item compatibility are realized for the first time. By fusing information on clothing model characteristics with user behavior characteristics, the diversification and accuracy of personalized recommendations are achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0089] Figure 1 Schematic diagram of the process for personalized clothing recommendations;

[0090] Figure 2 Schematic diagram of the CSTRecN framework for personalized clothing recommendation of the present invention;

[0091] Figure 3a Performance when recommended for a set, Figure 3b Performance when recommended for a single product. DETAILED DESCRIPTION

[0092] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0093] Example

[0094] The present invention provides a multimodal personalized clothing recommendation CSTRecN based on item compatibility, which consists of three basic modules: Figure 2 shown.

[0095] i) Single item feature modeling module: Modeling is performed using the features of different modes of clothing items;

[0096] ii) Item Pair Matching Coding Module: This module uses collaborative position coding to discover the associations between different types of items in the same set.

[0097] iii) User preference encoding module: It utilizes user historical interaction records and aims to integrate clothing features and user behavior features.

[0098] like Figure 1 The present invention also provides a personalized clothing recommendation method, the specific steps are as follows:

[0099] 1. Single product feature modeling module

[0100] In order to ensure the effectiveness of the model, this method considers modeling the features of multiple modalities of clothing items when modeling clothing items. This method uses the self-attention mechanism to build a clothing embedding model under different modalities. Given a set of clothing items, it is represented as a set of modality-aware features. Where, represents the original feature representation of the i-th item in clothing, dm Represents the feature dimension of a single product in m modalities (e.g., textual, visual). The textual and visual modality features of each single product come from different pre-trained models, namely: and The self-attention output of the clothing modality-aware features with multi-head attention is calculated as follows:

[0101] H m(l) =MultiHeadAtt(H m(l-1) ,H m(l-1) ,H m(1-1) )

[0102]

[0103] in, and is a trainable parameter in the modality m, and LayerNorm(·) represents the layer normalization operation. Stacking L self-attention blocks, we get the enhanced representation H of the input set. m(L) and aggregate them into the overall representation of the suit using a fully connected layer:

[0104]

[0105] Among them, LeakyReLU(·) is a nonlinear activation function, is the embedding representation of the suit o after the L-th multi-head self-attention block. So far, the overall representation of the suit in different model spaces has been obtained by using transformer blocks for each modality separately.

[0106] 2. Product pair matching degree coding module

[0107] The Item Pair Matching Encoding module aims to quantify the degree of match between different item types. Building on the Item Feature and User Preference Modules, this module interacts the user's feature vector with the feature vectors of each item to calculate a match score for each item. It also considers the correlation between items and, by integrating their feature information, constructs a multi-dimensional match encoding to ensure accurate and personalized recommendation results.

[0108] Specifically, we use the history of item matching information in the given set matching data set to construct a graph G = {(i1, ..i n )|,i n ∈I nThe feature vectors of all item nodes i in a set serve as ID embeddings within the matching behavior space, where the edges connecting all different types of items represent the interaction information in the dataset. Leveraging the collaborative filtering concept in recommendation systems, items with higher matching scores have their features closer in vector space, while other items near similar items may also have similar features. Using the GCN algorithm, information is propagated and local position features are extracted from the graph as collaborative position encoding signals:

[0109]

[0110] Collaborative behavior characteristics The l-hop neighbors of clothing item i are set N i and N u Represents the neighbor nodes of suit i∈I and user u∈U in the user-suite interaction graph G.

[0111] At the same time, multiple multi-head attention modules are used to model the matching degree of single items, and the previously obtained suit modality features and collaborative position encoding are integrated layer by layer. For each input clothing combination, all interactions generated by the single items in it are used as input, and the residual network unit is used to initialize The single product matching preference model is as follows:

[0112]

[0113]

[0114] Among them, Attention(·) is the attention mechanism function, softmax(·) is the activation function; Q, K, V are the query vector, key vector and value vector respectively; T is the transpose symbol; d k is the dimension of the key-value vector; is the state vector of the residual network of item i in layer l; w3 and b3 are trainable parameters; represents the collaborative position code of item i, Represents the overall representation of n historical items that user u has interacted with, that is,

[0115] After L multi-head attention modules, we get:

[0116]

[0117] in, Represents the matching characteristics of item k in modality m; w6, w7, b6, and b7 are all trainable parameters. Calculate the matching degree of other types of items i with item k in each modality m:

[0118]

[0119] in, Indicates the matching characteristics of candidate items in other categories;

[0120] The scores for all modalities are summed to get the final predicted preference score:

[0121]

[0122] Where M represents the total number of all modes.

[0123] 3. User preference encoding module

[0124] After modeling the multimodal characteristics of clothing items, user preferences are encoded using historical clothing items with which the user has interacted. Current approaches to clothing recommendation almost always model preferences for the entire outfit based on user behavioral characteristics (i.e., user-outfit interaction data). However, these approaches can cause the model to ignore the preference relationships between users and items, affecting the flexibility of recommendations. Therefore, this approach splits the outfit during modeling, establishing separate preference models for the user and each type of item, and utilizing a transformer-based network to capture the collaborative information in user behavior.

[0125] Specifically, similar to building the product matching module, we first construct a graph G = {(u,i)|u∈U,i∈I} based on the user's historical interactions with the product. Both the user node u and the product node i serve as ID embeddings within the user's behavior space, where the edge connecting user u and product i represents the historical interaction between them. Based on the concept of collaborative filtering in the recommendation system, users with similar preferences may show proximity in the behavior space, and similar features may be observed in the clothing near similar users. Using GCN, the user features are extracted as follows:

[0126]

[0127] Among them, the collaborative behavior characteristics represents the l-hop neighbors of user u, set N i and N u Represents the neighbor nodes of item i∈I and user u∈U in the user-item interaction graph G.

[0128] We use multiple multi-head attention modules to model user preferences, integrate the previously obtained set modality features with the collaborative position encoding layer by layer, take all historical items that user u interacts with as input, build a user preference model, apply the transformer layer to the user features, obtain the preference representation of user u, and obtain the final predicted preference score:

[0129]

[0130] Represents the matching score between user u and item i in modality m.

[0131] 4. Dataset Experimental Results and Analysis

[0132] In this example, we conducted experiments on the polyvore-u dataset and compared the performance of other existing open-source models. To assess the accuracy of the personalized package recommendation task, we used the area under the receiver operating characteristic (ROC) curve (AUC) and the normalized discounted cumulative gain (NDCG) as evaluation metrics. Essentially, AUC measures how much the model's predicted preference score for a given user-selected item exceeds its predicted preference score for a negative item. NDCG, on the other hand, measures the position of a positive item in the positive ranking list.

[0133] Figure 3 shows the performance comparison of different open source models on two indicators. The CSTRecN proposed in this paper has no obvious disadvantage compared with other models when recommending packages, and is always better than other open source models when recommending single products, which proves the superiority of this method.

[0134] Those skilled in the art will appreciate that, in addition to implementing the system, device, and various modules provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, and the like by logically programming the method steps. Therefore, the system, device, and various modules provided by the present invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; the modules for implementing various functions can also be considered both software programs for implementing the method and structures within the hardware component.

[0135] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. A multimodal personalized clothing recommendation method based on item compatibility, characterized in that: include: Step 1: Use a multi-head self-attention mechanism to model and represent clothing and users separately. Different types of items in the clothing set are separated and feature models are established for each type. For each type of item, a modality-aware encoder is used to independently model each modality to obtain multimodal fusion features of the item. Step 2: Leverage existing product combinations in historical data and combine them with the co-location codes of different types of products to model the matching between users and different types of products. Step 3: Based on the user's interaction history with multiple items, we use the user-item interaction graph to perform collaborative position encoding to enhance the representation of user behavior characteristics. We then combine the collaborative position encoding with the modal characteristics of the user's corresponding interest group to model the user's preferences. This allows us to make diversified recommendations based on the collaborative information between users and items. The step 2 includes: Use historical data to construct a graph of the historical interaction of the item matching information in the set matching , all items in one set The feature vectors of serve as ID embeddings in the collocation behavior space, where the edges connecting all different types of items represent the interactive information in the historical data; Using the GCN algorithm, information is propagated and local position features are extracted from the graph as collaborative position encoding signals, which are expressed as: Among them, the collaborative behavior characteristics Indicates clothing items of Hop Neighbor, Gather and Indicates a set and users In the user-product interaction diagram Neighbor nodes in ; Multiple multi-head attention modules are used to model the matching degree of individual items. The previously obtained suit modal features and the collaborative position encoding code are integrated layer by layer. For each input clothing combination, all interactions generated by the individual items are used as input and the residual network unit is used to initialize the model. , the single product matching preference model is as follows: in, Attention mechanism function, is the activation function; They are query vector, key-value vector and value vector respectively; is the transpose symbol; is the dimension of the key-value vector; It is a single product In the The state vector of the layer residual network; and is a trainable parameter; Indicates a single product Coordinated position coding, represents the overall representation of n historical items that user u has interacted with in modality m, that is, ; experience After the multi-head attention module, we get: Among them, among them, Representative items In modal Matching characteristics in ; 、 、 、 All are trainable parameters; calculate other types of different items In each mode The matching degree with item k: in, Indicates the matching characteristics of candidate items in other categories; The scores for all modalities are summed to get the final predicted preference score: in, represents the total number of all modes; The step 3 comprises: Build a graph based on the user's historical interactions with a single product ,user and single products Both act as ID embeddings within the user behavior space, connecting users and single products The edges indicate that there have been historical interactions between them; Using GCN, user features are extracted as follows: Among them, the collaborative behavior characteristics Represents a user of Hop Neighbor, Gather and Indicates a single product and users In the user-product interaction diagram Neighbor nodes in ; Multiple multi-head attention modules are used to model user preferences, and the previously obtained suit modality features and collaborative position encoding are integrated layer by layer to All historical items of interaction are used as input to build a user preference model, and the transformer layer is applied to the user features to obtain the user The preference representation of , the final predicted preference score is obtained: in, Represents a user The matching score with item i in modality m.

2. The multimodal personalized clothing recommendation method based on item compatibility according to claim 1, characterized in that: The step 1 comprises: Given a set of clothing items, it is represented as a set of modality-aware features , where Indicates that the i-th item in the clothing is in the mode The original feature representation in express The dimension of product characteristics on the modality; is the number of items in the clothing; The text and visual modality features of each item are expressed as: and ,in and Text and visual samples of the individual products respectively; and They are language representation function and deep residual network function respectively; Calculate the self-attention output of the clothing modality perception feature with multi-head attention, the expression is: in, is the multi-head attention function, is the first The output of the layer; and is modal The trainable parameters in Representation layer normalization operation; stacking self-attention blocks to obtain an enhanced representation of the input set and aggregate them into the overall representation of the suit using a fully connected layer: in, is a nonlinear activation function, It's a set After the The embedding representation after the multi-head self-attention block, and is modal The trainable parameters in .

3. The multimodal personalized clothing recommendation method based on item compatibility according to claim 1, characterized in that: The area under the receiver operating characteristic (ROC) curve (AUC) and normalized discounted cumulative gain (NDCG) are used as evaluation metrics to assess the accuracy of the personalized package recommendation task. AUC is used to measure the probability that the model's predicted preference score for a positive sample exceeds the predicted preference score for a negative sample for a given user's selected product. NDCG is used to evaluate the position of the positive sample in the positive ranking list.

4. A multimodal personalized clothing recommendation system based on item compatibility, characterized by: include: Module M1: Uses a multi-head self-attention mechanism to model and represent clothing and users separately. Different types of items in a clothing set are separated and feature models are established for each. For each type of item, a modality-aware encoder is used to independently model each modality to obtain multimodal fusion features of the item. Module M2: Utilizes existing product combinations in historical data and combines them with the co-location codes of different types of products to model the matching between users and different types of products. Module M3: Based on the user's interaction history with multiple items, the user-item interaction graph is used to perform collaborative position encoding to enhance the representation of user behavior characteristics. The collaborative position encoding is combined with the modal characteristics of the user's corresponding interest group to complete the modeling of user preferences. Diversified recommendations are made based on the collaborative information between users and items. The module M2 includes: Use historical data to construct a graph of the historical interaction of the item matching information in the set matching , all items in one set The feature vectors of serve as ID embeddings in the collocation behavior space, where the edges connecting all different types of items represent the interactive information in the historical data; Using the GCN algorithm, information is propagated and local position features are extracted from the graph as collaborative position encoding signals, which are expressed as: Among them, the collaborative behavior characteristics Indicates clothing items of Hop Neighbor, Gather and Indicates a set and users In the user-product interaction diagram Neighbor nodes in ; Multiple multi-head attention modules are used to model the matching degree of individual items. The previously obtained suit modal features and collaborative position encoding are integrated layer by layer. For each input clothing combination, all interactions generated by the individual items are used as input and initialized using residual network units. , the single product matching preference model is as follows: in, Attention mechanism function, is the activation function; They are query vector, key-value vector and value vector respectively; is the transpose symbol; is the dimension of the key-value vector; It is a single product In the The state vector of the layer residual network; and is a trainable parameter; Indicates a single product Coordinated position coding, represents the overall representation of n historical items that user u has interacted with in modality m, that is, ; experience After the multi-head attention module, we get: Among them, among them, Representative items In modal Matching characteristics in ; 、 、 、 All are trainable parameters; calculate other types of different items In each mode The matching degree with item k: in, Indicates the matching characteristics of candidate items in other categories; The scores for all modalities are summed to get the final predicted preference score: in, represents the total number of all modes; The module M3 includes: Build a graph based on the user's historical interactions with a single product ,user and single products Both act as ID embeddings within the user behavior space, connecting users and single products The edges indicate that there have been historical interactions between them; Using GCN, user features are extracted as follows: Among them, the collaborative behavior characteristics Represents a user of Hop Neighbor, Gather and Indicates a single product and users In the user-product interaction diagram Neighbor nodes in ; Multiple multi-head attention modules are used to model user preferences, and the previously obtained suit modality features and collaborative position encoding are integrated layer by layer to All historical items of interaction are used as input to build a user preference model, and the transformer layer is applied to the user features to obtain the user The preference representation of , the final predicted preference score is obtained: in, Represents a user The matching score with item i in modality m.

5. The multimodal personalized clothing recommendation system based on item compatibility according to claim 4 is characterized in that: The module M1 includes: Given a set of clothing items, it is represented as a set of modality-aware features , where Indicates that the i-th item in the clothing is in the modal The original feature representation in express The dimension of product characteristics on the modality; is the number of items in the clothing; The text and visual modality features of each item are expressed as: and ,in and Text and visual samples of the individual products respectively; and They are language representation function and deep residual network function respectively; Calculate the self-attention output of the clothing modality perception feature with multi-head attention, the expression is: in, is the multi-head attention function, is the first The output of the layer; and is modal The trainable parameters in Representation layer normalization operation; stacking self-attention blocks to obtain an enhanced representation of the input set and aggregate them into the overall representation of the suit using a fully connected layer: in, is a nonlinear activation function, It's a set After the The embedding representation after the multi-head self-attention block, and is modal The trainable parameters in .

6. The multimodal personalized clothing recommendation system based on item compatibility according to claim 4, characterized in that: The area under the receiver operating characteristic (ROC) curve (AUC) and normalized discounted cumulative gain (NDCG) are used as evaluation metrics to assess the accuracy of the personalized package recommendation task. AUC is used to measure the probability that the model's predicted preference score for a positive sample exceeds the predicted preference score for a negative sample for a given user's selected product. NDCG is used to evaluate the position of the positive sample in the positive ranking list.

Images