Fashion item recommendation method and system integrating fine-grained compatibility and multi-view correlation

Abstract

The present invention relates to the technical field of recommendation system and deep learning, and relates to a method and system for recommending fashion items integrating fine-grained compatibility and multi-view correlation. The method comprises: extracting visual features and text features of each item in M-1 existing complementary items; splicing the visual features and text features of each item to obtain multimodal features of each item; using a multi-space compatibility modeling module based on semantic conditional dynamic convolution to process the multimodal features of each item to obtain a multi-space compatibility score based on semantic conditional dynamic convolution; using a multi-view based deep correlation modeling module to process the multimodal features of each item to obtain a multi-view based deep correlation score; performing weighted summation on the two scores to obtain a final score of a complementary clothing recommendation task, and obtaining a recommendation result of a target item according to the final score of the complementary clothing recommendation task. In actual scenarios, a better recommendation effect is achieved.

Description

Technical Field

[0001] The present invention relates to the field of recommendation systems and deep learning technologies, and in particular to a fashion item recommendation method and system integrating fine-grained compatibility and multi-view correlation. Background Art

[0002] With the continuous development of the economy, various fashion items such as crossbody bags, floral dresses, Martin boots, and black denim shorts have emerged rapidly in the online and offline markets, greatly enriching consumers' choices and adding beauty to daily life. However, this diversity also brings challenges, especially for consumers who may lack aesthetic judgment. How to find fashion items that match the clothes you already have in the complex commodities has become a difficult problem. Therefore, the importance of complementary fashion recommendation has become increasingly prominent. It aims to recommend a complementary fashion item for a given incomplete suit that is missing one. For example, based on the selected black leather jacket, white printed T-shirt, and blue denim pants, a pair of boots with matching style is recommended. For online shopping platforms, implementing an effective clothing recommendation system can significantly improve consumers' shopping experience, because customers tend to choose items that match the clothes they have purchased. Therefore, developing a recommendation algorithm that can provide accurate matching suggestions at the right time has become a key strategy for e-commerce platforms to optimize user experience.

[0003] Previous methods have achieved certain results in pairwise complementary recommendation, which lays the foundation for understanding the mutual relationship between items. However, these methods do not explicitly deal with the compatibility relationship of multiple items. To fill this research gap, recent methods focus on predicting the compatibility of sets composed of multiple items. Although these methods perform well, they have the following two key limitations: These methods only focus on modeling the compatibility between items and ignore the correlation of the overall matching. In fact, when recommending complementary items, in addition to focusing on the compatibility of fine-grained attributes (such as color, material, etc.) between items, it is also necessary to consider the potential correlation between multiple items from the perspective of overall matching. Most existing techniques focus on using a single latent space to learn the compatibility relationship between items of different categories, while this paper believes that the compatibility of sets should be modeled in multiple semantic subspaces because it is essentially determined by multiple fine-grained attributes (such as color, material, style, etc.). In addition, these techniques implicitly learn the mapping parameters of fine-grained attributes in a predefined way, which affects the model's understanding of the relationship between features. Summary of the invention

[0004] In order to address the shortcomings of the prior art, the present invention provides a fashion item recommendation method and system that integrates fine-grained compatibility and multi-view correlation, which effectively addresses the problem that existing models only consider clothing matching from a single perspective and implicitly model fine-grained attributes between items in a single latent space, and achieves better results in actual scenarios.

[0005] On the one hand, a fashion item recommendation method integrating fine-grained compatibility and multi-view correlation is provided, including: obtaining images and text descriptions of M-1 existing complementary items, as well as the name of the target item; inputting the obtained data into a trained fashion item recommendation model to obtain a recommendation result of the target item; wherein the trained fashion item recommendation model extracts the visual features and text features of each item in the M-1 existing complementary items; concatenates the visual features and text features of each item to obtain the multimodal features of each item; uses a multi-space compatibility modeling module based on semantic conditional dynamic convolution to process the multimodal features of each item to obtain a multi-space compatibility score based on semantic conditional dynamic convolution; uses a deep correlation modeling module based on multiple views to process the multimodal features of each item to obtain a deep correlation score based on multiple views; performs weighted summation of the multi-space compatibility score based on semantic conditional dynamic convolution and the deep correlation score based on multiple views to obtain a final score of the complementary clothing recommendation task, and obtains the recommendation result of the target item according to the final score of the complementary clothing recommendation task.

[0006] On the other hand, a fashion item recommendation system integrating fine-grained compatibility and multi-view correlation is provided, including: an acquisition module, which is configured to: acquire images and text descriptions of M-1 existing complementary items, and the name of a target item; a recommendation module, which is configured to: input the acquired data into a trained fashion item recommendation model to obtain a recommendation result for the target item; wherein the trained fashion item recommendation model extracts visual features and text features of each item in the M-1 existing complementary items; splices the visual features and text features of each item to obtain a multimodal feature map of each item. feature; a multi-space compatibility modeling module based on semantic conditional dynamic convolution is used to process the multimodal features of each single product to obtain a multi-space compatibility score based on semantic conditional dynamic convolution; a deep correlation modeling module based on multi-view is used to process the multimodal features of each single product to obtain a deep correlation score based on multi-view; the multi-space compatibility score based on semantic conditional dynamic convolution and the deep correlation score based on multi-view are weightedly summed to obtain the final score of the complementary clothing recommendation task, and according to the final score of the complementary clothing recommendation task, the recommendation result of the target product is obtained.

[0007] The above technical solution has the following advantages or beneficial effects: 1) In order to comprehensively perform complementary fashion recommendation tasks from different angles, the present invention proposes a new method, namely, a complementary fashion recommendation method that integrates fine-grained compatibility and multi-view correlation, which recommends complementary fashion items by simultaneously modeling the fine-grained compatibility and overall matching correlation between items of different categories.

[0008] 2) A new multi-space compatibility modeling method based on semantic conditional dynamic convolution is proposed. This scheme dissociates and aligns the fine-grained attributes between items of different categories, while explicitly and dynamically generating convolution parameters and mapping them into multiple semantic subspaces, thereby learning the fine-grained compatibility between items.

[0009] 3) This paper proposes a new multi-view based deep correlation analysis method, which evaluates the correlation of the entire set of clothing by treating each item in the set as an independent view and maximizing the typical correlation information of these views in a common latent space. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0011] Figure 1 This is a flow chart of the method of embodiment 1.

[0012] Figure 2 This is a schematic diagram of fashion item recommendation in Example 1. DETAILED DESCRIPTION

[0013] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0014] Embodiment 1, as Figure 1As shown, this embodiment provides a fashion item recommendation method integrating fine-grained compatibility and multi-view correlation, including: S101: obtaining images and text descriptions of M-1 existing complementary items, as well as the name of the target item; S102: inputting the obtained data into the trained fashion item recommendation model to obtain the recommendation result of the target item; wherein the trained fashion item recommendation model extracts the visual features and text features of each item in the M-1 existing complementary items; splices the visual features and text features of each item to obtain the multimodal features of each item; adopts the basic The multi-space compatibility modeling module based on semantic conditional dynamic convolution is used to process the multimodal features of each item to obtain a multi-space compatibility score based on semantic conditional dynamic convolution. The multi-view based deep correlation modeling module is used to process the multi-modal features of each item to obtain a multi-view based deep correlation score. The multi-space compatibility score based on semantic conditional dynamic convolution and the deep correlation score based on multi-view are weightedly summed to obtain the final score of the complementary clothing recommendation task. According to the final score of the complementary clothing recommendation task, the recommendation result of the target item is obtained.

[0015] Assumptions express A collection of outfits put together by fashion experts. , both include For each fashion item , respectively use and to represent its visual image and text description. To this end, a complementary fashion recommendation model that combines fine-grained compatibility and multi-view correlation is designed. , which can integrate the multimodal information of fashion items and introduce a multi-space compatibility modeling score based on semantic conditional dynamic convolution and multi-view based depth correlation scores Improving the performance of complementary clothing recommendation tasks from multiple perspectives , which is expressed as follows:

[0016] ;

[0017] in, is the set of all parameters in the model.

[0018] Furthermore, if Figure 2As shown, S101: obtain images and text descriptions of M-1 existing complementary items, as well as the name of the target item; wherein the M-1 existing complementary items, for example, include: a set of clothing including a silk V-neck shirt, white suit pants, a black leather bag and silver pearl earrings, and the name of the target item, for example, includes: shoes.

[0019] Furthermore, the training process of the trained fashion item recommendation model includes:

[0020] Construct a training set, wherein the training set is M-1 images and text descriptions of existing complementary products for which the target product recommendation results are known;

[0021] The training set is input into the fashion item recommendation model to train the model. When the loss function value of the fashion item recommendation model no longer decreases or the number of iterations exceeds the set number, the training is stopped to obtain the trained fashion item recommendation model.

[0022] Furthermore, the loss function of the fashion item recommendation model is specifically expressed as:

[0023] ;

[0024] in, is the regularization coefficient, is the set of model parameters;

[0025] Cross Entropy Loss as follows:

[0026] ;

[0027] in, It is The true label of the set.

[0028] In multi-view based deep correlation modeling, optimization is performed by minimizing the correlation loss, and its loss function is as follows:

[0029] ;

[0030] in, is a matrix The trace norm of Represents the trace operation of a matrix.

[0031] ;

[0032] in, and Respectively and The covariance matrix of express and The cross-covariance matrix of ;

[0033] ; ;

[0034] ; ; ;

[0035] in, is the regularization term, and Is a view and Depth representation.

[0036] Furthermore, the trained fashion item recommendation model extracts visual features and text features of each item in the M-1 existing complementary items;

[0037] For each product image, the ResNet network is used to extract visual features. The feature map extracted by each layer of the ResNet network is converted to obtain a feature vector. The feature vectors of all layers are concatenated to obtain the visual features of each product.

[0038] The text description of each item is encoded to obtain an initial vector, and the initial vector is processed using 6 serially connected Transformer layers to obtain the text features of each item.

[0039] For example, for each fashion item Visual image , the pre-trained network ResNet-18, which has outstanding performance in the field of image representation learning, is used to extract its visual features. The pre-trained network ResNet-18 consists of 18 convolutional layers, and each layer can capture visual information of a different range. Among them, the shallower layers of the network mainly focus on capturing local fine-grained information (such as color, texture, etc.), while the deeper layers can learn more complex semantic information (such as style, style, etc.). In particular, the present invention uses global average pooling (Global Average Pooling, GAP) to convert the feature map of each layer into a feature vector, while reducing the data dimension while retaining its global information. Finally, the present invention splices the features extracted from different layers to obtain each single product. Visual features .in, is the dimension of visual features.

[0040] For example, to capture fashion items Based on the text features of the fashion item, the present invention adopts the BERT (Bidirectional Encoder Representations from Transformers) model, which has strong versatility and robustness in extracting deep semantic information from the text. First, the text description of each fashion item is The model is fed with data and each word is encoded into an initial vector through a word embedding layer. Subsequently, the initial vector is further extracted through 6 series-connected Transformer layers with a self-attention mechanism, which can not only learn the information of each word in the text, but also understand the complex relationship between words. Finally, the output of the last Transformer layer is used as a fashion item. Text features .in, is the dimension of text features.

[0041] Furthermore, the visual features and text features of each single product are spliced ​​to obtain multimodal features of each single product, and the splicing is a serial splicing.

[0042] For example, given complementary items and target items, through a set of parallel semantic conditions To align the fine-grained semantic features between items of different categories. First, for each item , its multimodal features It can be obtained by:

[0043] ;

[0044] in, Represents a concatenation operation.

[0045] Existing complementary clothing recommendation methods usually learn the compatibility relationship between items of different categories through a potential compatibility space. It is difficult to fully capture the complex and changeable compatibility factors between items and lacks interpretability. In fact, the compatibility between items of different categories is jointly determined by a variety of complex fine-grained attributes (such as color, material, style, etc.), and the dominant factors vary in different combinations. In addition, existing methods usually implicitly learn the mapping parameters of multimodal information to fine-grained attributes in a predefined manner and rely on a shared and unified encoding method. To solve the above problems, the present invention proposes a multi-space compatibility modeling method based on semantic conditional dynamic convolution, which dissociates and aligns the fine-grained attributes between items of different categories, and explicitly and dynamically generates convolution parameters and maps them to multiple semantic subspaces, thereby learning the fine-grained compatibility between items.

[0046] Furthermore, the multi-space compatibility modeling module based on semantic conditional dynamic convolution is used to process the multimodal features of each single product to obtain a multi-space compatibility score based on semantic conditional dynamic convolution, including:

[0047] (1-1) The multimodal features of different items are dissociated under different semantic conditions, and the aligned fine-grained semantic features are represented;

[0048] (1-2) Learn convolution weights and bias parameters for encoding fine-grained semantic features of different items according to different sets;

[0049] (1-3) Determine the aligned fine-grained semantic features based on the convolution weights, bias parameters and semantic features;

[0050] (1-4) Based on the aligned fine-grained semantic features, the fine-grained compatibility scores of M-1 existing complementary items and the target item in different semantic subspaces are calculated, and then the multi-space compatibility scores based on semantic conditional dynamic convolution are obtained.

[0051] For example, (1-1) the multimodal features of different items are dissociated under different semantic conditions, and the aligned fine-grained semantic features are represented, including:

[0052] Multimodal features for different items Fine-grained semantic features after dissociation and alignment under different semantic conditions It is expressed as follows:

[0053] ;

[0054] in, express Operation.

[0055] Exemplarily, (1-2) learning convolution weights and bias parameters for encoding fine-grained semantic features of different items according to different sets includes:

[0056] Explicitly encode learnable convolutional weights for fine-grained semantic features of different items according to different sets and bias Parameters are explicitly generated in the following way:

[0057] ;

[0058] in, is the activation function.

[0059] Currently, existing methods usually map all features of different items into the same space by randomly initializing model parameters and sharing implicit weight vectors, which affects the model's understanding of the relationship between features. To this end, a dynamic convolutional network is designed. .

[0060] Exemplarily, (1-3) determines the aligned fine-grained semantic features based on the convolution weights, bias parameters and semantic features, including:

[0061] The same fine-grained semantic features after separating complementary items Mapped into the same semantic subspace and further aligned as follows:

[0062] .

[0063] For example, (1-4) is based on the aligned fine-grained semantic features, and the fine-grained compatibility scores of M-1 existing complementary items and the target item in different semantic subspaces are calculated, and then the multi-space compatibility scores based on semantic conditional dynamic convolution are obtained, including:

[0064] Given Fine-grained compatibility scores of the item and the target item in different semantic subspaces Calculated as follows:

[0065] ,

[0066] in, , and are the weight matrices and biases.

[0067] Finally, the multi-space compatibility score based on semantic conditional dynamic convolution The calculation is as follows:

[0068] .

[0069] In actual matching, in addition to paying attention to the fine-grained compatibility between items, it is also necessary to pay attention to the overall correlation. For a set of clothing including a silk V-neck shirt, white suit pants, a black leather bag and silver pearl earrings, considering only from the perspective of compatibility, the present invention will find that silver pearl shoes, black Mary Jane shoes and white high heels all meet the complementary and compatible matching requirements. However, further considering the correlation between the overall clothing, the present invention will find that white high heels and other items form a more harmonious and unified suit. Therefore, further considering the correlation between items of different categories can improve the recommendation effect. In addition, when calculating multi-view correlation, existing methods usually need to consider the correlation between each pair of views separately, which will cause the computational complexity to increase significantly with the increase in the number of views. To solve this problem, the present invention proposes a deep correlation modeling method based on multiple views, which evaluates the correlation of the entire set of clothing by treating each item in the suit as an independent view and maximizing the typical correlation information of these views in a common latent space.

[0070] Furthermore, the multi-view based deep correlation modeling module is used to process the multimodal features of each single product to obtain a multi-view based deep correlation score, including:

[0071] (2-1) Combine the features of all the individual items to get the overall view of the set;

[0072] (2-2) Determine a new view based on the sparse matrix and the overall view of the set;

[0073] (2-3) Based on two independent convolutional neural networks, the feature representation of the overall view and the feature representation of the new view are learned respectively;

[0074] (2-4) Determine the centering matrix of the overall view and the centering matrix of the new view according to the feature representation of the overall view and the feature representation of the new view;

[0075] (2-5) Based on the centering matrix of the overall view and the centering matrix of the new view, define the covariance matrix and the cross-covariance matrix;

[0076] (2-6) Based on the covariance matrix and the cross-covariance matrix, a multi-view based depth correlation score is determined.

[0077] For example, (2-1) combines the features of all the individual items to obtain an overall view of the set, including:

[0078] In order to integrate the information of all views, the features of all items are spliced ​​together to obtain the overall view of the set. :

[0079] .

[0080] Exemplarily, the (2-2) determines a new view based on the sparse matrix and the overall view of the set, including: in order to better analyze the potential correlation between different single product views, the present invention replaces the splicing order of single product features to obtain different overall views. A commonly used method is to construct a permutation matrix, but this will result in huge memory consumption and high computational complexity. To solve this problem, the present invention uses a sparse matrix to replace the traditional permutation matrix. The sparse matrix only records the positions of non-zero elements, thereby significantly reducing memory consumption. The present invention constructs a sparse matrix To get a new view , which is expressed as follows:

[0081] ;

[0082] in, is the matrix multiplication operation.

[0083] Exemplarily, the (2-3) respectively learns the feature representation of the overall view and the feature representation of the new view according to two independent convolutional neural networks, including: using Deep Canonical Correlation Analysis (DCCA) to learn the nonlinear relationship between multiple views in different sequences and maximize the mutual correlation between them.

[0084] Specifically, two independent convolutional neural networks (CNNs) are used to learn and The deep representation of and , whose parameters are and By adjusting the relevant parameters and Maximizing network output and The typical correlation between them is as follows:

[0085] ;

[0086] The corr() function calculates the correlation between two outputs, and argmax means finding the parameter that maximizes the correlation.

[0087] Exemplarily, the step (2-4) determines the centering matrix of the overall view and the centering matrix of the new view according to the feature representation of the overall view and the feature representation of the new view, including:

[0088] Centralized matrix for a holistic view for: ;

[0089] The centering matrix of the new view for: .

[0090] in, represents a matrix of all 1s.

[0091] Exemplarily, the (2-5) defines a covariance matrix and a cross-covariance matrix according to a centralization matrix of the whole view and a centralization matrix of the new view, including:

[0092] definition and The covariance matrix and cross-covariance matrix of are as follows:

[0093]

[0094] in, and Respectively and The covariance matrix of express and The cross covariance matrix of is the regularization term, and Before The total correlation of the principal components is the matrix Before The sum of the singular values.

[0095] Exemplarily, the (2-6) determines the multi-view based depth correlation score according to the covariance matrix and the cross-covariance matrix, including:

[0096] Multi-view based depth correlation score The calculation is as follows:

[0097]

[0098] in, Indicates The relevance score of the set of clothing; Represents the calculation of two deep feature representations and The correlation between The overall view H is passed through the CNN network The obtained depth representation; The permuted feature sequence G is passed through the CNN network The obtained depth representation; : normalization factor, is the number of singular values ​​selected to normalize the final score to between 0 and 1; Represents the front of the matrix T The sum of the largest singular values;

[0099] ;

[0100] in, express The covariance matrix of express The covariance matrix of express and The cross covariance matrix of .

[0101] Furthermore, the multi-space compatibility score based on semantic conditional dynamic convolution and the depth correlation score based on multiple views are weightedly summed to obtain the final score of the complementary clothing recommendation task, including:

[0102] Calculate the final score for the complementary clothing recommendation task The calculation is as follows:

[0103]

[0104] in, is a non-negative balance parameter that measures the importance of compatibility and relevance.

[0105] In order to verify the rationality and performance of the present invention, experiments were conducted on two data sets. The present invention was compared with existing benchmark models on three tasks: complementary product recommendation, filling in the blanks, and complementary product retrieval.

[0106] (1) Complementary product recommendation: A binary classification method is used to determine whether the recommended product can form a complete set with the given M-1 items, and AUC (Area Under the ROC Curve) is used as the main evaluation indicator.

[0107] (2) Fill in the blanks: Given an incomplete set of M-1 items, select one item from a set of candidate items (including one positive sample and three negative samples) to complete the clothing matching task. In order to evaluate the performance of this task, the present invention uses the accuracy Acc for evaluation.

[0108] (3) Complementary product retrieval: Consider the given M-1 products as a “query” and retrieve the complementary products with the highest scores from a larger candidate set. To evaluate the performance of this task, the present invention uses Recall@k to evaluate the ranking position of positive samples in the retrieval results, and sets k=10, k=30 and k=50 respectively.

[0109] The present invention mainly includes two key modules, multi-space compatibility modeling based on semantic conditional dynamic convolution and deep correlation modeling based on multi-view. The specific details are as follows:

[0110] Multi-space compatibility modeling based on semantic conditional dynamic convolution aims to learn the fine-grained compatibility of different items. To this end, semantic conditions and dynamic convolution are used to dually align the fine-grained semantic features between different types of items. The specific steps include:

[0111] (1) Dissociation: Given M-1 complementary items and a target item, in order to learn and dissociate their complex fine-grained attributes, this paper uses a set of parallel semantic conditions To align the fine-grained semantic features between items of different categories.

[0112] (2) Encoding learnable convolutional parameters: Through dynamic convolutional networks, learnable convolutional weights are explicitly encoded for the fine-grained semantic features of different items according to different sets. and bias , the same fine-grained semantic features after the dissociation of complementary items are mapped into the same semantic subspace.

[0113] (3) Learning fine-grained compatibility scores: Use a two-layer multilayer perceptron to learn the fine-grained compatibility scores of M-1 items and the target item in different semantic subspaces, and finally sum and average them to obtain the overall compatibility score.

[0114] The deep correlation modeling based on multiple views regards each item in the suit as an independent view, and maximizes the typical correlation information of these views in a common latent space, thereby evaluating the correlation of the entire suit. The specific steps include: Integrate the information of all views: When calculating the multi-view correlation, the existing methods usually need to consider the correlation between each pair of views separately, which will significantly increase the computational complexity with the increase in the number of views. The present invention obtains different overall views by designing a sparse matrix to permute the splicing order of the item features, thereby significantly reducing memory consumption. Learn the potential correlation: Use deep canonical correlation analysis to learn the nonlinear relationship between different overall views and maximize the mutual correlation between them, so as to obtain the overall correlation score between M-1 items and the target item.

[0115] Embodiment 2: This embodiment provides a fashion item recommendation system integrating fine-grained compatibility and multi-view correlation, including: an acquisition module, which is configured to: acquire images and text descriptions of M-1 existing complementary items, and the name of the target item; a recommendation module, which is configured to: input the acquired data into the trained fashion item recommendation model to obtain the recommendation result of the target item; wherein the trained fashion item recommendation model extracts the visual features and text features of each item in the M-1 existing complementary items; splices the visual features and text features of each item to obtain a multi-modal recommendation result of each item. The multi-modal features of each item are processed by a multi-space compatibility modeling module based on semantic conditional dynamic convolution to obtain a multi-space compatibility score based on semantic conditional dynamic convolution; the multi-view based deep correlation modeling module is used to process the multi-modal features of each item to obtain a multi-view based deep correlation score; the multi-space compatibility score based on semantic conditional dynamic convolution and the deep correlation score based on multi-view are weightedly summed to obtain the final score of the complementary clothing recommendation task, and the recommendation result of the target item is obtained according to the final score of the complementary clothing recommendation task.

[0116] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A fashion item recommendation method integrating fine-grained compatibility and multi-view correlation, characterized by: include: Obtain images and text descriptions of M-1 existing complementary items, as well as the name of the target item; The acquired data is input into the trained fashion item recommendation model to obtain the recommendation result of the target item; wherein the trained fashion item recommendation model extracts the visual features and text features of each item in the M-1 existing complementary items; the visual features and text features of each item are spliced ​​to obtain the multimodal features of each item; the multi-space compatibility modeling module based on semantic conditional dynamic convolution is used to process the multimodal features of each item to obtain the multi-space compatibility score based on semantic conditional dynamic convolution; the multi-modal features of each item are processed by the deep correlation modeling module based on multi-view to obtain the deep correlation score based on multi-view; the multi-space compatibility score based on semantic conditional dynamic convolution and the deep correlation score based on multi-view are weightedly summed to obtain the final score of the complementary clothing recommendation task, and the recommendation result of the target item is obtained according to the final score of the complementary clothing recommendation task; The multi-space compatibility modeling module based on semantic conditional dynamic convolution is used to process the multimodal features of each single product to obtain a multi-space compatibility score based on semantic conditional dynamic convolution, including: (1-1) The multimodal features of different items are dissociated under different semantic conditions, and the aligned fine-grained semantic features are represented; (1-2) Learn convolution weights and bias parameters for encoding fine-grained semantic features of different items according to different sets; (1-3) Determine the aligned fine-grained semantic features based on the convolution weights, bias parameters and semantic features; (1-4) Based on the aligned fine-grained semantic features, the fine-grained compatibility scores of the M-1 existing complementary items and the target item in different semantic subspaces are calculated, and then the multi-space compatibility scores based on semantic conditional dynamic convolution are obtained; The multi-view based deep correlation modeling module is used to process the multimodal features of each single product to obtain a multi-view based deep correlation score, including: (2-1) Combine the features of all the individual items to get the overall view of the set; (2-2) Determine a new view based on the sparse matrix and the overall view of the set; (2-3) Based on two independent convolutional neural networks, the feature representation of the overall view and the feature representation of the new view are learned respectively; (2-4) Determine the centering matrix of the overall view and the centering matrix of the new view according to the feature representation of the overall view and the feature representation of the new view; (2-5) Based on the centering matrix of the overall view and the centering matrix of the new view, define the covariance matrix and the cross-covariance matrix; (2-6) Based on the covariance matrix and the cross-covariance matrix, a multi-view based depth correlation score is determined.

2. The fashion item recommendation method integrating fine-grained compatibility and multi-view correlation as claimed in claim 1 is characterized in that: The training process of the trained fashion item recommendation model includes: Construct a training set, wherein the training set is M-1 images and text descriptions of existing complementary products for which the target product recommendation results are known; The training set is input into the fashion item recommendation model to train the model. When the loss function value of the fashion item recommendation model no longer decreases, or the number of iterations exceeds the set number, the training is stopped to obtain the trained fashion item recommendation model. The loss function of the fashion item recommendation model is specifically expressed as: ； in, is the regularization coefficient, is the set of model parameters; Cross Entropy Loss as follows: ； in, It is The real label of the set, is the multi-space compatibility modeling score; In multi-view based deep correlation modeling, optimization is performed by minimizing the correlation loss, and its loss function is as follows: ； in, is a matrix The trace norm of Represents the trace operation of a matrix; ； in, and Respectively and The covariance matrix of express and The cross-covariance matrix of ; ； ； ； ； ； in, is the regularization term, and Is a view and Depth representation.

3. The fashion item recommendation method integrating fine-grained compatibility and multi-view correlation as claimed in claim 1 is characterized in that: The trained fashion item recommendation model extracts visual features and text features of each item in the M-1 existing complementary items; including: extracting visual features of each item image using a ResNet network, converting the feature map extracted from each layer of the ResNet network to obtain a feature vector, and concatenating the feature vectors of all layers to obtain the visual features of each item; The text description of each item is encoded to obtain an initial vector; the initial vector is processed using 6 serially connected Transformer layers to obtain the text features of each item.

4. The fashion item recommendation method integrating fine-grained compatibility and multi-view correlation as claimed in claim 1, characterized in that: (1-1) The multimodal features of different items are dissociated under different semantic conditions, and the aligned fine-grained semantic features are represented, including: Multimodal features for different items Fine-grained semantic features after dissociation and alignment under different semantic conditions It is expressed as follows: ； in, express Operation; ; (1-2) Encoding the fine-grained semantic features of different items according to different sets, learning convolution weights and bias parameters, including: Explicitly encode learnable convolutional weights for fine-grained semantic features of different items according to different sets and bias Parameters are explicitly generated in the following way: ； in, is the activation function; (1-3) Based on the convolution weights, bias parameters and semantic features, the aligned fine-grained semantic features are determined, including: The same fine-grained semantic features after separating complementary items Mapped into the same semantic subspace and further aligned as follows: ； (1-4) Based on the aligned fine-grained semantic features, the fine-grained compatibility scores of the M-1 existing complementary items and the target item in different semantic subspaces are calculated, and then the multi-space compatibility scores based on semantic conditional dynamic convolution are obtained, including: Given Fine-grained compatibility scores of the item and the target item in different semantic subspaces Calculated as follows: ； in, , is the weight matrix, is bias; Finally, the multi-space compatibility score based on semantic conditional dynamic convolution The calculation is as follows: 。 5. The fashion item recommendation method integrating fine-grained compatibility and multi-view correlation as claimed in claim 1, characterized in that: The above (2-1) combines the features of all the individual items to obtain the overall view of the set, including: in order to integrate the information of all the views, combine the features of all the individual items to obtain the overall view of the set : ； Constructed a sparse matrix To get a new view , which is expressed as follows: ； in, is the matrix multiplication operation; The method (2-3) uses two independent convolutional neural networks to learn the feature representation of the overall view and the feature representation of the new view respectively, including: first, learning the feature representation of the overall view and the feature representation of the new view respectively by using two independent convolutional neural networks. and The deep representation of and , whose parameters are and ; By adjusting the relevant parameters and Maximizing network output and The typical correlation between them is as follows: ； The corr() function calculates the correlation between two outputs, and argmax means finding the parameter that maximizes this correlation. The step (2-4) determines the centering matrix of the overall view and the centering matrix of the new view according to the feature representation of the overall view and the feature representation of the new view, including: The centralization matrix of the overall view is: ; The centering matrix of the new view is: ; in, represents a matrix of all ones.

6. The fashion item recommendation method integrating fine-grained compatibility and multi-view correlation as claimed in claim 5 is characterized in that: The above (2-5) defines the covariance matrix and the cross-covariance matrix according to the centering matrix of the overall view and the centering matrix of the new view, including: definition and The covariance matrix and cross-covariance matrix of are as follows: ； in, and Respectively and The covariance matrix of express and The cross covariance matrix of is the regularization term, and Before The total correlation of the principal components is the matrix Before The sum of singular values; The (2-6) determines the multi-view based depth correlation score according to the covariance matrix and the cross-covariance matrix, including: Multi-view based depth correlation score The calculation is as follows: ； in, Indicates The relevance score of the set of clothing; Represents the calculation of two deep feature representations and The correlation between The overall view H is passed through the CNN network The obtained depth representation; The overall view G is passed through the CNN network The obtained depth representation; represents the normalization factor, is the number of singular values ​​selected to normalize the final score to between 0 and 1; Represents the front of the matrix T The sum of the largest singular values; ； in, express The covariance matrix of express The covariance matrix of express and The cross covariance matrix of .

7. The fashion item recommendation method integrating fine-grained compatibility and multi-view correlation as claimed in claim 6 is characterized in that: The multi-space compatibility score based on semantic conditional dynamic convolution and the depth correlation score based on multiple views are weightedly summed to obtain the final score of the complementary clothing recommendation task, including: Calculate the final score for the complementary clothing recommendation task The calculation is as follows: ； in, is a non-negative balance parameter that measures the importance of compatibility and relevance.

8. A fashion item recommendation system integrating fine-grained compatibility and multi-view correlation, characterized by: include: An acquisition module is configured to: acquire images and text descriptions of M-1 existing complementary items, and a name of a target item; The recommendation module is configured to: input the acquired data into the trained fashion item recommendation model to obtain the recommendation result of the target item; wherein the trained fashion item recommendation model extracts the visual features and text features of each item in M-1 existing complementary items; splices the visual features and text features of each item to obtain the multimodal features of each item; uses a multi-space compatibility modeling module based on semantic conditional dynamic convolution to process the multimodal features of each item to obtain a multi-space compatibility score based on semantic conditional dynamic convolution; uses a deep correlation modeling module based on multiple views to process the multimodal features of each item to obtain a deep correlation score based on multiple views; performs weighted summation of the multi-space compatibility score based on semantic conditional dynamic convolution and the deep correlation score based on multiple views to obtain the final score of the complementary clothing recommendation task, and obtains the recommendation result of the target item according to the final score of the complementary clothing recommendation task; The multi-space compatibility modeling module based on semantic conditional dynamic convolution is used to process the multimodal features of each single product to obtain a multi-space compatibility score based on semantic conditional dynamic convolution, including: (1-1) The multimodal features of different items are dissociated under different semantic conditions, and the aligned fine-grained semantic features are represented; (1-2) Learn convolution weights and bias parameters for encoding fine-grained semantic features of different items according to different sets; (1-3) Determine the aligned fine-grained semantic features based on the convolution weights, bias parameters and semantic features; (1-4) Based on the aligned fine-grained semantic features, the fine-grained compatibility scores of the M-1 existing complementary items and the target item in different semantic subspaces are calculated, and then the multi-space compatibility scores based on semantic conditional dynamic convolution are obtained; The multi-view based deep correlation modeling module is used to process the multimodal features of each single product to obtain a multi-view based deep correlation score, including: (2-1) Combine the features of all the individual items to get the overall view of the set; (2-2) Determine a new view based on the sparse matrix and the overall view of the set; (2-3) Based on two independent convolutional neural networks, the feature representation of the overall view and the feature representation of the new view are learned respectively; (2-4) Determine the centering matrix of the overall view and the centering matrix of the new view according to the feature representation of the overall view and the feature representation of the new view; (2-5) Based on the centering matrix of the overall view and the centering matrix of the new view, define the covariance matrix and the cross-covariance matrix; (2-6) Based on the covariance matrix and the cross-covariance matrix, a multi-view based depth correlation score is determined.

Images