Multi-modal clothing recommendation method and equipment based on graph reconstruction representation learning
By constructing a user-item interaction graph and using a graph reconstruction representation learning method, the problem of noise processing in multimodal recommendation systems is solved, achieving more accurate and personalized clothing recommendations.
Patent Information
- Application Number
- CN202510493588.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-18
- Publication Date
- 2025-09-12
AI Technical Summary
Existing multimodal recommendation systems cannot effectively handle modal data noise, resulting in a decrease in the accuracy of recommendation results.
By constructing a user-item interaction graph and interaction matrix, the variational graph autoencoder (VGAE) is used to reconstruct the item-item graph. Combining graph convolution and graph transformer modules, global structural dependencies are captured, multimodal features are extracted and fused, and the final feature representation of the user-item is obtained. The interaction score is calculated for recommendation.
Effectively remove noise data, improve the accuracy and personalization level of the recommendation system, capture users' multimodal interests and preferences, and provide more accurate clothing recommendations.
Smart Images
Figure CN120632082A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of multimodal recommendation, and in particular to a multimodal clothing recommendation method and device based on graph reconstruction representation learning. Background Art
[0002] Multimodal recommender systems (MMRSs) are a rapidly emerging branch of recommender system research in recent years. Traditional recommendation systems typically rely on user behavioral data, such as clicks, browsing, and ratings. In some cases, this single-modal information may not be sufficient to fully capture user interests and preferences. With the rapid development of the internet and the widespread use of multimedia content, users have generated a large amount of multimodal data across various platforms, such as text, images, video, and audio. This provides a richer source of information for recommendation systems. Advances in deep learning technologies, particularly graph neural networks (GNNs) and graph transformer models, have made it possible to extract high-order relationships in graph structures from different modalities and fuse global features. Multimodal recommender systems have emerged in this context. By integrating information from multiple modalities, they can more comprehensively capture user interests, improving recommendation accuracy and user experience.
[0003] Multimodal recommendation systems involve more than simply combining and fusing data from various modalities; model design also requires fully considering the relevance and complementarity between these modalities. Specifically, text data can help understand users' semantic needs, image data can reveal visual preferences, video data can combine temporal information with visual content, and audio data can capture users' auditory preferences. By effectively fusing information from these different modalities, we can better understand users' interests and needs, thereby providing more personalized and precise recommendations.
[0004] Although multimodal recommendation systems have shown great potential in improving recommendation performance, the modal data used in real-world recommendation scenarios often contains noise. This noise can arise from the data collection process, random user behavior, or data quality issues. If left unaddressed, noisy data can interfere with model training and prediction, resulting in reduced recommendation accuracy. Therefore, effectively handling noisy data has become an important research topic in the design of multimodal recommendation systems. Summary of the Invention
[0005] The purpose of the present invention is to provide a multimodal clothing recommendation method based on graph reconstruction representation learning in order to solve the problem that existing multimodal recommendation systems cannot effectively handle modal data noise.
[0006] The above-mentioned purpose of this application is achieved through the following technical solutions: S1: Obtain text information and image information of clothing items, as well as user-clothing item interaction information, and construct a user-item interaction graph and a user-item interaction matrix; S2: Obtain multimodal features based on text information and image information; process the multimodal features and combine them with the KNN sparsification method to obtain an item-item graph for each modal feature; S3: Use the variational graph autoencoder (VGAE) to reconstruct the item-item graph and combine it with the user-item interaction matrix to obtain the user-item modal feature representation; perform graph convolution on the user-item interaction graph to obtain the user-clothing item feature representation; S4: Through the graph transformer module, the global structural dependencies of the feature space of different clothing items are captured to obtain the global feature representation of users and items; S5: Extract modality-specific features from the user-item modal feature representation, and obtain the final feature representation by fusing the user-clothing item feature representation with the modality-specific features and the global feature representation. Based on the final feature representation, calculate the user-item interaction score and obtain candidate recommendation results for clothing items.
[0007] Optionally, step S1 includes: Set user set , clothing item collection ; Construct a node set through text information and image information ; Build a user-item interaction edge set through interaction information represents the relationship between users and clothing items, where Represents a user and clothing items There is an edge between them, otherwise ; Using Node Collections and user-item interaction edge sets , build a user-item interaction graph ; Build Represents the user-item interaction matrix.
[0008] Optionally, step S2 includes: S21: Use the pre-trained deep learning model VGG to extract visual modal features of image information; use the pre-trained deep learning model Word2vec to extract text modal features of text information; set up represents a modal set, Indicates the visual modality, Indicates text mode; The modal characteristics are expressed as ,in Represents the dimension of the feature matrix, wherein the modal features include: text modal features and visual modal features; S22: Calculate the cosine similarity between all clothing items corresponding to each modal feature, obtain a similarity matrix and perform normalization processing; S23: Based on the similarity matrix and combined with the KNN sparsification method, a sparse item-item graph of each modal feature is obtained.
[0009] Optionally, step S3 includes: S31: Learning the mean and variance of each node in the item-item graph through the variational graph autoencoder VGAE: The node feature representation and graph structure of the item-item graph are encoded as distribution parameters in the latent space. The distribution parameters are the mean and variance as follows: Introducing a latent variable , used to describe the potential feature representation of the node, Representation node latent feature representation of ; Representation node latent feature representation of ; Update the latent feature representation:
[0010]
[0011]
[0012]
[0013] in represents the updated latent feature representation in the modality; Indicates that a multilayer perceptron is used to update the latent feature representation; It is a learnable parameter used to distinguish the central node from the neighboring nodes; represents the initial latent feature representation in the modality; Representation node The set of neighbor nodes of Indicates passing Model calculation ; represents the mean of the latent trait representation; Indicates use Model calculation ; represents the logarithm of the variance of the latent feature representation; is the sigmoid activation function, is the weight matrix, is the bias vector; S32: Representing latent features Use matrix T to construct the probability distribution of the potential feature representation of each node:
[0014] in is the mean of the latent feature representation; is the variance of the latent feature representation, is the normalized similarity matrix; The mean of the node potential feature representation of a certain modality; represents the probability density function of the Gaussian distribution; The function is used to extract the diagonal elements of a matrix; The variance of the node potential feature representation representing a certain modality; For each pair of nodes , by calculating the Euclidean distance between the potential feature representations of two nodes, the probability of the edge existing is obtained ,as follows:
[0015] in yes The elements in is the sigmoid function; S33: Using KL divergence loss function, combined with probability distribution and the probability , adjust the reconstruction error of the latent variable Z and the item-item graph; S34: Perform multi-layer graph convolution operations on the reconstructed item-item graph, combine it with the user-item interaction matrix, obtain the high-order representation of each node, and obtain the item modal features; S35: Obtaining user modal features by aggregating item modal features; concatenating the user modal features and item modal features to obtain a user-item modal feature representation; S36: By performing multi-layer graph convolution operations on the user-item interaction graph, high-order collaborative signals are captured to obtain the representation of the message propagation stage in each graph convolution layer; the representation of the message propagation stage in each graph convolution layer is aggregated to obtain the user-clothing item feature representation.
[0016] Optionally, step S33 includes:
[0017] in represents the KL divergence loss function; represents the expected value of the approximate posterior distribution; [ ] is used to measure the difference between the approximate posterior distribution and the prior distribution; represents the probability of generating a matrix given the latent variable Z, Indicates the number of nodes; Indicates that given parameters The approximate posterior distribution of the latent variable Z under ; Represents latent variables The prior distribution of .
[0018] Optionally, step S4 includes: The input of the Transformer layer of the graph Transformer module is represented as , central node is expressed as , Indicates the nodes; The representations of the neighbor nodes most relevant to the central node are stacked into a matrix ,in is the number of samples; Using three matrices Project the node feature representation of the user-item interaction graph to Q, K, and V respectively. According to the attention distribution, the aggregated information is:
[0019]
[0020]
[0021] in Represents the global feature representation of users and items, is the normalization function; represents the normalized exponential function, Use multi-head attention mechanism: The number of heads divided into multiple , each head uses the corresponding weight matrix , represents the concatenation function, represents the projection matrix; Represents the dimension of the vector; is the normalized exponential function.
[0022] Optionally, step S5 includes: Extracting modality-specific features from the user-item modality feature representation, where the modality-specific features include textual modality-specific features and visual modality-specific features; Compute attention scores for textual modality-specific features and visual modality-specific features; By adaptively fusing text modality-specific features and visual modality-specific features and combining them with attention scores, we obtain modality-specific features as follows:
[0023]
[0024] in is a modal-specific feature, is the user-item modality feature representation; is the weight matrix, is the bias vector, is the activation function, is the attention score.
[0025] An electronic device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to enable the electronic device to perform a multimodal clothing recommendation method based on graph reconstruction representation learning.
[0026] A computer-readable storage medium stores instructions. When the instructions are executed, a multimodal clothing recommendation method based on graph reconstruction representation learning is performed.
[0027] The beneficial effects of the technical solution provided by this application are: The project-project graph is reconstructed using a variational graph autoencoder (VGAE) to denoise the image and text information of the project, obtaining a user-project modal feature representation. The modality-specific features of the user-project modal feature representation are further extracted, allowing information from different modalities to complement each other and provide richer user preference features. A graph transformer module is introduced to capture global structural dependencies from different project modal feature spaces, learn global feature representations of users and projects, and combine the user-clothing project feature representation of the user-project interaction graph with the modality-specific features and global feature representation to obtain the final feature representation. Based on the final feature representation, the user-project interaction score is calculated to obtain candidate recommendation results for clothing projects. The technical solution of this application can fully capture user interests, improve recommendation accuracy, and improve user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] The present application will be further described below with reference to the accompanying drawings and embodiments, in which: Figure 11 is a step diagram of a multimodal clothing recommendation method based on graph reconstruction representation learning in an embodiment of the present application; Figure 2 is a model framework diagram of a multimodal clothing recommendation method based on graph reconstruction representation learning in an embodiment of the present application; Figure 3 Schematic diagram of the electronic device structure of the multimodal clothing recommendation method based on graph reconstruction representation learning in an embodiment of the present application. DETAILED DESCRIPTION
[0029] In order to have a clearer understanding of the technical features, purposes and effects of this application, the specific implementation methods of this application are now described in detail with reference to the accompanying drawings.
[0030] The embodiments of the present application provide a multimodal clothing recommendation method based on graph reconstruction representation learning.
[0031] Please refer to Figure 1 , Figure 1 This is a step diagram of a multimodal clothing recommendation method based on graph reconstruction representation learning in an embodiment of the present application, including: S1: Obtain text information and image information of clothing items, as well as user-clothing item interaction information, and construct a user-item interaction graph and a user-item interaction matrix; Step S1 includes: Set user set , clothing item collection ; Construct a node set through text information and image information ; Build a user-item interaction edge set through interaction information represents the relationship between users and clothing items, where Represents a user and clothing items There is an edge between them, otherwise ; Using Node Collections and user-item interaction edge sets , build a user-item interaction graph ; Build Represents the user-item interaction matrix.
[0032] Specifically, the interaction information describes the interaction between users and items, including the user's rating, clicking, and favorite behaviors of the items. The text information and image information are the item's attribute description, picture, title, comment, etc. User-item interaction diagram, such as Figure 2 shown.
[0033] S2: Obtain multimodal features based on text information and image information; process the multimodal features and combine them with the KNN sparsification method to obtain an item-item graph for each modal feature; Step S2 includes: S21: Use the pre-trained deep learning model VGG to extract visual modal features of image information; use the pre-trained deep learning model Word2vec to extract text modal features of text information; set up represents a modal set, Indicates the visual modality, Indicates text mode; The modal characteristics are expressed as ,in Represents the dimension of the feature matrix, wherein the modal features include: text modal features and visual modal features; S22: Calculate the cosine similarity between all clothing items corresponding to each modal feature, obtain a similarity matrix and perform normalization processing; S23: Based on the similarity matrix and combined with the KNN sparsification method, a sparse item-item graph of each modal feature is obtained.
[0034] Specifically, the sparse item-item graph retains the edges between each node and its K nearest neighbor nodes and removes the rest of the edges.
[0035] S3: Use the variational graph autoencoder (VGAE) to reconstruct the item-item graph and combine it with the user-item interaction matrix to obtain the user-item modal feature representation; perform graph convolution on the user-item interaction graph to obtain the user-clothing item feature representation; Step S3 includes: S31: Learning the mean and variance of each node in the item-item graph through the variational graph autoencoder VGAE: The node feature representation and graph structure of the item-item graph are encoded as distribution parameters in the latent space. The distribution parameters are the mean and variance as follows: Introducing a latent variable , used to describe the potential feature representation of the node, Representation node latent feature representation of ; Representation node latent feature representation of ; Update the latent feature representation:
[0036]
[0037]
[0038]
[0039] in represents the updated latent feature representation in the modality; Indicates that a multilayer perceptron is used to update the latent feature representation; It is a learnable parameter used to distinguish the central node from the neighboring nodes; represents the initial latent feature representation in the modality; Representation node The set of neighbor nodes of Indicates passing Model calculation ; represents the mean of the latent trait representation; Indicates use Model calculation ; represents the logarithm of the variance of the latent feature representation; is the sigmoid activation function, is the weight matrix, is the bias vector; Specifically, the GIN (Graph Isomorphism Network) model is a type of neural network based on graph isomorphism. It uses a variational graph autoencoder (VGAE) to reconstruct the item-item graph for each modality, learn latent representations for nodes, and remove noise from each modality. Reconstructing the item-item graph for each modality through the encoder effectively mitigates noise interference across modalities and improves the model's focus on important features. In the visual modality, the encoder can learn feature reconstruction to mitigate issues such as overexposure and color shift, allowing it to focus more on key clothing style and color features within the image. In the textual modality, the encoder can reduce the influence of irrelevant information (such as background descriptions or irrelevant advertising slogans), allowing the model to focus more on important descriptive information related to the clothing. This approach results in a distribution of clothing items in the latent space that is closer to reality and less affected by noise, thereby improving the accuracy and robustness of the recommendation system. This reconstruction approach ensures more representative information from each modality, providing a more reliable foundation for subsequent multimodal fusion.
[0040] S32: Representing latent features Use matrix T to construct the probability distribution of the potential feature representation of each node:
[0041] in is the mean of the latent feature representation; is the variance of the latent feature representation, is the normalized similarity matrix; The mean of the node potential feature representation of a certain modality; represents the probability density function of the Gaussian distribution; The function is used to extract the diagonal elements of a matrix; The variance of the node potential feature representation representing a certain modality; For each pair of nodes , by calculating the Euclidean distance between the potential feature representations of two nodes, the probability of the edge existing is obtained ,as follows:
[0042] in yes The elements in is the sigmoid function; S33: Using KL divergence loss function, combined with probability distribution and the probability , adjust the reconstruction error of the latent variable Z and the item-item graph; Step S33 includes:
[0043] in represents the KL divergence loss function; represents the expected value of the approximate posterior distribution; [ ] is used to measure the difference between the approximate posterior distribution and the prior distribution; represents the probability of generating a matrix given the latent variable Z, Indicates the number of nodes; Indicates that given parameters The approximate posterior distribution of the latent variable Z under ; Represents latent variables The prior distribution of .
[0044] S34: Perform multi-layer graph convolution operations on the reconstructed item-item graph, combine it with the user-item interaction matrix, obtain the high-order representation of each node, and obtain the item modal features; S35: Obtaining user modal features by aggregating item modal features; concatenating the user modal features and item modal features to obtain a user-item modal feature representation; S36: By performing multi-layer graph convolution operations on the user-item interaction graph, high-order collaborative signals are captured to obtain the representation of the message propagation stage in each graph convolution layer; the representation of the message propagation stage in each graph convolution layer is aggregated to obtain the user-clothing item feature representation.
[0045] S4: Through the graph transformer module, the global structural dependencies of the feature space of different clothing items are captured to obtain the global feature representation of users and items; Step S4 includes: The input of the Transformer layer of the graph Transformer module is represented as , central node is expressed as , Indicates the nodes; The representations of the neighbor nodes most relevant to the central node are stacked into a matrix ,in is the number of samples; Using three matrices Project the node feature representation of the user-item interaction graph to Q, K, and V respectively. According to the attention distribution, the aggregated information is:
[0046]
[0047]
[0048] in Represents the global feature representation of users and items, is the normalization function; represents the normalized exponential function, Use multi-head attention mechanism: The number of heads divided into multiple , each head uses the corresponding weight matrix , represents the concatenation function, represents the projection matrix; Represents the dimension of the vector; is the normalized exponential function.
[0049] S5: Extract modality-specific features from the user-item modal feature representation, and obtain the final feature representation by fusing the user-clothing item feature representation with the modality-specific features and the global feature representation. Based on the final feature representation, calculate the user-item interaction score and obtain candidate recommendation results for clothing items.
[0050] Step S5 includes: Extracting modality-specific features from the user-item modality feature representation, where the modality-specific features include textual modality-specific features and visual modality-specific features; Compute attention scores for textual modality-specific features and visual modality-specific features; By adaptively fusing text modality-specific features and visual modality-specific features and combining them with attention scores, we obtain modality-specific features as follows:
[0051]
[0052] in is a modal-specific feature, is the user-item modality feature representation; is the weight matrix, is the bias vector, is the activation function, is the attention score.
[0053] Specifically, the attention score is used to determine the user's preference for the unique features of each clothing item's textual modality, as well as the user's preference for the unique features of each clothing item's visual modality. Each preference is then refined and weighted accordingly, explicitly indicating the user's preference for the different modalities of the clothing item. The model can dynamically adjust the weights of each modality in the recommendation system based on the user's preference for each modality, thereby highlighting the modality with the strongest user preference when making recommendations. This dynamic adjustment can improve the accuracy and personalization of recommendations. The choices that users may make based on their preferences for different modalities further dynamically demonstrate their preferences for different modalities.
[0054] Specifically, by adaptively fusing features specific to the textual modality and features specific to the visual modality, we can more accurately predict user preferences for items. There are inherent differences between visual modality-specific features and textual modality-specific features. For example, an image may display the style and color of a garment, while text may describe the material or style. Neither image nor text alone can provide users with complete information about a clothing item. Extracting modality-specific information can better capture user preferences, i.e., their preference for a particular clothing item. This difference can lead users to have different preferences (degrees of preference) for the same item, resulting in users prioritizing different modalities. For example, a user may like the style of a garment shown in an image, but not choose it based on the material described in the text.
[0055] Specifically, modality-specific features refer to features that are unique to a certain modality and can represent the information of that modality. For example, in the visual modality, the unique features may include the color, style, texture, etc. in the image; while in the textual modality, the unique features may include the material, style, design details, etc. of the description. The purpose of extracting modality-specific features is to better understand the user's preferences for different modalities, and combine these features to accurately predict the user's overall preference for a certain clothing item. For example: Visual modality: An image showing a red dress may make the user feel good about the style and color of the dress. These are the unique features of the visual modality. Textual modality: The description of the dress as being made of cotton and suitable for summer wear is the unique features of the textual modality.
[0056] By extracting unique features from the text modality, we can further refine user preferences and help the system understand the specific reasons why users dislike an item. This allows the system to not only identify a user's dislike of an item due to the text modality but also gain a deeper understanding of the specific features that led to this dislike. For example, a user may see an item of clothing with an attractive image, but the text description mentions that the item is made of polyester. The user may dislike the feel or breathability of polyester, thus developing a negative preference for the item. By extracting the unique feature of material information from the text modality, the system can accurately identify the specific reason for the user's dislike as the material itself, rather than other features in the text description. This refinement process helps the recommendation system more accurately capture the user's true preferences, making recommendations more personalized and precise. The system can also use this information to prioritize clothing made of materials that the user dislikes in future recommendations, further improving user satisfaction.
[0057] This application also discloses an electronic device. Figure 3 , Figure 3 Schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. The electronic device 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.
[0058] The communication bus 502 is used to implement the connection and communication between these components.
[0059] The user interface 503 may include a display screen and a camera. Optionally, the user interface 503 may also include a standard wired interface and a wireless interface.
[0060] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).
[0061] The present application also discloses a computer-readable storage medium storing a plurality of instructions suitable for loading by a processor to execute the above-mentioned multimodal clothing recommendation method based on graph reconstruction representation learning.
[0062] The above are merely exemplary embodiments of the present disclosure and are not intended to limit the scope of the present disclosure. In other words, any equivalent variations and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the disclosure and the practical implications thereof.
[0063] This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not described herein. The description and examples are to be considered as exemplary only, and the scope and spirit of the present disclosure are to be defined by the claims.
Claims
1. A multimodal clothing recommendation method based on graph reconstruction representation learning, characterized by: The method comprises the following steps: S1: Obtain text information and image information of clothing items, as well as user-clothing item interaction information, and construct a user-item interaction graph and a user-item interaction matrix; S2: Obtain multimodal features based on text information and image information; process the multimodal features and combine them with the KNN sparsification method to obtain an item-item graph for each modal feature; S3: Use the variational graph autoencoder (VGAE) to reconstruct the item-item graph and combine it with the user-item interaction matrix to obtain the user-item modal feature representation; perform graph convolution on the user-item interaction graph to obtain the user-clothing item feature representation; S4: Through the graph transformer module, the global structural dependencies of the feature space of different clothing items are captured to obtain the global feature representation of users and items; S5: extracting modality-specific features from the user-item modality feature representation, and fusing the user-clothing item feature representation with the modality-specific features and the global feature representation to obtain the final feature representation; Based on the final feature representation, the user-item interaction score is calculated to obtain the candidate recommendation results for clothing items.
2. A multimodal clothing recommendation method based on graph reconstruction representation learning according to claim 1, characterized in that: Step S1 includes: Set user set , clothing item collection ; Construct a node set through text information and image information ; Build a user-item interaction edge set through interaction information represents the relationship between users and clothing items, where Represents a user and clothing items There is an edge between them, otherwise ; Using Node Collections and user-item interaction edge sets , build a user-item interaction graph ; Build Represents the user-item interaction matrix.
3. The multimodal clothing recommendation method based on graph reconstruction representation learning according to claim 1, characterized in that: Step S2 includes: S21: Use the pre-trained deep learning model VGG to extract visual modal features of image information; use the pre-trained deep learning model Word2vec to extract text modal features of text information; set up represents a modal set, Indicates the visual modality, Indicates text mode; The modal characteristics are expressed as ,in Represents the dimension of the feature matrix, wherein the modal features include: text modal features and visual modal features; S22: Calculate the cosine similarity between all clothing items corresponding to each modal feature, obtain a similarity matrix and perform normalization processing; S23: Based on the similarity matrix and combined with the KNN sparsification method, a sparse item-item graph of each modal feature is obtained.
4. The multimodal clothing recommendation method based on graph reconstruction representation learning according to claim 1, characterized in that: Step S3 includes: S31: Learning the mean and variance of each node in the item-item graph through the variational graph autoencoder VGAE: The node feature representation and graph structure of the item-item graph are encoded as distribution parameters in the latent space. The distribution parameters are the mean and variance as follows: Introducing a latent variable , used to describe the potential feature representation of the node, Representation node latent feature representation of ; Representation node latent feature representation of ; Update the latent feature representation: in represents the updated latent feature representation in the modality; Indicates that a multilayer perceptron is used to update the latent feature representation; It is a learnable parameter used to distinguish the central node from the neighboring nodes; represents the initial latent feature representation in the modality; Representation node The set of neighbor nodes of Indicates passing Model calculation ; represents the mean of the latent trait representation; Indicates use Model calculation ; represents the logarithm of the variance of the latent feature representation; is the sigmoid activation function, is the weight matrix, is the bias vector; S32: Representing latent features Use matrix T to construct the probability distribution of the potential feature representation of each node: in is the mean of the latent feature representation; is the variance of the latent feature representation, is the normalized similarity matrix; The mean of the node potential feature representation of a certain modality; represents the probability density function of the Gaussian distribution; The function is used to extract the diagonal elements of a matrix; The variance of the node potential feature representation representing a certain modality; For each pair of nodes , by calculating the Euclidean distance between the potential feature representations of two nodes, the probability of the edge existing is obtained ,as follows: in yes The elements in is the sigmoid function; S33: Using KL divergence loss function, combined with probability distribution and the probability , adjust the reconstruction error of the latent variable Z and the item-item graph; S34: Perform multi-layer graph convolution operations on the reconstructed item-item graph, combine it with the user-item interaction matrix, obtain the high-order representation of each node, and obtain the item modal features; S35: Obtaining user modal features by aggregating item modal features; concatenating the user modal features and item modal features to obtain a user-item modal feature representation; S36: By performing multi-layer graph convolution operations on the user-item interaction graph, high-order collaborative signals are captured to obtain the representation of the message propagation stage in each graph convolution layer; the representation of the message propagation stage in each graph convolution layer is aggregated to obtain the user-clothing item feature representation.
5. The multimodal clothing recommendation method based on graph reconstruction representation learning according to claim 4, characterized in that: Step S33 includes: in represents the KL divergence loss function; represents the expected value of the approximate posterior distribution; [ ] is used to measure the difference between the approximate posterior distribution and the prior distribution; represents the probability of generating a matrix given the latent variable Z, Indicates the number of nodes; Indicates that given parameters The approximate posterior distribution of the latent variable Z under ; Represents latent variables The prior distribution of .
6. The multimodal clothing recommendation method based on graph reconstruction representation learning according to claim 1, characterized in that: Step S4 includes: The input of the Transformer layer of the graph Transformer module is represented as , central node is expressed as , Indicates the nodes; The representations of the neighbor nodes most relevant to the central node are stacked into a matrix ,in is the number of samples; Using three matrices The node feature representation of the user-item interaction graph is projected onto Q, K, and V respectively. According to the attention distribution, the aggregated information is: in Represents the global feature representation of users and items, is the normalization function; represents the normalized exponential function, Use multi-head attention mechanism: The number of heads divided into multiple , each head uses the corresponding weight matrix , represents the concatenation function, represents the projection matrix; Represents the dimension of the vector; is the normalized exponential function.
7. The multimodal clothing recommendation method based on graph reconstruction representation learning according to claim 1, characterized in that: Step S5 includes: Extracting modality-specific features from the user-item modality feature representation, where the modality-specific features include textual modality-specific features and visual modality-specific features; Calculate attention scores for textual modality-specific features and visual modality-specific features; By adaptively fusing text modality-specific features and visual modality-specific features and combining them with attention scores, we obtain modality-specific features as follows: in is a modal-specific feature, is the user-item modality feature representation; is the weight matrix, is the bias vector, is the activation function, is the attention score.
8. An electronic device, characterized in that: The electronic device comprises a processor (501), a memory (505), a user interface (503) and a network interface (504), wherein the memory (505) is used to store instructions, the user interface (503) and the network interface (504) are used to communicate with other devices, and the processor (501) is used to execute the instructions stored in the memory (505) so that the electronic device executes the method according to any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed by a computer, the method according to any one of claims 1 to 7 is executed.