An attention mechanism-based multi-modal interpretable recommendation method

Abstract

The application discloses a kind of based on attention mechanism's multimodal explainable recommendation method, comprising:1. obtaining basic information under product recommendation scene;2. extract the semantic features of product text description and the visual features of image area;3. obtain the text, image representation after fusing user attention;4. utilize the improved gated recurrent unit (GRU) network to integrate multi-source information, predict the comment text of user and its probability;5. predict the preference degree of user to product.The application comprehensively utilizes the multimodal data of text and image, and introduces attention mechanism to identify specific visual and text elements that affect user preference, integrates and gives different attention to heterogeneous information, and then predicts the purchase behavior and comment text of user, gives efficient and explainable recommendation.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of recommendation system, and particularly relates to a multi-modal explainable recommendation method based on an attention mechanism. BACKGROUND

[0002] A recommendation system is a core technology of online user-oriented platforms such as e-commerce, social media and content sharing websites. The recommendation in such platforms usually needs to cater to the personalized needs of users, thereby increasing the possibility and satisfaction of users' purchase, and creating more revenue for the platform.

[0003] At present, a recommendation system uses multi-source heterogeneous data and deep learning and an attention mechanism to improve the accuracy and explainability of recommendation. The multi-source heterogeneous data includes a textual description and an image of a product, and a comment of a user, wherein the textual description of the product can provide detailed parameter information and product details; the image of the product can provide instant and intuitive visual information to assist the user in analyzing the applicability of the product; and the comment of the user can provide the evaluation and purchase basis of the user on the product. Although existing researches comprehensively use text and image to enhance the recommendation performance, and identify important text or image information affecting user decision through an attention mechanism, the fusion of multi-modal data and its different degrees of influence on user decision are often ignored, and it is difficult to provide a recommendation reason easy to understand. In view of the above background and technology, there is an urgent need for a recommendation method which can effectively extract and fuse multi-modal features related to a product, and fully utilize the synergistic effect of user preference and decision, and design a recommendation method with high accuracy and explainability. SUMMARY

[0004] The application is proposed to solve the above-mentioned problems in the prior art, and provides a multi-modal explainable recommendation method based on an attention mechanism, so as to comprehensively utilize multi-modal data of text and image, introduce an attention mechanism to identify specific visual and text elements affecting user preference, integrate and give different attention degrees to heterogeneous information, and then predict the purchase behavior and comment text of the user, and give an efficient and explainable recommendation.

[0005] In order to achieve the above-mentioned application purposes, the application adopts the following technical solutions.

[0006] The multi-modal explainable recommendation method based on the attention mechanism has the characteristics that the following steps are included.

[0007] Step 1, basic information in a product recommendation scene is obtained, including a textual description of a product, image information, and an interaction behavior of a user and the product; wherein a user set is denoted as , any user in is denoted as ; and a product set is denoted as , wherein Any property recorded as ;make This represents the set of user interactions with the product, where... Indicates user Interactive behavior of purchasing product i; Indicates product set The text description set of all products in the document, among which, The text description represents product i, and , This represents the c-th word in the text description of product i. Let represent the total number of words in the text description of product i. Text description of product i The j-th subword of the c-th word, ;n represents the text description of product i. The cth word The total number of subwords; let The image represents product i, where, Let h represent the k-th region of the image of product i, and h represent the total number of image regions; let Represent the characterization of user u; let This represents the characteristics of product i;

[0008] Step 2: Based on the basic information in the product recommendation scenario, use a text feature extractor to obtain the text description of product i. The cth word semantic features Use an image feature extractor to obtain the k-th region of product i. visual features ;

[0009] Step 3: Utilize the attention mechanism module to represent user u With semantic features Integration, user acquisition Text representation of product i after attention fusion The attention mechanism module is used to represent user u. With visual features To integrate and acquire users Image representation of product i after attention fusion ;

[0010] Step 4: Text Representation Image representation Interaction representation of user u and product i , obtain the Features of time step integration into user attention User generation using the improved GRU network In the prediction comments for product i, the first one word Predicted probability This serves as the basis for recommending product i to user u; among which, Indicates a linear transformation operation;

[0011] Step 5: Text Representation and image representation Predict user u's preference for product i. This is used to generate a list of products with high user preference for user u.

[0012] The multimodal interpretable recommendation method based on attention mechanism described in this invention is also characterized in that step 2 includes:

[0013] Step 2.1: Construct a text feature extractor based on the pre-trained BERT model;

[0014] Step 2.1.1: The text feature extractor uses equation (1) to calculate the text description of product i. The j-th subword of the c-th word Embedded vector ;

[0015] (1)

[0016] In equation (1), This indicates an embedded function based on Wordpiece; Represents a position-based embedding function;

[0017] Step 2.1.2: The text feature extractor utilizes the Transformer layer to... Process and obtain semantic features ;

[0018] Step 2.1.3: The text feature extractor uses equation (2) to obtain the text description of product i. Semantic features of the c-th word :

[0019] (2)

[0020] Step 2.2: Construct an image feature extractor based on the pre-trained VGGNet-19 network. The image feature extractor uses equation (3) to extract the k-th region of product i. visual features ;

[0021] (3).

[0022] Furthermore, step 3 includes:

[0023] Step 3.1: Calculate the user using equation (4). Text description of product i The cth word level of attention and use the softmax function to Normalization is performed to obtain the c-th word. Normalized attention level ;

[0024] (4)

[0025] In equation (4), and These represent the two weight parameters of the first fully connected text layer; Represents the ReLU function; This represents the weight parameters of the second fully connected text layer; Indicates the bias of the first fully connected text layer; Indicates the bias of the second fully connected text layer; T indicates transpose;

[0026] Step 3.2: Calculate the user using equation (5). Text representation of product i after attention fusion :

[0027] (5)

[0028] Step 3.3: Calculate the degree of attention user u pays to the k-th region in the product i image using equation (6). and use the softmax function to Normalization is performed to obtain the normalized attention level of the k-th region. ;

[0029] (6)

[0030] In equation (6), and These represent the two weight parameters of the first fully connected layer in the image; This represents the weight parameters of the second fully connected layer in the image; This indicates the bias of the first fully connected layer in the image; Indicates the bias of the second fully connected layer in the image;

[0031] Step 3.2.2: Calculate the user using equation (7). Image representation of product i after attention fusion :

[0032] (7).

[0033] Furthermore, step 4 includes:

[0034] Step 4.1: Define and initialize the current time step of the improved GRU network. ; Define the first The hidden state of the time step is and to Perform random initialization; define the first The comment text words predicted by time step are and to Initialize to an empty string; use the text feature extractor according to steps 2.1-2.3. Processing is performed to obtain semantic features ;

[0035] Step 4.2: The improved GRU network uses the folding stick method for text representation. Image representation Interactive representation By assigning different weights, the first... Features of time step integration into user attention and capture features and semantic features Update to a hidden state. , obtained the The hidden state of the time step is ;

[0036] Step 4.2.1: Calculate the d-th representation pair using equations (8)-(9). Features of time step integration into user attention weight Among them, text representation The weight is Image representation The weight is Interactive representation weight ,and :

[0037] (8)

[0038] (9)

[0039] In equations (8) and (9), Indicates the first The weight parameter of the d-th representation at time step, ; Represents the sigmoid function; Indicates the first Time step hidden state The weight matrix; Indicates the first The first step of time The weight parameters of each representation, ;

[0040] Step 4.2.2: Calculate the first step using equation (10). Features of time step integration into user attention :

[0041] (10)

[0042] In equation (10), RELU represents a nonlinear activation function; Indicates a linear transformation operation;

[0043] Step 4.2.3: The improved GRU network uses equations (11)-(14) to obtain the first... Hidden state of time step :

[0044] (11)

[0045] (12)

[0046] (13)

[0047] (14)

[0048] In equations (11)-(14), Indicates the first The time step update gate; Indicates the first Reset gate for time step; Indicates the first The candidate set of hidden states at each time step; tanh represents the hyperbolic tangent function; It represents the Hadamardi (or Hadama) stack; Indicates the first Comment words predicted by time step semantic features;

[0049] Step 4.3: Use equation (15) to obtain the first... Time step users Comments on product i The Middle one word Predicted probability :

[0050] (15)

[0051] In equation (15), Indicates hidden state The weight matrix; Indicates the first to the last The word sequence predicted at each time step; softmax represents the normalized exponential function.

[0052] Furthermore, step 5 includes:

[0053] Step 5.1: Obtain the final characterization of product i using equation (16). :

[0054] (16)

[0055] In equation (16), This represents the weight matrix used for dimensional transformation; Represents the parameter matrix used for dimensional transformation;

[0056] Step 5.2: Predict users using equation (17) Preference value for product i :

[0057] (17)

[0058] In equation (17), PredictionLayer represents a multilayer perceptron;

[0059] Step 5.3: Calculate the loss function using equation (18). According to the loss function Backpropagation is performed on the attention mechanism module and the improved GRU network to update the parameters, obtaining the optimal trained model, which is used to predict user preferences for products and review text.

[0060] (18)

[0061] In equation (18), This represents the set of products purchased by user u. This represents the set of products that user u has not purchased. It is a hyperparameter; It is the regularization coefficient; represents the regularization parameter; L represents the total number of layers in the improved GRU network.

[0062] The present invention provides an electronic device, including a memory and a processor, wherein the memory is used to store a program that supports the processor in executing the multimodal interpretable recommendation method, and the processor is configured to execute the program stored in the memory.

[0063] The present invention discloses a computer-readable storage medium on which a computer program is stored, wherein the computer program, when executed by a processor, performs the steps of the described multimodal interpretable recommendation method.

[0064] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0065] 1. This invention introduces multimodal data containing text and images, and uses feature extractors and attention mechanisms to identify specific visual and textual elements that influence user preferences. This helps merchants select appropriate display media for specific product information and makes it easier for users to obtain product information.

[0066] 2. This invention develops a novel gated recurrent unit (GRU) network, which introduces a folding stick method for efficient weight allocation, integrates and assigns different levels of importance to heterogeneous information, effectively predicts user review content, and provides suggestions for merchants to improve marketing strategies and update products.

[0067] 3. This invention integrates image representation and text representation to accurately predict users' preferences for products, thereby improving the efficiency of e-commerce platform recommendations and reducing recommendation costs. Attached Figure Description

[0068] Figure 1 This is a flowchart of the attention-based multimodal interpretable recommendation method of the present invention;

[0069] Figure 2 This is a flowchart of the text feature extraction process of the present invention;

[0070] Figure 3 This is a flowchart of the visual feature extraction process of the present invention.

[0071] Figure 4 The improved GRU network of the present invention Flowchart of time steps. Detailed Implementation

[0072] In this example, a multimodal interpretable recommendation method based on an attention mechanism is presented. This method is designed as an efficient and interpretable recommendation approach for multimodal data containing product text descriptions, images, and user reviews. It introduces a feature extractor and an attention mechanism to identify specific visual and textual elements that influence user preferences, thereby predicting the degree of user preference. An improved GRU network is used to integrate textual, image, and interaction representations to predict user review text. Specifically, as... Figure 1 As shown, the method is performed according to the following steps:

[0073] Step 1: Obtain basic information for the product recommendation scenario, including: the product's text description, image information, and user interaction behavior with the product; whereby the user set is denoted as... ,make Any user in the middle is recorded as The product set is denoted as ,make Any property recorded as ;make This represents the set of user interactions with the product, where... Indicates user Interactive behavior of purchasing product i; Indicates product set The text description set of all products in the document, among which, The text description represents product i, and , This represents the c-th word in the text description of product i. Let represent the total number of words in the text description of product i. Text description of product i The j-th subword of the c-th word, ;n represents the text description of product i. The cth word The total number of subwords; let The image represents product i, where, Let h represent the k-th region of the image of product i, and h represent the total number of image regions; let Represent the characterization of user u; let This represents the characteristics of product i;

[0074] Step 2: Based on the basic information in the product recommendation scenario, use a text feature extractor to obtain the text description of product i. The cth word semantic features Use an image feature extractor to obtain the k-th region of product i. visual features ;

[0075] Step 2.1: Construct a text feature extractor based on the pre-trained BERT model; BERT is a deep bidirectional encoder that uses self-supervised learning to represent text as a series of vectors, such as... Figure 2 As shown;

[0076] Step 2.1.1: The text feature extractor uses equation (1) to calculate the text description of product i. The j-th subword of the c-th word Embedded vector ;

[0077] (1)

[0078] In equation (1), This indicates an embedded function based on Wordpiece; Represents a position-based embedding function; embedding vector It is represented by a 768-dimensional vector.

[0079] Step 2.1.2: The text feature extractor utilizes the Transformer layer to... Process and obtain semantic features ;

[0080] Step 2.1.3: The text feature extractor uses equation (2) to obtain the text description of product i. Semantic features of the c-th word :

[0081] (2)

[0082] Step 2.2: Construct an image feature extractor based on the pre-trained VGGNet-19 network. The image feature extractor uses equation (3) to extract the k-th region of product i. visual features VGGNet-19 is a convolutional neural network widely used in computer vision to acquire complex image features, such as... Figure 3 As shown;

[0083] (3).

[0084] Step 3: Utilize the attention mechanism module to represent user u With semantic features Integration, user acquisition Text representation of product i after attention fusion The attention mechanism module is used to represent user u. With visual features To integrate and acquire users Image representation of product i after attention fusion ;

[0085] Step 3.1: Calculate the user using equation (4). Text description of product i The cth word level of attention and use the softmax function to Normalization is performed to obtain the c-th word. Normalized attention level ;

[0086] (4)

[0087] In equation (4), and This represents the two weight parameters of the first fully connected text layer, used to map user representations. and semantic features To the same dimension; Represents the ReLU function; This represents the weight parameters of the second fully connected text layer; Indicates the bias of the first fully connected text layer; This indicates the bias of the second fully connected text layer; T represents transpose; capturing fine-grained text preferences at the region level helps to clarify the impact of specific text in product descriptions on users' decision-making processes;

[0088] Step 3.2: Calculate the user using equation (5). Text representation of product i after attention fusion :

[0089] (5)

[0090] In equation (5), This represents the user after normalization using the softmax function. Text description of product i The cth word The level of attention;

[0091] Step 3.3: Calculate the degree of attention user u pays to the k-th region in the product i image using equation (6). and use the softmax function to Normalization is performed to obtain the normalized attention level of the k-th region. ;

[0092] (6)

[0093] In equation (6), and This represents the two weight parameters of the first fully connected layer, used to map user representations. and visual features To the same dimension; This represents the weight parameters of the second fully connected layer in the image; This indicates the bias of the first fully connected layer in the image; This indicates the bias of the second fully connected layer in the image; capturing fine-grained visual preferences of users at the region level helps to clarify the impact of specific regions in product images on users' decision-making process;

[0094] Step 3.2.2: Calculate the user using equation (7). Image representation of product i after attention fusion :

[0095] (7)

[0096] In equation (7), This represents the degree of attention that user u pays to the k-th region in the image of product i after normalization by the softmax function.

[0097] Step 4: Text Representation Image representation Interaction representation of user u and product i , obtain the Features of time step integration into user attention User generation using the improved GRU network In the prediction comments for product i, the first one word Predicted probability This serves as the basis for recommending product i to user u; among which, This indicates a linear transformation operation.

[0098] Step 4.1: Define and initialize the current time step of the improved GRU network. ; Define the first The hidden state of the time step is and to Perform random initialization; define the first The comment text words predicted by time step are and to Initialize to an empty string; use the text feature extractor according to steps 2.1-2.3. Processing is performed to obtain semantic features ;

[0099] Step 4.2: The improved GRU network uses the folding stick method for text representation. Image representation Interactive representation By assigning different weights, the first... Features of time step integration into user attention and capture features and semantic features Update to a hidden state. , obtained the The hidden state of the time step is ;

[0100] Step 4.2.1: Calculate the d-th representation pair using equations (8)-(9). Features of time step integration into user attention weight Among them, text representation The weight is Image representation The weight is Interactive representation weight ,and The weighting calculation employs the stick-breaking method, decomposing the generation process of comment terms into a weighted average of these three types of representations. This process begins with a unit-length "stick," representing the probability of allocation among different representations. In each step, a portion of the "stick" is assigned to a specific representation, while the remainder proceeds to the next step. Mathematically, this process can be represented as:

[0101] (8)

[0102] (9)

[0103] In equations (8) and (9), Indicates the first The weight parameter of the d-th representation at time step, ; Represents the sigmoid function; Indicates the first Time step hidden state The weight matrix; Indicates the first The first step of time The weight parameters of each representation, .

[0104] Step 4.2.2: Calculate the first step using equation (10). Features of time step integration into user attention :

[0105] (10)

[0106] In equation (10), RELU represents a nonlinear activation function; Indicates a linear transformation operation;

[0107] Step 4.2.3: The improved GRU network uses equations (11)-(14) to obtain the first... Hidden state of time step The improved GRU network not only relies on historical hidden states. and semantic features It also introduced features ,like Figure 4 As shown. This design enables the GRU's gating mechanism to dynamically adjust the contribution of multimodal information to the current word generation, enhancing the model's ability to model complex user behaviors. Mathematically, this process can be represented as:

[0108] (11)

[0109] (12)

[0110] (13)

[0111] (14)

[0112] In equations (11)-(14), Indicates the first Update gates at time steps help capture long-term dependencies in a sequence; Indicates the first Resetting gates for time steps can help capture short-term dependencies in a sequence; Indicates the first The candidate set of hidden states at each time step; tanh represents the hyperbolic tangent function; It represents the Hadamardi (or Hadama) stack; Indicates the first Comment words predicted by time step Semantic features.

[0113] Step 4.3: Use equation (15) to obtain the first... Time step users Comments on product i The Middle one word Predicted probability :

[0114] (15)

[0115] In equation (15), Indicates hidden state The weight matrix; Indicates the first to the last The word sequence predicted at each time step; softmax represents the normalized exponential function.

[0116] Step 5: Text Representation and image representation Predict user u's preference for product i. This is used to generate a list of products with high user preference for user u.

[0117] Step 5.1: Obtain the final characterization of product i using equation (16). :

[0118] (16)

[0119] In equation (16), This represents the weight matrix used for dimensional transformation; Represents the parameter matrix used for dimensional transformation; Indicates user Image representation of product i after attention fusion; Indicates user Text representation of product i after attention fusion;

[0120] Step 5.2: Predict users using equation (17) Preference value for product i :

[0121] (17)

[0122] In equation (17), PredictionLayer represents a multilayer perceptron;

[0123] Step 5.3: Calculate the loss function using equation (18). According to the loss function Backpropagation is performed on the attention mechanism module and the improved GRU network to update the parameters. , , , , , , , , , , , The optimal trained model is then used to predict user preferences for products and their comment text.

[0124] (18)

[0125] In equation (18), This represents the set of products purchased by user u. This represents the set of products that user u has not purchased. It is a hyperparameter used in modeling to balance user purchase feedback and user review information; It is the regularization coefficient, a hyperparameter used to control the complexity of the model; represents the regularization parameter; L represents the total number of layers in the improved GRU network.

[0126] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the above-described method, and the processor is configured to execute the program stored in the memory.

[0127] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above method.

[0128] In summary, this invention comprehensively utilizes multimodal data of text and images, and introduces an attention mechanism to identify specific visual and textual elements that influence user preferences. It integrates and assigns different levels of importance to heterogeneous information, thereby predicting user purchasing behavior and review text, and providing efficient and interpretable recommendations.

Claims

1. A multimodal interpretable recommendation method based on an attention mechanism, characterized in that, Includes the following steps: Step 1: Obtain basic information for the product recommendation scenario, including: the product's text description, image information, and user interaction behavior with the product; whereby the user set is denoted as... ,make Any user in the middle is recorded as The product set is denoted as ,make Any property recorded as ;make This represents the set of user interactions with the product, where... Indicates user Interactive behavior of purchasing product i; Indicates product set The text description set of all products in the document, among which, The text description represents product i, and , This represents the c-th word in the text description of product i. Let represent the total number of words in the text description of product i. Text description of product i The j-th subword of the c-th word, ;n represents the text description of product i. The cth word The total number of subwords; let The image represents product i, where, Let h represent the k-th region of the image of product i, and h represent the total number of image regions; let Represent the characterization of user u; let This represents the characteristics of product i; Step 2: Based on the basic information in the product recommendation scenario, use a text feature extractor to obtain the text description of product i. The cth word semantic features Use an image feature extractor to obtain the k-th region of product i. visual features ; Step 3: Utilize the attention mechanism module to represent user u With semantic features Integration, user acquisition Text representation of product i after attention fusion The attention mechanism module is used to represent user u. With visual features To integrate and acquire users Image representation of product i after attention fusion ; Step 4: Text Representation Image representation Interaction representation of user u and product i , obtain the Features of time step integration into user attention User generation using a GRU network improved by the folding stick method In the prediction comments for product i, the first one word Predicted probability This serves as the basis for recommending product i to user u; among which, Indicates a linear transformation operation; Indicates the first to the last Word sequences predicted by time steps; Step 5: Text Representation and image representation Predict user u's preference for product i. This is used to generate a list of products with high user preference for user u.

2. The multimodal interpretable recommendation method based on attention mechanism according to claim 1, characterized in that, Step 2 includes: Step 2.1: Construct a text feature extractor based on the pre-trained BERT model; Step 2.1.1: The text feature extractor uses equation (1) to calculate the text description of product i. The j-th subword of the c-th word Embedded vector ; (1) In equation (1), This indicates an embedded function based on Wordpiece; Represents a position-based embedding function; Step 2.1.2: The text feature extractor utilizes the Transformer layer to... Process and obtain semantic features ; Step 2.1.3: The text feature extractor uses equation (2) to obtain the text description of product i. Semantic features of the c-th word : (2) Step 2.2: Construct an image feature extractor based on the pre-trained VGGNet-19 network. The image feature extractor uses equation (3) to extract the k-th region of product i. visual features ; (3)。 3. The multimodal interpretable recommendation method based on attention mechanism according to claim 2, characterized in that, Step 3 includes: Step 3.1: Calculate the user using equation (4). Text description of product i The cth word level of attention and use the softmax function to Normalization is performed to obtain the c-th word. Normalized attention level ; (4) In equation (4), and These represent the two weight parameters of the first fully connected text layer; Represents the ReLU function; This represents the weight parameters of the second fully connected text layer; Indicates the bias of the first fully connected text layer; Indicates the bias of the second fully connected text layer; T indicates transpose; Step 3.2: Calculate the user using equation (5). Text representation of product i after attention fusion : (5) Step 3.3: Calculate the degree of attention user u pays to the k-th region in the product i image using equation (6). and use the softmax function to Normalization is performed to obtain the normalized attention level of the k-th region. ; (6) In equation (6), and These represent the two weight parameters of the first fully connected layer in the image; This represents the weight parameters of the second fully connected layer in the image; This indicates the bias of the first fully connected layer in the image; Indicates the bias of the second fully connected layer in the image; Step 3.2.2: Calculate the user using equation (7). Image representation of product i after attention fusion : (7)。 4. The multimodal interpretable recommendation method based on attention mechanism according to claim 2, characterized in that, Step 4 includes: Step 4.1: Define and initialize the current time step of the improved GRU network. ; Define the first The hidden state of the time step is and to Perform random initialization; define the first The comment text words predicted by time step are and to Initialize to an empty string; use the text feature extractor according to steps 2.1-2.

3. Processing is performed to obtain semantic features ; Step 4.2: The improved GRU network uses the folding stick method for text representation. Image representation Interactive representation By assigning different weights, the first... Features of time step integration into user attention and capture features and semantic features Update to a hidden state. , obtained the The hidden state of the time step is ; Step 4.2.1: Calculate the d-th representation pair using equations (8)-(9). Features of time step integration into user attention weight Among them, text representation The weight is Image representation The weight is Interactive representation weight ,and : (8) (9) In equations (8) and (9), Indicates the first The weight parameter of the d-th representation at time step, ; Represents the sigmoid function; Indicates the first Hidden state in time step The weight matrix; Indicates the first The first step of time The weight parameters of each representation, ; Step 4.2.2: Calculate the first step using equation (10). Features of time step integration into user attention : (10) In equation (10), ReLU represents a nonlinear activation function; Indicates a linear transformation operation; Step 4.2.3: The improved GRU network uses equations (11)-(14) to obtain the first... Hidden state of time step : (11) (12) (13) (14) In equations (11)-(14), Indicates the first Time step update gate; Indicates the first Reset gate for time step; Indicates the first The candidate set of hidden states at each time step; tanh represents the hyperbolic tangent function; It represents the Hadamardi (or Hadama) stack; Indicates the first Comment words predicted by time step semantic features; Step 4.3: Use equation (15) to obtain the first... Time step users Comments on product i The Middle one word Predicted probability : (15) In equation (15), Indicates hidden state The weight matrix; softmax represents the normalized exponential function.

5. The multimodal interpretable recommendation method based on attention mechanism according to claim 3, characterized in that, Step 5 includes: Step 5.1: Obtain the final characterization of product i using equation (16). : (16) In equation (16), This represents the weight matrix used for dimensional transformation; Represents the parameter matrix used for dimensional transformation; Step 5.2: Predict users using equation (17) Preference value for product i : (17) In equation (17), PredictionLayer represents a multilayer perceptron; Step 5.3: Calculate the loss function using equation (18). According to the loss function Backpropagation is performed on the attention mechanism module and the improved GRU network to update the parameters, obtaining the optimal trained model, which is used to predict user preferences for products and review text. (18) In equation (18), This represents the set of products purchased by user u. This represents the set of products that user u has not purchased. It is a hyperparameter; It is the regularization coefficient; represents the regularization parameter; L represents the total number of layers in the improved GRU network.

6. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports the processor in executing the multimodal interpretable recommendation method according to any one of claims 1-5, the processor being configured to execute the program stored in the memory.

7. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program, when executed by a processor, performs the steps of the multimodal interpretable recommendation method according to any one of claims 1-5.

Images