Method and system for predicting the usefulness of multimodal user reviews of intelligent connected vehicles
Through the neighbor-aware multimodal review usefulness prediction method, combined with the similarity-difference multi-view decomposition attention mechanism and dual-space contrastive learning strategy, the problem of the inability to accurately predict the usefulness of smart connected car reviews in existing technologies is solved, the prediction accuracy and feature consistency are improved, and consumers are helped to understand vehicle performance and user experience.
Patent Information
- Application Number
- CN202410744223.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-11
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2044-06-11
AI Technical Summary
Existing technologies only consider the similarity between multimodal information and cannot accurately predict the usefulness of smart connected car reviews.
A neighbor-aware multimodal review helpfulness prediction method is adopted. Through the similarity-difference multi-view decomposition attention mechanism and dual-space contrastive learning strategy, the review text and image features are combined, and the neighbor review information is introduced to build a model to predict the usefulness of the review.
The accuracy of review helpfulness prediction is improved, the complementarity and conflict between multiple views are considered, the consistent representation of features is enhanced, the inconsistency between views is reduced, and consumers are helped to better understand vehicle performance and user experience.
Smart Images

Figure CN118733988B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of review usefulness prediction, and in particular to a method, system, storage medium and electronic device for predicting the usefulness of multimodal reviews of users of intelligent connected vehicles. Background Art
[0002] The development of connected vehicles (ICVs) has provided consumers with more choices and convenience, making online car ordering a popular method of purchasing. Consumers can order cars directly through online platforms, avoiding the cumbersome and time-consuming process of traditional car purchases. However, relying solely on online ordering information may not fully understand a vehicle's performance, quality, and user experience. Online word-of-mouth (WOM), such as product reviews, has become an important channel for consumers to obtain authentic feedback on ICVs. Consumers can learn about vehicle performance, handling, safety, and reliability from the experiences of other car owners or professional reviewers. However, with the rapid increase in the number of reviews, filtering out relevant ones from the vast number of reviews has become a challenge.
[0003] In related technologies, the paper (Ren G, Diao L, Guo F, Hong TA co-attention based multi-modal fusion network for review helpfulness prediction [J]. Information Processing & Management, 2024, 61(1): 103573.) considers the deep features of text and images in multimodal reviews and incorporates manual features of text and images for helpfulness prediction. First, BERT and VGG are used to obtain the deep features of text and images. Then, the intra-modal and inter-modal interactions are considered. The self-attention mechanism is used to perform weighted summation of the deep features and manual features of text and images respectively to consider the intra-modal correlation between the two. A joint attention mechanism is designed to explore the correlation between text and image modalities. Finally, MFB is used to fuse multimodal information for helpfulness prediction.
[0004] However, the above studies only considered the similarity between multimodal information and could not accurately predict the usefulness of reviews. Summary of the Invention
[0005] (1) Technical problems solved
[0006] In response to the shortcomings of the existing technology, the present invention provides a method, system, storage medium and electronic device for predicting the usefulness of multimodal reviews of users of intelligent connected vehicles, which solves the technical problem that only considering the similarity between multimodal information cannot accurately predict the usefulness of reviews.
[0007] (2) Technical solution
[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:
[0009] A method for predicting the usefulness of multimodal user reviews of intelligent connected vehicles, based on a preset model, includes:
[0010] Obtain multiple historical multimodal review data and summary statistical information; any review data includes a review text and multiple associated review images, usefulness score labels, and numerical review information;
[0011] For the comment data at the current location, extract the text embedding representation of the comment text and the image embedding representation of all associated comment images, introduce a similarity-difference multi-view decomposition attention mechanism, and obtain a multimodal comment feature representation. Combine the multimodal comment feature representation with the features of the numerical comment information to obtain the comment feature representation of the comment data.
[0012] Obtaining a first initial belief from the aggregated statistical information; selecting a preset number of neighbor review data relative to the current location, fusing the review feature representations of all neighbor review data to obtain a second initial belief from the neighbors; adaptively fusing the first initial belief and the second initial belief to obtain a neighbor review feature representation;
[0013] Based on the comment feature representation and neighbor comment feature representation of the comment data, the similarity-difference multi-view decomposition attention mechanism is introduced again to obtain the usefulness score prediction result of the comment data;
[0014] Based on the usefulness score label and usefulness score prediction results of each review data, a main task loss is constructed; based on a dual-space contrastive learning strategy acting on the similarity subspace and the difference subspace, an auxiliary loss is constructed; the joint loss function of the main task loss and the auxiliary loss is minimized, and the model is trained until convergence;
[0015] The multimodal review data to be predicted is used as the input of the converged model to obtain the corresponding usefulness score.
[0016] Preferably, the method of introducing a similarity-difference multi-view decomposition attention mechanism to obtain a multimodal review feature representation includes:
[0017] For text embedding representation K t and image embedding representation K i Perform L2 normalization respectively and scale them to unit length to obtain the normalized features
[0018] Use dot product and Euclidean distance to calculate features respectively The similarity and difference scores between them are scaled to the range of [0,1] and normalized to obtain the similarity weight matrix S and the difference weight matrix D;
[0019] Embed the text into K t and image embedding representation K i Multiply the two matrices S and D to obtain weighted text-image similarity feature representations. Text-image difference feature representation Image-text similarity feature representation Image-text difference feature representation
[0020] The above four feature representations are weighted concatenated and the concatenation result is combined with the text embedding representation K t and image embedding representation K i Secondary concatenation and mapping through linear layers to obtain multimodal review feature representation K t_i .
[0021] Preferably, the method further introduces the similarity-difference multi-view decomposition attention mechanism based on the comment feature representation and the neighbor comment feature representation of the comment data to obtain the usefulness score prediction result of the comment data; including:
[0022] Based on the similarity-difference multi-view decomposition attention mechanism, the comment feature representation E of the comment data is fused. r ∈R d×1 , Neighborhood Comment Feature Representation E b , to obtain the weighted current review-neighborhood review similarity feature representation Feature representation of the difference between current review and neighbor review Neighbor review-current review similarity feature representation Neighbor comment-current comment difference feature representation
[0023] The above four feature representations are weighted concatenated and the concatenated result is combined with the comment feature representation E of the comment data. r ∈R d×1 , Neighborhood Comment Feature Representation E b Secondary splicing, and mapping through the linear layer to obtain the fusion view feature representation K r_b ;
[0024] The fusion view feature K r_b The usefulness score preset result s of the review data is obtained through multi-layer perceptron mapping.
[0025] Preferably, the numerical review information includes the release time and the overall star rating; then the step of combining the multimodal review feature representation with the features of the numerical review information to obtain the review feature representation of the review data includes:
[0026] The low-dimensional features of release time and overall star rating are mapped into high-dimensional features through a multi-layer perceptron, and the high-dimensional features are combined with the multimodal review features K t_i Splice and obtain the comment feature representation E of the comment data through linear layer mapping r ∈R d×1 , where d is the dimension of the vector.
[0027] Preferably, the summary statistical information includes average star rating, star rating distribution, and total number of reviews; and obtaining the first initial belief from the summary statistical information includes:
[0028] Calculate the standard deviation of star rating by using the average star rating and star rating distribution;
[0029] Combine the average star rating and standard deviation and get K through the embedding layer 0 ∈R 2×d ;
[0030] K 0 Transform, through two linear layers Tanh activation function, to obtain the first initial belief E from the summary statistics 0_sta ∈R d×1 , where d is the dimension of the vector.
[0031] Preferably, the step of selecting a preset number of neighbor review data relative to the current location and fusing the review feature representations of all neighbor review data to obtain a second initial belief from the neighbors comprises:
[0032] Select n neighbor comment data relative to the current location and obtain the comment feature representation of each neighbor comment data i∈1…n; where i is the index of the neighbor comment data;
[0033] The comment features of each neighbor's comment data are represented as nodes, and the dependency relationships between neighbor's comment data are used as edges to construct a fully connected neighbor graph;
[0034] For the neighbor graph, a new representation of each node is obtained through a two-layer GCN network, which is concatenated and linearly mapped to the first initial belief E 0_sta In the same feature space, the second initial belief E from the neighbors is obtained 0_nei ∈R d ×1 .
[0035] Preferably, the auxiliary loss is constructed based on a dual-space contrast learning strategy acting on the similarity subspace and the difference subspace; comprising:
[0036] Construct dual-space contrast loss between modalities separately and the dual-space contrast loss between reviews
[0037] The feature representation space obtained by the similarity weight matrix is named the similarity subspace The feature representation space obtained by the difference weight matrix is named the difference subspace Then the bi-space contrast loss between modalities is Expressed as:
[0038]
[0039] Among them, B represents batch size; Used to measure the similarity between A and B, the similarity function sim(A,B) = A·B, τ is the temperature hyperparameter; K i m ,K j m Representation of weighted text-image similarity feature Text-image difference feature representation Image-text similarity feature representation Image-text difference feature representation Any two feature representations in ;
[0040] Similarly, the bi-space contrast loss between reviews is Expressed as:
[0041]
[0042] Among them, K i o ,K j o Represents the weighted current review-neighborhood review similarity feature representation Feature representation of the difference between current review and neighbor review Neighbor review-current review similarity feature representation Neighbor comment-current comment difference feature representation Any two feature representations in ;
[0043] According to the neighbor review feature representation E b , we use deep learning to obtain the confidence c of neighbor comments, as shown in the following formula, and use it as a regulating factor for the auxiliary loss:
[0044] c=Conv(Pooling(MLP(E b )))
[0045] Among them, Conv represents convolution operation, Pooling represents pooling operation, and MLP represents multi-layer perceptron;
[0046] According to the dual spatial contrast loss and Combined with the confidence c of the neighbor comments, the auxiliary loss is constructed:
[0047]
[0048] Preferably, the process of obtaining the joint loss function includes:
[0049] According to the usefulness score label and usefulness score prediction results of each review data, the mean square error loss is used as the main task loss:
[0050]
[0051] Among them, s i 、y i Respectively represent the preset usefulness score result and usefulness score label of the i-th comment data; |B| represents the total number of comment data in batch B;
[0052] According to the main task loss and auxiliary losses Construct the joint loss function:
[0053]
[0054] Here, λ represents a hyperparameter used to tune the effect of auxiliary contrastive learning loss.
[0055] A system for predicting the usefulness of multimodal user reviews of intelligent connected vehicles, based on a preset model, includes:
[0056] The review data acquisition module is used to obtain multiple historical multimodal review data and summary statistical information; any review data includes a review text and multiple associated review images, usefulness score labels and numerical review information;
[0057] The review information modeling module is used to extract the text embedding representation of the review text and the image embedding representation of all associated review images for the review data at the current location, introduce a similarity-difference multi-view decomposition attention mechanism, and obtain a multimodal review feature representation; and combine the multimodal review feature representation with the features of the numerical review information to obtain the review feature representation of the review data;
[0058] A neighbor review modeling module is configured to obtain a first initial belief from the aggregated statistical information; select a preset number of neighbor review data relative to the current location, fuse the review feature representations of all neighbor review data, and obtain a second initial belief from the neighbors; and adaptively fuse the first initial belief and the second initial belief to obtain a neighbor review feature representation.
[0059] A usefulness score prediction module is configured to, based on the comment feature representation and neighbor comment feature representation of the comment data, reintroduce the similarity-difference multi-view decomposition attention mechanism to obtain a usefulness score prediction result for the comment data;
[0060] A model training module is configured to construct a primary task loss based on the usefulness score label and usefulness score prediction results of each review data; construct an auxiliary loss based on a dual-space contrastive learning strategy acting on the similarity subspace and the difference subspace; and train the model until convergence by minimizing the joint loss function of the primary task loss and the auxiliary loss.
[0061] The usefulness score acquisition module is used to take the multimodal review data to be predicted as the input of the converged model to obtain the corresponding usefulness score.
[0062] A storage medium stores a computer program for predicting the usefulness of multimodal user reviews of intelligent connected vehicles, wherein the computer program enables a computer to execute the method for predicting the usefulness of multimodal user reviews of intelligent connected vehicles as described above.
[0063] An electronic device, comprising:
[0064] One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, the programs including a method for executing the method for predicting the usefulness of multimodal reviews of users of smart connected vehicles as described above.
[0065] (3) Beneficial effects
[0066] The present invention provides a method, system, storage medium, and electronic device for predicting the usefulness of multimodal user reviews of intelligent connected vehicles. Compared with existing technologies, this method has the following advantages:
[0067] The proposed neighbor-aware multimodal review usefulness prediction method incorporates neighbor information into the model. By modeling the interaction between neighbor reviews and the current review, it considers the impact of neighbor reviews on review usefulness perception, improving the model's prediction performance. Furthermore, the designed similarity-difference multi-view decomposition attention mechanism considers the complementarity and conflict between multiple views, effectively extracting the specific semantic features of each view and fusing them with common semantic features to obtain a more comprehensive multi-view feature representation. Furthermore, the designed dual-space contrast learning strategy, operating on the similarity and difference subspaces, enables the model to learn discriminative similarity and difference vectors, strengthening the consistent representation of features. This allows multi-view features to better reflect commonalities in the data and reduce inconsistencies between views. BRIEF DESCRIPTION OF THE DRAWINGS
[0068] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0069] Figure 1 A schematic diagram of a model used in a method for predicting the usefulness of multimodal user reviews of intelligent connected vehicles provided by an embodiment of the present invention;
[0070] Figure 2 A block diagram of a method for predicting the usefulness of multimodal user reviews of intelligent connected vehicles provided by an embodiment of the present invention;
[0071] Figure 3 A schematic diagram of the structure of a similarity-difference multi-view decomposition attention mechanism provided by an embodiment of the present invention;
[0072] Figure 4 A schematic diagram of a dual-space contrast learning strategy acting on similarity subspace and difference subspace provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0073] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0074] The embodiments of the present application provide a method, system, storage medium, and electronic device for predicting the usefulness of multimodal reviews by users of intelligent connected vehicles. These methods address the technical problem of being unable to accurately predict the usefulness of reviews by only considering the similarities between multimodal information. The present application aims to explore the prediction of the usefulness of multimodal reviews in the context of intelligent connected vehicles, thereby helping consumers quickly find useful reviews from a large number of online reviews.
[0075] The technical solution in the embodiments of the present application is to solve the above technical problems, and the overall idea is as follows:
[0076] The embodiment of the present invention essentially proposes a neighbor-aware multimodal review usefulness prediction method. The model design used is as follows Figure 1 As shown in the figure, it includes three parts: review information modeling, neighbor review modeling, and usefulness prediction. The input is multimodal reviews (neighbor and own review text and review pictures), summary statistical information (average star rating, star rating distribution, total number of reviews), and other information (release time and overall star rating). The output is the multimodal review usefulness prediction score.
[0077] The embodiments of this invention will help consumers of connected vehicles gain a more comprehensive understanding of vehicle performance and user experience, providing decision support and vehicle purchase references. Furthermore, by accurately predicting the usefulness of reviews, automakers and online platforms can better understand user needs and feedback, improve products and services, and enhance consumer satisfaction.
[0078] Specifically, this embodiment of the present invention designs a similarity-difference multi-view decomposition attention mechanism that fully considers the complementarity and conflict between multiple views. By introducing similarity and difference attention weight matrices, the similarities and differences between different views can be modeled and quantified, thereby better capturing the associations between multiple views and the fine-grained differences and similarities between them.
[0079] The present invention also designs a dual-space contrast learning strategy that operates on both the similarity subspace and the difference subspace. This strategy enables the model to maximize similarity within the similarity subspace and difference within the difference subspace, thereby strengthening the consistent representation of features. This allows multi-view features to better reflect commonalities in the data and reduce inconsistencies between views.
[0080] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.
[0081] Example 1:
[0082] 1. A method for predicting the usefulness of multimodal reviews of intelligent connected vehicles, characterized by: Figure 1 The preset model shown in Figure 2 As shown, the method includes:
[0083] S1. Obtain multiple historical multimodal review data and summary statistical information; any review data includes a review text and multiple associated review images, usefulness score labels, and numerical review information;
[0084] S2. For the comment data at the current location, extract the text embedding representation of the comment text and the image embedding representation of all associated comment images, introduce a similarity-difference multi-view decomposition attention mechanism, and obtain a multimodal comment feature representation; combine the multimodal comment feature representation with the features of the numerical comment information to obtain a comment feature representation of the comment data;
[0085] S3. Obtain a first initial belief from the aggregated statistical information; select a preset number of neighbor comment data relative to the current location, fuse the comment feature representations of all neighbor comment data to obtain a second initial belief from the neighbors; adaptively fuse the first initial belief and the second initial belief to obtain a neighbor comment feature representation;
[0086] S4. Based on the comment feature representation and neighbor comment feature representation of the comment data, the similarity-difference multi-view decomposition attention mechanism is introduced again to obtain a usefulness score prediction result of the comment data;
[0087] S5. Construct a main task loss based on the usefulness score label and usefulness score prediction result of each review data; construct an auxiliary loss based on a dual-space contrastive learning strategy acting on the similarity subspace and the difference subspace; minimize the joint loss function of the main task loss and the auxiliary loss, and train the model until convergence;
[0088] S6. Use the multimodal review data to be predicted as the input of the converged model to obtain the corresponding usefulness score.
[0089] In an embodiment of the present invention, a neighbor-aware multimodal review usefulness prediction method is proposed. This method introduces neighbor information into the model and, through modeling the interaction between neighbor reviews and the current review, considers the impact of neighbor reviews on the perception of review usefulness, thereby improving the model's prediction performance. Furthermore, the designed similarity-difference multi-view decomposition attention mechanism takes into account the complementarity and conflict between multiple views, effectively extracting the specific semantic features of each view and fusing them with common semantic features to obtain a more comprehensive multi-view feature representation. Furthermore, the designed dual-space contrast learning strategy, which acts on the similarity subspace and the difference subspace, enables the model to learn distinctive similarity and difference vectors, strengthening the consistent representation of features and enabling multi-view features to better reflect the commonalities in the data and reduce inconsistencies between views.
[0090] The following sections describe the various steps of the above solution in detail:
[0091] In step S1, a plurality of historical multimodal review data and summary statistical information are obtained; any review data includes a review text and a plurality of associated review images, usefulness score labels and numerical review information.
[0092] Obtain and process the original comment data. Preprocessing methods include performing text cleaning, word segmentation, and stop word removal on the comment text, as well as image processing operations such as scaling and cropping on the image, to obtain the historical multimodal comment data required by the embodiment of the present invention.
[0093] Note: The summary statistics include the average star rating, star rating distribution and total number of reviews, as well as the numerical review information ( Figure 1 The usefulness score label can be divided into multiple score segments according to actual needs. For example, the number of votes can be mapped to 5 intervals based on an exponential function with base 2: [1,2), [2,4), [4,8), [8,16), [16,∞), and the five intervals correspond to 5 usefulness scores s, s∈{0,1,2,3,4}.
[0094] In step S2, for the comment data at the current location, the text embedding representation of the comment text and the image embedding representation of all associated comment images are extracted respectively, and a similarity-difference multi-view decomposition attention mechanism is introduced to obtain a multimodal comment feature representation; the multimodal comment feature representation is combined with the features of the numerical comment information to obtain the comment feature representation of the comment data.
[0095] It is understandable that this step is performed Figure 1Since each historical multimodal review data includes a review text and multiple associated review images, each input review text consists of multiple words.
[0096] Accordingly, this step specifically includes:
[0097] (1) About text embedding representation extraction
[0098] For the review text, we use the powerful pre-trained text representation model BERT to obtain text embedding features. The embedding layer converts the review text into a word vector and obtains its embedding from BERT's classification label space (a 768-dimensional latent space for document representation). The embedding matrix of the text sequence is denoted as K t , convert each word into an embedding vector The embedding length of all words is d. For a comment text with a sentence length of X, the embedding representation of the comment text is as follows:
[0099]
[0100] (2) About Image Embedding Representation Extraction
[0101] For comment images, a pre-trained VGG model is used to obtain image feature representations. The comment images are resized to pixels and then vectorized into RGB matrix images. The matrix images are input into the VGG-19 network and the image embedding matrix is obtained through the transformation of the convolution operation. Each multimodal comment has multiple comment images. The image vectors of each comment are stacked into a sequence embedding matrix. The image embedding features are padded with zero vectors according to a fixed sequence length. The matrix shape is then reshaped to obtain the image embedding matrix K. i , convert each patch into an embedding vector All patch embeddings have the same length (d). For a comment image containing Y patches, the embedding of its comment image is represented as follows:
[0102]
[0103] (3) Similarity-difference multi-view decomposition attention mechanism
[0104] When predicting the usefulness of a review, it is necessary to consider the similarities and differences between the review text and the review image. However, previous studies have focused more on similarity and less on the differences between text and images. In order to fully explore the interactive relationship between review text and review images, the embodiment of the present invention designs a similarity-difference decomposition attention mechanism, which considers the similarities and differences between text and images at the same time to obtain the comprehensive interactive information of text and images, and obtains the multimodal review feature representation K. t_i .
[0105] The overall attention network design is as follows Figure 3 As shown in Figure 2, the attention mechanism obtains the similarity-difference decomposition information between multiple views through two attention weight matrices: the similarity attention weight matrix and the difference attention weight matrix. This attention mechanism can be used to fuse multimodal and multi-viewpoint information. The multimodal information fusion process is as follows:
[0106] For text embedding representation K t and image embedding representation K i Perform L2 normalization respectively and scale them to unit length to obtain the normalized features
[0107] Use dot product and Euclidean distance to calculate features respectively The similarity and difference scores between them are scaled to the range of [0,1] and normalized to obtain the similarity weight matrix S and the difference weight matrix D;
[0108] Embed the text into K t and image embedding representation K i Multiply the two matrices S and D to obtain weighted text-image similarity feature representations. Text-image difference feature representation Image-text similarity feature representation Image-text difference feature representation
[0109] The above four feature representations are weighted concatenated and the concatenation result is combined with the text embedding representation K t and image embedding representation K i Secondary concatenation and mapping through linear layers to obtain multimodal review feature representation K t_i .
[0110] The above process can be calculated by formulas (3 to 6), where α1 and α2 are balancing factors and Linear represents a linear operation:
[0111]
[0112] In addition to the non-structural deep features (i.e., multimodal review feature representation K t_i ), such as comment text and comment pictures, the model in the embodiment of the present invention also takes into account the characteristics of the aforementioned numerical comment information, and maps the low-dimensional numerical features (i.e., the features of release time, star rating, and number of votes) into high-dimensional features through a multi-layer perceptron, and then concatenates the features with the deep features and obtains the final comment feature representation E through a linear layer. r ∈R d×1, where d is the dimension of the vector.
[0113] In step S3, a first initial belief is obtained from the summary statistical information; a preset number of neighbor comment data relative to the current position are selected, and the comment feature representation of all neighbor comment data is fused to obtain a second initial belief from the neighbors; the first initial belief and the second initial belief are adaptively fused to obtain the neighbor comment feature representation
[0114] It is understandable that this step is performed Figure 1 The neighbor review modeling section in
[15] is related to the operations. Neighbor reviews are primarily derived from the summary statistics of the e-commerce platform and the neighbor review data. As mentioned above, this embodiment of the present invention limits the summary statistics to include the average star rating and star rating distribution. For neighbor information, n neighbor review data are selected for feature modeling.
[0115] Accordingly, this step specifically includes:
[0116] (1) Modeling of aggregated statistical information
[0117] First, calculate the standard deviation of the star rating by the average star rating and star distribution; then combine the average star rating and standard deviation to get K through the embedding layer. 0 ∈R 2×d ; Finally, K 0 Transform, through two linear layers Tank activation function, to obtain the first initial belief E from the summary statistical information 0_sta ∈R d×1 ,in b1∈R d×1 ,b2∈R d×1 is the parameter matrix that needs to be optimized during the training process, as shown in formula (7):
[0118]
[0119] (2) Information Modeling of Neighborhood Review Data
[0120] Based on the order of comments displayed on the e-commerce platform, select n neighbor comment data relative to the current location, and obtain the comment feature representation of each neighbor comment data according to the comment information modeling method given in step S2 i∈1…n; where i is the index of the neighbor comment data.
[0121] To better integrate these n feature representations, the present embodiment adopts a graph convolutional neural network (GCN) approach. First, a fully connected neighbor graph is constructed, where nodes represent the feature representation of each neighbor's comments, edges represent the relationship between neighbor comments, and the adjacency matrix is A. Then, the features of each node are aggregated with the features of its neighbor nodes to generate a new representation of each node. This is shown in formula (8):
[0122]
[0123] Among them, H (l) It is a matrix that represents the feature representation of all neighbor comments in the lth layer. Each row corresponds to the feature representation of a neighbor comment in the lth layer. i∈1…n, represents the feature representation of the i-th neighbor comment (r) at layer l, H (0) is a matrix composed of the initial embedding representations of all neighbor comments, and each row corresponds to the initial embedding representation of a neighbor comment, that is, i∈1…n. represents the adjacency matrix after adding self-connection, is the degree of each node, W (l) is the learnable parameter matrix of the lth layer, where σ is the activation function. Specifically, a two-layer GCN is used to encode the neighbor nodes, as shown in formula (9):
[0124]
[0125] Among them, W (0) and W (1) is the weight matrix, The features of each node after passing through the two layers of GCN are concatenated and linearly mapped to the 0_sta In the same feature space, we can obtain the second initial belief E from the neighbors 0 _nei Then the initial belief E from the summary statistics 0_sta With the initial belief E from the neighbor 0_nei Perform adaptive fusion to obtain the neighbor review feature representation E b ∈R d×1 , where d is the dimension of the vector and γ is the adjustment factor that adjusts the importance between the first and second initial beliefs.
[0126] E b =γE o_sta +(1-y)E o_nei #(10)
[0127] In step S4, based on the comment feature representation and neighbor comment feature representation of the comment data, the similarity-difference multi-view decomposition attention mechanism is introduced again to obtain the usefulness score prediction result of the comment data.
[0128] It is understandable that this step is performed Figure 1 The relevant operations of the usefulness prediction part in . The complementary information and conflicting information between opinions will affect consumers' perception of the usefulness of reviews. When conducting multi-opinion interaction, it is necessary to model both aspects of information at the same time. Through the above process, the multimodal review feature representation E has been obtained. r and neighbor review feature representation E b , and then fuse multiple view representations through the similarity-difference multi-view decomposition attention mechanism to obtain the final fusion view feature representation K for prediction r_b , the feature representation is passed through a multi-layer perceptron to obtain the predicted review usefulness score s.
[0129] Referring to formulas (3-6) in step S2, this step specifically includes:
[0130] Based on the similarity-difference multi-view decomposition attention mechanism, the comment feature representation E of the comment data is fused. r ∈R d×1 , Neighborhood Comment Feature Representation E b , to obtain the weighted current review-neighborhood review similarity feature representation Feature representation of the difference between current review and neighbor review Neighbor review-current review similarity feature representation Neighbor comment-current comment difference feature representation
[0131] The above four feature representations are weighted concatenated and the concatenated result is combined with the comment feature representation E of the comment data. r ∈R d×1 , Neighborhood Comment Feature Representation E b Secondary splicing, and mapping through the linear layer to obtain the fusion view feature representation K r_b ;
[0132] The fusion view feature K r_b The usefulness score preset result s of the review data is obtained through multi-layer perceptron mapping.
[0133] s=MLP(K r_b )#(11)
[0134] In step S5, the main task loss is constructed based on the usefulness score label and usefulness score prediction result of each review data; the auxiliary loss is constructed based on the dual-space contrast learning strategy acting on the similarity subspace and the difference subspace; the joint loss function of the main task loss and the auxiliary loss is minimized, and the model is trained until convergence.
[0135] The process of obtaining the joint loss function includes:
[0136] (1) Construction of main task loss
[0137] The main task loss of this model is the mean square error (MSE) loss, as shown in formula (12).
[0138]
[0139] Among them, s i 、y i Respectively represent the preset usefulness score result and usefulness score label of the i-th comment data; |B| represents the total number of comment data in batch B;
[0140] (2) Construction of auxiliary loss
[0141] The auxiliary loss of this model is based on a dual-space contrast learning strategy that acts on the similarity subspace and the difference subspace. It strengthens the consistent representation of features, enables multi-view features to better reflect the commonalities in the data, maximizes the difference feature representation, and enables the model to learn more distinctive view representations.
[0142] Based on the mutual information theory, the above dual-space contrastive learning strategy acting on the similarity subspace and the difference subspace is as follows: Figure 4 This strategy trains the model by minimizing the distance of feature representations in the similarity subspace, maximizing the distance of feature representations in the difference subspace, and the distance of feature representations in different subspaces, which can enhance the representation learned by the model and improve the performance of downstream tasks.
[0143] Specifically, the auxiliary loss includes a bi-spatial contrast loss between modalities and the dual-space contrast loss between reviews Then the construction process of the auxiliary loss includes:
[0144] (2-1) Dual-space contrast loss between modalities Construction
[0145] The feature representation space obtained by the similarity weight matrix is named the similarity subspace The feature representation space obtained by the difference weight matrix is named the difference subspace Then the bi-space contrast loss between modalities is Expressed as:
[0146]
[0147]
[0148] Among them, B represents batch size; Used to measure the similarity between A and B, the similarity function sim(A,B) = A·B, τ is the temperature hyperparameter; K i m ,K j m Representation of weighted text-image similarity feature Text-image difference feature representation Image-text similarity feature representation Image-text difference feature representation Any two features in .
[0149] (2-2) Bi-space contrast loss between reviews Construction
[0150] Similarly, the bi-space contrast loss between reviews is Expressed as:
[0151]
[0152] Among them, K i o ,K j o Represents the weighted current review-neighborhood review similarity feature representation Feature representation of the difference between current review and neighbor review Neighbor review-current review similarity feature representation Neighbor comment-current comment difference feature representation Any two features in .
[0153] (2-3) Obtaining the confidence c of neighbor comments
[0154] The degree of influence of neighbor reviews on usefulness evaluation depends on the degree of consumers' confirmation that their neighbor reviews are correct, that is, the confidence level. The higher the confidence level, the greater the influence level; the lower the confidence level, the smaller the influence level. As shown in formula (19), according to the neighbor review feature representation E b , we use deep learning to obtain the confidence c of neighbor comments, as shown in the following formula, and use it as a regulating factor for the auxiliary loss:
[0155] c=Con(Pooling(MLP(E b )))#(19)
[0156] Among them, Conv represents convolution operation, Pooling represents pooling operation, and MLP represents multi-layer perceptron;
[0157] (2-4) Construction of auxiliary loss
[0158]
[0159] (3) Construction of joint loss function
[0160] According to the main task loss and auxiliary losses Construct the joint loss function:
[0161]
[0162] Here, λ represents a hyperparameter used to tune the effect of auxiliary contrastive learning loss.
[0163] After obtaining the joint loss function, the joint loss function is minimized and the model is trained until convergence.
[0164] In step S6, the multimodal review data to be predicted is used as the input of the converged model to obtain the corresponding usefulness score.
[0165] At this point, the embodiment of the present invention has completed the usefulness prediction work for multimodal review data.
[0166] To help better understand the embodiments of the present invention, the following examples are provided:
[0167] (1) Data collection: Using Python web crawler technology, we crawled review data related to intelligent connected car products on platforms such as Autohome and Dongchedi. The main crawled content includes product ID, star rating, review date, review text, review image, number of votes, etc., forming a multimodal review dataset.
[0168] (2) Preprocess the data:
[0169] ① Text data preprocessing: Perform operations such as text cleaning, word segmentation, and stop word removal on the review text to generate text feature representations. Image data preprocessing: Perform image processing operations such as scaling, cropping, and normalization on the product images to generate image feature representations.
[0170] ② Calculate the standard deviation of the star rating in the summary statistics based on the product ID and add it to each sample as a meta-attribute.
[0171] ③ Map the vote counts to five intervals using a base-2 exponential function: [1, 2), [2, 4), [4, 8), [8, 16), [16, ∞). The five intervals correspond to five usefulness scores s, s∈{0, 1, 2, 3, 4}. Remove reviews with 0 votes because they are in an unknown user feedback state.
[0172] (3) Divide the dataset: Divide the dataset into training set and test set in a ratio of 7:3 to evaluate and select the model.
[0173] (4) Use the pytorch framework to build a deep neural network model and train it on the training set.
[0174] ① Segment the comment text into words and convert the comment images into pixels, and then put them into the pre-trained BERT model and pre-trained VGG model respectively to learn text and image features.
[0175] ② The review text and review image are decomposed into a multimodal review representation through a similarity-difference multi-view attention mechanism, which is then concatenated with the manual review features and then linearly mapped to obtain the review feature representation.
[0176] ③ Select n neighbor comments in the order of platform display, and then embed the neighbor comments according to the above method to obtain n neighbor comment feature representations. These n neighbor comment feature representations constitute a neighbor graph, and then use GCN to extract neighbor information to obtain the neighbor comment representation from the neighbors. This representation is fused with the neighbor comment representation from the summary statistical information to generate the neighbor comment feature representation.
[0177] ④ The review feature representation obtained in step ② is fused with the neighbor review feature representation obtained in step ③ using the similarity-difference multi-view decomposition attention mechanism. Then, the fused features that take into account the customer confirmation bias are passed through MLP to generate the review usefulness prediction result.
[0178] ⑤ For the similarity and difference vectors generated in step ②, construct the inter-modality bi-space contrast loss. For the similarity and difference vectors generated in step ④, construct the inter-review bi-space contrast loss, and construct the mean square error loss function based on the true usefulness score of the review.
[0179] ⑥The mean square error loss generated in step ⑤ is used as the main loss of the model, and the inter-modality bi-space contrast loss and inter-review bi-space contrast loss are used as the auxiliary losses of the model. The auxiliary losses are adjusted using hyperparameters; the optimizer is used to backpropagate the model and update the gradient.
[0180] (5) The performance of the model is evaluated on the test set using the two metrics, Mean Average Precision (MAP) and Normalized Discounted Cumulative Gain (NDCG), where N = 3 and N = 5 are selected. The model parameters are tuned according to these metric values, and the tuned model is trained and continuously adjusted until the model performance is optimal. First, the reviews are sorted in descending order according to the usefulness prediction score s, and then the metric value is calculated.
[0181] The calculation formula of MAP indicator is shown in (22-24).
[0182]
[0183] where rel i represents the true usefulness score of the comment ranked at the i-th position, and rel represents the number of truly relevant samples, that is, comments whose true usefulness score is greater than or equal to the threshold β.
[0184] Example 2:
[0185] The embodiment of the present invention provides a system for predicting the usefulness of multimodal reviews of intelligent connected vehicles by users. Figure 1 The preset model shown, the system includes:
[0186] The review data acquisition module is used to obtain multiple historical multimodal review data and summary statistical information; any review data includes a review text and multiple associated review images, usefulness score labels and numerical review information;
[0187] The review information modeling module is used to extract the text embedding representation of the review text and the image embedding representation of all associated review images for the review data at the current location, introduce a similarity-difference multi-view decomposition attention mechanism, and obtain a multimodal review feature representation; and combine the multimodal review feature representation with the features of the numerical review information to obtain the review feature representation of the review data;
[0188] A neighbor review modeling module is configured to obtain a first initial belief from the aggregated statistical information; select a preset number of neighbor review data relative to the current location, fuse the review feature representations of all neighbor review data, and obtain a second initial belief from the neighbors; and adaptively fuse the first initial belief and the second initial belief to obtain a neighbor review feature representation.
[0189] A usefulness score prediction module is configured to, based on the comment feature representation and neighbor comment feature representation of the comment data, reintroduce the similarity-difference multi-view decomposition attention mechanism to obtain a usefulness score prediction result for the comment data;
[0190] A model training module is configured to construct a primary task loss based on the usefulness score label and usefulness score prediction results of each review data; construct an auxiliary loss based on a dual-space contrastive learning strategy acting on the similarity subspace and the difference subspace; and train the model until convergence by minimizing the joint loss function of the primary task loss and the auxiliary loss.
[0191] The usefulness score acquisition module is used to take the multimodal review data to be predicted as the input of the converged model to obtain the corresponding usefulness score.
[0192] Example 3:
[0193] An embodiment of the present invention provides a storage medium storing a computer program for predicting the usefulness of multimodal reviews by users of intelligent connected vehicles, wherein the computer program enables a computer to execute the method for predicting the usefulness of multimodal reviews by users of intelligent connected vehicles as described in Example 1.
[0194] Example 4:
[0195] An embodiment of the present invention provides an electronic device, including:
[0196] One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the method for predicting the usefulness of multimodal reviews of users of intelligent connected vehicles as described in Example 1.
[0197] It is understandable that the system, storage medium and electronic device for predicting the usefulness of multimodal reviews of users of intelligent connected vehicles provided in the embodiments of the present invention correspond to the method for predicting the usefulness of multimodal reviews of users of intelligent connected vehicles provided in the embodiments of the present invention. The explanations, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the method for predicting the usefulness of multimodal reviews of users of intelligent connected vehicles, and will not be repeated here.
[0198] In summary, compared with the existing technology, the present invention has the following beneficial effects:
[0199] 1. Based on the confirmation bias theory, the embodiment of the present invention considers the impact of multimodal neighbor reviews on usefulness perception, which can effectively improve the predictive performance of the model and make it more consistent with the human processing process of review information.
[0200] 2. Similarity-difference multi-view decomposition attention mechanism. The embodiment of the present invention takes into account the complementarity and conflict between multiple views at the same time, and can effectively extract the specific semantic features of each view and fuse them with the common semantic features to obtain a more comprehensive multi-view feature representation.
[0201] 3. The embodiment of the present invention uses a dual-space contrast learning strategy. By minimizing the distance of feature representations in similar subspaces and maximizing the distance of feature representations in different subspaces and the distance of feature representations in different subspaces, it can fully mine implicit semantic information, better express the structure and characteristics of the data, improve the distinguishability and robustness of feature representation, and promote better feature learning and task accuracy.
[0202] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.
[0203] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.
Claims
1. A method for predicting the usefulness of multimodal user reviews of intelligent connected vehicles, characterized by: Based on the preset model, the methods include: Obtain multiple historical multimodal review data and summary statistical information; any review data includes a review text and multiple associated review images, usefulness score labels, and numerical review information; For the comment data at the current location, extract the text embedding representation of the comment text and the image embedding representation of all associated comment images, introduce a similarity-difference multi-view decomposition attention mechanism, and obtain a multimodal comment feature representation. Combine the multimodal comment feature representation with the features of the numerical comment information to obtain the comment feature representation of the comment data. Obtaining a first initial belief from the aggregated statistical information; selecting a preset number of neighbor review data relative to the current location, fusing the review feature representations of all neighbor review data to obtain a second initial belief from the neighbors; adaptively fusing the first initial belief and the second initial belief to obtain a neighbor review feature representation; Based on the comment feature representation and neighbor comment feature representation of the comment data, the similarity-difference multi-view decomposition attention mechanism is introduced again to obtain the usefulness score prediction result of the comment data; Based on the usefulness score label and usefulness score prediction results of each review data, a main task loss is constructed; based on a dual-space contrastive learning strategy acting on the similarity subspace and the difference subspace, an auxiliary loss is constructed; the joint loss function of the main task loss and the auxiliary loss is minimized, and the model is trained until convergence; The multimodal review data to be predicted is used as the input of the converged model to obtain the corresponding usefulness score; The method introduces a similarity-difference multi-view decomposition attention mechanism to obtain multimodal review feature representations, including: For text embedding representation K t and image embedding representation K i Perform L2 normalization respectively and scale them to unit length to obtain the normalized features Use dot product and Euclidean distance to calculate features respectively The similarity and difference scores between them are scaled to the range of [0,1] and normalized to obtain the similarity weight matrix S and the difference weight matrix D; Embed the text into K t and image embedding representation K i Multiply the two matrices S and D to obtain weighted text-image similarity feature representations. Text-image difference feature representation Image-text similarity feature representation Image-text difference feature representation The above four feature representations are weighted concatenated and the concatenation result is combined with the text embedding representation K t and image embedding representation K i Secondary concatenation and mapping through linear layers to obtain multimodal review feature representation K t_i .
2. The method for predicting the usefulness of multimodal user reviews of intelligent connected vehicles according to claim 1, wherein: The method further introduces the similarity-difference multi-view decomposition attention mechanism based on the comment feature representation and the neighbor comment feature representation of the comment data to obtain the usefulness score prediction result of the comment data; including: Based on the similarity-difference multi-view decomposition attention mechanism, the comment feature representation E of the comment data is fused. r ∈R d ×1 , Neighborhood Comment Feature Representation E b , to obtain the weighted current review-neighborhood review similarity feature representation Feature representation of the difference between current review and neighbor review Neighbor review-current review similarity feature representation Neighbor comment-current comment difference feature representation The above four feature representations are weighted concatenated and the concatenated result is combined with the comment feature representation E of the comment data. r ∈R d ×1 , Neighborhood Comment Feature Representation E b Secondary splicing, and mapping through the linear layer to obtain the fusion view feature representation K r_b ; The fusion view feature K r_b The usefulness score preset result s of the review data is obtained through multi-layer perceptron mapping.
3. The method for predicting the usefulness of multimodal user reviews of intelligent connected vehicles according to claim 1, wherein: The numerical review information includes the release time and the overall star rating; the step of combining the multimodal review feature representation with the features of the numerical review information to obtain the review feature representation of the review data includes: The low-dimensional features of release time and overall star rating are mapped into high-dimensional features through a multi-layer perceptron, and the high-dimensional features are combined with the multimodal review features K t_i Splice and obtain the comment feature representation E of the comment data through linear layer mapping r ∈R d×1 , where d is the dimension of the vector.
4. The method for predicting the usefulness of multimodal user reviews of intelligent connected vehicles according to claim 1, wherein: The summary statistical information includes the average star rating, star rating distribution, and the total number of reviews; and obtaining the first initial belief from the summary statistical information includes: Calculate the standard deviation of star rating by using the average star rating and star rating distribution; Combine the average star rating and standard deviation and get K through the embedding layer 0 ∈R 2×d ; K 0 Transform, through two linear layers Tanh activation function, to obtain the first initial belief E from the summary statistics 0_sta ∈R d×1 , where d is the dimension of the vector.
5. The method for predicting the usefulness of multimodal user reviews of intelligent connected vehicles according to claim 1, wherein: The method of selecting a preset number of neighbor review data relative to the current location and fusing the review feature representations of all neighbor review data to obtain a second initial belief from the neighbors includes: Select n neighbor comment data relative to the current location and obtain the comment feature representation of each neighbor comment data Where i is the index of the neighbor comment data; The comment features of each neighbor's comment data are represented as nodes, and the dependency relationships between neighbor's comment data are used as edges to construct a fully connected neighbor graph; For the neighbor graph, a new representation of each node is obtained through a two-layer GCN network, which is concatenated and linearly mapped to the first initial belief E 0_sta In the same feature space, the second initial belief E from the neighbors is obtained 0_nei ∈R d×1 .
6. The method for predicting the usefulness of multimodal user reviews of intelligent connected vehicles according to claim 2, wherein the auxiliary loss is constructed based on a dual-space contrastive learning strategy acting on a similarity subspace and a difference subspace; comprising: Construct dual-space contrast loss between modalities separately and the dual-space contrast loss between reviews The feature representation space obtained by the similarity weight matrix is named the similarity subspace The feature representation space obtained by the difference weight matrix is named the difference subspace Then the bi-space contrast loss between modalities is Expressed as: Among them, B represents batch size; Used to measure the similarity between A and B, the similarity function sim(A,B) = A·B, τ is the temperature hyperparameter; K i m ,K j m Representation of weighted text-image similarity feature Text-image difference feature representation Image-text similarity feature representation Image-text difference feature representation Any two feature representations in ; Similarly, the bi-space contrast loss between reviews is Expressed as: Among them, K i o ,K j o Represents the weighted current review-neighborhood review similarity feature representation Feature representation of the difference between current review and neighbor review Neighbor review-current review similarity feature representation Neighbor comment-current comment difference feature representation Any two feature representations in ; According to the neighbor review feature representation E b , we use deep learning to obtain the confidence c of neighbor comments, as shown in the following formula, and use it as a regulating factor for the auxiliary loss: c=Conv(Pooling(MLP(E b ))) Among them, Conv represents convolution operation, Pooling represents pooling operation, and MLP represents multi-layer perceptron; According to the dual spatial contrast loss and Combined with the confidence c of the neighbor comments, the auxiliary loss is constructed:
7. The method for predicting the usefulness of multimodal user reviews of intelligent connected vehicles according to claim 1, wherein: The process of obtaining the joint loss function includes: According to the usefulness score label and usefulness score prediction results of each review data, the mean square error loss is used as the main task loss: Among them, s i 、y i Respectively represent the preset usefulness score result and usefulness score label of the i-th comment data; |B| represents the total number of comment data in batch B; According to the main task loss and auxiliary losses Construct the joint loss function: Here, λ represents a hyperparameter used to tune the effect of auxiliary contrastive learning loss.
8. A system for predicting the usefulness of multimodal user reviews of intelligent connected vehicles, characterized by: Based on the preset model, the system includes: The review data acquisition module is used to obtain multiple historical multimodal review data and summary statistical information; any review data includes a review text and multiple associated review images, usefulness score labels and numerical review information; The review information modeling module is used to extract the text embedding representation of the review text and the image embedding representation of all associated review images for the review data at the current location, introduce a similarity-difference multi-view decomposition attention mechanism, and obtain a multimodal review feature representation; and combine the multimodal review feature representation with the features of the numerical review information to obtain the review feature representation of the review data; A neighbor review modeling module is configured to obtain a first initial belief from the aggregated statistical information; select a preset number of neighbor review data relative to the current location, fuse the review feature representations of all neighbor review data, and obtain a second initial belief from the neighbors; and adaptively fuse the first initial belief and the second initial belief to obtain a neighbor review feature representation. A usefulness score prediction module is configured to, based on the comment feature representation and neighbor comment feature representation of the comment data, reintroduce the similarity-difference multi-view decomposition attention mechanism to obtain a usefulness score prediction result for the comment data; A model training module is configured to construct a primary task loss based on the usefulness score label and usefulness score prediction results of each review data; construct an auxiliary loss based on a dual-space contrastive learning strategy acting on the similarity subspace and the difference subspace; and train the model until convergence by minimizing the joint loss function of the primary task loss and the auxiliary loss. The usefulness score acquisition module is used to take the multimodal review data to be predicted as the input of the converged model and obtain the corresponding usefulness score; The method introduces a similarity-difference multi-view decomposition attention mechanism to obtain multimodal review feature representations, including: For text embedding representation K t and image embedding representation K i Perform L2 normalization respectively and scale them to unit length to obtain the normalized features Use dot product and Euclidean distance to calculate features respectively The similarity and difference scores between them are scaled to the range of [0,1] and normalized to obtain the similarity weight matrix S and the difference weight matrix D; Embed the text into K t and image embedding representation K i Multiply the two matrices S and D to obtain weighted text-image similarity feature representations. Text-image difference feature representation Image-text similarity feature representation Image-text difference feature representation The above four feature representations are weighted concatenated and the concatenation result is combined with the text embedding representation K t and image embedding representation K i Secondary concatenation and mapping through linear layers to obtain multimodal review feature representation K t_i .
9. A storage medium, characterized in that: It stores a computer program for predicting the usefulness of multimodal reviews by users of intelligent connected vehicles, wherein the computer program enables a computer to execute the method for predicting the usefulness of multimodal reviews by users of intelligent connected vehicles as claimed in any one of claims 1 to 7.
10. An electronic device, characterized in that: include: one or more processors; Memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the method for predicting the usefulness of multimodal reviews of users of intelligent connected vehicles according to any one of claims 1 to 7.