Double-tower recommendation method fusing BERT and LSTM

Abstract

The invention relates to a BERT and LSTM fused double-tower recommendation method, which combines the advantages of BERT and LSTM, effectively fuses user comments and article information, processes the user comments through a BERT model, extracts deep features from the user comments, effectively captures user interest changes, obtains article features through an embedded layer, and then fuses the features in the LSTM, so that the user comments and the article information can be recommended. Capturing the time sequence dependence of the user behavior, and converting the prediction score; according to the recommendation method, the score prediction accuracy is improved, the information fusion capability is enhanced, a more accurate, more transparent and more personalized recommendation process is provided for the user, and the user satisfaction and the user stickiness are improved.

Description

Technical Field

[0001] The present invention relates to the field of recommendation systems, and in particular to a dual-tower recommendation method integrating BERT and LSTM. Background Art

[0002] In the digital age, recommendation systems play an important role in e-commerce, social media, and content platforms. By analyzing user behavior and preferences, recommendation systems can provide personalized content and product recommendations, improve user experience, and promote sales.

[0003] Common recommendation systems use collaborative filtering algorithms or content-based recommendation algorithms to recommend information, products or services similar to user interests. However, current recommendation methods still face many challenges. Traditional collaborative filtering and content-based recommendation systems have limitations in processing sparse data and capturing changes in user interests. At the same time, existing models often fail to fully utilize user text comments and item features, resulting in insufficient information fusion, which affects the accuracy of recommendations. In addition, many models have poor interpretability, making it difficult for users to understand the reasons behind the recommendation results.

[0004] In recent years, the rapid development of deep learning technology has provided new opportunities for the optimization of recommendation systems, especially in processing large-scale data and complex features. BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained language representation model based on the Transformer architecture. It captures the complex relationship between words and sentences and generates rich language feature representations through unsupervised learning of large amounts of text data. LSTM (Long Short-Term Memory) is a special recurrent neural network (RNN) architecture that can effectively capture and retain long-term dependencies by introducing "memory cells" and three gating mechanisms (input gate, forget gate, and output gate), thereby accurately capturing and effectively utilizing long-distance information in sequence data; the two-tower architecture, as an effective model design, enhances the flexibility and accuracy of the recommendation system by independently processing user and item features.

[0005] How to enhance the information fusion capability of the recommendation system, accurately capture user interests, and improve recommendation accuracy and explainability is the technical problem faced by the present invention. Summary of the invention

[0006] In view of the defects of the prior art, the present invention provides a dual-tower recommendation method that integrates BERT and LSTM; this method uses BERT to extract deep semantic features in user comments and captures the temporal dependency of item features through LSTM, thereby effectively improving the accuracy of rating prediction and enhancing the personalization and interpretability of recommendations.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] A dual-tower recommendation method integrating BERT and LSTM includes the following steps:

[0009] S1. Use the BERT model to perform semantic analysis on user comments and item details to obtain user comment word vectors and item detail word vectors; obtain the embedding vectors corresponding to the item ID and item ID group through the embedding layer; concatenate the user comment word vector and the embedding vector of the item ID group into a user vector to form a user tower; concatenate the detail word vector and the embedding vector of the item ID into an item vector to form an item tower;

[0010] S2, concatenate the user vector and the item vector into the LSTM, add the time dimension, and extract the hidden state of the last time step of the LSTM for subsequent rating prediction;

[0011] S3: The hidden state of the last time step output by LSTM is output through a fully connected layer to output the predicted score, and the predicted score is converted, and the recommendation is determined based on the converted score.

[0012] As a further optimization, the step S1 is specifically as follows:

[0013] S11. User comments and item details are used as input and input into the BERT model for semantic analysis, which is expressed as:

[0014] v review =mean(BERT(Input review ));

[0015] v detail =mean(BERT(Input detail ));

[0016] Among them, v review Represented as user comment word vector, v detail Represented as item detail word vector, mean means averaging all word vectors to obtain the aggregate representation of the entire review and details;

[0017] S12, item ID and item ID grouping are passed through the embedding layer to obtain the corresponding embedding vector. For a single item ID, its embedding vector is represented as eitem ; For item ID group G = {g1, g2, ..., g k}, and calculate the average value of its embedding vector as e group :

[0018] e item =E item [id i ]

[0019]

[0020] Among them, E item is the item embedding matrix, id i is a unique identifier for an item; is the i-th item in the g-th item group in item ID group G, and k is the number of item IDs in the group;

[0021] S13. Vectorize the comment words and the embedding vector grouped by item ID Spliced ​​into user vector v user ; Transform the details word vector and the embedding vector of the item ID Splice into item vector v item :

[0022]

[0023]

[0024] represents a real vector space of dimension d, i.e., it contains all real vectors of length d.

[0025] As a further optimization, step S2 is specifically as follows:

[0026] S21. Stack the user vector and item vector into a batch input:

[0027]

[0028] S22. Convert the combined vector into a format suitable for LSTM input and add the time dimension:

[0029]

[0030] v LSTM =v combined *unsqueeze(1)

[0031] Where B is the batch size, 1 is the tensor index used to add the time dimension, and unsqueeze(1) is used to add a dimension to a specific dimension of the tensor.

[0032] S23, the combined vector v LSTM After input into the LSTM model, the model calculates the hidden state and cell state at each time step:

[0033] h LSTM ,c LSTM =LSTM(v LSTM )

[0034] is the hidden state output of the LSTM, T is the number of time steps, is the cell state; the LSTM model updates the hidden state and cell state of the current time step based on the input vector and its hidden state of the previous time step;

[0035] S24. After processing all time steps, extract the hidden state of the last time step of LSTM for subsequent rating prediction. The extraction operation can be expressed as:

[0036]

[0037] Among them, h last is a matrix, each row of which corresponds to the hidden state of the last time step of a sample, containing the context information of the sample, d hidden Represents the dimension of the LSTM hidden state.

[0038] As a further optimization, step S3 is specifically as follows:

[0039] S31, the fully connected layer takes the hidden state of the last time step as input and performs a linear transformation, which is specifically expressed as:

[0040]

[0041] in, is the predicted rating of user u for item i; is the weight matrix of the fully connected layer, responsible for mapping the hidden state to the dimension of the score; is a bias term used to increase the flexibility of the model;

[0042] S32. Get prediction score Convert it into an actual rating value suitable for the recommendation system;

[0043] S33: Generate recommendation results based on the converted scores.

[0044] As a further optimization, the process of converting the prediction score in step S32 is specifically as follows:

[0045] S32.1, calculate the integer part int_part of the predicted score to obtain the closest lower bound:

[0046]

[0047] here, Represents a round-down operation to obtain a value that is less than or equal to The largest integer of ;

[0048] S32.2. Calculate the decimal part of the predicted score to determine whether the final score result needs to be adjusted upward:

[0049]

[0050] Among them, float_part is the result of subtracting the integer part from the original prediction score;

[0051] S32.3. Based on the value of the decimal part, the final score result S is determined by rounding off:

[0052]

[0053] S32.4 To ensure that the scores are within an acceptable range, the scores are limited to a minimum score and a maximum score:

[0054]

[0055] in, Represents the converted scoring result.

[0056] As a further optimization, the process of generating the recommendation result in step S33 is specifically as follows:

[0057] S33.1 defines a score threshold θ for recommendation, which is used to determine which items should be recommended:

[0058] like If the score is greater than the rating threshold θ, the item i is recommended to user u; otherwise, it is not recommended. The rating threshold θ is a value between the minimum rating and the maximum rating.

[0059] S33.2. Generate a set of recommended items i for each user u:

[0060] Beneficial effects:

[0061] The present invention uses a dual-tower model to construct the overall framework of the recommendation algorithm, which can process large-scale data in parallel and reduce the running time of the algorithm. The present invention combines the advantages of BERT and LSTM to effectively integrate user comments and item information. User comments are processed by the BERT model to extract deep features and effectively capture changes in user interests, while item features are obtained through the embedding layer. Then, these features are fused in LSTM to capture the temporal dependency of user behavior and effectively fuse the information. This not only improves the accuracy of rating prediction, but also enhances the ability to integrate information and pays attention to explainability, providing users with a more accurate and transparent recommendation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is the overall architecture diagram of the method recommended by the present invention;

[0063] Figure 2 It is an overall flow chart of the method recommended by the present invention. DETAILED DESCRIPTION

[0064] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0065] like Figure 1-2 As shown, the present invention provides a dual-tower recommendation method integrating BERT and LSTM for e-commerce services, aiming to effectively integrate user comments and item features to improve the performance of the recommendation system. The basic process of the recommendation method is: first, a pre-trained BERT model is used to process user comments to extract their deep semantic features to capture subtle information in the text; then, the vector representation of the item ID is obtained by searching the embedding matrix, and the item ID groups are averaged to obtain a unified item feature representation, and then the extracted user features are concatenated with the item features to form a joint vector, which is input into the LSTM layer to model the temporal dependency between features; finally, a score prediction is generated through a fully connected layer to achieve personalized recommendation, which specifically includes the following steps:

[0066] S1, perform semantic analysis on user comments and item details, obtain corresponding embedding vectors for item ID and item ID grouping to form a sequence; concatenate the comment word vector and the embedding of item ID grouping into the user vector, and form a user tower; concatenate the detail word vector and the embedding of item ID into the item vector, and form an item tower. Step S1 specifically includes:

[0067] S11. User comments and item details are used as input and input into the BERT model for semantic analysis, which can be expressed by the following specific formula:

[0068] v review =mean(BERT(Input review)) (1)

[0069] v detail =mean(BERT(Input detail )) (2)

[0070] Among them, v review Represented as user comment word vector, v detail Represented as item detail word vector; mean means averaging all word vectors to obtain an aggregate representation of the entire review and details.

[0071] S12, item ID and item ID group get the corresponding embedding vector through the embedding layer; for a single item ID, its embedding vector can be expressed as e item ; For item ID group G = {g1, g2, ..., g k}, the average value of its embedding vector can be calculated as e group .

[0072] e item =E item [id i ] (3)

[0073]

[0074] Among them, E item is the item embedding matrix, id i is a unique identifier for the item, It represents the i-th item in the g-th item group in the item ID group G, where g represents a specific group in the item ID group and i represents a specific item in the group; k is the number of item IDs in the group.

[0075] S13. Vectorize the comment words and item ID grouping embedding Concatenate into the user's vector v user ; The details word vector and item ID embedding The vector v of the item spliced ​​together item :

[0076]

[0077]

[0078] represents a real vector space of dimension d, that is, it contains all real vectors of length d, and so on. represents real vector spaces of different dimensions, d1 is the dimension of the user comment word vector; d2 is the dimension of the item ID group embedding vector; d3 is the dimension of the item detail word vector; d4 is the dimension of the item ID embedding vector; in step S13, these vectors are concatenated to form user vectors and item vectors, the dimension of the user vector is d1+d2, and the dimension of the item vector is d3+d4. The concatenation operation is to integrate different types of feature information into one vector.

[0079] S2: Input the user vector and item vector concatenated in step S1 into LSTM and add the time dimension; then, extract the hidden state of the last time step of LSTM for subsequent rating prediction. Step S2 specifically includes:

[0080] S21. Concatenate the user vector and the item vector into a combined vector and stack them into a batch input:

[0081]

[0082] Among them, the symbol represents concatenation, that is, concatenating two vectors into a new vector. It is a real vector space with dimension d1+d2+d3+d4. The vectors in this space are obtained by concatenating multiple feature vectors of users and items.

[0083] S22, the combined vector v combined To convert it into a format suitable for LSTM input, we need to convert it into a 3D tensor and add the time dimension:

[0084]

[0085] v LSTM =v combined *unsqueeze(1 )(9)

[0086] In this step, B×1×(d1+d2+d3+d4) represents the dimension of the three-dimensional tensor, B is the batch size, which indicates the number of data samples processed at a time, 1 is the tensor index used to add the time dimension, and nusqueeze(1) is used to add a dimension to a specific dimension of the tensor. combined Input to LSTM.

[0087] S23. After the combined vector is input into the LSTM model, the model calculates the hidden state and cell state at each time step:

[0088] h LSTM ,c LSTM =LSTM(v LSTM ) (10)

[0089] in, is the hidden state output of LSTM, is a three-dimensional real tensor, T is the number of time steps (1 in this case), d huddsn Represents the dimension of the LSTM hidden state, that is, the size of the hidden layer inside the LSTM, is the cell state; the LSTM model updates the hidden state and cell state of the current time step based on the input vector and its hidden state of the previous time step.

[0090] S24. After processing all time steps, it is necessary to extract the hidden state of the last time step of LSTM for subsequent rating prediction. The hidden state of the last time step can represent the information of the entire sequence. The extraction operation can be expressed as:

[0091]

[0092] Among them, h last It is a matrix, each row of which corresponds to the hidden state of the last time step of a sample and contains the context information of the sample.

[0093] S3, after extracting the hidden state of the last time step of LSTM in step S2, output the predicted score through the fully connected layer (linear layer); convert the predicted score, and decide the recommendation based on the converted score. The specific process is as follows:

[0094] S31. The predicted score is output through the fully connected layer (linear layer). The fully connected layer takes the hidden state of the last time step as input and performs a linear transformation, which is specifically expressed as:

[0095]

[0096] in, represents the predicted rating of user u for item i; is the weight matrix of the fully connected layer, responsible for mapping the hidden state to the dimension of the score; is a bias term used to increase the flexibility of the model.

[0097] S32. Get prediction score Convert it into an actual rating value suitable for the recommendation system. The conversion process includes the following steps:

[0098] S32.1. Calculate the integer part int_part of the predicted score to obtain the closest lower bound:

[0099]

[0100] in, Represents a round-down operation to obtain a value that is less than or equal to The calculation of the integer part can effectively isolate the overall value of the score and provide a basis for the subsequent comparison of the decimal part.

[0101] S32.2. Calculate the decimal part float_part of the predicted score to determine whether the final score result needs to be adjusted upward. This process can be expressed as:

[0102]

[0103] Among them, float_part is the result of subtracting the integer part from the original score. The purpose of this step is to facilitate rounding through the value of the decimal part to ensure the rationality and accuracy of the score.

[0104] S32.3. According to the value of the decimal part float_part, the final score result S is determined by rounding off. If the decimal part is less than 0.5, the integer part is retained. Otherwise, if the decimal part is greater than or equal to 0.5, the integer part is increased by 1, which is expressed as:

[0105]

[0106] S32.4. To ensure that the score is within an acceptable range, the result is limited to between the minimum score and the maximum score. The converted score value is:

[0107] Among them, min_score = 1, max_score = 5, limiting the score to the range of 1 to 5 is to ensure the consistency and acceptability of the score. This restriction can prevent extreme scores, such as scores that are too high or too low.

[0108] In step S32, the predicted score is first rounded to S, and then S is converted into a scoring standard between the minimum score and the maximum score. The actual score value determined by calculating the decimal part and applying the rounding rule is equivalent to first determining a benchmark (integer part), and then deciding whether to adjust the benchmark based on the decimal part. This is more precise than directly rounding the predicted score, and the converted score has better generalization ability and robustness than the direct score. By converting and limiting the range of the predicted score, extreme scores caused by model prediction bias can be reduced, thereby improving the fairness and reliability of the score.

[0109] S33: After the scores are converted, a recommendation result is generated based on the converted scores.

[0110] S33.1. Define a score threshold θ for recommendation, which is used to determine which items should be recommended:

[0111]

[0112] like If the score is greater than the score threshold θ, Recommendation(u,i) takes the value of 1, indicating that item i is recommended to user u. Otherwise, it means that it is not recommended. In practical applications, the threshold θ is set according to needs. For example, in the embodiment of the present invention, the threshold is set to 3, then items with a score greater than 3 will be recommended to the user.

[0113] S33.2. For each user u, generate a set R of items i recommended to user u u :

[0114]

[0115] The above is the overall process of the recommended method in the embodiment of the present invention.

[0116] The dual-tower recommendation method integrating BERT and LSTM provided by the present invention significantly improves the accuracy and personalization of recommendations, provides explainability for the recommendation process, and effectively enhances user satisfaction and user stickiness, which is mainly reflected in: (1) using the BERT model to perform in-depth semantic analysis on user comments and item details, and extracting rich information on user preferences and item features. This deep understanding enables the recommendation system to more accurately capture the personalized needs of users and the unique attributes of items, thereby improving the personalization of recommendations; (2) item sequence features are integrated in both the user tower and the item tower. This integration helps to enhance the expression effect of feature interaction, increase feature richness, improve the accuracy and diversity of recommendation results, and significantly improve the personalization of recommendations; (3) the dual-tower structure allows the model to independently learn the features of users and items, and then match users and items through vector similarity calculation, thereby improving the personalization of recommendations and providing more personalized recommendations.

[0117] The recommendation method of the present invention can also effectively alleviate the cold start problem of new users or new items. In the dual-tower model, by converting the item ID and its grouping information into an embedding vector and combining them, the inherent attributes of the item and the relationship between them are captured, and items can be recommended to users without user data; through the BERT model, in-depth semantic analysis of user comments and item details can be performed to extract rich information about user preferences and item characteristics. This information provides additional contextual information for new users or new items, and recommendations can be made based on the additional information in the absence of data.

[0118] The present invention uses a dual-tower structure to encode users and items respectively. The model learns the characteristics of users and items independently. The learning results of the user tower and the item tower can be viewed separately to understand how recommendations are generated based on the similarities of user and item characteristics, which provides explainability for the recommendation process and increases user satisfaction.

[0119] In order to further demonstrate the performance and effect of the recommendation method of the present invention, experimental comparisons are conducted on BERT, BERT+multi-head attention mechanism and the present invention. Public datasets are used for experimental verification, and indicators such as mean square error (MSE), mean absolute error (MAE) and explained variance score (EVS) are used to evaluate model performance and conduct quantitative analysis to ensure the effectiveness and reliability of the recommendation system. The corresponding formulas are as follows.

[0120]

[0121]

[0122]

[0123] In formulas (19)-(21), n ​​is the total number of samples; is the predicted score of the i-th sample; y i is the true score of the i-th sample; is the variance of the prediction error; Var(y) is the variance of the true value.

[0124] There are three comparison models:

[0125] 1) model_1: BERT+Multi-Head Attention, the input is user comments, item details, item ID, and item ID grouping;

[0126] 2) model_2: BERT+LSTM, input is user comments and item details;

[0127] 3) model_3: BERT, input is user comments, item details, item ID and item ID group;

[0128] 4) our_model: BERT+LSTM, representing the model of the present invention, the input is user comments, item details, item ID and item ID grouping.

[0129] The experimental results are shown in Tables 1 and 2.

[0130] Table 1 Data of four groups of models in each round under three evaluation criteria

[0131] Table 1 shows the data of each round of the four models under three evaluation indicators. The best model in each group is highlighted in bold and the suboptimal model is highlighted in underline. As shown in Table 1, in terms of MAE values, in Epoch1 and Epoch2, the MAE values ​​of our_model are 0.5250 and 0.5225 respectively, which are significantly lower than other models, indicating that our_model has the smallest error and has a higher prediction accuracy; in Epoch3, the MAE value of our_model is 0.8925, which is close to other models, but still performs well.

[0132] From the MSE value, our_model also performs well. In all three epochs, the MSE value of our_model is always the lowest, which is 1.1300, 1.1875 and 1.1425 respectively. The lower MSE indicates that the prediction error is smaller and the model is less affected by outliers. In contrast, other models, especially model_3, have the highest MSE in Epoch1 and Epoch3.

[0133] Although Model_1 and Model_2 also have good performance, they cannot surpass our_model in MAE and MSE, especially the gap between Epoch1 and Epoch2 is more obvious. Although Model_3 performs well in MAE of Epoch2, the overall error is high, indicating that its prediction effect is not ideal when combining item ID and ID grouping.

[0134] EVS reflects the model's ability to explain the data, and the closer the score is to 1, the better. Our_model outperforms other models in EVS in all cycles, especially in the third cycle, which reaches 0.9050, which is significantly higher than other comparison models. This shows that our model has a good performance in explaining data and can more accurately capture the relationship between user comments and item features.

[0135] Table 2 Analysis of the training loss of each round of four models

[0138] Table 2 is the training loss analysis of the four groups of models in each round of the data set experiment, in which the lowest value in each group is highlighted in bold and the second lowest value is underlined. As shown in Table 2, the training loss value of our_model in Epoch1 is slightly higher than that of model_1 and model_2, but it converges better in subsequent cycles, especially reaching the lowest value in Epoch3, indicating that the prediction accuracy and reliability are high.

[0139] In summary, the present invention introduces BERT and LSTM in the dual-tower structure, uses the BERT model to extract deep semantic features in user reviews, and captures the temporal dependency of item features through LSTM, effectively improving the accuracy of rating prediction, enhancing the personalization and interpretability of recommendations, and helping to improve user satisfaction and stickiness.

[0140] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A dual-tower recommendation method integrating BERT and LSTM, characterized in that: The following steps are involved: S1. Use the BERT model to perform semantic analysis on user comments and item details to obtain user comment word vectors and item detail word vectors; obtain the embedding vectors corresponding to the item ID and item ID group through the embedding layer; concatenate the user comment word vector and the embedding vector of the item ID group into a user vector to form a user tower; concatenate the detail word vector and the embedding vector of the item ID into an item vector to form an item tower; S2, concatenate the user vector and the item vector into the LSTM, add the time dimension, and extract the hidden state of the last time step of the LSTM for subsequent rating prediction; S3: The hidden state of the last time step output by LSTM is output through a fully connected layer to output the predicted score, and the predicted score is converted, and the recommendation is determined based on the converted score.

2. According to claim 1, a dual-tower recommendation method integrating BERT and LSTM is characterized in that: Step S1 is specifically as follows: S11. User comments and item details are used as input and input into the BERT model for semantic analysis, which is expressed as: v review =mean(BERT(Input review )); (1) v detail =mean(BERT(Input detail )); (2) In the formula, v review Represented as user comment word vector, v detail Represented as item detail word vector, mean means averaging all word vectors to obtain the aggregate representation of the entire review and details; S12, item ID and item ID grouping are passed through the embedding layer to obtain the corresponding embedding vector. For a single item ID, its embedding vector is represented as e item ; For item ID group G = {g1, g2, ..., g " }, and calculate the average value of its embedding vector as e $r％up : it is item =E item [id i ] (3) Among them, E item is the item embedding matrix, id i is a unique identifier for an item; is the i-th item in the g-th item group in item ID group G, and k is the number of item IDs in the group; S13. Vectorize the comment words and the embedding vector grouped by item ID Spliced ​​into user vector v u5er ; Transform the details word vector and the embedding vector of the item ID Splice into item vector v item : represents a real vector space of dimension d, i.e., it contains all real vectors of length d.

3. According to claim 2, a dual-tower recommendation method integrating BERT and LSTM is characterized in that: Step S2 is specifically as follows: S21. Stack the user vector and item vector into a batch input: S22. Convert the combined vector into a format suitable for LSTM input and add the time dimension: in ＝>T@ =in :％m；i<ed *unsqueeze(1) (9) Where B is the batch size, 1 is the tensor index used to add the time dimension, and unsqueeze(1) is used to add a dimension to a specific dimension of the tensor. S23, the combined vector v ＝>T@ After input into the LSTM model, the model calculates the hidden state and cell state at each time step: h ＝>T@ ,c ＝>T@ =LSTM(v ＝>T@ ) (10) is the hidden state output of the LSTM, T is the number of time steps, is the cell state; the LSTM model updates the hidden state and cell state of the current time step based on the input vector and its hidden state of the previous time step; S24. After processing all time steps, extract the hidden state of the last time step of LSTM for subsequent rating prediction. The extraction operation can be expressed as: Among them, h la5t is a matrix, each row of which corresponds to the hidden state of the last time step of a sample, containing the context information of the sample, d hidden Represents the dimension of the LSTM hidden state.

4. According to claim 3, a dual-tower recommendation method integrating BERT and LSTM is characterized in that: Step S3 is specifically as follows: S31, the fully connected layer takes the hidden state of the last time step as input and performs a linear transformation, which is specifically expressed as: in, is the predicted rating of user u for item i; is the weight matrix of the fully connected layer, responsible for mapping the hidden state to the dimension of the score; is a bias term used to increase the flexibility of the model; S32. Get prediction score Convert it into an actual rating value suitable for the recommendation system; S33. Generate recommendation results based on the converted scores.

5. According to claim 4, a dual-tower recommendation method integrating BERT and LSTM is characterized in that: The specific process of converting the prediction score in step S32 is as follows: S32.1, calculate the integer part int_part of the predicted score to obtain the closest lower bound: here, Represents a round-down operation to obtain a value that is less than or equal to The largest integer of ; S32.

2. Calculate the decimal part of the predicted score to determine whether the final score result needs to be adjusted upward: Among them, float_part is the result of subtracting the integer part from the original prediction score; S32.

3. Based on the value of the decimal part, the final score result S is determined by rounding off: S32.4 To ensure that the scores are within an acceptable range, the scores are limited to a minimum score and a maximum score: in, Represents the converted scoring result.

6. According to claim 5, a dual-tower recommendation method integrating BERT and LSTM is characterized in that: The specific process of generating the recommendation result in step S33 is as follows: S33.1 defines a score threshold θ for recommendation, which is used to determine which items should be recommended: like If the score is greater than the rating threshold θ, the item i is recommended to user u; otherwise, it is not recommended. The rating threshold θ is a value between the minimum rating and the maximum rating. S33.

2. Generate a set of recommended items i for each user u:

Images