A sequence recommendation method based on the fusion of positive and negative feedback and logical rules
By combining positive and negative feedback from user behavior with logical rules, and utilizing the Transformer model and self-cross attention mechanism, the problems of neglecting negative feedback and insufficient interpretability in existing recommendation algorithms are solved, thus realizing an efficient and highly interpretable recommendation system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TIANJIN UNIV
- Filing Date
- 2025-02-26
- Publication Date
- 2026-05-26
AI Technical Summary
Existing recommendation algorithms are mostly based on positive feedback data modeling, ignoring negative feedback. This makes it difficult for the models to accurately capture users' true preferences and lacks interpretability and dynamic adaptability, especially in fields such as healthcare and finance.
Combining positive and negative feedback information from user behavior with logical rules, a deep learning model built using Transformer is used for sequence recommendation. Self-attention and cross-attention mechanisms are used to capture the relationship between user behavior and feedback, and the model is dynamically adjusted in conjunction with the logical rule module to construct a comprehensive loss function optimization model.
It significantly improves the accuracy, interpretability, and adaptability of recommendation systems, enabling them to dynamically respond to changes in user interests, provide intuitive explanations, and be applicable to various scenarios.
Smart Images
Figure CN120067449B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of personalized recommendation system technology, and in particular to a sequence recommendation method based on the fusion of positive and negative feedback and logical rules. Background Technology
[0002] Recommender systems are crucial tools for addressing information overload and are widely used in e-commerce, video platforms, and social media. Traditional recommendation algorithms often model based on positive feedback data (such as clicks and purchases), while paying insufficient attention to negative feedback data (such as skips and disinterest), making it difficult for models to accurately capture users' true preferences. Furthermore, existing recommendation algorithms are largely data-driven and lack sufficient interpretability, which is particularly critical in fields like healthcare and finance where interpretability and transparency are paramount.
[0003] Logical rules, as an interpretable framework, can intuitively express the constraints between users and projects. However, combining logical rules with deep learning presents the following challenges: (1) Difficulty in integrating logical rules with numerical computation: Logical rules are usually expressed in Boolean form, which is incompatible with the continuous features of deep learning. (2) Insufficient dynamic adaptability: Logical rules are mostly static and difficult to adapt to the dynamic changes in user interests. (3) Limited support for complex scenarios: A single rule is difficult to cope with complex business logic and diverse user behaviors.
[0004] Therefore, how to effectively utilize positive and negative feedback information and integrate logical rules to construct recommendation algorithms that combine accuracy and interpretability has become an important research direction. Summary of the Invention
[0005] The purpose of this invention is to overcome the shortcomings and defects of existing technologies by providing a sequence recommendation method based on the fusion of positive and negative feedback and logical rules. By combining positive and negative feedback information of user behavior with logical rules, the performance, interpretability, and adaptability of the recommendation system are significantly improved.
[0006] A sequence recommendation method based on the fusion of positive and negative feedback and logical rules includes:
[0007] The collected user behavior data is divided into positive feedback and negative feedback, and a feedback sequence is constructed according to the time sequence of user behavior, including positive feedback sequence and negative feedback sequence;
[0008] The user behavior sequence and feedback sequence are processed separately by the input layer to obtain the corresponding embedded features;
[0009] By utilizing the self-attention mechanism of a deep learning model based on Transformer to process the embedded features of the input, the internal dependency features of the original user behavior sequence are captured. Through the cross-attention mechanism, the positive and negative feedback sequences are cross-correlated with the input user behavior sequence to capture the dynamic relationship between user behavior and feedback.
[0010] The output layer maps the model's intermediate features to user preference ratings for target items, ultimately outputting a list of user ratings for all candidate items for recommendation ranking.
[0011] The model's loss function is a comprehensive loss function, which includes Loss1 generated by the deep learning model and Loss2 generated by the logical rule module. By minimizing the comprehensive loss, the deep learning model and the logical rule module can work together to optimize, so that the model can learn the statistical features in the data while following the logical rules. A dynamic weight adjustment mechanism is adopted to dynamically adjust the weights of Loss1 and Loss2, so that the logical rules can adapt to the dynamic changes in user interests.
[0012] The logical rules are automatically mined from historical data using the decision tree method, and the Sigmoid function is used to fuzzify the logical rules, transforming discrete rules into continuous values to adapt to the calculation of deep learning models.
[0013] The logical rules include positive rules for determining whether a preset condition is met, and negative rules for determining whether a condition is not met or the opposite condition is met. The positive and negative rules correspond to specific conditions, and each specific condition corresponds to a weight.
[0014] Positive feedback includes actions such as liking, purchasing, saving, and giving high ratings; negative feedback includes actions such as skipping, not being interested, giving low ratings, or explicitly marking as disliked.
[0015] The process of processing the features of the user behavior sequence and the feedback sequence separately through the input layer to obtain the corresponding embedding features involves processing the features of the user behavior sequence and the feedback sequence separately through independent embedding layers, including:
[0016] The original features of items in the user behavior sequence / feedback sequence are used as the first branch, and the user features, item features and time features are used as the second branch. These are represented by vectors through independent embedding layers to obtain two different implicit feature results. The two different implicit feature results are concatenated column by column and processed by linear transformation to obtain the corresponding implicit features, which are the embedding features to be input into the deep learning model.
[0017] The user characteristics include the user's age, gender, region, and historical preferences; the item characteristics include the product category, price, brand, and popularity.
[0018] The first branch includes a first embedding layer function φ, and the second branch includes a second embedding layer function ψ; the first embedding layer function φ extracts implicit features z. i The expression is as follows:
[0019]
[0020] The second embedding layer function ψ extracts the implicit feature q. i The expression is as follows:
[0021] q i =ψ(i i ,u,t i ) = concat col (i i ,u,t i W ψ +b ψ W ψ ∈R (j+l+k)×g ,b ψ ∈R g ,
[0022] The implicit feature e is obtained by concatenating two different implicit feature results column by column. i The expression is as follows:
[0023] e i =ω(z) i ,q i ) = concat col (z i ,q i W ω +b ω W ω ∈R (g+d)×d ,b ω ∈R d ;
[0024] i i ,u,t i x represents item characteristics, user characteristics, and time characteristics, respectively. i Indicates the original characteristics of the item, W represents the weights and offsets of the first embedding layer function. ψ b ψ W represents the weights and offsets of the first embedding layer function. ω b ω This indicates the weights and offsets of the splicing function.
[0025] The deep learning model based on Transformer includes at least one preprocessing module and an output module. The preprocessing module, from input to output, includes a multi-head attention module, a normalization module, and an FFN feedforward network module. The multi-head attention module models the user behavior sequence data input to the embedding layer. The output of the preprocessing module is normalized and then input to the output module. The output module, from input to output, includes an output cross-attention module, an FFN-OUT network module, and a Sigmoid function. The cross-attention module models the normalized data output from the preprocessing module and the feedback sequence data input to the embedding layer. The Sigmoid function maps the input to between 0 and 1 and calculates the output score. The obtained score, combined with the true value, is used to calculate the loss using the loss function Loss1 to evaluate the model's prediction results and guide the update of model parameters during backpropagation to optimize model performance.
[0026] The multi-head attention mechanism transforms the input sequence through a linear transformation to obtain the query, key, and value. It then calculates the dot product of the query and key, obtains the attention weights through a softmax function, and multiplies them with the values to obtain a weighted output. By computing multiple self-attention mechanisms in parallel, it can capture various dependencies at different positions in the input sequence.
[0027] This invention significantly improves the performance, interpretability, and adaptability of recommendation systems by combining positive and negative feedback information of user behavior with logical rules. Attached Figure Description
[0028] Figure 1 This is a flowchart illustrating the sequence recommendation method based on the fusion of positive and negative feedback and logical rules of the present invention.
[0029] Figure 2 This is a schematic diagram of the overall architecture of the sequence recommendation method based on the fusion of positive and negative feedback and logical rules of the present invention.
[0030] Figure 3 This is a schematic diagram of the architecture of the sequence feature fusion module of the present invention. Detailed Implementation
[0031] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0032] See Figure 1 As shown, a sequence recommendation method based on the fusion of positive and negative feedback and logical rules includes:
[0033] The collected user behavior data is divided into positive feedback and negative feedback, and feedback sequences are constructed based on user behavior and time sequence, including positive feedback sequences and negative feedback sequences.
[0034] The user behavior sequence and feedback sequence are processed separately by the input layer to obtain the corresponding embedded features;
[0035] By utilizing the self-attention mechanism of a deep learning model based on Transformer to process the embedded features of the input, the internal dependency features of the original user behavior sequence are captured. Through the cross-attention mechanism, the positive and negative feedback sequences are cross-correlated with the input user behavior sequence to capture the dynamic relationship between user behavior and feedback.
[0036] The output layer maps the model's intermediate features to user preference ratings for target items, ultimately outputting a list of user ratings for all candidate items for recommendation ranking.
[0037] In this embodiment of the application, the data collection is carried out by collecting user behavior data through log data or third-party platform interfaces, which covers various types of behaviors such as clicks, purchases, skips, and dislikes.
[0038] When classifying feedback, data is divided into positive and negative feedback based on behavioral characteristics. For example, positive feedback includes user clicks, purchases, favorites, and high ratings, while negative feedback includes user skips, disinterest, low ratings, or being explicitly marked as "disliked".
[0039] When constructing time series data, positive and negative feedback are organized into time series according to the time sequence of user behavior to facilitate subsequent modeling.
[0040] In this embodiment of the application, the process of processing the features of the user behavior sequence and the feedback sequence separately through the input layer to obtain the corresponding embedded features involves processing the features of the user behavior sequence and the feedback sequence separately through independent embedding layers, including:
[0041] The original features of items in the user behavior sequence / feedback sequence are used as the first branch, and the user features, item features and time features are used as the second branch. These are represented by vectors through independent embedding layers to obtain two different implicit feature results. The two different implicit feature results are concatenated column by column and processed by linear transformation to obtain the corresponding implicit features, which are the embedding features to be input into the deep learning model.
[0042] In this embodiment of the application, the user features involved in the feature extraction of the sequence through the input layer include the user's age, gender, region, historical preferences, etc., and the item features include product category, price, brand, popularity, etc.
[0043] Specifically, when representing user behavior sequences using embedded vectors, an embedding layer maps discrete user behaviors to a continuous vector space. For positive and negative feedback sequences, separate embedding layers are used to process the sequences respectively.
[0044] The first branch includes a first embedding layer function φ, and the second branch includes a second embedding layer function ψ; the first embedding layer function φ extracts implicit features z. i The expression is as follows:
[0045]
[0046] In the above formula, R I →R d , z i ∈R d ,
[0047] The second embedding layer function ψ extracts the implicit feature q. i The expression is as follows:
[0048] q i =ψ(i i ,u,t i ) = concat col (i i ,u,t i W ψ +b ψ W ψ ∈R (j+l+k)×g ,b ψ ∈R g ,
[0049] In the above formula, R j+l+k →R g ,
[0050] The implicit feature e is obtained by concatenating two different implicit feature results column by column. i The expression is as follows:
[0051] e i =ω(z) i ,q i ) = concat col (z i ,q i W ω +b ω W ω ∈R (g+d)×d ,b ω ∈R d ;
[0052] In the above formula, R g+d →Rd ,
[0053] x i Indicates the original characteristics of an item, i i ,u,t i x represents item characteristics, user characteristics, and time characteristics, respectively. i Indicates the original characteristics of the item, W represents the weights and offsets of the first embedding layer function. ψ b ψ W represents the weights and offsets of the first embedding layer function. ω b ω This indicates the weights and offsets of the splicing function.
[0054] For details, see Figure 3 As shown, i1, i2, ..., i on the left n Represents the original characteristics of an item; the right-hand group contains a. i a u a t The feature groups represent item features, user features, and time features, respectively.
[0055] Original features i1, i2, ..., i of the items on the left n After item embedding, The item, user, and time features on the right undergo another embedding (ψ) process, transforming the high-dimensional, sparse original features into low-dimensional, dense vector representations. The embedded item feature vector is then merged with the item, user, and time feature vectors through a concatenation operation. This concatenated feature vector is then passed to a linear layer (ω) for linear transformation; after the linear transformation, the final embedded features i′1, i′2, ..., i′ are obtained. n These features can be used for subsequent machine learning model training or prediction tasks.
[0056] In this embodiment, the deep learning model built on Transformer includes at least one preprocessing module and an output module. The preprocessing module, from input to output, includes a multi-head attention module, a normalization module, and an FFN feedforward network module. The multi-head attention module models the user behavior sequence data input to the embedding layer. The output of the preprocessing module is normalized and then input to the output module. The output module, from input to output, includes an output cross-attention module, an FFN-OUT network module, and a Sigmoid function. The cross-attention module models the normalized data output from the preprocessing module and the feedback sequence data input to the embedding layer. The Sigmoid function maps the input to between 0 and 1 and calculates the output score. The obtained score, combined with the true value, is used to calculate the loss using the loss function Loss1 to evaluate the model's prediction results and guide the updating of model parameters during backpropagation to optimize model performance.
[0057] The multi-head attention mechanism transforms the input sequence through a linear transformation to obtain the query, key, and value. It then calculates the dot product of the query and key, obtains the attention weights through a softmax function, and multiplies them with the values to obtain a weighted output. By computing multiple self-attention mechanisms in parallel, it can capture various dependencies at different positions in the input sequence.
[0058] In this application, a self-attention mechanism can capture the internal dependencies of the input original behavior sequence and measure the importance of each behavior in the input original behavior sequence. Moreover, the multi-head self-attention mechanism simultaneously focuses on different behavior patterns, extracting richer feature information.
[0059] The specific mathematical expression of the multi-head self-attention mechanism's processing flow is as follows:
[0060]
[0061] In the proposed model, the symbols Q, K, and V are used to represent the query vector set, the key vector set, and the value vector set, respectively. This represents a set of learnable parameter matrices in the model. The notation `concatcol` is used to denote the operation of concatenating vectors column-wise. An adjustment factor is introduced to appropriately regulate the size of the inner product.
[0062] In this embodiment of the application, all training parameters are shared across elements in the feedforward neural network structure. The specific mathematical expression is as follows:
[0063]
[0064] In the model architecture presented in this invention, W (1) W (2) ∈R d×d These represent the weight parameter matrices of the two layers in a two-layer feed-forward network.
[0065] In this embodiment, the output module's cross-attention module models the cross-correlation between the positive and negative feedback sequences and the input sequence to capture the dynamic relationship between user behavior and feedback. A weighted mechanism is used to highlight the portions of the positive and negative feedback that contribute significantly to user preference prediction.
[0066] The detailed calculation process of the cross-attention module is as follows:
[0067]
[0068] σ represents the Sigmoid activation function, W O ∈R d×1 and b O Let R represent the weight parameter matrix and its corresponding bias vector, respectively. Based on the above, a loss function for the negative feedback deep learning module can be constructed to further evaluate and optimize the model's performance.
[0069]
[0070] In this embodiment, the logical rule module uses a decision tree to mine explicit logical rules from historical data, such as "age > 30 and category = electronic products". The Sigmoid function is used to fuzzify the rules, smoothing the rule output values into continuous numerical values, which facilitates integration with deep learning models.
[0071]
[0072] The model uses the sigmoid function, where variable x represents the specific value of a component in the feature vector of an object, parameter s controls the slope of the sigmoid function, determining the steepness of the curve, and parameter v represents the median of the curve, corresponding to the comparison value set by a specific rule on that component. In a single matching, the class with a higher matching degree is considered the more suitable target recommendation result.
[0073] In the embodiments of this application, such as Figure 3As shown, R+ and R- represent positive rules and negative rules, respectively. R+ might be used to determine positive rules that satisfy certain conditions, while R- is used to determine negative rules that do not satisfy or the opposite conditions. w1,wk, wm, w1,wk,wq represent weights. These weights (w1,wk,wm and w1wk,wq) derived from R+ and R- represent the importance of each condition in the rule and are used to weight different conditions during the calculation process. R posconds C represents the set of positive rule conditions. 11 C 1k C 1n These are specific conditions under the positive rule. R neqconds C represents the set of negative rule conditions. 11 C 1k C 1q These are the specific conditions under the negative rule.
[0074] The conversion method from logical computation to numerical computation in this application embodiment is detailed in the table below.
[0075]
[0076] By dynamically adjusting the rule weights, the model can flexibly adapt to the impact of the rules according to changes in the scenario.
[0077] In this embodiment, the output layer maps the intermediate features of the model to the user's preference rating for the target item through a multi-layer fully connected network, and finally outputs a list of user ratings for all candidate items for recommendation ranking.
[0078] Experimental verification
[0079] 1. Experimental Dataset
[0080] E-commerce data: User purchase and browsing history.
[0081] Movie recommendation data: user ratings and viewing history.
[0082] 2. Evaluation Indicators
[0083] Accuracy metrics: such as HR, NDCG.
[0084] Diversity index: assesses the diversity of recommendation results.
[0085] Explanatory metrics: user satisfaction and understanding of the recommendation results.
[0086] 3. Experimental Setup
[0087] The comparison models include traditional collaborative filtering, deep learning-based sequence recommendation models, and models that do not incorporate logical rules. Parameter tuning and ablation experiments were conducted to analyze the impact of the positive and negative feedback ratio and the weights of logical rules in the models.
[0088] 4. Experimental Results
[0089] The model of this invention outperforms the comparative models on all evaluation metrics, particularly demonstrating significant advantages in negative feedback utilization and logical rule fusion. The table below compares the experimental results with mainstream sequence recommendation models. PosNegRec++ is the model proposed by the method of this invention.
[0090]
[0091] The technology of the embodiments of the present invention has the following beneficial effects:
[0092] 1. Improve recommendation accuracy
[0093] The introduction of negative feedback significantly improves the accuracy of user preference modeling and solves the problem of one-sidedness in positive feedback information. Self-attention and cross-attention mechanisms effectively capture key features in user behavior and improve the predictive ability of user interests.
[0094] 2. Enhance system interpretability
[0095] The fusion of logical rules provides an intuitive explanation for the recommendation results, helping to increase users' trust in them. The fuzzy processing of logical rules allows the system to maintain the flexibility of deep learning while ensuring the interpretability of the rules.
[0096] 3. Strong dynamic adaptability
[0097] The model can adjust its recommendation strategy based on dynamic changes in user interests, preventing recommended content from becoming outdated. In the face of sudden shifts in user interests, the system can quickly respond and adjust recommended content.
[0098] 4. Wide applicability
[0099] The model is applicable to various scenarios, such as e-commerce, streaming media, and education platforms, and has high promotional value; it provides flexible interfaces to support integration with various business logics and industry requirements.
[0100] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention.
[0101] Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of the equivalents of the claims be included within the invention.
[0102] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.
Claims
1. A sequence recommendation method based on the fusion of positive and negative feedback and logical rules, characterized in that, include: The collected user behavior data is divided into positive feedback and negative feedback, and a feedback sequence is constructed according to the time sequence of user behavior, including positive feedback sequence and negative feedback sequence; The user behavior sequence and feedback sequence are processed separately by the input layer to obtain the corresponding embedded features; By utilizing the self-attention mechanism of a deep learning model based on Transformer to process the embedded features of the input, the internal dependency features of the original user behavior sequence are captured. Through the cross-attention mechanism, the positive and negative feedback sequences are cross-correlated with the input user behavior sequence to capture the dynamic relationship between user behavior and feedback. The output layer maps the model's intermediate features to user preference ratings for target items, ultimately outputting a list of user ratings for all candidate items for recommendation ranking. The model's loss function is a comprehensive loss function, including the loss generated by the deep learning model. Loss1 and the logic rules module generated Loss2 By minimizing the overall loss, the deep learning model and the logical rule module are optimized collaboratively, enabling the model to learn statistical features from the data while adhering to logical rules. A dynamic weight adjustment mechanism is employed to dynamically adjust... Loss1、 Loss2 The weights allow the logical rules to adapt to the dynamic changes in user interests.
2. The sequence recommendation method based on the fusion of positive and negative feedback and logical rules according to claim 1, characterized in that, The logical rules are automatically mined from historical data using the decision tree method, and the Sigmoid function is used to fuzzify the logical rules, transforming discrete rules into continuous values to adapt to the calculation of deep learning models.
3. The sequence recommendation method based on the fusion of positive and negative feedback and logical rules according to claim 1, characterized in that, The logical rules include positive rules for determining whether a preset condition is met, and negative rules for determining whether a condition is not met or the opposite condition is met. The positive and negative rules correspond to specific conditions, and each specific condition corresponds to a weight.
4. The sequence recommendation method based on the fusion of positive and negative feedback and logical rules according to claim 1, characterized in that, Positive feedback includes actions such as liking, purchasing, saving, and giving high ratings; negative feedback includes actions such as skipping, not being interested, giving low ratings, or explicitly marking as disliked.
5. The sequence recommendation method based on the fusion of positive and negative feedback and logical rules according to claim 1, characterized in that, The process of processing the features of the user behavior sequence and the feedback sequence separately through the input layer to obtain the corresponding embedded features involves processing the features of the user behavior sequence and the feedback sequence separately through independent embedding layers, including: The original features of items in the user behavior sequence / feedback sequence are used as the first branch, and the user features, item features and time features are used as the second branch. These are represented by vectors through independent embedding layers to obtain two different implicit feature results. The two different implicit feature results are concatenated column by column and processed by linear transformation to obtain the corresponding implicit features, which are the embedding features to be input into the deep learning model.
6. The sequence recommendation method based on the fusion of positive and negative feedback and logical rules according to claim 5, characterized in that, The user characteristics include the user's age, gender, region, and historical preferences; the item characteristics include the product category, price, brand, and popularity.
7. The sequence recommendation method based on the fusion of positive and negative feedback and logical rules according to claim 5, characterized in that, The first branch includes the first embedding layer function. The second branch includes the second embedding layer function. The first embedding layer function Extracting implicit features The expression is as follows: ； Second embedding layer function Extracting implicit features The expression is as follows: ， The implicit feature is obtained by concatenating two different implicit feature results column by column. The expression is as follows: ； These represent item characteristics, user characteristics, and time characteristics, respectively. Indicates the original characteristics of the item, This represents the weights and offsets of the first embedding layer function. This represents the weights and offsets of the first embedding layer function. This indicates the weights and offsets of the splicing function.
8. The sequence recommendation method based on the fusion of positive and negative feedback and logical rules according to claim 1, characterized in that, The deep learning model based on Transformer includes at least one preprocessing module and an output module. The preprocessing module, from input to output, includes a multi-head attention module, a normalization module, and an FFN feedforward network module. The multi-head attention module models the user behavior sequence data input to the embedding layer. The output of the preprocessing module is normalized and then input to the output module. The output module, from input to output, includes an output cross-attention module, an FFN-OUT network module, and a Sigmoid function. The cross-attention module models the normalized data output from the preprocessing module and the feedback sequence data input to the embedding layer. The Sigmoid function maps the input to between 0 and 1 and calculates the output score. The obtained scores are combined with the true values and processed by the loss function. Loss1 Calculate the loss, evaluate the quality of the model's predictions, and guide the updating of model parameters during backpropagation to optimize model performance.
9. The sequence recommendation method based on the fusion of positive and negative feedback and logical rules according to claim 1, characterized in that, Deep learning models employ a multi-head attention mechanism, which transforms the input sequence into queries, keys, and values through a linear transformation, calculates the dot product of the queries and keys, obtains attention weights through a softmax function, and then multiplies them with the values to obtain a weighted output. By computing multiple self-attention mechanisms in parallel, it can capture various dependencies at different positions in the input sequence.