Universal interpretable sequence recommendation method and system based on multi-task self-prompting technology

Abstract

The invention discloses a universal interpretable sequence recommendation method and system based on a multi-task self-prompt technology, and belongs to the technical field of sequence recommendation. A user interaction sequence is obtained, and a sequence recommendation module and a self-prompt generator are pre-trained based on the user interaction sequence; constructing a short input sequence based on the latest interaction of the user interaction sequence, inputting the short input sequence into a pre-trained sequence recommendation module to obtain an embedded representation of the short input sequence, inputting the embedded representation into a pre-trained self-prompt generator, extracting user features and generating personalized prompts of the user features; and on the basis of personalized prompts of user characteristics, alternately performing prompt fine adjustment on the pre-trained sequence recommendation module and the pre-trained explanation generation module, predicting a user recommendation result and generating natural language explanation. A flexible and efficient multi-task interpretable sequential recommendation framework ARTS is provided, accurate personalized recommendation is provided for recommendation results, and high-quality natural language interpretation is generated.

Description

Technical Field

[0001] The present invention belongs to the technical field of sequence recommendation, and in particular to a general interpretable sequence recommendation method and system based on multi-task self-prompting technology. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] The rapid development of deep learning technology has significantly enhanced the modeling capabilities of recommender systems, enabling them to provide users with more personalized and accurate recommendations. However, deep recommendation models are often regarded as black-box models, and their recommendation results lack interpretability. In recent years, large language models (LLMs) have attracted widespread attention from academia and industry due to their excellent semantic understanding and language generation capabilities. They have been widely used in various fields, especially natural language processing (NLP) and computer vision (CV). The success of pre-trained LLMs provides a new perspective for achieving high-quality interpretability in recommender systems.

[0004] Existing methods that use LLMs to generate natural language explanations for recommendation results usually focus on converting pre-trained user and item representations into explanatory text. These independent explanation generation models only focus on a single explanation task, ignoring the collaborative relationship between sequential recommendation and explanation generation tasks, and thus fail to achieve knowledge sharing and mutual reinforcement between multi-task modules.

[0005] There are two important challenges in achieving knowledge sharing and mutual reinforcement between the multi-task modules, namely the sequential recommendation and explanation generation tasks: First, it is extremely challenging to jointly learn the sequential recommendation and explanation generation tasks. The goal of the sequential recommendation model is to predict the user's next interactive item based on the user's historical interaction sequence, while the explanation generation task is to generate corresponding natural language explanations based on the target user and the recommended items, explaining why the item is recommended to the target user. The two tasks use different training data and have different task objectives. In addition, sequential recommendation models and LLMs usually have different numbers of parameters, which leads to significant differences in prior knowledge and learning capabilities between modules in the multi-task framework, making collaborative training more difficult.

[0006] Second, sequential recommendation models and LLMs have strong memory capabilities for processing input sequences, which enables them to effectively explore and exploit the interdependencies between items in long sequences. However, their running time and memory consumption are usually affected by the length of the input sequence, and longer input sequences require more time and space resources to process. Therefore, although longer input sequences can improve model performance, they pose a significant challenge to time and space efficiency. Summary of the invention

[0007] To overcome the shortcomings of the above-mentioned prior art, the present invention provides a general interpretable sequence recommendation method and system based on multi-task self-prompt technology, and provides a flexible and efficient multi-task interpretable sequential recommendation framework ARTS. The ARTS framework uses the excellent semantic understanding ability of LLMs to achieve mutual enhancement between sequential recommendation and explanation generation tasks. The ARTS framework effectively enhances user trust and interactive experience by providing accurate personalized recommendations and generating high-quality natural language explanations for recommendation results.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0009] In a first aspect, the present invention provides a general interpretable sequence recommendation method based on multi-task self-prompting technology, comprising:

[0010] Obtaining a user interaction sequence, and pre-training a sequence recommendation module and a self-prompt generator based on the user interaction sequence;

[0011] Constructing a short input sequence based on the latest interaction of the user interaction sequence, inputting the short input sequence into a pre-trained sequence recommendation module to obtain an embedded representation of the short input sequence, inputting the embedded representation into a pre-trained self-prompt generator, extracting user features and generating personalized prompts of the user features;

[0012] Personalized prompts based on user features alternately fine-tune the pre-trained sequence recommendation module and the pre-trained explanation generation module to predict user recommendation results and generate natural language explanations.

[0013] According to a further technical solution, the embedded representation is obtained by a random embedding method.

[0014] A further technical solution is to add an embedding vector in front of the embedding representation of the short input sequence and then input it into the self-prompt generator.

[0015] In a further technical solution, the self-prompt generator introduces a position embedding matrix during pre-training to learn the sequence relationship of user interaction sequences.

[0016] A further technical solution uses contrastive learning to enhance the personalized prompts generated by the prompt generator.

[0017] A further technical solution is to perform prompt fine-tuning on the pre-trained sequence recommendation module, specifically comprising: adding the personalized prompt to the front of the short input sequence to obtain a short input sequence with enhanced prompts, and inputting the short input sequence with enhanced prompts into the pre-trained sequence recommendation module for prompt fine-tuning to obtain an optimized sequence recommendation module.

[0018] A further technical solution is to perform prompt fine-tuning on the pre-trained explanation generation module, specifically including: constructing an input prompt sequence based on item embedding, user prompts and personalized prompts of the sequence recommendation module, inputting the input prompt sequence into the pre-trained explanation generation module for prompt fine-tuning, and obtaining an optimized explanation generation module.

[0019] In a second aspect, the present invention provides a general interpretable sequence recommendation system based on multi-task self-prompting technology, comprising:

[0020] A pre-training module configured to: obtain a user interaction sequence, and pre-train a sequence recommendation module and a self-prompt generator based on the user interaction sequence;

[0021] A prompt generation module is configured to: construct a short input sequence based on the latest interaction of the user interaction sequence, input the short input sequence into a pre-trained sequence recommendation module to obtain an embedded representation of the short input sequence, input the embedded representation into a pre-trained self-prompt generator, extract user features and generate personalized prompts of the user features;

[0022] A prompt fine-tuning module is configured to: alternately fine-tune the pre-trained sequence recommendation module and the pre-trained explanation generation module based on personalized prompts of user features, predict user recommendation results and generate natural language explanations.

[0023] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the general interpretable sequence recommendation method based on multi-task self-prompting technology as described in the first aspect.

[0024] In a fourth aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the general interpretable sequence recommendation method based on multi-task self-prompting technology as described in the first aspect are implemented.

[0025] One or more of the above technical solutions have the following beneficial effects:

[0026] This paper proposes a general and efficient multi-task self-hint framework ARTS for explainable sequential recommendations, which achieves efficient simultaneous generation of precise sequential recommendations and high-quality explanations. The ARTS framework introduces a novel self-hint generator that automatically generates personalized hints with comprehensive semantics using user interaction sequences, thereby facilitating efficient knowledge sharing between multi-task modules. For the first time, hint learning is applied to the work of simultaneously generating recommendations and explanations, successfully resolving the trade-off between performance and time and space efficiency.

[0027] This paper proposes a pre-training and prompt fine-tuning strategy for training multi-task modules, which effectively improves the knowledge sharing efficiency between multiple tasks and realizes effective joint learning and mutual reinforcement between the sequential recommendation and explanation generation modules in the ARTS framework. The short sequence input strategy designed based on the generated personalized prompts significantly improves the time and space efficiency of the multi-task sequential framework.

[0028] The present invention also proposes a short sequence input strategy based on generated personalized prompts and user's recent interactions. This strategy effectively increases the information density of the input sequence and significantly reduces the length of the input sequence. It effectively solves the time and space efficiency issues of the multi-task sequential model while ensuring the performance of the model.

[0029] Extensive experiments on four real-world datasets show that ARTS outperforms the state-of-the-art by 5% to 10% in sequential recommendation accuracy and by 10% in explanation generation performance. In addition, various experiments are designed to verify the effectiveness and efficiency of each module and the versatility of the proposed ARTS, demonstrating that it enhances the performance and time and space efficiency capabilities of sequential recommendation and explanation generation models. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0031] Figure 1 is a flowchart of a general interpretable sequence recommendation method according to an embodiment of the present invention;

[0032] Figure 2 ARTS is a framework diagram of a multi-task self-prompt framework according to an embodiment of the present invention;

[0033] Figure 3 is the comparative experimental result of ARTS in the embodiment of the present invention;

[0034] Figure 4 It is the result of the influence of the prompt length in the input length of the embodiment of the present invention on the model efficiency;

[0035] Figure 5 This is the result of the influence of the sequence data length in the input length on the model efficiency in the embodiment of the present invention. DETAILED DESCRIPTION

[0036] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0037] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0038] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0039] Terminology explanation:

[0040] Explainable recommendations refer to personalized recommendation algorithms that not only provide users with recommendation results, but also provide explanations to clarify why specific items are recommended. These explanations can cover the interpretability of recommendation inputs (such as user models), recommendation processes (such as algorithms), and / or recommendation outputs (such as products).

[0041] Embodiment 1

[0042] See attached Figure 1 As shown, this embodiment discloses a general interpretable sequence recommendation method based on multi-task self-prompting technology, which includes the following steps:

[0043] Obtaining a user interaction sequence, and pre-training a sequence recommendation module and a self-prompt generator based on the user interaction sequence;

[0044] Constructing a short input sequence based on the latest interaction of the user interaction sequence, inputting the short input sequence into a pre-trained sequence recommendation module to obtain an embedded representation of the short input sequence, inputting the embedded representation into a pre-trained self-prompt generator, extracting user features and generating personalized prompts of the user features;

[0045] Personalized prompts based on user features alternately fine-tune the pre-trained sequence recommendation module and the pre-trained explanation generation module to predict user recommendation results and generate natural language explanations.

[0046] The above general interpretable sequential recommendation method is implemented using the proposed ARTS framework, which is a general and efficient multi-task self-prompting framework for interpretable sequential recommendation. The ARTS framework mainly consists of three modules: (1) The sequential recommendation module is responsible for predicting the user's potential next interaction based on the provided user interaction sequence. (2) The explanation generation module is used to synchronously generate natural language explanations of the recommendation results. (3) The self-prompting generator is a module built on the Transformer architecture that aims to extract global user behavior features from user interaction sequences and generate personalized continuous prompts. As the core component of the ARTS framework, the self-prompting generator promotes efficient knowledge sharing among multiple tasks and significantly improves the time and space efficiency of ARTS.

[0047] In order to achieve collaborative optimization and mutual enhancement among multiple tasks, a pre-training and prompt fine-tuning strategy is designed based on the proposed self-prompt generator. Specifically, in the pre-training stage, a sequential recommendation task is designed to fully pre-train the sequential recommendation model and the self-prompt generator using all available sequence data. In the prompt fine-tuning stage, the pre-trained self-prompt generator is used to generate user-personalized prompts that represent the global semantic features of the user. Then the pre-trained sequential recommendation model and the explanation generation module are alternately fine-tuned based on the generated personalized prompts. This strategy can effectively improve the prediction accuracy of sequential recommendation and the quality of explanation generation, while significantly improving the time and space efficiency of the multi-task sequential framework.

[0048] It should be noted that ATRS is a highly general interpretable recommendation framework that is compatible with most existing sequential recommendation models and language generation models.

[0049] 1. Pre-training phase of the ARTS framework

[0050] 1) Pre-training of sequential recommendation module

[0051] In this embodiment, based on the target user's interaction sequence, the next item that the user is most likely to interact with is predicted, and a natural language explanation is generated to intuitively explain why these items are recommended.

[0052] Specifically, the set of users and items are represented as U and V respectively. Each user u∈U has a corresponding comment for each interactive item v∈V, expressed as:

[0053]

[0054] Among them, |E u,v | is the length of the comment, e i Represents a word in a comment.

[0055] Each user has a chronological interaction sequence expressed as:

[0056]

[0057] Among them, |S u | represents the number of interactive items, represents the item that user u interacts with at step t.

[0058] In particular, in order to avoid the problem of gradient vanishing or gradient exploding during model training, the length of the user interaction sequence is normalized to a maximum length l s If the length of the user interaction sequence exceeds the maximum length l s , only the user's most recent (latest) l is retained s interactive items to form a user interaction sequence. Otherwise, fill the sequence.

[0059] ARTS is a general framework that can support the most popular sequential recommendation models as its sequential recommendation module. In this embodiment, in order to ensure the clarity and conciseness of the method description, the well-known and simple SASRec method is used as the sequential recommendation module. In the experimental part, the universality and effectiveness of the framework will be verified by replacing the sequential recommendation module with a more advanced model. In sequential recommendation, SASRec is a pioneering model that integrates the self-attention mechanism into the sequential recommendation algorithm, which can effectively capture the user behavior patterns in the user's historical interaction sequence.

[0060] The sequential recommendation module stacks L layers of Transformer blocks to encode the historical behavior sequence. The corresponding embedding representation of the input sequence is obtained by the random embedding method, and the expression is:

[0061]

[0062] Among them, Z θ is a random initialization function used to convert item IDs into high-level features. represents the random embedding of the input layer sequence, Indicates interactive items The initial representation of , d is the dimension of the item representation.

[0063] The following embedding representation of the lth layer can be obtained The expression is:

[0064]

[0065] in, represents the representation of the t-th behavior of user u at the l-th layer.

[0066] After L layers, the embedding representation from L layers is used To represent the latest preference of the target user, it is represented as u o , and use it to predict the next interactive item of the target user.

[0067] The pre-training optimization objective of the sequential recommendation module can be expressed as:

[0068]

[0069] Among them, S + ={(u,v i )} represents the positive example set, user u and item v i There has been interaction; S - ={(u,v j )} represents the negative example set, which consists of randomly sampled items v that user u has not interacted with. j composition; σ(·) is the sigmoid function.

[0070] 2) Pre-training of the explanation generation module

[0071] The ARTS framework provides a high degree of flexibility in selecting natural language generation models and supports major generative network architectures such as Transformer, BERT, and GPT. With the rapid development of generative artificial intelligence, the GPT series of models have received widespread attention in the fields of education, mathematics, medicine, and physics. Considering that the GPT series of models have been extensively pre-trained on large corpora and have excellent natural language understanding and generation capabilities, and also considering the consumption of computing resources during model fine-tuning, this embodiment selects the pre-trained GPT-2 as the explanation generator. Since GPT-2 has been pre-trained, it will not be further trained during the pre-training stage.

[0072] 3) Pre-training of the self-prompt generator

[0073] Prompt learning is an emerging technique that introduces specific prompt information to guide the learning of the model and has demonstrated success in research fields such as NLP and CV. Traditional prompt learning methods usually use manually constructed discrete prompts to guide the model to perform specific tasks. These prompts may include text excerpts or structured inputs. Although traditional discrete prompt methods have made progress in guiding model learning, they have always faced difficulties in adapting to the specific needs of individual users and generating personalized prompts. This limitation reduces their adaptability in different task scenarios. Unlike discrete prompts, continuous prompts do not rely on manually designed fixed text templates. Instead, they obtain optimized prompt vectors through an automated training process.

[0074] This embodiment proposes a self-prompt generator, the purpose of which is to generate personalized continuous prompts for each user that integrate global features extracted from user interaction sequences. The self-prompt generator automatically supplements and extracts global user behavior features based on the user's historical interaction sequence, and then converts them into personalized prompts. The self-prompt generator effectively improves the efficiency of knowledge sharing between multiple tasks and improves the time and space efficiency of the sequential framework.

[0075] In the implementation of the self-prompt generator, considering the outstanding performance of Transformer in sequence data mining, a self-prompt generator was constructed based on the Transformer encoder structure.

[0076] Specifically, a random embedding method is used to initialize the user interaction sequence S u To the input embedding, the expression is:

[0077] H u =f θ (S u )

[0078]

[0079] Among them, H u is a sequence embedding, represents the representation of the t-th behavior of user u, f θ Represents a random initialization function.

[0080] Similar to the [class] tag of ViT, a learnable cls-token embedding cls∈R is randomly generated d , and add it to the front of the user interaction sequence embedding, which can be expressed as The output of the cls-state from the self-hint generator contains the general features of the user interaction sequence, reflecting the user's global behavior pattern, which is used as user-specific continuous hints for downstream multi-task learning.

[0081] The Transformer-based self-prompt generator architecture is as follows:

[0082] In order to preserve the position information, a learnable position embedding matrix is ​​introduced as additional information, which enables the model to learn the sequential relationship between user interactions. The expression of the position embedding matrix is:

[0083]

[0084] After integrating the position embedding, the resulting feature representation M is expressed as:

[0085]

[0086] The Transformer-based self-hint generator consists of n layers, each of which includes two sublayers, namely multi-head self-attention and position-by-position feedforward network. Multi-head self-attention enables the model to simultaneously focus on information from different positions, across various representation subspaces. Assuming that the multi-head self-attention sublayer contains c attention heads, the calculation of the pth head in the lth layer can be calculated as follows:

[0087]

[0088] Q=MW Q ,K=MW K ,V=MW V

[0089]

[0090] The matrix M is obtained by W Q , W K , W V Convert to MW Q MW K and MW V ,and

[0091] Among them, d k represents the dimension of the pth head, B l,p is the normalized A l,p The result is Z l,p A representation representing the final aggregation.

[0092] Represents the mask matrix, which is used to enable different items to pay attention to each other, thereby capturing the contextual relationship between them. In detail, the autoregressive masking technique is used in the self-attention layer to capture the sequential characteristics of the data. During model training, this strategy ensures that user behavior prediction depends only on the user's historical behavior. By masking future positions and giving them negative infinity values, queries are restricted to only consider the previous key-value pairs and their current positions. Therefore, the lower triangular part of the matrix is ​​0, and the other positions are negative infinity, which means that the current sequence only pays attention to the previous sequence and does not pay attention to the future sequence.

[0093] According to the formula, get the long output value They are then concatenated and projected to obtain a multifaceted representation of the lth layer. The process is as follows:

[0094] Z l =Concat(z 1 ,z 2 ,...,z c )W o

[0095]

[0096] Among them, W O is the mapping matrix.

[0097] In order to stabilize the training process and improve the generalization ability of the model, a normalization layer and a dropout layer are introduced to obtain the embedding representation F of the items in the lth layer. l Representing Z from multiple aspects of input l , the expression is:

[0098] F l =Dropout(LayerNorm(Z l )).

[0099] In the pre-training stage, a sequential recommendation task is designed to fully pre-train the self-prompt generator, so that it can fully learn the sequential pattern of user behavior and transfer the user's global behavior characteristics to the generated personalized prompts. Specifically, the output F of the n-th layer of the self-prompt generator is used t n To represent the user's preference based on a given t item (i.e. ) preference. Then, an MF layer is attached after the self-hint generator to predict the relevance of the candidate item v, expressed as:

[0100]

[0101] N∈R v×d

[0102] Among them, r v,t represents the probability of item v becoming the next item given the previous t items, N is an item embedding matrix, and |v| is the number of items.

[0103] The objective function based on cross entropy can be expressed as:

[0104]

[0105] Among them, S is the set of input sequences, and σ represents the sigmoid function.

[0106] Given a sequence of user interactions It is preceded by a (cls) tag, and the self-prompt generator can generate corresponding sequence outputs based on the user behavior patterns learned in the pre-training phase. Specifically, p contains global feature information of user sequences and can be used as the user's global behavioral preferences. In this paper, p is used as personalized user-specific hints, which plays a vital role in the subsequent multi-task hint fine-tuning and efficiency optimization. The proposed user-specific hints enable the ARTS framework to surpass sequential recommendation methods that use full sequence inputs by only leveraging the user's most recent (latest) interactions, thereby achieving the goal of enhancing recommendation performance while effectively improving the time and space efficiency of the model. Finally, the item embeddings and parameters of the trained sequential recommendation model and self-hint generator are retained for future use.

[0107] 4) Contrastive learning is used to achieve data augmentation of input sequences in both the pre-training stage and the prompt fine-tuning stage of the self-prompt generator.

[0108] Contrastive learning (CL) has demonstrated its effectiveness in recommender systems. Existing CL-based recommendation models usually leverage self-supervisory signals to augment supervised data and improve the robustness and effectiveness of user representation. Inspired by these methodologies, a personalized augmentation method for user-specific cues based on contrastive learning is proposed. Specifically, two data augmentation methods are applied to the interaction sequences. On one hand, random masking is performed to generate an augmented view of the interaction sequence. On the other hand, random noise is introduced to generate another augmented view of the interaction sequence.

[0109] The two enhanced views are defined as follows:

[0110]

[0111] Then, the cls tag is added at the beginning of both sequence views, as follows:

[0112]

[0113] And the labeled sequence view is input into the self-prompt generator, represented as:

[0114]

[0115] in, and is the output of the self-hint generator for the two augmented sequence graphs, p R and p D Enhanced global behavior features, f PG It is a self-prompt generator.

[0116] In model training, the original sequence representation of user (u) and its corresponding augmented counterpart are considered as positive pairs Other users in the same batch The remaining enhancement hint indicates that Negative pair

[0117] To simplify the calculation, the cosine similarity function sim(·,·) is used to measure the similarity between representations. Then the contrastive learning loss function can be defined as follows:

[0118]

[0119] Where N is the training batch and τ is the temperature hyperparameter. A prompt-guided contrastive learning loss is used as an auxiliary task to fully train personalized prompts. Finally, the optimization objective of the self-prompt generator is defined as follows:

[0120] L PG =L PG_predict +γ*L CL

[0121] Among them, γ is a hyperparameter used to control the weight of the contrastive learning task.

[0122] 2. Tips fine-tuning stage of the ARTS framework

[0123] The prompt-tuning method proposed by the ARTS framework is introduced in detail below. In the prompt-tuning stage, three pre-trained modules of the framework are collaboratively prompted-tuned based on personalized user-specific prompts: the sequence recommendation module, the self-prompt generator, and the explanation generator.

[0124] Since the pre-trained self-prompt generator can construct personalized prompts representing the user's global interest characteristics from user sequences, a short sequence input strategy based on the generated personalized prompts is designed. This strategy effectively improves the performance of sequence recommendation and explanation generation, while also significantly improving the time and space efficiency of the entire framework. Specifically, the maximum input sequence length is set to l t ,l t <<l s In the prompt-tuning phase, the user's last (most recent) interaction is selected to construct a short input sequence Then, we use the pre-trained sequence recommendation module to obtain embedding representations of short input sequences:

[0125]

[0126] Among them, Z θ is the pre-trained encoder for the sequence recommendation module, Next, we add the well-trained cls-token to the front of the sequence representation to get the input sequence representation Similar to the pre-training stage, the position information is integrated into the representation of the input sequence, which is then fed into the pre-trained self-hint generator and the output representation is obtained.

[0127] In order to fully utilize the prior knowledge learned by the pre-trained explanation generator and better utilize short input sequences to construct personalized prompts, the contrastive learning design in the pre-training stage is applied to the prompt-tuning of the self-prompt generator. The details are not repeated here.

[0128] For the prompt-tuning of the pre-trained sequence recommendation module, the output p of the self-prompt generator is used as the personalized prompt for the user and added to the front of the input sequence to obtain a short input sequence with enhanced prompts. Then Input into the pre-trained sequence recommendation module for prompt-tune. The objective function and optimization method are the same as those in the pre-training stage and will not be described in detail here.

[0129] Considering the time and space efficiency of running large language models online, as well as the need for highly personalized explanation generation, it is crucial to fully utilize the global user behavior features in the generated personalized prompts p to construct the input prompt sequence, with the goal of providing the most personalized information while keeping the prompt sequence as short as possible. In addition, despite containing a lot of semantic information related to user behavior features, it still cannot fully describe other potential factors that affect user decisions. Therefore, we introduce user and item IDs as auxiliary prompt information into the input prompt sequence of the explanation generator. Intuitively, we can regard user and item IDs as special tag words and add them to the vocabulary of the explanation generator. However, there may be millions or even billions of users and items in a recommender system. It is very difficult and time-consuming to predict an accurate ID word in such a large set of IDs. Therefore, instead of adding IDs to the vocabulary, we treat them as two different additional types of tags. Specifically, in order to effectively transfer item features to the explanation generator, promote the generation of personalized explanations, and facilitate the co-optimization of multiple tasks, we directly use the item embedding i learned by the pre-trained sequence recommendation model. In addition, we randomly embed userID to represent other potential factors considered in the user's decision process and use it as an additional user prompt, denoted as u.

[0130] Then, the input prompt sequence of the self-prompt generator can be represented as [u, p, i]. It is worth noting that the prompt sequence consists of only 3 tokens, but they can provide rich user and item feature details to the explanation generator in the online recommendation stage. Therefore, our framework can significantly improve the time and space efficiency of pre-training large language models while generating high-quality personalized explanations.

[0131] When prompt-tuning the explanation generator, the prompt [u, p, i] is compared to the representation of user u’s comment on item i. Combined, the token representation of the input sequence is constructed as:

[0132] Interpreting the input token representation of the generator involves combining the input sequence with the positional representation Combined, it incorporates the position information for each item in the sequence, represented as X = [x1, x2, ..., x |X| ] The output of the explanation generator is represented as And add a linear layer to convert X o Each mark represents Mapped to a size of On the vector of represents the number of words in the vocabulary. The process can be expressed as:

[0133]

[0134] in and are the projection weight and bias, c i represents the probability distribution of words. Then the objective function based on negative log likelihood can be written as:

[0135]

[0136] in, Indicates that e is generated at step i i Since specific hints are included in the sequence fed to the model, these hints occupy certain positions in the sequence. Therefore, when predicting the probability of a specific word, the index position needs to be adjusted by the length of these hints to properly align the predicted word with the actual word. Therefore, Corresponding to the offsets of the 3 cues, namely u, i and p.

[0137] The goal of this embodiment is to recommend content that the user may be interested in, while providing intuitive and easy-to-understand natural language explanations for the recommendations. Therefore, the ARTS framework includes two task modules: a sequential recommendation module and an explanation generation module. Traditional multi-task learning methods usually merge the loss functions of multiple tasks for joint optimization. However, due to the differences in task objectives and training data, it is challenging to merge the losses of the two modules in the ARTS framework for joint optimization.

[0138] To solve this problem, a multi-task alternating optimization strategy is proposed. This strategy uses the self-hint generator as a shared module and the generated personalized hints as a knowledge sharing channel, allowing the sequential recommendation module and the explanation generation module to perform alternating hint fine-tuning. Specifically, the sequential recommendation module and the self-hint generator are first optimized based on the loss function of the sequential recommendation task:

[0139] L SR_PG =λ*L SR +γ*L CL

[0140] Among them, λ and γ are hyperparameters.

[0141] Next, we optimize the explanation generator and the self-hint generator at the same time. The optimization loss function for this part can be expressed as:

[0142] L EG_PG =(1-λ)*L EG .

[0143] The proposed alternating hint fine-tuning strategy enables the self-hint generator to simultaneously learn and share the latest training knowledge from the sequential recommendation module and the explanation generator. It alleviates the differences in optimization objectives between different tasks, effectively promotes knowledge sharing between multi-task modules, and achieves mutual reinforcement between sequential recommendation and explanation generation tasks.

[0144] The goal of this aspect is to provide accurate sequential recommendations while generating personalized and easy-to-understand natural language explanations for the recommendation results. To this end, a general and efficient multi-task self-prompting framework (ARTS) for interpretable sequential recommendations is proposed, which improves the collaborative performance and time-space efficiency of multi-task modules based on the generated comprehensive semantic prompts. Specifically, a novel self-prompting generator is proposed to generate personalized prompts based on user interaction sequences to effectively express the global interest characteristics of users. Based on these prompt information, a short sequence input strategy is designed, and pre-training and prompt fine-tuning strategies are used to optimize the parameters of the framework, achieving simultaneous improvement in sequential recommendation and explanation generation performance. At the same time, due to the generation of personalized prompts and short sequence input strategies, the present invention also improves the time and space efficiency of the sequential recommendation model and the explanation generation model, effectively solving the trade-off between recommendation performance and time-space efficiency. A large number of experiments verify the effectiveness of the ARTS framework proposed in the present invention in improving recommendation accuracy and explanation generation quality. In addition, experiments are designed to fully verify the versatility of the proposed framework and its effectiveness in improving the time and space efficiency of the sequence model.

[0145] experiment:

[0146] About the data: Experiments are conducted on four real-world datasets collected on two platforms: Amazon and Yelp to comprehensively evaluate the ARTS framework. For all datasets, 10-core filtering is first performed, which means removing unpopular items and low-interaction users with less than 10 interaction records. Then, in order to fine-tune the prompts of the explanation generator, a large language model Alpaca is used to summarize the reviews in the dataset, extract key information from the reviews, and form refined real reviews limited to 20 words. Finally, the dataset is divided into training and test sets with a ratio of 8:2.

[0147] About evaluation metrics: Metrics for two types of tasks were used to evaluate the multi-task performance of ARTS. A traditional full ranking protocol was adopted. This approach offers several advantages, such as comprehensive consideration of all items, alignment with real-world recommendation scenarios, and independence from negative sampling strategies. In order to verify the effectiveness of the method of this embodiment in recommendation tasks, two widely recognized metrics were adopted: hit rate (HR@K) and normalized discounted cumulative gain (NDCG@K). These metrics provide valuable insights into the performance of ARTS in various recommendation scenarios. In addition, in order to evaluate the quality of natural language explanations, mature metrics such as BLEU and ROUGE, which are commonly used in explainable recommendation and text summarization tasks, are used. These metrics are able to comprehensively evaluate the quality of generated explanations.

[0148] About the comparison model: The ARTS framework involves two types of tasks: sequential recommendation and explanation generation. In order to comprehensively evaluate the performance of the framework, two types of methods, namely sequential recommendation and generation-based explainable recommendation, are selected as the baseline for comparison. Sequential recommendation methods include SASRec, ICLRec, FMLPRec, and MELTRec, and generation-based explainable recommendation methods include Att2Seq, NRT, PETER, and PEPLER.

[0149] Regarding the experimental results: Figure 2 As shown in the figure, by observing the experimental results, it is found that the proposed ARTS significantly outperforms the most advanced sequential recommendation methods on each dataset. In terms of NDCG and HR, ARTS improves by at least 5% on all datasets. The performance improvement is attributed to the user-specific prompts generated by the self-prompt generator, which contain rich semantic details and effectively reflect the user's global interest preferences. Since the pre-trained prompt generator contains prior knowledge related to user interests, the generated prompts can provide rich user preference information for the sequential recommendation model. This enables the sequential recommendation model to effectively predict the user's next action based only on the user's recent (latest) interactions.

[0150] ARTS also significantly outperforms the state-of-the-art generation-based explainable recommendation methods, with at least 6% improvement on all datasets, attributed to the rich semantic knowledge contained in the continuous prompts generated by the self-prompt generator and the alternating optimization strategy we designed. To improve the online time and space efficiency of the explanation generator, the input prompt sequence contains only 3 tokens. In this setting, the performance improvement of explanation generation fully demonstrates the effectiveness of our learned continuous prompts.

[0151] We attribute the performance improvement to our multi-task alternating optimization strategy. This strategy treats the self-hint generator as a shared module and the personalized hints it generates as a knowledge sharing channel. This enables the backbone sequential recommendation model and explanation generation model to be fine-tuned with hints alternately. This effectively bridges the gap between the two different tasks and improves the performance of explanation generation.

[0152] By comparing the performance of SASRec with ARTSS, and ICLRec with ARTSI, it is clear that the ARTS framework can effectively improve the recommendation performance of the sequential recommendation module. For explanation generation, ATRS has a significant performance improvement over PEPLER in all indicators. PEPLER also uses GPT-2 to generate natural language explanations, which verifies the effectiveness of the prompt fine-tuning strategy based on the proposed self-prompt generator in the explanation generation task. In addition, the experimental results also verify the versatility and scalability of the proposed framework.

[0153] For the explanation generation task, relying solely on quantitative evaluation metrics cannot fully evaluate the quality of the generated explanation language. Therefore, we provide examples of explanation generation for the baseline model and the proposed ATRS. Qualitative evaluation of the explanations generated by ATRS shows that it outperforms other baselines in terms of accuracy, readability, and detail richness.

[0154] In terms of time and space efficiency, for time complexity, both SASRec and GPT-2 use a self-attention mechanism to calculate the correlation between all items in the sequence at each time step. Therefore, their time complexity is proportional to the sequence length n and the size d of each mini-batch, usually O(n 2 *d). For space complexity, both models have two main factors: the embedding matrix and the self-attention weight matrix. First, they both need an embedding matrix to store the representation of each item or token. The size of the embedding matrix is ​​usually O(n*d). Second, since they both use the self-attention mechanism, the space occupied by the self-attention weight matrix is ​​O(n 2 ). Therefore, the overall space complexity of SASRec and GPT-2 is the embedding matrix size O(n*d) and the self-attention weight matrix size O(n 2 ), that is, O(n 2+n*d).

[0155] In actual recommendation scenarios, existing sequential recommendation methods usually use longer user behavior sequences so that the model can fully capture user behavior preferences and improve recommendation performance. At the same time, explanation generation models also require detailed long sequence prompt information as input to generate natural language explanations that meet user needs. However, based on the theoretical analysis of time and space complexity above, it is clear that the time and space efficiency of sequential recommendation models and explanation generation models are sensitive to the length of the input sequence.

[0156] In order to verify the impact of input length on model efficiency, a series of experiments were conducted on the real-world Tampa dataset and a virtual dataset. In the Tampa dataset, the length of user interaction sequences varies greatly, providing an effective method to simulate real-world recommendation scenarios. At the same time, the synthetic dataset generates user interaction sequences of equal length through a random generation method to verify the ideal case of improved time and space efficiency. Figure 3 , Figure 4 In the experimental results, it is noted that the time and memory costs of the sequential recommendation model and the explanation generation model have increased significantly. Therefore, in the actual recommendation scenario, the increase in the length of the input sequence seriously affects the time and space efficiency of the model. In this embodiment, the ATRS framework is proposed, which uses a self-prompt generator to construct an effective short sequence input strategy, achieves a simultaneous improvement in model performance and efficiency, and effectively solves the above problems.

[0157] Embodiment 2

[0158] This embodiment provides a general interpretable sequence recommendation system based on multi-task self-prompt technology, including:

[0159] A pre-training module configured to: obtain a user interaction sequence, and pre-train a sequence recommendation module and a self-prompt generator based on the user interaction sequence;

[0160] A prompt generation module is configured to: construct a short input sequence based on the latest interaction of the user interaction sequence, input the short input sequence into a pre-trained sequence recommendation module to obtain an embedded representation of the short input sequence, input the embedded representation into a pre-trained self-prompt generator, extract user features and generate personalized prompts of the user features;

[0161] A prompt fine-tuning module is configured to: alternately fine-tune the pre-trained sequence recommendation module and the pre-trained explanation generation module based on personalized prompts of user features, predict user recommendation results and generate natural language explanations.

[0162] Embodiment 3

[0163] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of embodiment 1 when executing the program.

[0164] Embodiment 4

[0165] The purpose of this embodiment is to provide a computer-readable storage medium, a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of the method of embodiment 1 are performed.

[0166] The steps involved in the apparatus of the above embodiments 3 and 4 correspond to the method embodiment 1, and the specific implementation method can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0167] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0168] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0169] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A general interpretable sequence recommendation method based on multi-task self-prompting technology, characterized in that: include: Obtaining a user interaction sequence, and pre-training a sequence recommendation module and a self-prompt generator based on the user interaction sequence; Constructing a short input sequence based on the latest interaction of the user interaction sequence, inputting the short input sequence into a pre-trained sequence recommendation module to obtain an embedded representation of the short input sequence, inputting the embedded representation into a pre-trained self-prompt generator, extracting user features and generating personalized prompts of the user features; Personalized prompts based on user features alternately fine-tune the pre-trained sequence recommendation module and the pre-trained explanation generation module to predict user recommendation results and generate natural language explanations.

2. The general interpretable sequence recommendation method based on multi-task self-prompting technology according to claim 1, characterized in that: The embedded representation is obtained by a random embedding method.

3. The general interpretable sequence recommendation method based on multi-task self-prompting technology according to claim 1, characterized in that: The embedding vector is added in front of the embedding representation of the short input sequence and then fed into the self-hint generator.

4. The general interpretable sequence recommendation method based on multi-task self-prompting technology according to claim 1, characterized in that: The self-prompt generator introduces a position embedding matrix during pre-training to learn the sequence relationship of user interaction sequences.

5. The general interpretable sequence recommendation method based on multi-task self-prompting technology according to claim 1, characterized in that: Enhance personalized prompts generated by self-prompt generator using contrastive learning.

6. The general interpretable sequence recommendation method based on multi-task self-prompting technology according to claim 1, characterized in that: The prompt fine-tuning of the pre-trained sequence recommendation module specifically includes: adding the personalized prompt to the front of the short input sequence to obtain a short input sequence with enhanced prompts, and inputting the short input sequence with enhanced prompts into the pre-trained sequence recommendation module for prompt fine-tuning to obtain an optimized sequence recommendation module.

7. The general interpretable sequence recommendation method based on multi-task self-prompting technology according to claim 1, characterized in that: The prompt fine-tuning of the pre-trained explanation generation module specifically includes: constructing an input prompt sequence based on the item embedding, user prompts and personalized prompts of the sequence recommendation module, inputting the input prompt sequence into the pre-trained explanation generation module for prompt fine-tuning, and obtaining an optimized explanation generation module.

8. A general interpretable sequence recommendation system based on multi-task self-prompting technology, characterized by: include: A pre-training module configured to: obtain a user interaction sequence, and pre-train a sequence recommendation module and a self-prompt generator based on the user interaction sequence; A prompt generation module is configured to: construct a short input sequence based on the latest interaction of the user interaction sequence, input the short input sequence into a pre-trained sequence recommendation module to obtain an embedded representation of the short input sequence, input the embedded representation into a pre-trained self-prompt generator, extract user features and generate personalized prompts of the user features; A prompt fine-tuning module is configured to: alternately fine-tune the pre-trained sequence recommendation module and the pre-trained explanation generation module based on personalized prompts of user features, predict user recommendation results and generate natural language explanations.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the general interpretable sequence recommendation method based on multi-task self-prompt technology as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the general interpretable sequence recommendation method based on multi-task self-prompt technology are implemented as described in any one of claims 1 to 7.

Images