A personalized search and explanation generation method based on unified hint perception

Abstract

The application realizes a personalized search and explanation generation method based on unified prompt perception through a method in the field of artificial intelligence. The user, the submitted query and the user's behavior sequence are taken as input, the unified personalized product search and explanation generation prompt perception framework is input, the search task and the explanation task are output, and the unified personalized product search and explanation generation prompt perception framework is composed of three parts: a base pre-training language model, a personalized retrieval component and an explanation generation component. The application proposes a unified training framework for personalized product search and explanation generation based on prompt perception, and designs specific prompts for each task in a unified manner, and the model finally outputs a natural language form of explanation while outputting the target product.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a personalized search and explanation generation method based on unified prompt perception. Background Technology

[0002] Product search systems are widely used on various online platforms and services, allowing users to access products they are potentially interested in. With the rapid development of online shopping, efficient and accurate product search methods have become increasingly crucial. In a typical product search scenario, users submit queries to a product search engine and then receive a list of products sorted according to their purchase probability. By improving the accuracy of product search algorithms, users can find the products they need more quickly and easily, thereby helping to increase the platform's revenue. Past algorithms mainly included non-personalized algorithms, personalized product search algorithms, and interpretable product search algorithms.

[0003] In the past, non-personalized algorithms typically returned the same search results to all users for the same query. Because different users have diverse interests, researchers have focused on personalized product search methods in recent years. These methods utilize users' search history to model their interests and preferences, making the search results more aligned with their needs. Some early personalized models used language models to model user reviews to obtain user interests and preferences. With the advent of transformers, more models have adopted this architecture to sequentially model user history and extract purchase preferences. To address the issue of unexplained search results, researchers have proposed explainable product search models. Existing explainable product search models construct knowledge graphs, using the knowledge representations of user and product nodes to calculate purchase probabilities and rank products. They also construct the path from the user to the product as an explanation of the search results. A small number of works also receive search results from black-box search models and build specialized models to generate explanations.

[0004] However, while these personalized methods have proven to significantly improve the accuracy of search results, they suffer from the problem of failing to explain to users why these products were retrieved. The lack of reliable explanations for search results can confuse users and further impact their trust in e-commerce platforms. Existing models extract multi-hop paths between users and products as explanations, but these path-based explanations can be difficult for users to understand and read, thus creating a comprehension barrier. Using predefined templates to translate paths into sentence explanations may result in information loss, and the high similarity between template explanations makes it difficult for users to differentiate between them. Summary of the Invention

[0005] To this end, this invention first proposes a personalized search and explanation generation method based on unified prompt awareness, which integrates user u, submitted query q, and user behavior sequence H = {q1, i1, ..., q}. k i k As input, a unified personalized product search and explanation generation prompt perception framework is input, and the output is a search task and an explanation task. The search task is to select the most suitable product i that meets the user's search intent under a given query q, and the explanation task is to generate a reasonable natural language explanation E for the previously generated product i that meets the user's search intent.

[0006] The unified personalized product search and explanation generation prompt-aware framework consists of three parts: a base pre-trained language model, a personalized retrieval component, and an explanation generation component. The base pre-trained language model completes both search and explanation tasks within a unified framework. The personalized retrieval component and the explanation generation component each include a prompt generator to generate task-specific prompts to assist the pre-trained language model in retrieving suitable products or generating reliable explanations. The search semantic signal from the personalized retrieval component is used as part of the explanation task input to narrow the semantic gap between search results and generated explanations. f(·|θ) is used... PLM G) represents a θ PLM The process of generating a contextualized representation for each tag using a base pre-trained language model with the tag sequence G as input, taking the tag as parameters.

[0007] The personalized search component consists of an encoder and a prompt generator representing the search task. A prompt generator specific to the search task is developed to assist the base model in retrieval. The prompt generator for the search task takes user u, current query q, and user's historical search sequence H as input and generates prompts for the search task as input to the base model, which are then used for prompt learning.

[0008] The goal of the encoder is to obtain a high-level semantic representation of u, q, and H. For a text-based encoder, BERT is directly used to encode text information such as user comments and product titles / descriptions from the input sequence {u, q, H} to obtain a text-based semantic representation.

[0009]

[0010] Where TextEncoder(·) is BERT, T(·) represents the input text sequence, that is, T(u) is the user review text of the purchased product, and T(i) is the product review text received from the user. These are the words in the query; [;] is a concatenation operation. It is the [CLS] representation of user u, query q, and product i output by the encoder.

[0011] For a knowledge-based encoder, high-order connectivity information of users, queries, and products is extracted as a knowledge representation. The knowledge graph-based product search method DREM is used as the knowledge-based encoder, and it is defined as follows:

[0012]

[0013] Where KnowledgeEncoder(·) is the DREM model. It is the representation of user u, query q, and product i in the knowledge graph.

[0014] The prompt generator for the search task uses the text-based and knowledge-based representations obtained from the two encoders mentioned above as input to generate a series of continuous search task-specific vectors, which are then used as input to the subsequent PLM to make them suitable for the search task. In the design of the search prompt, the prompt generator fuses the representations of the text-based and knowledge-based input sequence {u, q, H} to generate user preferences. Specifically, it includes two steps: first, using the fusion mechanism of a gate network, an interaction mechanism is learned to adaptively fuse the two semantic representations; then, the fused representation is converted into a task-specific prompt, and the prompt is obtained by transforming the fused representation through learning a multi-layer MLP.

[0015] For an input sequence {u, q, H}, the text-based and knowledge-based representation sequences can be described as follows:

[0016]

[0017]

[0018] The fusion and interaction mechanism of gate networks can be represented as:

[0019] h text =tanh(W k S text ),

[0020] h KG =tanh(W t S KG ),

[0021] α=σ(W f [h KG h text ]),

[0022] h fuse =α⊙h KG +(1-α)⊙h text ,

[0023] α is an adaptive parameter that controls the ratio of knowledge representation to text representation in the fused representation;

[0024] A transformation is performed after the fusion step to convert the fused representation into a continuous cue, and a multilayer perceptron is used in the transformation step:

[0025] P sch =ReLU(W n (...ReLU(W1·h fuse +b1)...)+b n ),

[0026] in It is the hint vector for the generated search task.

[0027] The prompt learning process integrates the fused representation of the user and the query with the prompt P for the search task. sch As input for prompting learning:

[0028]

[0029] in and It is the direct input of PLM. The search task hints Psch are added as prefix tokens to each layer of the base pre-trained language model using a prefix-tuning method.

[0030] Using cross-entropy loss as the objective function:

[0031]

[0032] The explanation generation component consists of a prompt generator for the explanation task and semantic signals from the search component;

[0033] The input to the explanation component includes u, q, i, and H, which are encoded using the same encoder as the search component to obtain text-based and knowledge-based representations of the input. These representations are then used as input to the prompt generator for the explanation task, and the same methods as the search component are used to fuse and transform these representations into prompts for the explanation task.

[0034] P exp =ReLU(W n′ ′(...ReLU(W1′·h fuse +b1′)...)+b n ′)

[0035] in It is the generated hint vector for the explanation task. The hint generator for the explanation task has the same architecture as the hint generator for the search task, but with different parameters.

[0036] The fused representation of users, queries, and products is used as part of the input, employing the last hidden state vector h of the search component. s As part of the input to the base pre-trained language model in the explanation component:

[0037]

[0038] To interpret the generated components, it is necessary to learn the generated hints P. exp and fusion representation and Represent them as θ exp The output vector of the last layer of the pre-trained language model is used and processed through a linear layer to create a probability distribution containing all words in the dataset. This distribution is then used to predict the next word by minimizing the negative log-likelihood.

[0039]

[0040] The technical effects to be achieved by this invention are as follows:

[0041] (1) A unified training framework based on prompt awareness is proposed for personalized product search and explanation generation. Specific prompts are designed for each task in a unified manner. The model outputs the target product and an explanation in natural language form at the same time.

[0042] (2) By adaptively fusing text and knowledge information, a task-specific prompt generator is designed to generate specific prompt inputs for the base pre-trained language model.

[0043] (3) Incorporating search results as part of the prompts for the explanation generation component can facilitate explanation generation and bridge the gap between the two tasks. Attached Figure Description

[0044] Figure 1 A personalized search and explanation generation method architecture based on unified prompt awareness; Detailed Implementation

[0045] The following are preferred embodiments of the present invention, which are described in conjunction with the accompanying drawings. However, the present invention is not limited to these embodiments.

[0046] This invention proposes a personalized search and explanation generation method based on unified prompt awareness. Its core is a unified prompt awareness framework, P-PEG, for personalized product search and explanation generation, which implements search and explanation generation tasks in a consistent manner. The architecture of P-PEG is as follows: Figure 1As shown, the model mainly consists of three parts: a base pre-trained language model, a personalized retrieval component, and an explanation generation component. The base pre-trained language model is the foundation of the entire model, enabling search and explanation tasks to be completed within a unified framework. Both the search and explanation components include a prompt generator, which generates task-specific prompts to assist the pre-trained language model in retrieving suitable products or generating reliable explanations. Furthermore, to enhance the connection between the two tasks, the search semantic signal from the search component is used as part of the input to the explanation task, thus narrowing the semantic gap between search results and generated explanations. Based on this, the model can complete search and explanation tasks in a unified manner, obtaining both accurate search results and credible explanations.

[0047] P-PEG aims to use a pre-trained language model to find suitable products and generate explanations. To better extract user search intent and preferences, P-PEG combines user u, submitted query q, and user behavior sequence H = {q1, i1, ..., q}. k i k {As input, perform the following tasks: (1) The search task is to select the most suitable product i that satisfies the user's search intent under a given query q; (2) The interpretation task is to generate a reasonable natural language interpretation E for the previously generated product i that satisfies the user's search intent.

[0048] Base pre-trained language model:

[0049] We chose GPT-2 as our base pre-trained language model. GPT-2 is a Transformer-based model pre-trained on a large-scale corpus. GPT-2 can generate coherent and context-sensitive text, making it suitable for various natural language processing tasks, such as summarization and question answering. Therefore, GPT-2 has been widely used in interpretable recommendation to generate reasonable and coherent explanations. This invention uses f(·|θ) PLM G) represents a θ PLM The process involves a pedestal-pretrained language model, with a sequence of tags G as input, generating a contextualized representation for each tag, using G as parameters. In our task, G is a cue-based learned input obtained from the search and interpretation components. We use the contextualized representation of the last tag to complete the search and interpretation generation tasks.

[0050] Personalized search component:

[0051] The core of personalized retrieval is the development of a search task-specific suggestion generator to assist the base model in retrieval. The search task suggestion generator takes user u, current query q, and user's historical search sequence H as input, and generates suggestions for the search task, which are then used as input to the base model. To enrich the suggestion information, this component integrates text-based and knowledge-based encoders to generate textual and knowledge-based information about u, q, and H. These suggestions are then used as input to the base model, enabling the retrieval of appropriate products based on the user's current query.

[0052] Encoder representation:

[0053] The goal of the encoder is to obtain high-level semantic representations of u, q, and H, which can be fed into the cue generator to produce high-quality cue messages. For the text-based encoder, we directly use BERT to encode textual information such as user reviews and product titles / descriptions from the input sequence {u, q, H} to obtain text-based semantic representations.

[0054]

[0055] Where TextEncoder(·) is BERT, T(·) represents the input text sequence, that is, T(u) is the user review text of the product he purchased, and r(i) is the product review text I received from the user. These are the words in the query; [;] is a concatenation operation. It is the [CLS] representation of user u, query q, and product i output by the encoder.

[0056] For the knowledge-based encoder, we consider the structural characteristics of the user-product knowledge graph and extract high-order connectivity information of users, queries, and products as knowledge representation. We utilize the knowledge graph-based product search method DREM as our knowledge-based encoder, and define it as:

[0057]

[0058] Where KnowledgeEncoder(·) is the DREM model. It is the representation of user u, query q, and product i in the knowledge graph.

[0059] Search task suggestion generator:

[0060] The prompt generator for the search task mainly uses the text-based and knowledge-based representations obtained from the two encoders mentioned above as input to generate a series of continuous search task-specific vectors, which are then used as input to the subsequent PLM to make it suitable for the search task. Therefore, for different tasks, only one copy of the PLM needs to be stored and task-specific prompt vectors need to be learned to optimize different tasks. In the design of the search prompt, the prompt generator integrates the representations of the text-based and knowledge-based input sequence {u, q, H} to generate user preferences. Specifically, it includes two steps: (1) using the fusion mechanism of a gate network, learning an interaction mechanism to adaptively fuse the two semantic representations, and (2) converting the fused representation into a task-specific prompt by learning a multi-layer MLP to transform the fused representation to obtain the prompt. For the input sequence {u, q, H}, the text-based and knowledge-based representation sequences can be described as:

[0061]

[0062]

[0063] The fusion and interaction mechanism of gate networks can be represented as:

[0064] h text =tanh(W k S text ),

[0065] h KG =tanh(W t S KG ),

[0066] α=σ(W f [h KG h text ]),

[0067] h fuse =α⊙h KG +(1-α)⊙h text ,

[0068] As can be seen from this formula, α is an adaptive parameter that controls the ratio of knowledge representation to text representation in the fused representation.

[0069] Following the fusion step, we perform a transformation that converts the fused representation into l consecutive cues. Here, we use a Multilayer Perceptron (MLP) in this transformation step:

[0070] P sch =ReLU(W n (...ReLU(W1·h fuse +b1)...)+b n ),

[0071] in It is the hint vector for the generated l search tasks.

[0072] Learning Tips:

[0073] The fusion representation of users and queries and the hints for the search task P sch As input for prompting learning:

[0074]

[0075] in and It's a direct input to PLM, a search for task prompts. sch Prefix-tuning is used as a prefix marker added to each layer of GPT-2, thereby enabling better tuning without changing the base model parameters.

[0076] In the search task component above, it is only necessary to learn the generated hint P. sch and fusion representation and We denote them as θ sch In order to learn θ sch We use cross-entropy loss as the objective function:

[0077]

[0078] Explanation of generated components:

[0079] The explanation generation component primarily provides users with acceptable natural language explanations based on relevant user information and query content. Its architecture mainly consists of a suggestion generator for the explanation task and semantic signals from the search component. The suggestion generator for the explanation task is similar to that for the search task.

[0080] The inputs to the explanation component include u, q, i, and H. We encode them using the same encoder as the search component, obtaining text-based and knowledge-based representations of the input. We then use these representations as input to the hint generator for the explanation task, and fuse and transform these representations into hints for the explanation task using the same method as the search component.

[0081] P exp =ReLU(W n′ ′(...ReLU(W1′·h fuse +b1′)...)+b n ′)

[0082] in These are the generated l′ hint vectors for the explanation task. The hint generator for the explanation task has the same architecture as the hint generator for the search task, but with different parameters.

[0083] Since the purpose of the explanation task is to generate explanations for the retrieved products for the user, we use a fused representation of the user, query, and product as part of the input. Furthermore, to bridge the semantic gap between the search and explanation tasks and to effectively utilize information from the search task, we use the last hidden state vector h of the search component (representing the user's search intent and preferences). s As part of the GPT-2 input in the interpretation component:

[0084]

[0085] For interpreting generated components, it is only necessary to learn the generated hints P. exp and fusion representation and We denote them as θ exp In order to learn θ exp We use the output vector of the last layer of GPT-2 and process it through a linear layer to create a probability distribution containing all words in the dataset. This distribution is then used to predict the next word, which can be achieved by minimizing the negative log-likelihood.

[0086]

[0087] Training and optimization:

[0088] The parameters of P-PEG consist of two task-specific cue generators and a base pre-trained language model, denoted as θ. sch θ exp and θ PLM In our training process, the parameters of our pre-trained language model are fixed, and we perform pipelined training for two other tasks. (1) First, we use a pre-trained text-based encoder and a knowledge-based encoder to encode the input information of users, queries, items, and historical behavior sequences to obtain two types of representations. (2) We generate prompts for the search task and a fusion representation of users and queries, and input them into the pre-trained language model, and process the parameters θ in the search task. sch Training is performed. (3) Finally, prompts for the explanation task and a fused representation of the user, query, and item are generated and coupled with the search semantic signal vector h. s Together, they are input into the pre-trained language model to train the parameters θ. exp .

[0089] Experimental verification:

[0090] To verify the effectiveness of the model, we constructed several experiments. We chose the Amazon dataset for the experiments, and specifically selected three subsets: Electronics, CDs, and...

[0091] Vinyl, and Cell Phones & Accessories. To better evaluate the experimental results, for the search task, we selected MAP, MRR, and NDCG@10 as evaluation metrics. For the interpretable generation task, we adopted the same evaluation method as the interpretable recommendation works PETER and PEPLER, selecting BLUE-1, BLUE-4, ROUGE-1, and ROUGE-2 as metrics to evaluate the quality of the generated text, and USR, FMR, FCR, and DIV as metrics to evaluate the interpretability of the text.

[0092] We selected several benchmark models for performance comparison. For the search task, we chose two non-personalized models, BM25 and LSE, and several personalized product search models, HEM, DREM, DREM-HGN, and CAMI. For the explanation and generation task, we selected the recurrent neural network-based methods NRT and Att2Seq, the Transformer-based methods Transformer and PETER, and the prompt-based method PEPLER.

[0093] The experimental results for the search task and the explanation generation task are shown in the table below.

[0094] Search task:

[0095]

[0096] Explanation of the generated task:

[0097]

[0098] The experimental results from the search task show that: (1) Our model outperforms existing personalized and non-personalized models, with NDCG@10 showing relative performance improvements of at least 13.4%, 4.1%, and 14.7% on the three datasets, respectively. This is mainly because our search suggestion generator can capture accurate user interests by fusing text-based and knowledge-based representations. (2) Our model outperforms graph-based methods (DREM, DREM-HGN, CAMI). This indicates that our adaptive semantic fusion representation can capture users' personalized search preferences and further retrieve suitable products for users.

[0099] The results from the interpretation generation task show that: (1) Our model significantly outperforms the five baseline models in text quality metrics on the three datasets because we utilize search information to facilitate interpretation generation, making the generated sentences more consistent with the search intent. (2) Our model performs better than Transformer-based and RNN-based methods. This is because GPT-2 has stronger generation capabilities than RNN and Transformer, generating more accurate and consistent sentences, and the prompt generator can help the model generate better interpretations through prompts. (3) Our model outperforms the prompt-based method PEPLER because we utilize search information to bridge the gap between the two tasks and facilitate information interaction. (4) Our model achieves the best results on almost all interpretability metrics on the three datasets. This is because we utilize interpretation task prompts and search information to help GPT-2 generate diverse and accurate sentences. (5) Compared to RNN-based models, our model utilizes fast information that fuses text-based and knowledge-based representations, effectively assisting GPT-2 in generating text, thus achieving better results.

Claims

1. A personalized search and explanation generation method based on unified prompt awareness, characterized in that: User Submitted query and user behavior sequence As input, a unified personalized product search and explanation prompt awareness framework is used; the output is the results of the search task and explanation task, whereby the search task is based on a given query. Choose the most suitable product that meets the user's search intent. The interpretation task is to interpret the previously generated product that meets the user's search intent. Generate reasonable natural language interpretations ; The unified personalized product search and explanation generation prompt-aware framework consists of three parts: a base pre-trained language model, a personalized retrieval component, and an explanation generation component. The base pre-trained language model completes both search and explanation tasks within a unified framework. The personalized retrieval component and the explanation generation component each include a prompt generator to generate task-specific prompts to assist the pre-trained language model in retrieving suitable products or generating reliable explanations. The search semantic signal from the personalized retrieval component is used as part of the explanation task input to narrow the semantic gap between search results and generated explanations. Indicates a As parameters, to label the sequence The process of generating contextualized representations for each token using a pre-trained language model based on the input base; The personalized search component consists of an encoder and a prompt generator representing the search task. It assists the base model in retrieval by developing a search task-specific prompt generator. The search task-specific prompt generator is user-defined. Current query and user search history The input is used to generate clues for the search task, which are then used as input to the base model and subsequently used for clue learning. The explanation generation component consists of a prompt generator for the explanation task and semantic signals from the search component; The input to the interpreter component includes The input is encoded using the same encoder as the search component to obtain text-based and knowledge-based representations, which are then used as input to the prompt generator for the explanation task. These representations are then fused and transformed into prompts for the explanation task using the same method as the search component. in It is the generated hint vector for the explanation task. The hint generator for the explanation task has the same architecture as the hint generator for the search task, but with different parameters. The fused representation of users, queries, and products is used as part of the input, employing the last hidden state vector of the search component. As part of the input to the base pre-trained language model in the explanation component: To interpret generated components, you need to learn the generated hints. and fusion representation , and Represent them as The output vector of the last layer of the pre-trained language model is used and processed through a linear layer to create a probability distribution containing all words in the dataset. This distribution is then used to predict the next word by minimizing the negative log-likelihood. 。 2. The personalized search and explanation generation method based on unified prompt awareness as described in claim 1, characterized in that: The goal of the encoder is to obtain For text-based encoders, BERT provides high-level semantic representations of the input sequence directly. User reviews and product titles / descriptions are encoded to obtain a text-based semantic representation: in It's BERT. This represents the input text sequence, i.e. It is text of user reviews from those who purchased the product. It's from the product review text received by the user. These are the words in the query; [;] represents the concatenation operation. User Query and goods The [CLS] representation is output by the encoder.

3. The personalized search and explanation generation method based on unified prompt awareness as described in claim 2, characterized in that: For a knowledge-based encoder, high-order connectivity information of users, queries, and products is extracted as a knowledge representation. The knowledge graph-based product search method DREM is used as the knowledge-based encoder, and it is defined as follows: in It is a DREM model. User Query and goods The representation of in a knowledge graph.

4. The personalized search and explanation generation method based on unified prompt awareness as described in claim 3, characterized in that: The prompt generator for the search task uses the text-based and knowledge-based representations obtained from the two encoders mentioned above as input to generate a series of consecutive search task-specific vectors, which are then used as input to the subsequent PLM to adapt them to the search task. In the design of the search prompts, the prompt generator integrates the text-based and knowledge-based input sequences. The method of generating user preferences involves two steps: First, using a gate network fusion mechanism, an interaction mechanism is learned to adaptively fuse the two semantic representations. Then, the fused representation is converted into task-specific cues, and cues are obtained by learning a multi-layer MLP to transform the fused representation. For the input sequence Text-based and knowledge-based representation sequences can be described as: The fusion and interaction mechanism of gate networks can be represented as: It is an adaptive parameter that controls the ratio of knowledge representation to text representation in the fused representation; A transformation is performed after the fusion step to convert the fused representation into a continuous cue, and a multilayer perceptron is used in the transformation step: in It is the hint vector for the generated search task.

5. The personalized search and explanation generation method based on unified prompt awareness as described in claim 4, characterized in that: The prompt learning process integrates the fused representation of the user and query with prompts for the search task. As input for prompting learning: in and It's a direct input to PLM, providing prompts for searching tasks. Prefix-tuning is used as a prefix marker added to each layer of the base pre-trained language model. Using cross-entropy loss as the objective function:

Images