Multi-domain recommendation method based on adaptive knowledge enhancement based on large language model

Abstract

The present invention provides a multi-domain recommendation method based on adaptive knowledge enhancement of a large language model, including: obtaining sample data of a target user; constructing domain-shared prompt templates based on text information corresponding to multiple recommendation domains; constructing a unique domain prompt template based on the text information corresponding to the target user in the target domain and a user knowledge base; calling a large language model to perform feature learning based on the domain-shared prompt template and the unique domain prompt template to obtain domain-invariant features and unique domain preference features of the target user; constructing a multi-task model based on the domain-invariant features and unique domain preference features of the target user, optimizing and training the multi-task model to generate a multi-domain recommendation model, and completing multi-domain recommendations for the target user through the multi-domain recommendation model. By designing two types of prompt templates to train the large language model, the domain-invariant features and unique domain preference features of the target user are learned, and the prediction performance of the multi-domain recommendation model is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a multi-domain recommendation method based on adaptive knowledge enhancement of a large language model. Background Art

[0002] Multi-domain Recommendation (MDR) aims to utilize various types of data from multiple domains (ID-based entities, tags, attributes, etc.) to improve the recommendation performance in each domain at the same time, and has attracted widespread attention from academia and industry. Generally speaking, online platforms can provide various items from different domains, such as books, clothing, toys, and products from other fields contained on shopping website pages; users on the platform can browse or purchase any goods from different fields, and their multi-domain interactive behaviors can provide rich training samples and preference knowledge for the recommendation model. Although many existing works are dedicated to improving the prediction performance of multi-domain recommendation, its prediction effect is still difficult to meet the application requirements of actual scenarios.

[0003] Recently, large language models (LLMs) have shown impressive capabilities in many downstream tasks such as text generation, intelligent questions, and mathematical reasoning, and are considered a key step towards achieving general artificial intelligence. Some pioneering works have also shown extraordinary capabilities in exploring the application of LLMs in recommendation systems. These works attempt to directly use LLMs to generate the final recommendation results based on different prompts or instruction tuning. However, due to the lack of extensive recommendation domain data in specific task training, LLMs find it difficult to make accurate predictions. To this end, researchers further fine-tune LLMs by injecting synergistic signals into the model, and found that these methods still cannot surpass previous specific domain models.

[0004] Specifically, the existing multi-domain recommendation methods have the following main defects: First, there is the semantic gap problem between the collaborative information and text information in the recommendation field. Existing recommendation models mainly rely on collaborative interactions based on user or item embedding IDs (such as UserID, ItemID, etc.) for recommendations, while LLMs usually use text-based corpora for model training. Therefore, directly inputting collaborative embedding data into LLMs for training may greatly reduce the recommendation performance. On the other hand, discarding a large amount of recommendation data and only using the provided text data for LLMs training will also greatly limit the recommendation efficiency. Therefore, the semantic gap challenge between training data makes it difficult for existing models to fully utilize the generation capabilities of LLMs to solve recommendation tasks.

[0005] The second is the sparse interaction problem in traditional recommendation models. Most models for specific domains can generate recommendation results with a small number of interaction samples. However, these models perform poorly in sparse scenarios due to insufficient supervision signals, and some user profiles and side information are not effectively utilized to improve recommendation performance. In addition, existing LLMs-based recommendations mainly focus on sequence recommendation and news recommendation tasks, and no research has explored the performance of LLM technology in multi-domain recommendation scenarios. Summary of the invention

[0006] The present invention provides a multi-domain recommendation method based on adaptive knowledge enhancement of a large language model, which is used to solve the defects in the prior art that the semantic gap between the collaborative information and text information in the recommendation field leads to poor recommendation performance and limited recommendation scenarios. The method realizes training the large language model by designing two types of prompt templates, learning the domain-invariant features and unique domain preference features of the target users, and effectively improving the prediction performance of the multi-domain recommendation model.

[0007] The present invention provides a multi-domain recommendation method based on adaptive knowledge enhancement of a large language model, comprising the following steps.

[0008] Acquire text information corresponding to the target user in multiple recommendation domains and text information corresponding to the target domain, wherein the text information includes text description information and collaborative information;

[0009] Building domain sharing prompt templates based on the text information corresponding to the target user in multiple recommended domains;

[0010] Building a unique domain prompt template based on the text information corresponding to the target user in the target domain and the user knowledge base;

[0011] Calling the large language model to perform feature learning based on the domain-shared prompt template to obtain the domain-invariant features of the target user, and calling the large language model to perform feature learning based on the domain-specific prompt template to obtain the domain-specific preference features of the target user;

[0012] A multi-task model is constructed based on the domain-invariant features and unique domain preference features of the target user, the multi-task model is optimized and trained to generate a multi-domain recommendation model, and multi-domain recommendations for the target user are completed through the multi-domain recommendation model.

[0013] According to a multi-domain recommendation method based on adaptive knowledge enhancement of a large language model provided by the present invention, the method for constructing the user knowledge base includes:

[0014] Acquire historical search instances of the target user, and construct an initial user knowledge base based on the historical search instances;

[0015] The historical search instance is entity linked with the instance in the external common sense knowledge base, and the instance in the external common sense knowledge base linked with the historical search instance is added to the initial user knowledge base to obtain the user knowledge base.

[0016] According to a multi-domain recommendation method based on adaptive knowledge enhancement of a large language model provided by the present invention, the text information also includes project information in which the target user interacts in multiple recommendation domains, and the method further includes:

[0017] Performing word segmentation processing on the collaborative information, the text description information, and the project information respectively corresponding to the target user in multiple recommendation domains;

[0018] The word segmentation results are subjected to hybrid encoding processing to obtain a multi-domain embedding vector sequence in the first sequence representation space;

[0019] The calling of the large language model to perform feature learning based on the domain-shared prompt template to obtain the domain-invariant features of the target user includes:

[0020] The large language model is called to perform feature learning based on the multi-domain embedding vector sequence to obtain the domain-invariant features of the target user.

[0021] According to a multi-domain recommendation method based on adaptive knowledge enhancement of a large language model provided by the present invention, the method further includes:

[0022] Performing feature mapping processing on the multi-domain embedding vector sequence to obtain a multi-domain embedding vector sequence in the same feature space;

[0023] The calling of the large language model to perform feature learning based on the multi-domain embedding vector sequence to obtain the domain-invariant features of the target user includes:

[0024] The large language model is called to perform feature learning based on the multi-domain embedding vector sequence in the same feature space to obtain the domain-invariant features of the target user.

[0025] According to a multi-domain recommendation method based on adaptive knowledge enhancement of a large language model provided by the present invention, the text information also includes project information corresponding to the target domain, and the method further includes:

[0026] Performing word segmentation processing on the collaborative information, the user knowledge base, the text description information and the project information corresponding to the target user in the target domain respectively;

[0027] The word segmentation results are subjected to hybrid encoding processing to obtain a unique domain embedding vector sequence in the second sequence representation space;

[0028] The calling of the large language model to perform feature learning based on the unique domain prompt template to obtain the unique domain preference features of the target user includes:

[0029] The large language model is called to perform feature learning based on the unique domain embedding vector sequence to obtain the unique domain preference features of the target user.

[0030] According to a multi-domain recommendation method based on adaptive knowledge enhancement of a large language model provided by the present invention, the method further includes:

[0031] Performing feature mapping processing on the independent domain embedding vector sequence to obtain an independent domain embedding vector sequence in the same feature space;

[0032] The calling of the large language model to perform feature learning based on the unique domain embedding vector sequence to obtain the unique domain preference features of the target user includes:

[0033] The large language model is called to perform feature learning based on the unique domain embedding vector sequence of the same feature space to obtain the unique domain preference features of the target user.

[0034] According to a multi-domain recommendation method based on adaptive knowledge enhancement of a large language model provided by the present invention, the collaborative information includes entity identifier information of the target user and entity identifier information of items with which the target user interacts in multiple recommendation domains.

[0035] The present invention also provides a multi-domain recommendation device based on adaptive knowledge enhancement of a large language model, comprising the following modules:

[0036] An acquisition module, used to acquire text information corresponding to the target user in multiple recommendation domains and text information corresponding to the target domain, wherein the text information includes text description information and collaborative information;

[0037] A first template building module, used to build domain sharing prompt templates based on the text information corresponding to the target user in multiple recommended domains;

[0038] A second template construction module is used to construct a unique domain prompt template based on the text information corresponding to the target user in the target domain and the user knowledge base;

[0039] A feature learning module is used to call the large language model to perform feature learning based on the domain-shared prompt template to obtain the domain-invariant features of the target user, and call the large language model to perform feature learning based on the domain-specific prompt template to obtain the domain-specific preference features of the target user;

[0040] The multi-domain recommendation module is used to construct a multi-task model based on the domain-invariant features and unique domain preference features of the target user, optimize and train the multi-task model to generate a multi-domain recommendation model, and complete multi-domain recommendations for the target user through the multi-domain recommendation model.

[0041] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a multi-domain recommendation method based on adaptive knowledge enhancement of a large language model as described above is implemented.

[0042] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the multi-domain recommendation methods based on adaptive knowledge enhancement of a large language model as described above.

[0043] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the multi-domain recommendation methods based on adaptive knowledge enhancement of a large language model as described above.

[0044] The present invention provides a multi-domain recommendation method based on adaptive knowledge enhancement of a large language model. The method obtains the text description information and collaborative information corresponding to the target user in multiple recommendation domains and the target domain, and then constructs domain-shared prompt templates based on the text information corresponding to the multiple recommendation domains, and constructs a unique domain prompt template based on the text information corresponding to the target user in the target domain and the user knowledge base; and calls the large language model to perform feature learning based on the domain-shared prompt template and the unique domain prompt template to obtain the domain-invariant features and unique domain preference features of the target user, and finally constructs a multi-task model based on the domain-invariant features and unique domain preference features of the target user, optimizes and trains the multi-task model to generate a multi-domain recommendation model, and completes multi-domain recommendations for the target user through the multi-domain recommendation model. The large language model is trained by designing two types of prompt templates to learn the domain-invariant features and unique domain preference features of the target user, and the prediction performance of the multi-domain recommendation model is effectively improved by combining the existing recommendation model with the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1This is one of the flow charts of the multi-domain recommendation method based on adaptive knowledge enhancement of a large language model provided by the present invention.

[0047] Figure 2 It is a schematic diagram of the domain sharing prompt template provided by the present invention.

[0048] Figure 3 It is a schematic diagram of the unique domain prompt template provided by the present invention.

[0049] Figure 4 This is the second flowchart of the multi-domain recommendation method based on adaptive knowledge enhancement of a large language model provided by the present invention.

[0050] Figure 5 It is a structural schematic diagram of a multi-domain recommendation device based on adaptive knowledge enhancement of a large language model provided by the present invention.

[0051] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Figure 1 This is one of the flow charts of the multi-domain recommendation method based on adaptive knowledge enhancement of a large language model provided by the present invention. Figure 1 As shown, the method includes the following steps 101 to 105.

[0054] Step 101: Obtain text information of a target user corresponding to multiple recommendation domains and text information corresponding to the target domain.

[0055] The text information includes text description information and collaborative information. The collaborative information includes entity identifier information of the target user and entity identifier information of items that the target user interacts with in multiple recommendation domains.

[0056] Specifically, the text description information includes detailed description information of the recommendation-related items that the target user has searched or viewed in multiple recommendation fields, such as the attributes, categories, tags, descriptions, etc. of the items. For example, in video recommendations, the information of the items may include the type of movie (such as action, comedy), director, actors, etc.; in music recommendations, the information of the items may include the singer, composer, genre, etc. of the song. The text description information can also include the attribute information of the target user, such as gender information. The entity identifier information of the target user is the entity identifier information based on the target user ID, such as <uid721> 、 <uid295>etc., the entity identifier information of the items that the target user interacts with in multiple recommendation domains, e.g. <iid793> 、 <iid942763>wait.

[0057] Step 102: construct domain sharing prompt templates based on the text information corresponding to the target user in multiple recommended domains.

[0058] In some embodiments, domain sharing prompt templates are respectively constructed based on the text information corresponding to the target user in multiple recommended domains, that is, the domain sharing prompt template is constructed based on the text description information and the collaborative information.

[0059] For example, see Figure 2 , Figure 2 is a schematic diagram of a domain sharing prompt template provided by the present invention, Figure 2 The domain sharing prompt template shown in the question is "Question: User <uid721>Watched a movie "Shooting Stars" in the movie field <iid1793>, this movie is a sports movie about basketball. <uid721>In the clothing category, I bought a "Los Angeles Lakers jersey" <iid942763>clothes. Predict whether the user will buy the book "Return of the King: LeBron James" <iid33809>. Answer: "Yes" or "No".

[0060] In the above domain sharing prompt template, collaborative information can be added as a special word to the domain sharing prompt template. For example, <uid721>Indicates the target user, <iid1793> 、 <iid942763>and <iid33809>Represents items in different domains interacted by the target user. Based on the designed domain-shared prompt template, the present invention attempts to tokenize the collaborative information into a representation space of language semantics. In addition, in order to further maintain semantic consistency, the present invention integrates the text description into the prompt template together with the user ID and the project ID as much as possible. Therefore, unlike the general single-domain prompt template that only contains entity descriptions and answer identifiers, the above-mentioned domain-shared prompt template can incorporate more entity embeddings, historical behaviors, and attribute information from multiple domains to form trainable samples.

[0061] Step 103: construct a unique domain prompt template based on the text information corresponding to the target user in the target domain and the user knowledge base.

[0062] In some embodiments, the method for constructing a user knowledge base includes: obtaining historical search instances of the target user, and constructing an initial user knowledge base based on the historical search instances;

[0063] The historical search instance is entity linked with the instance in the external common sense knowledge base, and the instance in the external common sense knowledge base linked with the historical search instance is added to the initial user knowledge base to obtain the user knowledge base.

[0064] Specifically, the historical retrieval of the target user in different fields can reveal his true intention and preference. Therefore, the historical retrieval instances involved by the target user can be embedded and aggregated as different instances of the user knowledge base to construct the initial user knowledge base, which is expressed as . Usually, the target user's retrieval and behavior on the web page are the knowledge that the target user is really interested in. For example, if the target user has made multiple queries including "Golden State Warriors", "Basketball scoring champion", and "CBA", and browsed some web pages in the above queries, then it can be considered that the target user is interested in the instantiation concept of "basketball game".

[0065] In order to effectively aggregate different instances, the embodiment of the present invention designs a query-based knowledge aggregation scheme, which can capture and store various concept instances involved in retrieval. A very lightweight initial user knowledge base is initially constructed based on the aggregated knowledge instances and injected into the LLMs for training. However, the initial user knowledge base The available knowledge in the user knowledge base may still be relatively sparse, making it difficult to effectively support the large language model to obtain meaningful prediction results. In addition, the aimless browsing behavior of the target user may introduce a large amount of query noise, further interfering with the training of LLMs. Therefore, the embodiment of the present invention constructs an instantiation method based on entity linking to form a user knowledge base. Specifically, through entity linking technology, the entities appearing in the target user's historical search are tokenized and linked to entities in external common sense knowledge bases (such as Baidu Encyclopedia, Wikipedia, etc.), and the entities in these external common sense knowledge bases can be added to the external common sense knowledge base. Therefore, the initial user database constructed and external common sense knowledge base Combined with the knowledge, we can get the user knowledge base after knowledge enhancement , providing LLMs with more knowledge to obtain the user's unique domain preference features. The specific merging formula is as follows:

[0066]

[0067] It should be noted that due to the user knowledge base It is built based on relevant instances of the target user, so it is relatively lightweight and can be easily injected into the LLM. However, although the explicit knowledge in the user knowledge base can be directly injected into the large language model, it is difficult for the LLM to effectively understand and utilize data similar to the ID-based embedding. Therefore, by designing domain-specific prompt templates based on the user knowledge base of the target user in different fields, the LLM can maximize the use of explicit knowledge and achieve better training results.

[0068] In some embodiments, since the domain-shared features (domain-invariant features) of the target users may not fully reveal their recent interests, some unique preference signals in the target domain can be further mined to improve the final preference features. Similar to the design of the multi-domain prompt template (domain-shared prompt template), the text data of the target domain and the constructed user knowledge base can also be integrated into the unique domain prompt template at the same time to achieve language and collaborative knowledge training.

[0069] Specifically, see Figure 3 , Figure 3 is a schematic diagram of a unique domain prompt template provided by the present invention, Figure 3 The single-domain prompt template shown in the example is "Question: User <uid213>Watched the movie "Jurassic Park" in the movie field <iid3309>, this movie <iid3309>In the user knowledge base it was directed by Steven Spielberg, who also directed another film in the film field "Schindler's List" <iid8209>. Predict whether the user will watch the movie "Schindler's List" <iid8209>. Answer: "Yes" or "No".

[0070] In the above domain-specific prompt template, in addition to text segmentation, some ID-based entity segmentation and user knowledge base in the target domain can also be used. The entity segmentation in is integrated into the unique domain prompt template to jointly train the large language model. <uid213> 、 <iid3309>and <iid8209>Represents ID-based entity segmentation, which are collaborative embedding vectors of the recommendation system. The entity "Steven Spielberg" is an entity segmentation in the user knowledge base, which can be associated with the entity <iid3309>and <iid8209>Through the above unique domain prompt template, the semantics of collaborative information and the semantics of natural language are synergistically integrated into the same latent space to learn the unique domain preference features of target users in specific fields.

[0071] Step 104: Call the large language model to perform feature learning based on the domain-shared prompt template to obtain the domain-invariant features of the target user, and call the large language model to perform feature learning based on the unique domain prompt template to obtain the unique domain preference features of the target user.

[0072] In some embodiments, the text information further includes information about items that the target user interacts with in multiple recommended domains. After constructing the domain sharing prompt template, the following steps are further included:

[0073] Performing word segmentation processing on the collaborative information, the text description information, and the project information respectively corresponding to the target user in multiple recommendation domains;

[0074] The word segmentation results are subjected to hybrid encoding processing to obtain a multi-domain embedding vector sequence in the first sequence representation space.

[0075] The calling of the large language model to perform feature learning based on the domain-shared prompt template to obtain the domain-invariant features of the target user includes:

[0076] The large language model is called to perform feature learning based on the multi-domain embedding vector sequence to obtain the domain-invariant features of the target user.

[0077] Specifically, in order to smoothly convert the information of the domain-shared prompt template into a latent vector that can be trained by LLMs, the embodiment of the present invention proposes a hybrid word segmentation encoding method to achieve the word segmentation of collaborative information and text content. Specifically, the LLMs word segmenter is first used to word segment all the information from multiple fields (including collaborative information, text description information and project information) to obtain the word segmentation processing result, and then the word segmentation processing result is further mixed and encoded into the first sequence representation space In the above example, we obtain a multi-domain embedding vector sequence, which is defined as follows.

[0078]

[0079] in, For target users The collaborative embedding vector of Represents text Tokenized embedding vectors in large language models, Represents the collaborative embedding vector of items from multiple fields. It should be noted that the collaborative embedding of users and items can be obtained through collaborative models, such as the Matrix Factorization (MF) model.

[0080] However, the effect of directly training hybrid encoding is not good, because it is difficult for the model to converge if various word segmentations are directly introduced into the large language model without pre-training; and because the number of entities (users or projects) from multiple fields is huge, the parameters that need to be fine-tuned are beyond the controllable training range. Therefore, the embodiment of the present invention can update the sequence representation , by aggregating various embeddings of target users from multiple domains, and thus learning their domain-invariant features for word segmentation prediction.

[0081] According to the above strategy, the embodiment of the present invention can make full use of side information from multiple fields (including collaborative information and text description information) to accelerate the convergence efficiency of word segmentation training. The embodiment of the present invention uses the supervised fine-tuning (SFT) method to train word segmentation representation and LLM parameters. Assume that each sample is composed of a domain shared prompt template Share feature labels with a domain The final training goal of LLMs is to model the possibility of parameter distribution, as shown below.

[0082]

[0083] in, Represents the trainable parameters of LLM. The target user's collaborative segmentation and text segmentation basically come from multiple fields. These segmentations and text segmentations are fed into the output layer of LLMs together to obtain the target user's domain-shared features (domain-invariant features) Therefore, the domain sharing loss function is defined as follows.

[0084]

[0085] In some embodiments, the text information further includes project information corresponding to the target domain. After constructing the above-mentioned single domain prompt template, the following steps are further included:

[0086] Performing word segmentation processing on the collaborative information, the user knowledge base, the text description information and the project information corresponding to the target user in the target domain respectively;

[0087] The word segmentation results are subjected to hybrid encoding processing to obtain a unique domain embedding vector sequence in the second sequence representation space;

[0088] The calling of the large language model to perform feature learning based on the unique domain prompt template to obtain the unique domain preference features of the target user includes:

[0089] The large language model is called to perform feature learning based on the unique domain embedding vector sequence to obtain the unique domain preference features of the target user.

[0090] Similar to the domain sharing feature learning process, the hybrid word segmentation encoding method proposed in the embodiment of the present invention can segment ID-based entity information, entity knowledge in the user knowledge base, and text information, that is, encode these data together in a sequence representation space It is formally defined as follows.

[0091]

[0092] in, represents the entity embedding in the user knowledge base, and Represent the text word embedding and item embedding in the target domain respectively.

[0093] Likewise, each sample consists of a unique domain prompt template and a unique feature label The SFT method is used to train the LLM parameters. Therefore, its modeling function is defined as follows.

[0094]

[0095] in, Represents the trainable parameters of LLM.

[0096] Then, the embodiment of the present invention inputs the collaborative segmentation and text segmentation of the target domain and the entity segmentation in the user knowledge base into LLMs through the unique domain prompt template, so as to learn the unique domain preference characteristics of the target user, and the unique domain loss function is defined as follows.

[0097]

[0098] In some embodiments, the method provided by the embodiment of the present invention further includes:

[0099] Performing feature mapping processing on the multi-domain embedding vector sequence to obtain a multi-domain embedding vector sequence in the same feature space;

[0100] The calling of the large language model to perform feature learning based on the multi-domain embedding vector sequence to obtain the domain-invariant features of the target user includes:

[0101] The large language model is called to perform feature learning based on the multi-domain embedding vector sequence in the same feature space to obtain the domain-invariant features of the target user.

[0102] In some embodiments, the method provided by the embodiment of the present invention further includes:

[0103] Performing feature mapping processing on the independent domain embedding vector sequence to obtain an independent domain embedding vector sequence in the same feature space;

[0104] The calling of the large language model to perform feature learning based on the unique domain embedding vector sequence to obtain the unique domain preference features of the target user includes:

[0105] The large language model is called to perform feature learning based on the unique domain embedding vector sequence of the same feature space to obtain the unique domain preference features of the target user.

[0106] Specifically, in order to aggregate more samples related to multi-domain recommendations and avoid introducing unnecessary noise, an embodiment of the present invention proposes an adaptive feedback model to utilize the provided collaborative information or text content. First, an arbitrary language model is trained in an end-to-end manner to learn and output special segmentations (i.e., feedback segmentations) to evaluate the generated predictions. If the feedback segmentation can effectively improve the generation quality, it can be input into the large language model together with the original domain-shared prompt template or the unique domain prompt template.

[0107] Assume that the training data is , the initial feedback word set is , use the pre-trained language model LM (such as Llama2-7B) to initialize , feedback learning is implemented through a conditional language modeling function with maximum likelihood, the specific function is shown below.

[0108]

[0109] in, is the original training data The sample instance in It is a trainable feedback segmentation. The generated feedback segmentation can dynamically evaluate whether the selected training data can better help LLM generate high-quality predictions, thereby improving the training effect of downstream tasks.

[0110] In order to effectively train sample semantic pairs in LLM, an embodiment of the present invention uses the Transformer decoder as the backbone of a large language model, where each Transformer layer is composed of multiple multi-head self-attention modules. Then, the domain-shared embedding vector sequence corresponding to the domain-shared prompt template and the unique domain embedding vector sequence corresponding to the unique domain prompt template are input into the front neural network to generate the output probability of the target word segmentation. However, since the above vector sequences represent different side information from multiple fields, it is necessary to perform an alignment operation between language and collaborative semantics.

[0111] Specifically, in order to further unify the sample semantic features in the above-mentioned domain sharing prompt template and the independent domain prompt template, the embodiment of the present invention proposes a multi-domain mutual learning method to map different word segmentation representations to the same feature space. In order to facilitate sharing of the same data, the embodiment of the present invention defines the mutual learning loss in the multi-domain mutual learning method to limit the embedding divergence, and the specific definition is as follows.

[0112]

[0113]

[0114] Among them, || ∙ ||2 represents the Mean Squared Error (MSE) loss model; and They are the tokenized representations of users, texts, and entities in LLMs respectively; and Representing the embeddings of users and items in traditional recommendation domain-specific models (e.g., matrix factorization); is the embedding obtained by the one-shot encoding model.

[0115] Then, according to the mutual learning model defined above, the domain shared feature loss of the target user and unique domain feature learning loss is further optimized as shown below.

[0116]

[0117]

[0118] in, and They are the weight parameters for adjusting the mutual learning loss.

[0119] Step 105: construct a multi-task model based on the domain-invariant features and unique domain preference features of the target user, optimize and train the multi-task model to generate a multi-domain recommendation model, and complete multi-domain recommendations for the target user through the multi-domain recommendation model.

[0120] In some embodiments, after the language semantics in the LLM and the collaborative semantics in the recommendation system are aligned in the same space, the multi-domain recommender can be combined with the domain-invariant feature learning loss and the domain-specific feature learning loss in multiple domains to jointly train the proposed multi-task model. Since the large language model used in the embodiment of the present invention can learn user preference features by predicting the word distribution probability of the target item, the multi-domain Bayesian personalized ranking method can be used to construct a multi-task model. Assume is the domain label, is a data sample in a unique domain, then the task model is defined as follows.

[0121]

[0122] in, represents the Sigmoid function, and Respectively represent domains The actual and predicted interaction effects in . is the regularization parameter, are trainable model parameters.

[0123] In addition, the multi-task model adopted by the present invention combines the multi-domain prediction loss and the feature learning loss based on LLM to jointly train the model proposed in the embodiment of the present invention. Therefore, the final objective function for training the model is defined as follows.

[0124]

[0125] in, is a hyperparameter in LLMs feature learning, is the hyperparameter of the model in multi-task learning, Represents the trainable parameters of the model.

[0126] In some embodiments, see Figure 4 , Figure 4 This is the second flow chart of the multi-domain recommendation method based on adaptive knowledge enhancement of a large language model provided by the present invention. Figure 4 As shown, an embodiment of the present invention proposes a multi-domain recommendation method based on adaptive knowledge enhancement of a large language model. The core idea is to use a large language model as a feature extractor, learn user preference features according to the designed prompt template and multi-domain Bayesian personalized ranking method, and jointly train the proposed multi-task model. Specifically, firstly, based on sample data from multiple fields, the embodiment of the present invention designs a domain-sharing prompt template to learn the domain-invariant features of the target user, and proposes a hybrid word segmentation encoding strategy to convert the input prompt into the latent representation space of LLM. Then, the available knowledge, collaborative semantic word segmentation and text semantic word segmentation in the constructed user knowledge base are mixed and encoded into the domain-specific prompt template in LLMs to learn the user-specific domain preference features. In order to avoid the interference of irrelevant noise samples, an embodiment of the present invention proposes an adaptive feedback model to aggregate more sample word segmentations related to the topic. In addition, an embodiment of the present invention also proposes a multi-domain mutual learning model to align the semantic features of various samples in the joint representation space of LLMs. Finally, a multi-domain recommendation model is generated through multi-task learning and training, and multi-domain recommendation for target users is completed through the multi-domain recommendation model.

[0127] The multi-domain recommendation device based on adaptive knowledge enhancement of a large language model provided by the present invention is described below. The multi-domain recommendation device based on adaptive knowledge enhancement of a large language model described below and the multi-domain recommendation method based on adaptive knowledge enhancement of a large language model described above can be referenced to each other.

[0128] like Figure 5 As shown, Figure 5 It is a structural schematic diagram of a multi-domain recommendation device based on adaptive knowledge enhancement of a large language model provided by the present invention. Figure 5 The multi-domain recommendation device 500 based on adaptive knowledge enhancement of a large language model shown in the figure comprises an acquisition module 501, a first template construction module 502, a second template construction module 503, a feature learning module 504, and a multi-domain recommendation module 505; wherein the acquisition module 501 is used to acquire text information corresponding to a target user in multiple recommendation domains, and text information corresponding to the target domain, wherein the text information includes text description information and collaborative information; the first template construction module 502 is used to construct domain sharing prompt templates based on the text information corresponding to the target user in multiple recommendation domains; the second template construction module 503 is used to construct domain sharing prompt templates based on the text information corresponding to the target user in multiple recommendation domains; A domain-specific prompt template is constructed based on the text information corresponding to the target user in the target domain and the user knowledge base; the feature learning module 504 is used to call the large language model to perform feature learning based on the domain-shared prompt template to obtain the domain-invariant features of the target user, and call the large language model to perform feature learning based on the domain-specific prompt template to obtain the domain-specific preference features of the target user; the multi-domain recommendation module 505 is used to construct a multi-task model based on the domain-invariant features and the domain-specific preference features of the target user, optimize and train the multi-task model to generate a multi-domain recommendation model, and complete multi-domain recommendations for the target user through the multi-domain recommendation model.

[0129] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6 As shown, the electronic device may include: a processor (processor) 610 , a communication interface (Communications Interface) 620 , a memory (memory) 630 and a communication bus 640 , wherein the processor 610 , the communication interface 620 , and the memory 630 communicate with each other through the communication bus 640 . The processor 610 can call the logic instructions in the memory 630 to execute a multi-domain recommendation method based on adaptive knowledge enhancement of a large language model, the method comprising: obtaining text information corresponding to a target user in multiple recommendation domains, and text information corresponding to a target domain, wherein the text information comprises text description information and collaborative information; constructing domain-sharing prompt templates based on the text information corresponding to the target user in multiple recommendation domains; constructing a unique domain prompt template based on the text information corresponding to the target user in the target domain and a user knowledge base; calling a large language model to perform feature learning based on the domain-sharing prompt template to obtain domain-invariant features of the target user, calling the large language model to perform feature learning based on the unique domain prompt template to obtain unique domain preference features of the target user; constructing a multi-task model based on the domain-invariant features and unique domain preference features of the target user, optimizing and training the multi-task model to generate a multi-domain recommendation model, and completing multi-domain recommendations for the target user through the multi-domain recommendation model.

[0130] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0131] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multi-domain recommendation method based on the adaptive knowledge enhancement of the large language model provided by the above methods, the method including: obtaining text information corresponding to the target user in multiple recommendation domains, and text information corresponding to the target domain, wherein the text information includes text description information and collaborative information; constructing domain-sharing prompt templates based on the text information corresponding to the target user in multiple recommendation domains; constructing a unique domain prompt template based on the text information corresponding to the target user in the target domain and a user knowledge base; calling a large language model to perform feature learning based on the domain-sharing prompt template to obtain the domain-invariant features of the target user, calling the large language model to perform feature learning based on the unique domain prompt template to obtain the unique domain preference features of the target user; constructing a multi-task model based on the domain-invariant features and unique domain preference features of the target user, optimizing and training the multi-task model to generate a multi-domain recommendation model, and completing multi-domain recommendations for the target user through the multi-domain recommendation model.

[0132] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the processor executes the multi-domain recommendation method based on the adaptive knowledge enhancement of the large language model provided by the above methods, the method comprising: obtaining text information corresponding to the target user in multiple recommendation domains, and text information corresponding to the target domain, wherein the text information includes text description information and collaborative information; constructing domain-sharing prompt templates based on the text information corresponding to the target user in multiple recommendation domains; constructing a unique domain prompt template based on the text information corresponding to the target user in the target domain and a user knowledge base; calling the large language model to perform feature learning based on the domain-sharing prompt template to obtain the domain-invariant features of the target user, calling the large language model to perform feature learning based on the unique domain prompt template to obtain the unique domain preference features of the target user; constructing a multi-task model based on the domain-invariant features and unique domain preference features of the target user, optimizing and training the multi-task model to generate a multi-domain recommendation model, and completing multi-domain recommendations for the target user through the multi-domain recommendation model.

[0133] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0134] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention. < / uid213> < / iid1793> < / iid793> < / uid721>

Claims

1. A multi-domain recommendation method based on adaptive knowledge enhancement of a large language model, characterized in that: include: Acquire text information corresponding to the target user in multiple recommendation domains and text information corresponding to the target domain, wherein the text information includes text description information and collaborative information; Building domain sharing prompt templates based on the text information corresponding to the target user in multiple recommended domains; Building a unique domain prompt template based on the text information corresponding to the target user in the target domain and the user knowledge base; Calling the large language model to perform feature learning based on the domain-shared prompt template to obtain the domain-invariant features of the target user, and calling the large language model to perform feature learning based on the domain-specific prompt template to obtain the domain-specific preference features of the target user; A multi-task model is constructed based on the domain-invariant features and unique domain preference features of the target user, the multi-task model is optimized and trained to generate a multi-domain recommendation model, and multi-domain recommendations for the target user are completed through the multi-domain recommendation model; The text information also includes information about items that the target user interacts with in multiple recommendation domains, and the method further includes: Performing word segmentation processing on the collaborative information, the text description information, and the project information respectively corresponding to the target user in multiple recommendation domains; The word segmentation results are subjected to hybrid encoding processing to obtain a multi-domain embedding vector sequence in the first sequence representation space; The calling of the large language model to perform feature learning based on the domain-shared prompt template to obtain the domain-invariant features of the target user includes: The large language model is called to perform feature learning based on the multi-domain embedding vector sequence to obtain the domain-invariant features of the target user.

2. The multi-domain recommendation method based on adaptive knowledge enhancement of a large language model according to claim 1, characterized in that: The method for constructing the user knowledge base includes: Acquire historical search instances of the target user, and construct an initial user knowledge base based on the historical search instances; The historical search instance is entity linked with the instance in the external common sense knowledge base, and the instance in the external common sense knowledge base linked with the historical search instance is added to the initial user knowledge base to obtain the user knowledge base.

3. The multi-domain recommendation method based on adaptive knowledge enhancement of a large language model according to claim 1, characterized in that: The method further comprises: Performing feature mapping processing on the multi-domain embedding vector sequence to obtain a multi-domain embedding vector sequence in the same feature space; The calling of the large language model to perform feature learning based on the multi-domain embedding vector sequence to obtain the domain-invariant features of the target user includes: The large language model is called to perform feature learning based on the multi-domain embedding vector sequence in the same feature space to obtain the domain-invariant features of the target user.

4. The multi-domain recommendation method based on adaptive knowledge enhancement of a large language model according to claim 1, characterized in that: The text information also includes project information corresponding to the target domain, and the method further includes: Performing word segmentation processing on the collaborative information, the user knowledge base, the text description information and the project information corresponding to the target user in the target domain respectively; The word segmentation results are subjected to hybrid encoding processing to obtain a unique domain embedding vector sequence in the second sequence representation space; The calling of the large language model to perform feature learning based on the unique domain prompt template to obtain the unique domain preference features of the target user includes: The large language model is called to perform feature learning based on the unique domain embedding vector sequence to obtain the unique domain preference features of the target user.

5. The multi-domain recommendation method based on adaptive knowledge enhancement of a large language model according to claim 4, characterized in that: The method further comprises: Performing feature mapping processing on the independent domain embedding vector sequence to obtain an independent domain embedding vector sequence in the same feature space; The calling of the large language model to perform feature learning based on the unique domain embedding vector sequence to obtain the unique domain preference features of the target user includes: The large language model is called to perform feature learning based on the unique domain embedding vector sequence of the same feature space to obtain the unique domain preference features of the target user.

6. The multi-domain recommendation method based on adaptive knowledge enhancement of a large language model according to claim 1, characterized in that: The collaboration information includes entity identifier information of the target user and entity identifier information of items with which the target user interacts in multiple recommendation domains.

7. A multi-domain recommendation device based on adaptive knowledge enhancement of a large language model, characterized in that: include: An acquisition module, used to acquire text information corresponding to the target user in multiple recommendation domains and text information corresponding to the target domain, wherein the text information includes text description information and collaborative information; A first template building module is used to build domain sharing prompt templates based on the text information corresponding to the target user in multiple recommended domains; A second template construction module is used to construct a unique domain prompt template based on the text information corresponding to the target user in the target domain and the user knowledge base; A feature learning module is used to call the large language model to perform feature learning based on the domain-shared prompt template to obtain the domain-invariant features of the target user, and call the large language model to perform feature learning based on the domain-specific prompt template to obtain the domain-specific preference features of the target user; A multi-domain recommendation module is used to construct a multi-task model based on the domain-invariant features and unique domain preference features of the target user, optimize and train the multi-task model to generate a multi-domain recommendation model, and complete multi-domain recommendations for the target user through the multi-domain recommendation model; The text information also includes information about items that the target user interacts with in multiple recommendation domains, and the device is further used for: Performing word segmentation processing on the collaborative information, the text description information, and the project information respectively corresponding to the target user in multiple recommendation domains; The word segmentation results are subjected to hybrid encoding processing to obtain a multi-domain embedding vector sequence in the first sequence representation space; The calling of the large language model to perform feature learning based on the domain-shared prompt template to obtain the domain-invariant features of the target user includes: The large language model is called to perform feature learning based on the multi-domain embedding vector sequence to obtain the domain-invariant features of the target user.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the multi-domain recommendation method based on adaptive knowledge enhancement of a large language model as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-domain recommendation method based on adaptive knowledge enhancement of a large language model as claimed in any one of claims 1 to 6 is implemented.

Images