User privacy protection cross-domain service recommendation method for large language model enhancement

Abstract

The invention discloses a large language model enhanced user privacy protection cross-domain service recommendation method, which comprises the following steps of: firstly, screening data in a data set, and designing a text prompt to apply a large language model enhanced service description text and user attribute information; a BERT language model is used for coding the service description text, and embedding is generated; embedding and mapping the text into discrete codes by using a PQ method, and maintaining a code dictionary; reading user historical service interaction data, and generating a user behavior sequence file according to a timestamp sequence; service description information in the user behavior sequence is converted into discrete code representation, the discrete code representation is input into a sequence encoder for local model training, and a model with the optimal performance is stored; the parameter gradient of the local model is clipped and quantified and then is uploaded to a server; and performing fine adjustment on the model by adopting soft prompt and hard prompt. According to the method, the semantic difference in cross-domain recommendation is reduced, the model is finely adjusted through soft prompt and hard prompt, and the recommendation individuation and accuracy are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical fields of data mining and recommendation systems, and particularly to a cross-domain service recommendation method for enhancing user privacy protection with a large language model. Background Art

[0002] Nowadays, with the rapid development of information technology and the advent of the big data era, the phenomenon of data concentration in some popular fields is quite common. Cross-domain service recommendation is dedicated to using user service interaction data collected from the source domain to mine implicit user preferences and recommend suitable services to target domain users, thereby meeting the multi-domain personalized needs of users. It is an effective way to overcome information imbalance. However, traditional recommendation methods face multiple challenges in cross-domain scenarios, including insufficient user privacy protection, low data utilization efficiency, and lack of interpretability in the recommendation process.

[0003] Recently, many researchers have proposed many cross-domain recommendation models based on knowledge transfer mapping, collaborative matrix factorization, and representation sharing, which transfer user preferences for services according to the commonalities between domains. Most existing recommendation methods integrate data from multiple domains in a centralized manner, which not only poses a serious risk of privacy leakage but also increases users' suspicion of the system, thus affecting the recommendation effect. In addition, it is difficult for recommendation models to effectively model users' preferences in multiple domains, especially in the case of sparse data or lack of unified semantic representation, which further limits the performance of the recommendation system. Summary of the Invention

[0004] The purpose of the present invention is to address the deficiencies of the prior art and propose a cross-domain service recommendation method for enhancing user privacy protection with a large language model, improving the recommendation performance of the in-domain model, and designing two fine-tuning methods: soft prompts based on the common preferences of domain users and hard prompts optimized using the user's historical interaction sequence.

[0005] To solve the above technical problems, the technical solution of the present invention is as follows:

[0006] A cross-domain service recommendation method for enhancing user privacy protection with a large language model, comprising the following steps:

[0007] Step 1: Screen the data in the dataset and design prompts to enhance the service description text and user attribute information using the GPT language model;

[0008] Step 2: Encode the service description text using the BERT language model to generate embeddings;

[0009] Step 3: Use the PQ method to map the text embeddings into discrete codes and maintain a code dictionary;

[0010] Step 4: Read and process the historical service interaction data of users, sort the historical service interaction records of each user in chronological order, and generate corresponding user behavior sequence files. Each line in the file represents the service interaction sequence of a user. Then use the leave-N-out strategy to divide the training, validation, and test data sets;

[0011] Step 5: Represent each service description information in the sequence file using discrete codes, and then perform local training using the processed historical service interaction data, and save the model with the best performance according to the metrics;

[0012] Step 6: Locally upload only the gradients of the model parameters to the server. After the server fuses the gradients, it updates the code dictionary and synchronizes it to the local;

[0013] Step 7: Soft prompt and hard prompt fine-tuning

[0014] Preferably, in step 1, before enhancement, it is necessary to filter the service data set, remove the interaction data with few interactions and the service metadata with missing data, and design appropriate prompts to enhance and fuse through a large language model:

[0015]

[0016]

[0017] Among them, the text prompt and are used for attribute optimization and content enhancement of user u and item i respectively. A u and A i are the user / item features enhanced by a large language model (LLM), and D u and D i represent the text attributes of the data set itself respectively.

[0018] Preferably, in step 2, when using the Bidirectional Encoder Representations from Transformers (BERT) to encode the service description text, add the special input token [CLS] before the initial encoding T i ={w1, w2, …, w j , …, w c} of the description text of each service, that is, x i =BERT([CLS; w1; …; w c ), where [;] represents the concatenation operation, w j is the content word in natural language, c is the truncation length of the item text, and the text encoding vector represents the representation of the given text.

[0019] Preferably, in step 3, the Product Quantization (PQ) method:

[0020] 1) Sub - vector division: Define D groups of vectors, where each vector corresponds to an embedding of dimension d W / D and contains M c centroids. The text - encoding vector x i is first divided into D sub - vectors, namely:

[0021] x i =[x i,1 ,…,x i,D

[0022] 2) Discrete mapping: Let α k,j ∈d W / D represent the j - th centroid embedding in the k - th group of vectors. For each k - th sub - vector x i,k , PQ selects the nearest centroid embedding from the corresponding set of centroids to generate the discrete encoding of x i,k :

[0023]

[0024] where c i,k is the k - th dimensional discrete encoding vector of item i.

[0025] 3) Maintain the code dictionary: The text - encoding vector x i is divided into D sub - vectors, and each sub - vector represents the projection of the original vector in a different subspace. A codebook (a set of predefined centroids) is created for each subspace, and these codebooks together form the code - embedding dictionary. Each sub - vector is quantized to the centroid closest to it in the codebook, and the index of the centroid is used as the "code" for that sub - vector. The quantization results of all sub - vectors are combined into a code vector to represent the original high - dimensional vector.

[0026] Through a lookup operation, the corresponding centroids are retrieved from the global code - embedding dictionary. These centroids are then combined into the final item - representation vector by the average - pooling method. These embedding representations are stored in the code - embedding dictionary for subsequent retrieval and matching.

[0027] Each domain maintains a code - embedding dictionary where d v represents the dimension of the item embedding. There are D code - embedding matrices in T, and each matrix is shared among all items in the server (not limited to a single domain). This feature allows us to align different domains and embed general - domain information into the item embeddings.

[0028] For the retrieval operation of T, the code - embedding of item i can be represented as​ Among them is the matrix T (k) in the embedding vector of the

[0029]

[0030] where v i is the final representation of the item, is the average pooling method in D dimensions, and the code embedding of item i can be expressed as

[0031] Preferably, in step 4, in each service interaction sequence, the service IDs corresponding to the user user's interactions and the corresponding categories are sorted in the order of interaction time, separated by commas; the form is S u ={(ID of service 1),(ID of service 2),…,(ID of service n)}={(i u,1 ),(i u,2 ),…,(i u,n ). The training, validation, and test data sets are divided according to the number of interaction services and the leave-N-out strategy. Set an appropriate value of N, retain N interaction data as the validation set, retain N interaction data as the test set, and the rest as the training set.

[0032] Preferably, in step 5, after each service description information in the behavior sequence file undergoes non-linear transformation through the multi-head attention layer (MH) and the feed-forward neural network (FFN) in the Transformer, at the last position n of the sequence, the generated hidden state is used as the representation form of the entire sequence:

[0033]

[0034] Then use the softmax function for the sequence According to the formula:

[0035]

[0036] to perform score prediction, where v A represents all items in A. Finally, use the cross-entropy loss function to train in the domain A. Save the model with the best performance on the specified metrics, and the same applies to other domains.

[0037] Preferably, in step 6, the local needs to first clip the gradient to the range [-τ, τ] and quantize it into discrete values:

[0038]

[0039] Then upload it to the server, and the server flattens and concatenates the gradients from various fields:

[0040]

[0041] The server determines the weights according to the data volume ratio of the fields and uses the weighted sum average algorithm for gradient fusion:

[0042]

[0043] Before updating the code dictionary in the server and synchronizing it to the local client to strengthen local training, an operation of reshaping G is required, reshaping it to the initial shape and decoding it to the original range.

[0044] Preferably, in step 7, there are two prompting methods: soft prompting and hard prompting.

[0045] Soft prompting: that is, domain prompting P domain , which is composed of d W context words and a domain prompt encoder, and is encoded through a multi-head attention layer (MA). In the encoding process, the sequence embedding h in the pre-trained model is used as the query, and calculations are combined with the prompt content to generate multiple attention heads, and the final prompt encoding is obtained through a linear transformation. The calculation of each attention head includes the linear transformation of the query, key, and value and the calculation of the attention weights.

[0046] Hard prompting: On the basis of soft prompting, it is simulated and constructed in combination with user preferences. Through the LLM as a knowledge-aware sampler, user-item pairs are generated from the training data. That is, after soft prompting, identify the hard samples with high prediction scores in the candidate set C u . These hard samples contain valuable positive and negative samples. It mainly includes the following sub-steps:

[0047] Step 7-2-1: Generate positive and negative samples

[0048] The LLM selects positive samples (including task description, historical interaction, candidate items, and output format) from the candidate items of user u and negative samples to construct an enhanced user-item interaction triple

[0049] Step 7-2-2: Construct an enhanced dataset

[0050] The generated enhanced dataset ε A contains positive and negative sample triples These samples are sampled by the LLM and integrated into the original training set ε to form a joint training set ε∪ε A .

[0051] Step 7-2-3: Train the model

[0052] Optimize the contrastive ranking-based loss function L using the joint dataset BPR . The loss function is optimized by the difference between the predicted scores of the positive samples and the predicted scores of the negative samples , and the formula is as follows:

[0053] where σ(·) is the sigmoid activation function, |Θ| 2 is the weight decay regularization, and the weights are weighted by the parameter ω1. The augmented dataset ε A is a subset of the data generated by the LLM, and its size is controlled by the batch size B and the rate ω2.

[0054] The present invention has the following features and beneficial effects:

[0055] By adopting the above technical solution, the present invention makes full use of the semantic information based on the large language model (LLM) to achieve cross-domain knowledge transfer, and improve the performance and user experience of the recommendation system in multiple domains. The LLM is used to deeply enhance the semantic information of the project description text. On the basis of retaining the original information, domain-specific vocabulary is removed, general explanations are added, and the text representation effect is optimized, so as to improve the application ability of semantic IDs in different domains and significantly enhance the knowledge transfer efficiency between domains. In order to uniformly model projects in different domains, the text descriptions of projects are quantified and mapped into the same semantic space, and semantic IDs are used to represent projects. While improving the generalization of project representation, this method effectively protects the privacy information of users by combining the training framework of federated learning, and avoids the privacy leakage problem that may occur in traditional centralized training. In order to make full use of domain expertise, two fine-tuning strategies are designed: soft prompt and hard prompt. The soft prompt fine-tunes the recommendation results by extracting the common preferences shared by all users to improve the overall recommendation quality. The hard prompt combines the LLM to screen the candidate recommendation results, thereby filtering out inappropriate results, effectively reducing the impact of false positives (FP) and false negatives (FN), and further improving the accuracy and personalization performance of the recommendation system. The present invention solves the problems of insufficient utilization of service text data, easy leakage of privacy data, and low interpretability of recommendation results. Experiments on real datasets show that the cross-domain service recommendation method with enhanced user privacy protection based on large language models has better recommendation capabilities than traditional single-domain and cross-domain recommendation methods, and exceeds traditional service recommendation algorithms in multiple metrics (Precision, Recall, etc.). Especially in dealing with complex domain differences and protecting user privacy, this method shows stronger robustness and practicality, proving its superiority and broad prospects in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings without creative efforts.

[0057] Figure 1 Frame diagram of an embodiment of a cross-domain service recommendation method with enhanced user privacy protection based on a large language model of the present invention.

[0058] Figure 2 Schematic diagram of the content designed for the large language model prompt in the implementation of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments may be combined with each other.

[0060] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, the meaning of "a plurality" is two or more.

[0061] In the description of the present invention, it should be noted that, unless otherwise clearly specified and defined, the terms "mounted", "connected", "coupled" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0062] The present invention provides a large language model enhanced cross - domain service recommendation method for user privacy protection. By using the world knowledge of the large language model (LLM), the domain similarity between the source domain and the target domain is improved. In order not to touch the sensitive privacy information of users when transferring user preferences, we use the federated learning framework, represent items with semantic IDs, and only upload the semantic ID information to the central server for fusion. Finally, in order to further improve the recommendation performance of the in - domain model, we design two prompt fine - tuning methods: soft prompt and hard prompt. The soft prompt is fine - tuned using the common preferences of all users in the domain, and the hard prompt is fine - tuned according to the historical interaction sequence of each user. Its model diagram is as Figure 1 shown and includes the following steps:

[0063] Step 1: Screen the historical service interaction data and service metadata of users. The interaction data includes user ID, service ID, service information (such as ratings), and the time of interaction; the metadata contains descriptive information about the service. Remove the interaction data with an interaction volume less than the specified number and the services without metadata. Design a prompt and input it into the LLM together with the historical service interaction data and service metadata in sequence to obtain the historical service interaction data that the user may prefer and the extended and enhanced service metadata.

[0064] Specifically, the text prompt includes user attribute optimization prompts and item content enhancement prompts The user attribute optimization prompt and item content enhancement prompt generate user enhancement feature A u and item enhancement feature A i through the large language model. Finally, the user enhancement feature A u and item enhancement feature A i are fused with the original features D u and D i of the service dataset. The formula is:

[0065]

[0066] where, represents the feature concatenation or weighted fusion operation, and LLM(·) represents the large language model.

[0067] Step 2: When using the Bidirectional Encoder Representations from Transformers (BERT) to encode the service description text, add the special input token [CLS] before the initial encoding T i ={w1, w2, …, w j , …, w c} of the description text of each service, and then input it into the BERT large language model to obtain the text embedding, that is, x i =BERT([CLS; w1; …; w c ), where [;] represents the concatenation operation, w j is the content word in natural language, c is the truncation length of the item text, and the vector represents the representation of the given text.

[0068] Step 3: Convert the text embedding into discrete codes using the Product Quantization (PQ) method, and represent the service with the codes. The product quantization method includes: dividing the text embedding vector into D sub-vectors, selecting the nearest neighbor centroid index from the centroid set as the discrete code for each sub-vector, and pooling through the centroid embeddings in the code dictionary to generate the final item representation vector.

[0069] Specifically, the Product Quantization (PQ) method:

[0070] 1) Divide sub - vectors: Define D groups of vectors, where each vector corresponds to an embedding of M centroids with a dimension of d W / D and contains M c centroids. The text encoding vector x i is first divided into D sub - vectors, that is:

[0071] x i =[x i,1 , …, x i,D

[0072] 2) Discrete mapping: Let α k,j ∈d W / D represent the j - th centroid embedding in the k - th group of vectors. For each k - th sub - vector x i,k , PQ selects the nearest centroid embedding from the corresponding set of centroids to generate the discrete encoding of x i,k :

[0073]

[0074] where c i,k is the k - th dimensional discrete encoding vector of item i.

[0075] 3) Maintain the code dictionary: The text encoding vector x i is divided into D sub - vectors, and each sub - vector represents the projection of the original vector in different sub - spaces. A codebook (a set of predefined centroids) is created for each sub - space, and these codebooks together form the code embedding dictionary. Each sub - vector is quantized to the centroid closest to it in the codebook, and the index of the centroid is used as the "code" for that sub - vector. The quantization results of all sub - vectors are combined into a code vector to represent the original high - dimensional vector.

[0076] Through a lookup operation, the corresponding centroids are retrieved from the global code embedding dictionary. These centroids are then combined into the final item representation vector by the average pooling method. These embedding representations are stored in the code embedding dictionary for subsequent retrieval and matching.

[0077] Each domain maintains a code embedding dictionary where d v represents the dimension of the item embedding. There are D code embedding matrices in T, and each matrix is shared among all items in the server (not limited to a single domain). This feature allows us to align different domains and embed general domain information into the item embeddings.

[0078] For the retrieval operation of T, the code embedding of item i can be represented as where​ is the embedding vector of the (k) row in matrix T The code embedding obtains the final representation vector of item i through average pooling:

[0079]

[0080] where, v i is the final representation of the item, is the average pooling method over D dimensions, and the code embedding of item i can be expressed as

[0081] Step 4: Read and process the historical service interaction data of users, sort the historical service interaction records of each user in chronological order of timestamps, and generate the corresponding user behavior sequence file. Each line in the file represents the service interaction sequence of a user. Then use the leave-N-out strategy to divide the training, validation, and test data sets.

[0082] Specifically, the service IDs and corresponding categories interacted by the corresponding user user are sorted in chronological order of interaction, separated by commas; the form is:

[0083] S u ={(ID of service 1),(ID of service 2),…,(ID of service n)}={(i u,1 ),(i u,2 ),…,(i u,n )}

[0084] Divide the training, validation, and test data sets according to the number of interaction services and the leave-N-out strategy, set an appropriate value of N, reserve N interaction data as the validation set, reserve N interaction data as the test set, and the rest as the training set.

[0085] Step 5: Convert the service description information in the user behavior sequence into discrete code representations, input it into the Transformer-based sequence encoder for local model training, and save the model with the optimal performance.

[0086] Specifically, in the step 5, each service description information in the behavior sequence file is represented by discrete codes and converted into the encoding {v1, v2, …, v i , …, v n} that can be used by the Transformer sequence encoder. For each input item representation v i , add the corresponding position embedding p j (j is the position of item i in the sequence): Then It is passed to the multi-head attention layer (MH) and the feed-forward neural network (FFN) in the Transformer for non-linear transformation. The process is as follows: H l+1 = FFN(MH(H l ))), where l represents a layer in the feed-forward neural network. After the transformation is completed, at the last position n of the sequence, the generated hidden state is used as the representation of the entire sequence. This representation combines all the information in the input sequence and takes into account the sequence dependencies between positions:

[0087]

[0088] Then, the softmax function is used for the sequence According to the formula:

[0089]

[0090] Score prediction is performed, where v A represents all items in A. Finally, the cross-entropy loss function is used for training in the domain A. The model with the best performance on the specified metrics is saved, and the same applies to other domains.

[0091] Step 6: Under the federated learning framework, each client clips and quantizes the parameter gradients of the local model and uploads them to the server. The server fuses the gradients according to the data volume weights and updates the code dictionary, and synchronizes it to each client.

[0092] Specifically, the local needs to first clip the gradient to the range [-τ, τ] and quantize it to discrete values:

[0093]

[0094] Then upload it to the server, and the server flattens and concatenates the gradients from each domain:

[0095]

[0096] The server determines the weights according to the data volume ratio of the domain and uses the weighted sum average algorithm for gradient fusion:

[0097]

[0098] Update the code dictionary in the server and synchronize it to the local client. Before strengthening the local training, a reshaping operation needs to be performed on G, reshaping it to the initial shape and decoding it to the original range. Update the code dictionary in the server and synchronize it to the local client to strengthen the local training. Repeat the above operations continuously until the model converges in the pre-training stage.

[0099] Step 7: Fine-tune the model using soft prompts and hard prompts.

[0100] Specifically, for soft prompts: Extract specific content from the common preferences of all users in each domain, and use context words and domain prompt encoders to form domain prompt P domain , which consists of d W context words and is encoded through a multi-head attention layer (MA). In the encoding process, the sequence embedding h in the pre-trained model is used as a query, and calculations are combined with the prompt content to generate multiple attention heads, and the final prompt encoding is obtained through a linear transformation. The calculation of each attention head includes linear transformations of the query, key, and value and the calculation of attention weights, thereby capturing the correlation between the sequence embedding and the domain prompt.

[0101] For hard prompts: Hard prompts are constructed by simulating and combining user preferences on the basis of soft prompts. Using the LLM as a knowledge-aware sampler, user-item pairs are generated from the training data. Specifically, hard prompts provide auxiliary information (such as year, category) of historical interactions and candidate item pools for each user, and are represented in text form as C u ={i u,1 , i u,2 ,..., i u,|Cu| ,}. Since the LLM cannot comprehensively rank all items, it will identify hard samples with high prediction scores in the candidate set C u after the soft prompt. These hard samples contain valuable positive and negative samples, providing enhanced training signals for model training.

[0102] The main steps of the hard prompt process include:

[0103] 1. Generate positive and negative samples

[0104] The LLM selects positive samples (including task descriptions, historical interactions, candidates, and output formats) and negative samples from the candidate items of user u to construct enhanced user-item interaction triples

[0105] 2. Construct an enhanced dataset

[0106] The generated enhanced dataset ε A contains positive and negative sample triples These samples are sampled by the LLM and integrated into the original training set ε to form a joint training set ε ∪ ε A .

[0107] 3. Train the model

[0108] Optimizing the contrastive ranking-based loss function L using a combined dataset BPR . The loss function optimizes the difference between the predicted scores of positive samples and the predicted scores of negative samples as follows:

[0109] where σ(·) is the sigmoid activation function, |Θ| 2 is weight decay regularization, and the weights are weighted by the parameter ω1. The augmented dataset ε A is a subset of the data generated by the LLM, and its size is controlled by the batch size B and the rate ω2.

[0110] Based on a user privacy protection cross-domain service recommendation method enhanced by a large language model provided in this embodiment, a specific simulation example is provided: The cross-domain recommendation model used therein is denoted as PCR-LLM.

[0111] The original data used in this embodiment are five widely used real-world scenario datasets: Scientific, Arts, Pantry, Movies, Online Retail. Among them, Scientific, Art, Pantry, and Movie are all from Amazon. The Online dataset is used to verify the robustness of the model to avoid all data coming from Amazon. In the Scientific dataset, 8,443 users interacted with 4,385 services for 50,981 times, and the data sparsity rate was 99.8623%; in the Arts dataset, 45,487 users interacted with 21,019 services for 349,664 times, and the data sparsity rate was 99.9634%; in Pantry, 13,102 users interacted with 4,898 users for 113,861 times, and the data sparsity rate was 99.8226%; in the Movies dataset, 281,701 users interacted with 59,204 services for 2,942,031 times, and the data sparsity rate was 99.9925%; in the Online Retail dataset, 16,521 users interacted with 3,469 services for 503,386 times, and the data sparsity rate was 99.1217%. All datasets have been filtered to remove user interaction records with few interactions and services with missing metadata.

[0112] In this embodiment, some parameters of PCR-LLM are selected as follows: Each input Token in the pre-trained language model is mapped to a 768-dimensional vector; the number of subspaces in the product quantization method is set to 32; the iteration termination condition set in the pre-training stage of the cross-domain recommendation model is that the total loss function value converges or the number of iteration rounds reaches 100 rounds; the iteration termination condition in the model fine-tuning stage is that the evaluation index does not improve within 10 rounds or the number of iteration rounds reaches 100 rounds.

[0113] In addition, the experiments in this embodiment also compared the method of the present invention with several traditional prediction methods. The single-domain traditional prediction methods used as controls are: (1) SASRec: A sequential recommendation method based on self-attention; (2) BERT4Rec: A sequential recommendation method based on the bidirectional encoder representation of the transformer.

[0114] (3) GRU4Rec: A session-based recommendation method using a recurrent neural network. The multi-domain traditional prediction methods are: (1) CCDR: A contrastive cross-domain recommendation method in matching; (2) RecGURU: An adversarial learning method for generalized user representation in cross-domain recommendation. This embodiment uses Recall and NDCG as evaluation metrics for the prediction model. Recall measures the proportion of the top K relevant items recommended by the model among all relevant items and is used to evaluate the coverage ability of the recommendation system. NDCG (Normalized Discounted Cumulative Gain) is a ranking quality evaluation metric that measures the relevance and ranking quality of the recommendation results. In the experiments, K = {10, 50} is set for both.

[0115] Table 1 and Table 2 Comparison of Experimental Results between the Method of the Present Invention and the Control Methods

[0116] Table 1: Performance Comparison on Pantry and Online

[0117]

[0118] Table 2: Performance Comparison on Scientific and Arts

[0119]

[0120] The final experimental results are shown in Table 1 and Table 2. It can be seen that PCR-LLM in the method of the present invention achieves better results than the control model on the Pantry-Online and Scientific-Arts datasets. Specifically, in terms of the Recall metric, the PCR-LLM model has a minimum improvement of 1.30% and a maximum improvement of 32.43% compared to the single-domain recommendation method, and a minimum improvement of 5.50% and a maximum improvement of 35.46% compared to the multi-domain recommendation method; in terms of the NDCG metric, it has a minimum improvement of 1.68% and a maximum improvement of 37.92% compared to the single-domain recommendation method, and a minimum improvement of 2.22% and a maximum improvement of 37.95% compared to the multi-domain recommendation method. It is worth mentioning that after two fine-tunings with soft prompts and hard prompts, the performance has been steadily improved, with a minimum improvement of 9.77% and a maximum improvement of 36.90%. Thus, it can be seen that large language models significantly improve the model recommendation performance.

[0121] The embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, without departing from the principle and spirit of the present invention, various changes, modifications, substitutions, and variations to these embodiments including components still fall within the protection scope of the present invention.

Claims

1. A large language model-enhanced user privacy protection cross-domain service recommendation method, characterized in that: The following steps are involved: Step 1: Filter the service data sets of the source domain and the target domain, wherein the service data sets include service description text and user attribute information. First, filter the service data sets, and then use the preset text prompts to semantically enhance and fuse the filtered service data sets through a large language model; Step 2: Encode the enhanced service description text using the transformer represented by the bidirectional encoder to generate a high-dimensional text embedding; Step 3: Mapping the high-dimensional text embedding into discrete codes by a product quantization method, and constructing a code dictionary, wherein the code dictionary includes a centroid set of multiple subspaces; Step 4: Read the user's historical service interaction data, generate a user behavior sequence file in timestamp order, and use the leave-N-out strategy to divide the training set, validation set, and test set; Step 5: Convert the service description information in the user behavior sequence into discrete code representation, input it into the Transformer-based sequence encoder for local model training, and save the model with the best performance; Step 6: Under the federated learning framework, each client clips and quantizes the parameter gradient of the local model and uploads it to the server. The server fuses the gradient according to the data volume weight and updates the code dictionary, which is synchronized to each client. Step 7: Fine-tune the model using soft and hard prompts.

2. The method according to claim 1, characterized in that In step 1, the filtering method is to remove interaction data with interaction volume lower than a preset threshold and services with missing metadata.

3. The method according to claim 1, characterized in that In step 1, the text prompt includes a user attribute optimization prompt And item content enhancement prompt P i P , the user attribute optimization tips And item content enhancement prompt P i P User-enhanced features A generated by a large language model u and item enhancement feature A i Finally, the user enhanced feature A u and item enhancement feature A i The original features D of the service dataset u and D i The fusion formula is: in, represents feature concatenation or weighted fusion operation, and LLM(·) represents a large language model.

4. The method according to claim 1, characterized in that: In step 2, when the transformer represented by the bidirectional encoder is used to encode the service description text, the initialization code T of the description text of each service is i ={w1,w2,…,w j ,…,,w c } Add a special input tag [CLS] before it, that is, x i =BERT([CLS; w1; ...; w c ]), where [;] represents the concatenation operation, w j is the content word in natural language, c is the cutoff length of the item text, and the vector Representation for the given text.

5. The method according to claim 1, characterized in that In step 3, the product quantization method includes: dividing the text embedding vector into D sub-vectors, selecting the nearest neighbor centroid index from the centroid set as a discrete code for each sub-vector, and pooling through the centroid embedding in the code dictionary to generate a final item representation vector.

6. The method according to claim 5, characterized in that The pooling operation is average pooling, and the formula is: Among them, v i is the final representation of the item, is an average pooling method in D dimensions. The code embedding of item i can be expressed as 7. The method according to claim 1, characterized in that In step 4, in each user behavior sequence file, the service IDs and corresponding categories that the corresponding user has interacted with are sorted in the order of interaction time, separated by commas; the format is: S u ={(ID of service 1),(ID of service 2),…,(ID of service n)}={(i u,1 ),(i u,2 ),…,(i u,n )} Divide the training, validation, and test data sets according to the number of interactive services and the leave-N-out strategy, set an appropriate N value, retain N interactive data as the validation set, retain N interactive data as the test set, and the rest as the training set.

8. The method according to claim 1, characterized in that In step 6, the local needs to first convert the gradient Clipped to the range [-τ,τ] and quantized to discrete values: Then upload it to the server, which flattens and concatenates the gradients from each field: The server determines the weights based on the data volume ratio of the fields and uses the weighted sum average algorithm to perform gradient fusion: Update the code dictionary in the server and synchronize it to the local client. Before strengthening local training, G needs to be reshaped to its initial shape and decoded to its original range.

9. The method according to claim 1, characterized in that: In step 7, the soft prompt fine-tuning is to generate prompt context words based on the common preferences of users in the field, and optimize the recommendation results after encoding through a multi-head attention layer.

10. The method according to claim 1, characterized in that In step 7, the specific steps of fine-tuning the hard prompt include: Use a large language model to generate positive and negative sample pairs based on user historical interactions and candidate service information Then construct an enhanced dataset ε A , where the enhanced dataset ε A Contains positive and negative sample triplets And combined with the original training set to optimize the comparison ranking loss function L BPR .

11. The method according to claim 10, characterized in that The formula of the comparison ranking loss function is: in, and denote the predicted scores of positive and negative samples respectively, ω1 is the weight parameter, σ(·) is the sigmoid activation function, |Θ| 2 Heavy decay regularization.

12. The method according to claim 1, characterized in that In step 5, the sequence encoder uses a multi-head self-attention layer and a position-aware feedforward neural network, and is trained using a cross-entropy loss function.

13. The method according to any one of claims 1 to 12, characterized in that: The large language model includes GPT, BERT or their variants, which are used for text enhancement, semantic encoding or candidate service screening.

Images