A recommendation method and apparatus based on a large language model

Abstract

This invention discloses a recommendation method and apparatus based on a large language model. The method includes: acquiring user question information; processing the user question information to obtain question answer information. This invention provides a recommendation method and apparatus based on a large language model, utilizing knowledge graph data and real-time updated retrieval-enhanced question-and-answer data. By integrating knowledge graph question-and-answer and retrieval-enhanced question-and-answer, it invokes auxiliary tools, addressing the need for specialized computing tools to assist in question-and-answer processes in decision-making scenarios.

Description

Technical Field

[0001] This invention relates to the field of recommender systems, and more particularly to a recommender method and apparatus based on a large language model. Background Technology

[0002] Currently, generative recommendation schemes based on large models have become an important research direction in the field of recommender systems. These schemes aim to combine the natural language understanding and generation capabilities of large models to improve the personalization, interpretability, and generalization ability of recommendations. Some commonly used technical solutions are shown below:

[0003] (1) Generative recommendation technology based on sequence recommendation

[0004] Generative recommendation technology based on sequence recommendation is a recommendation paradigm that dynamically models user behavior sequences using generative models. Its core idea is to directly predict items a user might interact with in the future using their historical interaction sequences (such as clicks, browsing, and purchases), rather than relying on traditional candidate set filtering based on matching or ranking. This method overcomes the limitations of traditional sequence models in modeling long dependencies and supports the generation of diverse recommendation results.

[0005] (2) Generative recommendation technology based on semantic ID

[0006] Generative recommendation technology based on semantic IDs is a cutting-edge paradigm that reconstructs item representations using structured semantic identifiers (Semantic IDs) and dynamically generates recommendation results using generative models. Its core idea is to encode the semantic information of items (such as category, attributes, and content features) into hierarchical, interpretable discrete identifiers (Semantic IDs), replacing traditional continuous embedding vectors or single ID representations. This explicitly models the semantic relationships between items, supports fine-grained inference and cross-domain generalization of generative models to user interests, and can capture subtle semantic relationships between products.

[0007] As new items constantly emerge and users' interests and behaviors change dynamically, discrete semantic IDs are not flexible enough in a dynamic environment and cannot quickly adapt to these changes, thus affecting the recommendation effect and stability, resulting in low accuracy and practicality of the recommendation results. Summary of the Invention

[0008] The technical problem to be solved by this invention is to provide a recommendation method and apparatus based on a large language model. From the perspective of practical problems and applications, it adopts a hybrid semantic ID based on contrastive learning and a large model training method based on GRPO, which effectively improves the semantic association between user information and item description, as well as the reliability of recommendations generated by the large model.

[0009] To address the aforementioned technical problems, a first aspect of this invention discloses a recommendation method based on a large language model, characterized in that the method includes:

[0010] S1, Obtain training data;

[0011] S2, The training data is used to train the large language model to obtain an optimized hybrid semantic ID model and a generative recommendation large language model;

[0012] S3, obtain user demand data;

[0013] S4. Using the optimized hybrid semantic ID model and the generative recommendation large language model, the user demand data is processed to obtain recommendation sequence data.

[0014] As an optional implementation, in the first aspect of the present invention, the step of training the large language model using the training data to obtain an optimized hybrid semantic ID model and a generative recommendation large language model includes:

[0015] S21, the training data is encoded to obtain first user latent vector data, first item latent vector data and first semantic ID latent vector data;

[0016] S22, perform vector concatenation processing on the first user potential vector data and the first project potential vector data to obtain the first user mixed vector data;

[0017] S23, perform residual quantization processing on the first user hybrid vector data to obtain the first hybrid semantic ID data;

[0018] S24, the large model is trained using the first hybrid semantic ID data and the first semantic ID latent vector data to obtain the optimized hybrid semantic ID model and the generative recommendation large language model.

[0019] As an optional implementation, in a first aspect of the present invention, the residual quantization processing of the first user hybrid vector data to obtain the first hybrid semantic ID data includes:

[0020] S231, preset the total number of residual processing stages N and the number of residual processing stages i = 0;

[0021] S232, when the number of residual processing levels is less than the total number of residual processing levels, perform residual quantization processing on the first user mixed vector data to obtain quantization result data, and execute S233; otherwise, execute S234;

[0022] S233, increase the residual processing level by 1, and execute S232;

[0023] S234, the quantization result data is encoded and reconstructed to obtain the first hybrid semantic ID data.

[0024] As an optional implementation, in a first aspect of the present invention, the step of performing stepwise residual quantization processing on the first user hybrid vector data to obtain quantization result data includes:

[0025] S2321, the default codebook for each level is {B1,B2,…,B} N}; Initialize the original residual l0 = x0;

[0026] S2322, using the residual processing level i and the codebook R corresponding to the residual processing level i. i The residual calculation process is performed on the first user's mixed vector data to obtain vector residual result data;

[0027] The residual calculation processing expression is as follows:

[0028] l i =l i-1 -r i ;

[0029]

[0030] Among them, l i x represents the residual value of the i-th level quantization process; i Represents the i-th mixed vector data of the first user; r i Indicates the relationship between the current codebook and the residual l i-1 The closest vector; r represents codebook B i The learnable vectors included;

[0031] S2323, Quantize the vector residual result data to obtain quantized result data;

[0032] The quantization expression is:

[0033]

[0034] Among them, L i This represents the quantization result of the i-th level.

[0035] As an optional implementation, in a first aspect of the present invention, training a large model using the first hybrid semantic ID data and the first semantic ID latent vector data to obtain an optimized hybrid semantic ID model and a generative recommendation large language model includes:

[0036] S241, Based on the contrastive loss function, the hybrid semantic ID model is compared and learned by the first hybrid semantic ID data and the first semantic ID latent vector data to obtain an optimized hybrid semantic ID model;

[0037] The expression for the contrastive loss function is:

[0038]

[0039] Among them, L con Indicates comparative loss; This represents the semantic loss of the user; Let U represent the semantic loss of the project, where U represents the user, I represents the project, and M represents the large language model.

[0040] The contrastive learning expression is:

[0041]

[0042] Where N represents the number of positive samples, 2N represents the number of samples after augmentation; sim(·,·) represents the cosine similarity; z i z represents the i-th latent vector data of the first semantic ID; j Let represent the j-th latent vector data of the first semantic ID; τ represents the temperature parameter used to control the shape of the loss function; k represents the sample number; z k This represents the k-th potential vector data of the first semantic ID;

[0043] S242, the first mixed semantic ID data is parsed to obtain the first mixed SID data and the first item sequence label mixed SID data;

[0044] S243, using the first mixed SID data and the first item sequence label mixed SID data, the large language model is trained to obtain a generative recommendation large language model.

[0045] As an optional implementation, in a first aspect of the present invention, the step of processing the user demand data using the optimized hybrid semantic ID model and the generative recommendation large language model to obtain recommendation sequence data includes:

[0046] S41, using the optimized hybrid semantic ID model, the user demand data is processed to obtain the second hybrid semantic ID data;

[0047] S42, using the generative recommendation large language model, the second mixed semantic ID data and sequence prediction prompt word data are processed to obtain recommendation sequence data.

[0048] As an optional implementation, in a first aspect of the present invention, processing the user demand data using the optimized hybrid semantic ID model to obtain second hybrid semantic ID data includes:

[0049] S411, The user demand data is parsed and processed to obtain user ID data, user profile data, project ID data, project title data and project description data;

[0050] S412, using the optimized hybrid semantic ID model, the user ID data and the user profile data are processed to obtain the second user potential vector data;

[0051] S413, using the optimized hybrid semantic ID model, the project ID data, the project title data, and the project description data are processed to obtain the second project potential vector data;

[0052] S414, perform vector concatenation processing on the second user potential vector data and the second item potential vector data to obtain the second mixed semantic ID data.

[0053] A second aspect of the present invention discloses a recommendation device based on a large language model, the device comprising: a first acquisition module, a model training module, and a second acquisition model and data processing module;

[0054] The first acquisition module is used to acquire training data;

[0055] The model training module is used to train the large language model using the training data to obtain an optimized hybrid semantic ID model and a generative recommendation large language model.

[0056] The second acquisition module allows users to acquire user demand data;

[0057] The data processing module is used to process the user demand data using the optimized encoder model and the generative recommendation language model to obtain recommendation sequence data.

[0058] A third aspect of the present invention discloses another recommendation device based on a large language model, the device comprising:

[0059] Memory containing executable program code;

[0060] A processor coupled to the memory;

[0061] The processor calls the executable program code stored in the memory to execute some or all of the steps in the recommendation method based on a large language model disclosed in the first aspect of the present invention.

[0062] The fourth aspect of the present invention discloses a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, which, when invoked, execute some or all of the steps in the recommendation method based on a large language model disclosed in the first aspect of the present invention.

[0063] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0064] In this embodiment of the invention, a hybrid semantic ID model is adopted to co-encode user behavior information and item information, so that the generated semantic ID remains stable even when user behavior fluctuates frequently, thereby ensuring the consistency and robustness of the recommendation results. Contrastive learning is used to train the hybrid semantic ID model, which improves the correlation between user and item information and the semantic understanding ability of the encoder, thereby improving the accuracy of the recommendation generation results. Reinforcement learning is used to train the large language model, which improves the accuracy and interpretability of the recommendation system in sequence prediction. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 This is a schematic diagram of a scenario for a recommendation device based on a large language model provided in an embodiment of the present invention;

[0067] Figure 2 This is a flowchart illustrating a recommendation method based on a large language model disclosed in an embodiment of the present invention;

[0068] Figure 3 This is a schematic diagram of the structure of a recommendation system based on a large language model disclosed in an embodiment of the present invention;

[0069] Figure 4 This is a schematic diagram of another recommendation device based on a large language model disclosed in an embodiment of the present invention. Detailed Implementation

[0070] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0072] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0073] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0074] It should be noted that since the method in this application embodiment is executed in a computer device, the processing objects of each computer device exist in the form of data or information, such as time, which is essentially time information. It is understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data that exist so that the computer device can process them. Specific details will not be elaborated here.

[0075] It should be noted that the term "large model" in this application refers to an artificial neural network model with a very large number of parameters. In the field of artificial intelligence, a large model typically refers to a model with hundreds of millions to trillions of parameters. These models usually require training on large-scale datasets and necessitate significant computational resources for optimization and tuning. Large models are commonly used to solve complex tasks such as natural language processing, computer vision, and speech recognition. Generative AI is a type of AI capable of creating new content and ideas, including dialogues, stories, images, videos, and music. In the embodiments of this application, the large model can be a language model of the scale of ChatGPT, BERT, XLNet, Zhipu model, Claude, MoonshotAI model, ChatGLM model, Qianyitongwen model, MiniMax model, Xinghuo model, Llama model, 360GPT model, Qwen model, Baichuan model, Yunque model, vivoLM model, and Wenxin Yiyan, etc., and this application does not limit the scope of the large model.

[0076] This application provides a recommendation method, apparatus, computer device, and computer-readable storage medium based on a large language model, which will be described in detail below.

[0077] Please see Figure 1 , Figure 1 This is a schematic diagram of a decision management system provided in an embodiment of this application. The decision management system based on a large language model may include a computer device 100, which integrates a recommendation device based on a large language model, such as... Figure 1 Computer equipment in the country.

[0078] In this embodiment, the computer device 100 can be a standalone server, a server network, or a server cluster. For example, the computer device 100 described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0079] It is understood that the computer device 100 used in the embodiments of this application can be a device that includes both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such a device may include: cellular or other communication devices having a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display. Specifically, the computer device 100 may be a desktop terminal or a mobile terminal, and may also be one of a mobile phone, tablet computer, laptop computer, etc.

[0080] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include those that are more specific to this application. Figure 1 The number of computer devices shown is more or less, for example Figure 1 Only one computer device is shown in the diagram. It is understood that the system may also include one or more other services, which are not limited here.

[0081] In addition, such as Figure 1 As shown, the decision management system may also include a storage device 200 for historical data, such as user demand data, recommendation process data, and recommendation result data.

[0082] It should be noted that, Figure 1 The schematic diagram of the question-answering decision system shown is merely an example. The recommendation device and scenario based on the large language model described in this application embodiment are for the purpose of more clearly illustrating the technical solutions of this application embodiment and do not constitute a limitation on the technical solutions provided in this application embodiment. As those skilled in the art will know, with the evolution of question-answering decision systems and the emergence of new business scenarios, the technical solutions provided in this application embodiment are also applicable to similar technical problems.

[0083] This invention discloses a recommendation method and apparatus based on a large language model. Utilizing knowledge graph data and real-time updated retrieval-enhanced question-answering data, it integrates knowledge graph question-answering and retrieval-enhanced question-answering, and invokes auxiliary tools to address the need for specialized computing tools in conjunction with question-answering in decision-making scenarios. Detailed explanations follow.

[0084] Example 1

[0085] Please see Figure 2 , Figure 2 This is a flowchart illustrating a recommendation method based on a large language model disclosed in an embodiment of the present invention. Figure 2 The described recommendation method based on a large language model is applied to question-answering decision systems, such as local servers or cloud servers used in question-answering decision systems; however, this embodiment of the invention is not limited to such applications. Figure 2 As shown, this recommendation method based on a large language model can include the following operations:

[0086] S1, Obtain training data;

[0087] It should be noted that the training data is used to train the large language model; it includes training user ID data, training user profile data, training item ID data, training item title data, training item description data, and training item sequence label data.

[0088] It should be noted that, in this embodiment, the training data includes user behavior information and product information; the user behavior information includes, but is not limited to, user search record data, user click data, user purchase history data, and user profile data; the product information includes, but is not limited to, product ID data, product title data, product description information, and user review information.

[0089] S2, The training data is used to train the large language model to obtain an optimized hybrid semantic ID model and a generative recommendation large language model;

[0090] S3, obtain user demand data;

[0091] It should be noted that the user demand data includes user ID data, user profile data, project ID data, project title data, and project description data;

[0092] It should be noted that, in this embodiment,

[0093] S4. Using the optimized hybrid semantic ID model and the generative recommendation large language model, the user demand data is processed to obtain recommendation sequence data.

[0094] As can be seen, by implementing the recommendation method based on a large language model described in the embodiments of the present invention, using contrastive learning to train the hybrid semantic ID model and reinforcement learning to train the large language model, the consistency, robustness and accuracy of the recommendation results are improved, thereby enhancing the accuracy and interpretability of the recommendation system in sequence prediction.

[0095] Optionally, the step of training the large language model using the training data to obtain an optimized hybrid semantic ID model and a generative recommendation large language model includes:

[0096] S21, the training data is encoded to obtain first user latent vector data, first item latent vector data and first semantic ID latent vector data;

[0097] It should be noted that in this embodiment, user search record data, user click data, user purchase history data and user profile data are encoded to obtain the first user potential vector data;

[0098] The product ID data, product title data, product detailed description information, and user review information are encoded to obtain the first item's potential vector data;

[0099] S22, perform vector concatenation processing on the first user potential vector data and the first project potential vector data to obtain the first user mixed vector data;

[0100] It should be noted that the splicing process refers to splicing data in sequence;

[0101] S23, perform residual quantization processing on the first user hybrid vector data to obtain the first hybrid semantic ID data;

[0102] S24, the large model is trained using the first hybrid semantic ID data and the first semantic ID latent vector data to obtain the optimized hybrid semantic ID model and the generative recommendation large language model.

[0103] As can be seen, implementing the recommendation method based on a large language model as described in the embodiments of the present invention, and using the training data to train the large language model to obtain an optimized hybrid semantic ID model and a generative recommendation large language model, lays the foundation for improving the consistency, robustness, and accuracy of the recommendation results, and enhancing the accuracy and interpretability of the recommendation system in sequence prediction.

[0104] Optionally, the encoding process of the training data to obtain the first user latent vector data, the first item latent vector data, and the first semantic ID latent vector data includes:

[0105] S211, The training data is parsed and processed to obtain training user ID data, training user profile data, training project ID data, training project title data, training project description data, and training project sequence label data;

[0106] It should be noted that the parsing process refers to extracting the corresponding data according to the field type;

[0107] S212, using the text encoder of the large language model, the training user ID data and the training user profile data are encoded to obtain the first user latent vector data;

[0108] S213, using the text encoder of the large language model, the training item ID data, the training item title data and the training item description data are encoded to obtain the first item latent vector data;

[0109] S214, using the text encoder of the large language model, the training item sequence label data is encoded to obtain the first semantic ID potential vector data;

[0110] As can be seen, by implementing the recommendation method based on a large language model as described in the embodiments of the present invention, the training data is encoded to obtain the first user latent vector data, the first item latent vector data, and the first semantic ID latent vector data. This provides data support for the subsequent training of the hybrid semantic ID model and the large language model, and lays the foundation for improving the consistency, robustness, and accuracy of the recommendation results, as well as enhancing the accuracy and interpretability of the recommendation system in sequence prediction.

[0111] Optionally, the step of performing residual quantization processing on the first user hybrid vector data to obtain the first hybrid semantic ID data includes:

[0112] S231, preset the total number of residual processing stages N and the number of residual processing stages i = 0;

[0113] S232, when the number of residual processing levels is less than the total number of residual processing levels, perform residual quantization processing on the first user mixed vector data to obtain quantization result data, and execute S233; otherwise, execute S234;

[0114] S233, increase the residual processing level by 1, and execute S232;

[0115] S234, Encode and reconstruct the quantization result data to obtain the first hybrid semantic ID data;

[0116] As can be seen, by implementing the recommendation method based on a large language model described in this embodiment of the invention, residual quantization is performed on the first user hybrid vector data to obtain the first hybrid semantic ID data. The hybrid semantic ID model is used to encode user behavior information and item information together, so that the generated semantic ID remains stable even when user behavior fluctuates frequently. This ensures the consistency and robustness of the recommendation results, and lays the foundation for improving the consistency, robustness and accuracy of the recommendation results, as well as enhancing the accuracy and interpretability of the recommendation system in sequence prediction.

[0117] Optionally, the step of performing stepwise residual quantization processing on the first user hybrid vector data to obtain quantization result data includes:

[0118] S2321, the default codebook for each level is {B1,B2,…,B} N}; Initialize the original residual l0 = x0;

[0119] S2322, using the residual processing level i and the codebook R corresponding to the residual processing level i. i The residual calculation process is performed on the first user's mixed vector data to obtain vector residual result data;

[0120] The residual calculation processing expression is as follows:

[0121] l i =l i-1 -r i ;

[0122]

[0123] Among them, l i x represents the residual value of the i-th level quantization process; i Represents the i-th mixed vector data of the first user; r i Indicates the relationship between the current codebook and the residual l i-1 The closest vector; r represents codebook B i The learnable vectors included;

[0124] S2323, Quantize the vector residual result data to obtain quantized result data;

[0125] The quantization expression is:

[0126]

[0127] Among them, L i This represents the quantization result of the i-th level.

[0128] As can be seen, by implementing the recommendation method based on the large language model described in the embodiments of the present invention, the first user's hybrid vector data is subjected to stepwise residual quantization processing to obtain quantized result data, which provides data support for the subsequent training of the hybrid semantic ID model and the large language model. This lays the foundation for improving the consistency, robustness and accuracy of the recommendation results, and enhancing the accuracy and interpretability of the recommendation system in sequence prediction.

[0129] Optionally, the step of training the large model using the first hybrid semantic ID data and the first semantic ID latent vector data to obtain an optimized hybrid semantic ID model and a generative recommendation large language model includes:

[0130] S241, Based on the contrastive loss function, the hybrid semantic ID model is compared and learned by the first hybrid semantic ID data and the first semantic ID latent vector data to obtain an optimized hybrid semantic ID model;

[0131] The expression for the contrastive loss function is:

[0132]

[0133] Among them, L con Indicates comparative loss; This represents the semantic loss of the user; Let U represent the semantic loss of the project, where U represents the user, I represents the project, and M represents the large language model.

[0134] The contrastive learning expression is:

[0135]

[0136] Where N represents the number of positive samples, 2N represents the number of samples after augmentation; sim(·,·) represents the cosine similarity; z i z represents the i-th latent vector data of the first semantic ID; j Let represent the j-th latent vector data of the first semantic ID; τ represents the temperature parameter used to control the shape of the loss function; k represents the sample number; z k This represents the k-th potential vector data of the first semantic ID;

[0137] S242, the first mixed semantic ID data is parsed to obtain the first mixed SID data and the first item sequence label mixed SID data;

[0138] S243, using the first mixed SID data and the first item sequence label mixed SID data, the large language model is trained to obtain a generative recommendation large language model.

[0139] It should be noted that in this embodiment, the large language model is trained using the GRPO algorithm.

[0140] As can be seen, by implementing the recommendation method based on the large language model described in the embodiments of the present invention, the first user's hybrid vector data is subjected to stepwise residual quantization processing to obtain quantized result data, which provides data support for the subsequent training of the hybrid semantic ID model and the large language model, improves the consistency, robustness and accuracy of the recommendation results, and lays the foundation for improving the accuracy and interpretability of the recommendation system in sequence prediction.

[0141] Optionally, the step of training the large language model using the first mixed SID data and the first item sequence label mixed SID data to obtain a generative recommendation large language model includes:

[0142] Based on the reward evaluation model, the GRPO algorithm is used to train the large language model using the first mixed SID data and the first item sequence label mixed SID data to obtain a generative recommendation large language model.

[0143] The expression for the reward evaluation model is:

[0144] J = αJ o +βJ f +μJ c ;

[0145]

[0146] Where J represents the reward value; α represents the sequential reward coefficient; J o Represents the ordinal reward value; β represents the formatted reward coefficient; J f The format reward value is represented by μ; the accuracy reward coefficient is represented by J. c Represents the accuracy reward value; q represents the index of the current recommendation in the recommendation sequence; Q represents the number of elements in the recommendation sequence; U represents the index of the current recommendation in the recommendation sequence. q S represents the current recommended user order value; q Indicates the currently recommended system ordinal value; F p S represents the p-th recommended format conformance value; p This represents the accuracy value of the p-th recommendation;

[0147] It should be noted that in this embodiment, α is set to 0.3; β is set to 0.2; and μ is set to 0.5.

[0148] It should be noted that, in this embodiment, when the recommended item format meets the user's requirements, the format reward value J is... f Set to 1, otherwise set to 0;

[0149] It should be noted that, in this embodiment, the accuracy reward value J c Set to 1-5 points;

[0150] It should be noted that the GRPO (Group Relative Policy Optimization) algorithm is a reinforcement learning algorithm that improves upon traditional proximal policy optimization by eliminating the reliance on a separate value network. Specifically, in each state, GRPO samples a set of candidate outputs from the old policy and then measures the relative advantage of each output by normalizing the reward of this set of outputs (calculating a standardized score for each output relative to the average reward within the group). Next, the algorithm uses a pruning mechanism similar to PPO to update the policy, i.e., while keeping the update magnitude from being too large, it makes the model more inclined to generate outputs with higher relative advantages. To prevent the new policy from deviating excessively from the reference policy, a KL divergence penalty term is also added. This method not only reduces training variance and saves computational resources, but also achieves excellent results in reinforcement learning tuning of large-scale language models.

[0151] It should be noted that in this embodiment, three rewards are defined for reinforcement learning: order reward, format reward, and accuracy reward. The order reward is used to improve the accuracy of the large model in predicting the order of sequences in recommendation where there is a sequence. The format reward is used to constrain the semantic IDs generated by the large language model to conform to the required format, such as the semantic ID format indicating that the output result reflects a thought process.<a_4> The accuracy reward mentioned above is key to improving the prediction accuracy of large language models; large language models trained using the GPRO reinforcement learning algorithm have strong reasoning capabilities, which alleviates the interpretability of large language models to some extent; in addition, large language models trained based on reinforcement learning algorithms can achieve better generalization capabilities and obtain better stability and robustness in recommendation systems.

[0152] As can be seen, by implementing the recommendation method based on a large language model described in the embodiments of the present invention, the large language model is trained using the first mixed SID data and the first item sequence label mixed SID data to obtain a generative recommendation large language model. This lays the foundation for subsequent optimization of the mixed semantic ID model and the generative large language model to obtain recommendation sequence data, improves the consistency, robustness and accuracy of the recommendation results, and enhances the accuracy and interpretability of the recommendation system in sequence prediction.

[0153] Optionally, the step of processing the user demand data using the optimized hybrid semantic ID model and the generative recommendation large language model to obtain recommendation sequence data includes:

[0154] S41, using the optimized hybrid semantic ID model, the user demand data is processed to obtain the second hybrid semantic ID data;

[0155] S42, using the generative recommendation large language model, the second mixed semantic ID data and sequence prediction prompt word data are processed to obtain recommendation sequence data.

[0156] As can be seen, implementing the recommendation method based on a large language model described in the embodiments of the present invention, using the generative recommendation large language model to process the user demand data to obtain recommendation sequence data, improves the consistency, robustness and accuracy of the recommendation results, and lays the foundation for improving the accuracy and interpretability of the recommendation system in sequence prediction.

[0157] Optionally, the step of processing the user demand data using the optimized hybrid semantic ID model to obtain second hybrid semantic ID data includes:

[0158] S411, The user demand data is parsed and processed to obtain user ID data, user profile data, project ID data, project title data and project description data;

[0159] S412, using the optimized hybrid semantic ID model, the user ID data and the user profile data are processed to obtain the second user potential vector data;

[0160] It should be noted that the aforementioned processing refers to encoding the user ID data and the user profile data using the optimized hybrid semantic ID model.

[0161] S413, using the optimized hybrid semantic ID model, the project ID data, the project title data, and the project description data are processed to obtain the second project potential vector data;

[0162] It should be noted that the aforementioned processing refers to encoding the project ID data, the project title data, and the project description data using the optimized hybrid semantic ID model.

[0163] S414, perform vector concatenation processing on the second user potential vector data and the second item potential vector data to obtain the second mixed semantic ID data.

[0164] As can be seen, by implementing the recommendation method based on a large language model as described in the embodiments of the present invention, the user demand data is processed to obtain the second hybrid semantic ID data, which improves the consistency, robustness and accuracy of the recommendation results, and lays the foundation for improving the accuracy and interpretability of the recommendation system in sequence prediction.

[0165] Optionally, the step of processing the second hybrid semantic ID data using the generative recommendation large language model to obtain recommendation sequence data includes:

[0166] S421, Obtain sequence prediction prompt word data;

[0167] S422, the second mixed semantic ID data and the sequence prediction prompt word data are concatenated to obtain mixed SID data;

[0168] S423, using the generative recommendation large language model, the mixed SID data is processed to obtain recommendation sequence data.

[0169] As can be seen, by implementing the recommendation method based on a large language model described in the embodiments of the present invention, the second hybrid semantic ID data is processed using the generative recommendation large language model to obtain recommendation sequence data, which improves the consistency, robustness and accuracy of the recommendation results, and lays the foundation for improving the accuracy and interpretability of the recommendation system in sequence prediction.

[0170] Optionally, recommendation methods based on large language models can be applied to online video platforms to provide users with personalized movie recommendations, including:

[0171] 1. Obtain user information and movie information

[0172] 2. Encode the user information and the movie information to obtain a hybrid semantic ID;

[0173] The user information is encoded using a hybrid semantic ID model to obtain a user latent vector;

[0174] It should be noted that the user behavior data includes, but is not limited to, user viewing history, rating information, and preference tag information;

[0175] The movie information is encoded using a hybrid semantic ID model to obtain a movie latent vector;

[0176] It should be noted that the movie information includes, but is not limited to, movie ID information, movie title information, plot synopsis information, and actor information;

[0177] The user latent vector and the movie latent vector are concatenated to obtain a mixed vector;

[0178] The hybrid vector is processed using the residual quantization algorithm model of RQ-VAE to obtain the hybrid semantic ID;

[0179] This ensures that the semantic ID remains stable even if the user's behavior information fluctuates;

[0180] 3. Receive prompt word information;

[0181] 4. Concatenate the hybrid semantic ID with the prompt word information to obtain the large model input data;

[0182] The hybrid semantic ID is concatenated with the prompt words predicted by the sequence to form the input of the large language model. This step ensures that the generated recommendation sequence not only takes into account the user's historical behavior, but also incorporates the multi-dimensional information of the movie itself.

[0183] 5. Input the large model input data into the large language model to generate recommendation sequence data;

[0184] Using a large model trained based on the GRPO algorithm, a predicted movie recommendation sequence is generated. During the training process, order reward (to ensure that the recommendation order is reasonable), format reward (to ensure that the output format meets expectations, such as including recommendation reasons), and accuracy reward (to improve the matching accuracy of the recommendations) are defined. After reinforcement learning optimization, the model can reason and generate a movie recommendation list that not only meets user interests but also has a certain explanatory power.

[0185] As can be seen, the recommendation method based on a large language model described in the embodiments of the present invention uses contrastive learning to train a hybrid semantic ID model with user information and movie information, and reinforcement learning to train a large language model to provide personalized movie recommendations for users, thereby improving the consistency, robustness and accuracy of the recommendation results, and thus enhancing the accuracy and interpretability of the recommendation system in sequence prediction.

[0186] Optionally, recommendation methods based on large language models can be applied to e-commerce platforms to push suitable products to users based on their recent browsing and purchasing behavior, as well as the characteristics of the products themselves.

[0187] Step 1: Obtain user behavior information and product information;

[0188] Step 2: Process the user behavior information and the product information to obtain a hybrid semantic ID;

[0189] The user behavior information is processed using a hybrid semantic ID model to obtain the user's latent vector information;

[0190] The user behavior information includes, but is not limited to, user search history, user click information, user purchase history, and user profile information;

[0191] The product information is processed using the hybrid semantic ID model to obtain potential product vector information;

[0192] It should be noted that the product information includes, but is not limited to, product ID information, product title information, product description information, and user review information;

[0193] The user potential vector information and the product potential vector information are concatenated to obtain hybrid vector information;

[0194] The residual quantization algorithm model of RQ-VAE is used to process the hybrid vector to obtain hybrid semantic ID information;

[0195] It should be noted that this method can effectively mitigate the impact of changes in user behavior on the stability of semantic IDs;

[0196] Step 3: Obtain scene prompt information;

[0197] For example, the scenario prompt information is: a continuous sequence of recently popular and highly rated products recommended for you;

[0198] Step 4: Concatenate the hybrid semantic ID information and the scene prompt word information to obtain the input data for the large language model;

[0199] Step 5: Input the large model input data into the large language model to generate recommendation sequence data;

[0200] A large model is trained using a reinforcement learning method based on GRPO, which sets up order rewards (considering the coherence of user purchase decisions), format rewards (ensuring the output structure is standardized, such as labeling product categories and recommendation reasons), and accuracy rewards (ensuring that recommendations match user interests). The final product recommendation sequence generated by the model can not only reflect the user's current shopping preferences, but also capture the inherent relationships between products, thereby improving the overall recommendation effect.

[0201] Step 6: Using real-time user behavior information and real-time product information, perform feedback iteration on the large language model to obtain an updated large language model;

[0202] By performing online feedback iteration on the large language model, an updated large language model is obtained, ensuring that the recommended content always matches the user's current needs.

[0203] As can be seen, the recommendation method based on a large language model described in the embodiments of the present invention trains the hybrid semantic ID model using contrastive learning based on user behavior information and product information, and trains the large language model using reinforcement learning. Based on the user's recent browsing and purchasing behavior, and combined with the characteristics of the product itself, it pushes suitable products to the user, improving the consistency, robustness and accuracy of the recommendation results, thereby enhancing the accuracy and interpretability of the recommendation system in sequence prediction.

[0204] Example 2

[0205] Please see Figure 3 , Figure 3 This is a schematic diagram of a recommendation device based on a large language model disclosed in an embodiment of the present invention. Figure 3 The described apparatus can be applied to question-answering decision systems, such as local servers or cloud servers used in question-answering decision systems, and the embodiments of the present invention are not limited thereto. Figure 3 As shown, the device may include: a first acquisition module 101, a model training module 102, a second acquisition module 103, and a data processing module 104;

[0206] The first acquisition module 101 is used to acquire training data;

[0207] The model training module 102 is used to train the large language model using the training data to obtain an optimized hybrid semantic ID model and a generative recommendation large language model.

[0208] The second acquisition module 103 allows the user to acquire user demand data;

[0209] The data processing module 104 is used to process the user demand data using the optimized encoder model and the generative recommendation large language model to obtain recommendation sequence data.

[0210] Example 3

[0211] Please see Figure 4 , Figure 4 This is a schematic diagram of a recommendation device based on a large language model disclosed in an embodiment of the present invention. Figure 4 The described apparatus can be applied to multi-sample simulation control and management systems, such as local servers or cloud servers used in multi-sample simulation control systems, and the embodiments of the present invention are not limited thereto. Figure 4 As shown, the device may include:

[0212] Memory 201 storing executable program code;

[0213] Processor 202 coupled to memory 201;

[0214] The processor 202 calls the executable program code stored in the memory 201 to execute the steps in the recommendation method based on a large language model described in Embodiment 1.

[0215] Example 4

[0216] This invention discloses a computer-readable storage medium that stores a computer program for electronic data interchange, wherein the computer program causes a computer to perform the steps in the recommendation based on a large language model as described in Embodiment 1.

[0217] Example 5

[0218] This invention discloses a computer program product comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform the steps in the recommendation method based on a large language model described in Embodiment 1.

[0219] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0220] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0221] Finally, it should be noted that the recommendation method and apparatus based on a large language model disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A recommendation method based on a large language model, characterized in that, The method includes: S1, Obtain training data; S2, The training data is used to train the large language model to obtain an optimized hybrid semantic ID model and a generative recommendation large language model; S3, obtain user demand data; S4. Using the optimized hybrid semantic ID model and the generative recommendation large language model, the user demand data is processed to obtain recommendation sequence data; The step of training the large language model using the training data to obtain an optimized hybrid semantic ID model and a generative recommendation large language model includes: S21, the training data is encoded to obtain first user latent vector data, first item latent vector data and first semantic ID latent vector data; S22, perform vector concatenation processing on the first user potential vector data and the first project potential vector data to obtain the first user mixed vector data; S23, perform residual quantization processing on the first user hybrid vector data to obtain the first hybrid semantic ID data; S24, the large model is trained using the first hybrid semantic ID data and the first semantic ID latent vector data to obtain the optimized hybrid semantic ID model and the generative recommendation large language model; The step of training a large model using the first hybrid semantic ID data and the first semantic ID latent vector data to obtain an optimized hybrid semantic ID model and a generative recommendation large language model includes: S241, Based on the contrastive loss function, the hybrid semantic ID model is compared and learned using the first hybrid semantic ID data and the first semantic ID latent vector data to obtain an optimized hybrid semantic ID model; The expression for the contrastive loss function is: Among them, L con Indicates comparative loss; This represents the semantic loss of the user; Let U represent the semantic loss of the project, where U represents the user, I represents the project, and M represents the large language model. The contrastive learning expression is: Where N represents the number of positive samples, 2N represents the number of samples after augmentation; sim(·,·) represents the cosine similarity; z i z represents the i-th latent vector data of the first semantic ID; j Let represent the j-th latent vector data of the first semantic ID; τ represents the temperature parameter used to control the shape of the loss function; k represents the sample number; z k This represents the k-th potential vector data of the first semantic ID; S242, the first mixed semantic ID data is parsed to obtain the first mixed SID data and the first item sequence label mixed SID data; S243, using the first mixed SID data and the first item sequence label mixed SID data, the large language model is trained to obtain a generative recommendation large language model.

2. The recommendation method based on a large language model according to claim 1, characterized in that, The step of performing residual quantization processing on the first user hybrid vector data to obtain the first hybrid semantic ID data includes: S231, preset the total number of residual processing stages N and the number of residual processing stages i = 0; S232, when the number of residual processing levels is less than the total number of residual processing levels, perform residual quantization processing on the first user mixed vector data to obtain quantization result data, and execute S233; otherwise, execute S234; S233, increase the residual processing level by 1, and execute S232; S234, the quantization result data is encoded and reconstructed to obtain the first hybrid semantic ID data.

3. The recommendation method based on a large language model according to claim 1, characterized in that, The step-by-step residual quantization processing of the first user's hybrid vector data to obtain quantization result data includes: S2321, the default codebook for each level is {B1,B2,…,B} N }; Initialize the original residual l0 = x0; S2322, using the residual processing level i and the codebook R corresponding to the residual processing level i. i The residual calculation process is performed on the first user's mixed vector data to obtain vector residual result data; The residual calculation processing expression is as follows: l i ＝l i-1 -r i ； Among them, l i x represents the residual value of the i-th level quantization process; i Represents the i-th mixed vector data of the first user; r i Indicates the current codebook R i Mean and residual l i-1 The closest vector; r represents the codebook R. i China B i The learnable vectors included; S2323, Quantize the vector residual result data to obtain quantized result data; The quantization expression is: Among them, L i This represents the quantization result of the i-th level.

4. The recommendation method based on a large language model according to claim 1, characterized in that, The process of using the optimized hybrid semantic ID model and the generative recommendation large language model to process the user demand data to obtain recommendation sequence data includes: S41, using the optimized hybrid semantic ID model, the user demand data is processed to obtain the second hybrid semantic ID data; S42, using the generative recommendation large language model, the second mixed semantic ID data and sequence prediction prompt word data are processed to obtain recommendation sequence data.

5. The recommendation method based on a large language model according to claim 4, characterized in that, The process of using the optimized hybrid semantic ID model to process the user demand data to obtain second hybrid semantic ID data includes: S411, The user demand data is parsed and processed to obtain user ID data, user profile data, project ID data, project title data and project description data; S412, using the optimized hybrid semantic ID model, the user ID data and the user profile data are processed to obtain the second user potential vector data; S413, using the optimized hybrid semantic ID model, the project ID data, the project title data, and the project description data are processed to obtain the second project potential vector data; S414, perform vector concatenation processing on the second user potential vector data and the second item potential vector data to obtain the second mixed semantic ID data.

6. A recommendation device based on a large language model, characterized in that, The device includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the recommendation method based on a large language model as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when invoked by a processor, are used to execute the recommendation method based on a large language model as described in any one of claims 1-5.

Images