Recommendation data processing method and device, computer equipment and storage medium

Abstract

The embodiment of the invention discloses a recommendation data processing method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring triple training data constructed by a recommendation description instruction, a positive sample and a negative sample; obtaining a prediction probability based on a recommendation prediction processing process of the triple training data; the prediction probability is used for representing the probability that the recommendation large model rejects a negative sample and biases to select a positive sample according to the recommendation description instruction; obtaining deterministic parameters when recommendation prediction processing is carried out according to the triple training data; the deterministic parameter is used for indicating the uncertainty that the recommendation large model rejects a negative sample and deviates to select a positive sample according to the recommendation description instruction; determining a loss value according to the prediction probability and the deterministic parameter, and training the big recommendation model by adopting the loss value to obtain a trained big recommendation model; the trained recommendation large model is used for outputting the target recommendation data according to the recommendation instruction, and the recommendation accuracy of the recommendation large model can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a recommended data processing method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the continuous development of computer technology, training corresponding models to assist people's production and life has become a current trend. For example, training a recommendation model to assist in data recommendation can effectively improve the efficiency of current production and life. Currently, the method used to train recommendation models is to input user interaction data into the model and carry out a supervised training process. However, the recommendation model trained in this way has weak training effectiveness because it does not adequately consider the factors affecting the model's capabilities during supervised training. This results in a lack of accuracy in the training of the recommendation model and insufficient understanding of user preferences. Summary of the Invention

[0003] This invention provides a recommendation data processing method, apparatus, computer device, and storage medium, which can effectively identify user preferences, thereby improving the training effectiveness of large recommendation models and enhancing the recommendation accuracy of large recommendation models.

[0004] On one hand, embodiments of the present invention provide a recommendation data processing method, including:

[0005] Obtain training data consisting of triplets constructed from recommendation description instructions, positive samples associated with the recommendation description instructions, and negative samples;

[0006] Based on the recommendation prediction process of the large recommendation model on the training data of the triplet, the prediction probability is obtained; the prediction probability is used to represent the probability that the large recommendation model rejects the negative sample and favors the positive sample according to the recommendation description instruction.

[0007] Obtain the deterministic parameters corresponding to the recommendation model when performing recommendation prediction processing based on the triplet training data; the deterministic parameters are used to indicate the uncertainty of the recommendation model in rejecting the negative sample and favoring the positive sample according to the recommendation description instruction;

[0008] Based on the predicted probability and the deterministic parameter, a loss value is determined for the large recommendation model, and the large recommendation model is trained using the loss value to obtain a trained large recommendation model; the trained large recommendation model is used to output target recommendation data according to the recommendation instruction.

[0009] In another aspect, embodiments of the present invention provide a recommendation data processing apparatus, comprising:

[0010] The acquisition unit is used to acquire triplet training data constructed from the recommendation description instruction, positive samples and negative samples associated with the recommendation description instruction;

[0011] The acquisition unit is further configured to acquire a prediction probability based on the recommendation prediction processing of the triplet training data by the recommendation big model; the prediction probability is used to represent the probability that the recommendation big model, according to the recommendation description instruction, rejects the negative sample and favors the positive sample.

[0012] The acquisition unit is further configured to acquire the deterministic parameters corresponding to the recommendation model when performing recommendation prediction processing based on the triplet training data; the deterministic parameters are used to indicate the uncertainty of the recommendation model rejecting the negative sample and favoring the positive sample according to the recommendation description instruction;

[0013] The processing unit is configured to determine the loss value for the large recommendation model based on the predicted probability and the deterministic parameter, and to train the large recommendation model using the loss value to obtain the trained large recommendation model; the trained large recommendation model is used to output target recommendation data according to the recommendation instruction.

[0014] In one embodiment, the recommended large model is a large model that has undergone fine-tuning training;

[0015] The acquisition unit is further configured to acquire a recommendation description instruction and a positive sample associated with the recommendation description instruction; wherein the positive sample consists of M sequentially arranged word units; M≥1 and is a positive integer;

[0016] The processing unit is further configured to call a pre-trained large model to identify the recommendation description instruction, and combine the t words of the positive sample to perform data prediction, thereby obtaining the likelihood probability of the (t+1)th word output by the pre-trained large model; where 1≤t≤M and is a positive integer;

[0017] Based on the likelihood probability, the pre-trained large model is fine-tuned to obtain the recommended large model.

[0018] In one embodiment, the model parameters of the pre-trained large model include pre-training parameters and fine-tuning parameters;

[0019] The fine-tuning process for the pre-trained large model includes adjusting the fine-tuning parameters and obtaining the adjusted parameters.

[0020] The model parameters of the fine-tuned large model are the pre-training parameters and the adjustment parameters, and the model parameters of the fine-tuned large model are used as the initial model parameters of the recommended large model.

[0021] In one embodiment, the processing unit is further configured to invoke the large recommendation model to identify and process the recommendation description instruction, thereby obtaining the recommendation data output by the large recommendation model;

[0022] The processing unit is further configured to perform data matching processing between the recommended data and the positive sample, and when the matching fails, use the recommended data as a negative sample of the recommended description instruction;

[0023] The processing unit is further configured to use the data triples consisting of the recommendation description instruction, the positive sample, and the negative sample as training data for the triples of the recommendation model.

[0024] In one embodiment, the processing unit is further configured to obtain a candidate dataset, the candidate dataset including the positive sample and at least one candidate negative sample;

[0025] The processing unit is further configured to extract the semantic features of each candidate negative sample in the candidate dataset, and extract the semantic features of the recommendation description instruction;

[0026] The processing unit is further configured to determine the semantic similarity between the semantic features of each candidate negative sample and the semantic features of the recommended description instruction, and select a negative sample from the at least one candidate negative sample based on the semantic similarity.

[0027] In one embodiment, the processing unit is specifically used for:

[0028] Obtain at least one historical positive sample associated with the recommendation description instruction, and extract the semantic features of each historical positive sample;

[0029] The semantic similarity between any candidate negative sample and the semantic similarity between each historical positive sample is calculated by comparing the semantic features of any candidate negative sample with the semantic features of each historical positive sample.

[0030] The mean of the semantic similarities of any candidate negative sample is calculated to obtain the mean similarity of the candidate negative sample, and the mean similarity is used as the semantic similarity between the candidate negative sample and the recommendation description instruction.

[0031] In one embodiment, the processing unit is specifically used for:

[0032] Based on the semantic similarity between each candidate negative sample and the recommended description instruction, the candidate negative sample with the highest semantic similarity is selected.

[0033] The candidate negative sample corresponding to the highest semantic similarity is selected as the selected negative sample.

[0034] In one embodiment, the acquisition unit is further configured to acquire recommendation requirements and determine alternative data associated with the recommendation requirements; the recommendation requirements are used to indicate the data types that the trained recommendation model should reject.

[0035] The processing unit is further configured to perform data selection processing on the alternative data associated with the recommendation requirement, and to use the selected alternative data as negative samples.

[0036] In one embodiment, the processing unit is specifically used for:

[0037] The first probability that the recommendation model will favor the positive sample and the second probability that it will favor the negative sample when performing recommendation prediction processing based on the triplet training data are obtained.

[0038] Probability statistics are performed based on the first probability and the second probability, and the results of the statistical calculations are used as deterministic parameters.

[0039] In one embodiment, the processing unit is specifically used for:

[0040] Perform exponential operations on the first probability and the second probability respectively, and sum the results of the exponential operations to obtain the probability sum;

[0041] Calculate the ratio between the exponential result of the second probability and the sum of the probabilities, and use the obtained ratio as the statistical result.

[0042] In one embodiment, the processing unit is specifically used for:

[0043] Based on the predicted probability and the deterministic parameter, the first loss value of the recommendation model in the recommendation prediction process, which is to reject the negative sample and favor the positive sample according to the recommendation description instruction, is determined, and the second loss value is to reject the positive sample and favor the negative sample according to the recommendation description instruction is determined.

[0044] The sum of the first loss value and the second loss value is used as the loss value for the recommendation model.

[0045] In one embodiment, the processing unit is specifically used for:

[0046] Based on the deterministic parameter, the deterministic of whether the large recommendation model will reject the negative sample and favor the positive sample according to the recommendation description instruction is determined.

[0047] Based on the certainty and the prediction probability, the first loss value is determined for the recommendation model during the recommendation prediction process, in accordance with the recommendation description instruction, to reject the negative sample and favor the positive sample.

[0048] In one embodiment, the processing unit is specifically used for:

[0049] Based on the predicted probability, determine the reference probability that the recommendation model, according to the recommendation description instruction, rejects the positive sample and favors the negative sample;

[0050] Based on the uncertainty and the reference probability, a second loss value is determined for the recommendation model to reject the positive sample and favor the negative sample according to the recommendation description instruction.

[0051] In one embodiment, the processing unit is specifically used for:

[0052] Obtain the recommendation instruction, and call the trained recommendation model to perform data recommendation processing on the recommendation instruction to obtain one or more recommendation data related to the input instruction;

[0053] Based on at least one piece of recommendation data associated with the recommendation instruction, output target recommendation data.

[0054] In another aspect, embodiments of the present invention provide a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described in the first aspect.

[0055] In another aspect, embodiments of the present invention provide a computer program product, the computer program product including a computer program, the computer program including program instructions, and when the program instructions are invoked by a processor, causing the processor to execute the method described in the first aspect.

[0056] In this embodiment, when training a large recommendation model, the computer device can acquire triplet data containing recommendation description instructions and positive and negative samples as triplet training data. Then, the large recommendation model can be invoked to process this triplet training data, thereby obtaining the predicted probability that the large recommendation model, based on the constraints of the recommendation description instructions, prefers to select positive samples for output recommendations rather than negative samples, and obtaining the deterministic parameters when the computer invokes the large recommendation model to make data recommendations according to the triplet training data. Based on the obtained predicted probability and deterministic parameters, the computer device can determine the loss value for the large recommendation model and train the large recommendation model according to the loss value. Since the deterministic parameters obtained by the computer device are used to indicate the uncertainty of the recommendation model in outputting recommendations by rejecting negative samples and preferring positive samples, the uncertainty of the triple training data is introduced into the loss value. Therefore, training the recommendation model based on this loss value can reduce this uncertainty. Based on the reduction of the uncertainty of the triple training data, the trained recommendation model can effectively reflect the user's preference needs, that is, achieve an accurate understanding of the user's preferences. This can effectively improve the training effectiveness of the computer device on the recommendation model, thereby improving the robustness of the trained recommendation model. Attached Figure Description

[0057] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1a This is a schematic diagram of a direct preference optimization training method provided in an embodiment of the present invention;

[0059] Figure 1b This is a schematic diagram of a recommendation data processing system provided in an embodiment of the present invention;

[0060] Figure 2 This is a schematic flowchart of a recommendation data processing method provided in an embodiment of the present invention;

[0061] Figure 3a This is a schematic diagram of negative sampling provided in an embodiment of the present invention;

[0062] Figure 3b This is a schematic diagram of a model fine-tuning process provided in an embodiment of the present invention;

[0063] Figure 3c This is a schematic diagram illustrating an efficient supervised fine-tuning of parameters provided in an embodiment of the present invention;

[0064] Figure 4 This is a schematic diagram illustrating a data recommendation using a pre-trained large recommendation model, as provided in an embodiment of the present invention.

[0065] Figure 5 This is a schematic block diagram of a recommendation data processing device provided in an embodiment of the present invention;

[0066] Figure 6 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0067] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0068] This application proposes a recommendation data processing method. When training a large recommendation model, a computer device can obtain a predicted probability P of the large recommendation model's recommendation processing based on the positive and negative samples in the triplet training data, according to the process of the large recommendation model performing recommendation prediction processing on the triplet training data. Based on this predicted probability P, the ability of the large recommendation model to make correct recommendations based on the positive and negative samples in the triplet training data can be evaluated, thus characterizing the large recommendation model's ability to distinguish between positive and negative samples in the current triplet training data. Furthermore, the computer device will also determine the deterministic parameters of the large recommendation model's recommendation processing based on the process of the large recommendation model using the triplet training data. This allows the computer device to predict based on the determined probability P and deterministic parameters. The loss value for the large recommendation model is determined, and the model is trained based on the determined loss value. The computer device obtains deterministic parameters based on the recommendation prediction process. This is used to characterize the uncertainty of the recommendation model in making recommendation predictions based on the positive and negative samples in the training samples after recognizing the training samples. This introduces the uncertainty of the samples into the loss value. By introducing the uncertainty into the loss value, the computer device can effectively quantify the difficulty of the recommendation model in making recommendation predictions based on the training samples when determining the loss value. This allows the computer device to dynamically adjust the training process of the recommendation model based on the difficulty of the training samples, thereby effectively improving the effectiveness of the training process and enhancing the robustness of the trained recommendation model.

[0069] It's understandable that training a large recommendation model aims to enable it to effectively distinguish between positive and negative samples in the triplet training data through the recognition and processing of training samples. Therefore, by determining the predicted probability P, the computer incorporates the ability to distinguish between positive and negative samples into the loss value. Furthermore, the computer uses deterministic parameters... The determination of the probability density function (P) allows computer equipment to incorporate the difficulty levels of positive and negative samples, thereby introducing the ability of the recommendation model to dynamically adjust based on the difficulty levels of positive and negative samples into the loss value. Therefore, based on the prediction probability P and the deterministic parameters... The determined loss value can effectively reflect the difficulty of the recommendation model based on positive and negative samples, and its ability to effectively distinguish between positive and negative samples for recommendations. This enables the recommendation model trained based on the loss value to effectively distinguish different data and achieve accurate recommendations, thereby improving the recommendation accuracy and effectiveness of the trained recommendation model.

[0070] The large recommendation model trained using triplet training data by computer devices is called a large model recommendation system (LLM-based Recommender). This refers to a large model that directly uses a Large Language Model (LLM) for recommendation processing or uses a Large Language Model for recommendation enhancement. In other words, the large recommendation model trained in this application embodiment is essentially a Large Language Model. To enable the Large Language Model to provide recommendation services, user preferences need to be incorporated into the Large Language Model during the model training phase. This allows the trained Large Language Model to accurately identify and output recommendation data that meets user preferences based on input commands. The process of incorporating user preferences into the Large Language Model during the model training phase mainly involves the following three steps ① to ③.

[0071] ①Supervised fine-tuning process (SFT).

[0072] A pre-trained large language model is fine-tuned under supervised guidance on downstream tasks of interest (such as dialogue tasks, recommendation tasks, etc.) using a triplet training dataset to obtain the fine-tuned large language model. This fine-tuned large language model is the recommendation model, and its model parameters can be denoted as π. SFT This model parameter is also used as the initial model parameter for subsequent processes.

[0073] ②Reward model.

[0074] This process involves generating preference data pairs based on given prompts (such as data pairs consisting of positive and negative samples from the preceding triplet training data), determining which output the user prefers in the generated preference data pairs through user evaluation, and then combining the preference predictions output by the large language model to perform reinforcement learning on the large language model.

[0075] ③ Strengthen the learning process.

[0076] Reinforcement learning is used to improve the model parameters π of a large language model. SFT Further optimization is then performed. During this optimization process, feedback is provided by the reward model to guide the large language model in generating predictive outputs that align with user preferences, thus completing the final adjustment of the large model and integrating user preferences into it.

[0077] Based on the aforementioned three-step model training process, it is known that to integrate user preferences into the large language model, the training is guided by an iterative model reward feedback loop. Simplifying the iterative model reward feedback training can effectively improve the training process of the large language model. Therefore, this application primarily employs a personalized preference optimization (DPO) framework (an algorithm improved based on Direct Preference Optimization) to adjust the large language model. Training the large language model under the DPO framework refers to using preference data to directly optimize the model output to match the user's preference response. That is, when training the large language model based on the DPO framework, the computer device can directly fine-tune the model using the preference data of the recommended description instruction x (i.e., the positive and negative samples of the aforementioned triplet training data). This transforms the reinforcement learning process into a large language model-like SFT process, effectively improving the stability and efficiency of the large language model training. The method for training a large language model based on a personalized preference optimization framework, as used in this application embodiment, fine-tunes the model using preference data of the recommended description instruction x, and further utilizes instance-level smoothing coefficients (i.e., the aforementioned deterministic parameters). The loss function is used to achieve personalized preference optimization, thereby further enhancing the recommendation capabilities of the large language model.

[0078] The training process of a large language model within the framework of personalized preference optimization can be as follows: Figure 1a As shown, the training process of the large language model based on the personalized preference optimization framework mainly includes the aforementioned step ① supervised fine-tuning of SFT. After the supervised fine-tuning of the large language model is completed, the preference data (i.e. the aforementioned positive and negative samples) is directly used to optimize the large language model that has been fine-tuned.

[0079] The process of directly optimizing a fine-tuned large language model using preference data is essentially the process of training and optimizing this model using triple training data (containing recommendation description instructions x and positive and negative samples). In this process, the recommendation description instruction x is used as the input instruction to the large language model, while the positive and negative samples are used as preference data for that input instruction. Positive samples used as preference data are the data that the model should choose to output, while negative samples are the data that the model should reject from output. In other words, optimizing a fine-tuned large language model (i.e., the large recommendation model) directly using preference data involves inputting triple training data into the large recommendation model. This allows the model to learn the ability to recommend data based on the preference data corresponding to the recommendation description instruction after recognizing it. In other words, the large recommendation model learns to be more biased (or prefer) to output positive samples than negative samples in the preference data, thus incorporating the user's preferences into the large model.

[0080] The training data consists of a triplet consisting of a recommendation description instruction and positive and negative samples. The recommendation description instruction x is the input instruction, such as "Please recommend women's clothing to the user", while the positive sample can be a floral dress, a white tulle dress, etc., and the negative sample can be a tracksuit, etc.

[0081] Based on the recommendation data processing method proposed in the embodiments of this application, the recommendation data processing system applied to this method will be described below.

[0082] Please see Figure 1b This is a schematic diagram of the network architecture of a recommendation data processing system provided in an embodiment of this application, as shown below. Figure 1b As shown, the network architecture of this recommended data processing system may include a server 200 and a terminal device cluster. The terminal device cluster may include one or more terminal devices; the number of terminal devices will not be limited here. Figure 1b As shown, multiple terminal devices can specifically include terminal device 1, terminal device 2, terminal device 3, ..., terminal device n, where n is a positive integer. For example... Figure 1bAs shown, terminal device 1, terminal device 2, terminal device 3, ..., terminal device n can all connect to server 200 via the network, so that each terminal device can interact with server 200 through the network connection.

[0083] like Figure 1b The server 200 shown can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The terminal device can be a smartphone, tablet, laptop, desktop computer, smart TV, in-vehicle terminal, smart home device, or other smart terminal. The following description uses the communication between terminal device 1 and server 200 as an example to illustrate the specific implementation of this application.

[0084] (1) Users collect triple training data through terminal device 1 in different recommendation scenarios. These recommendation scenarios include, but are not limited to: e-commerce recommendation scenarios and content recommendation scenarios from different business clients. Specific content recommendation scenarios may include: playback clients recommending playback content, game clients recommending game objects, transportation clients recommending routes, and medical clients recommending treatments, etc. When users collect triple training data on terminal device 1, they can collect triple training data from these business clients. The number of triple training data collected by terminal device 1 is multiple, and each triple training data set contains a recommendation description instruction, a positive sample, and a negative sample.

[0085] (2) The terminal device 1 sends the collected training data of multiple triples to the server 200. After receiving the training data of multiple triples, the server 200 identifies and processes the recommendation description instructions in each training data of triples by calling the recommendation big model.

[0086] (3) Based on the identification of the recommendation description instruction by the recommendation big model, the server 200 determines that the recommendation big model is more inclined to output positive samples as the prediction probability of recommendation than negative samples, and determines the uncertainty of the recommendation big model using positive and negative samples in the training data of the triplet for recommendation prediction.

[0087] (4) Server 200 uses the obtained predicted probabilities and uncertainties to generate a loss value;

[0088] (5) Server 200 adjusts the model parameters of the recommendation model based on the loss value, and uses other triplet training data to repeatedly execute steps (3) to (5) until a well-trained recommendation model is obtained.

[0089] It should be noted that the data interaction process described above for recommendation data processing is for illustrative purposes only and does not limit the specific execution process of the terminal device and the server. Optionally, the process of collecting triple training data can also be performed by the server; or, the process of training the large recommendation model based on the triple training data can also be performed by the terminal device; or, the complete process of recommendation data processing described above can be performed independently by any terminal device or server in the recommendation data processing system.

[0090] Based on the training process of the large recommendation model mentioned in the recommendation data processing system, by introducing the uncertainty of positive and negative samples into the loss value, the difficulty of the large recommendation model in identifying positive and negative samples is effectively trained, thereby enabling the large recommendation model to distinguish user preferences and thus improving the training effect of the model.

[0091] It should also be noted that the relevant data involved in the model training process of this application (such as recommended description instructions, positive samples, negative samples, etc. in the triplet training data) requires the permission or consent of the target user when the above embodiments of this application are applied to specific products or technologies. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the region, and must adhere to the principles of legality, legitimacy, and necessity, without involving the acquisition of data types prohibited or restricted by laws and regulations. In some optional embodiments, the relevant data involved in the embodiments of this application is obtained after separate authorization from the target user. Additionally, when obtaining separate authorization from the target user, the purpose of the relevant data is explained to the target user.

[0092] Please see Figure 2 This is a schematic flowchart illustrating a recommendation data processing method proposed in an embodiment of this application. This recommendation data processing method can be executed by a computer device, which can be... Figure 1b The terminal device in the middle can also be Figure 1b Servers in, such as Figure 2 As shown, the method may include:

[0093] S201, Obtain training data consisting of triples constructed from the recommendation description instruction, positive samples associated with the recommendation description instruction, and negative samples.

[0094] The triplet training data acquired by the computer device is a preference triplet dataset. This preference triplet dataset contains a recommendation description instruction x and a corresponding preference data set. The recommendation description instruction x is an instruction used to indicate the user's recommendation needs and is input into the recommendation model so that the model can recognize the recommendation description instruction x and perform recommendation processing based on the recognition result. This recommendation description instruction can generally be instruction text; specifically, the recommendation description instruction x could be: "Please recommend the next song to the user based on their music playback history," or "Please reduce the amount of sports content played for the user," etc.

[0095] In one embodiment, the recommendation description instruction x can be automatically generated by the computer device. For example, the computer device can automatically generate the aforementioned recommendation description instruction "Please recommend the next song to the user based on the user's historical music playback history" during the current music playback process, so that the computer device can display one or more recommended next songs to the user after the current music playback is completed, or directly play the recommended next song. Alternatively, the recommendation description instruction x can also be entered by the user and sent to the computer device. For example, the user can input playback requests as recommendation description instruction x into the terminal device during content playback (such as video and / or text and images), so that the computer device can obtain the recommendation description instruction from the terminal device. The playback request entered by the user through the terminal device can be, for example, the aforementioned recommendation description instruction "Please reduce the playback content of sports." After obtaining the recommendation description instruction, the computer device will reduce the playback content of sports and increase the playback content of other types when playing content in the future.

[0096] By directly generating recommendation description instructions from the computer device, the smoothness of content playback can be improved. Furthermore, by acquiring the user's input playback requests (i.e., recommendation description instructions), the computer device can dynamically adjust the playback content in real time according to changes in user needs, thereby increasing user satisfaction with the output recommendations. The aforementioned method of acquiring recommendation description instructions by the computer device is merely illustrative; other methods may also be used to obtain these instructions, and this application does not limit the specific methods used.

[0097] The preference data set within the preference triple refers to the user preference data corresponding to the recommendation description instruction x. This preference data set includes positive samples (denoted as yw) and negative samples (denoted as yl). Positive samples are those actually selected by the user based on the recommendation description instruction x, while negative samples are those not actually selected by the user based on the recommendation description instruction x. Specifically, if the recommendation description instruction is "Please recommend the next song to the user based on their music playback history," and the recommended music displayed to the user under this instruction includes music A, music B, and music C, then if music A is selected by the user for playback, the computer device can treat music A as a positive sample and music B (or music C) as a negative sample. For music D that is not recommended to be displayed to the user, the computer device can also select music D as a negative sample. That is to say, when the computer device selects positive samples, it is based on user interaction. However, when selecting negative samples, it can be based on certain selection rules. These selection rules can be used to constrain one or more of the selection range, selection method, and selection conditions of negative samples. It can be understood that the embodiments of this application do not constrain the selection process of negative samples. In the embodiments of this application, data that has not been interacted with by the user can be used as negative samples.

[0098] For acquiring triple training data, computer devices can acquire it based on the recommendation scenario of the large recommendation model. The following describes two methods involved in acquiring triple training data by computer devices.

[0099] (1) Acquisition of triplet training data based on a single recommendation scenario.

[0100] For example, when the recommendation scenario is a game scenario, data from the game scenario can be selected as the training data for the triples. The recommendation description instructions, positive and negative samples contained in the training data for the triples in this game scenario could be: "Please recommend game equipment to the user," "Equipment X," and "Equipment Y." This can be understood as: Equipment X is the game equipment recommended by the computer device that was selected by the user, while Equipment Y is the equipment that was not selected by the user. Similarly, when the recommendation scenario is a medical scenario, a content playback scenario, or an e-commerce scenario, the description text and positive and negative samples for that scenario can be obtained, and the resulting triple data can be used as the training data for the triples in that scenario.

[0101] (2) Acquire triple training data based on at least two recommended scenarios.

[0102] Since there are multiple triple training data sets used to train the large recommendation model, the computer device can also collect triple data sets from different recommendation scenarios as triple training data sets. In other words, the multiple triple training data sets used to train the large recommendation model can include triple data sets collected from at least two scenarios.

[0103] Computer devices can achieve different training effects on recommendation models by using different methods of acquiring triplet training data. Therefore, when acquiring triplet training data, computer devices can consider the desired training effect of the recommendation model. Specifically, if the computer device uses triplet training data from a specific recommendation scenario for model training, the recommendation model will exhibit good recommendation capabilities in that scenario, thus improving its recommendation performance. Furthermore, if the computer device uses triplet training data from at least two different scenarios for model training, the trained recommendation model will be more adaptable to different recommendation scenarios, enabling flexible recommendations.

[0104] In this embodiment, the triplet training data collected during the training of the large recommendation model includes negative samples, which are obtained through negative sampling. In one embodiment, since the large recommendation model relies on historical user feedback to model user interests, the positive samples that the computer device needs to acquire are data that the user likes, while the negative samples are data that the user dislikes. However, in the actual data acquisition process, it is difficult for the computer device to obtain explicit user feedback; that is, the computer device cannot determine whether a certain piece of data is preferred (or liked) by the user. Therefore, implicit feedback information can be used to describe whether the user likes it. This implicit feedback information is whether there is interaction with the user. Thus, based on implicit feedback information, the process by which the computer device acquires positive and negative samples based on whether the user likes them becomes acquiring positive and negative samples based on whether there is interaction with the user.

[0105] like Figure 3a As shown, a computer device can use data that has been interacted with by the user as a positive sample, and select a portion of data from the data that the user has not interacted with as a negative sample. The process of selecting negative samples from the data that has not been interacted with based on a certain strategy is called negative sampling.

[0106] Based on the acquisition of triple training data by computer equipment, the computer equipment can further call the recommendation big model to identify the triple training data. The identification and processing of the triple training data by the computer equipment calling the recommendation big model is the identification and processing of the recommendation description instruction x contained in the triple training data. Based on the identification and processing process of the recommendation description instruction x in the triple training data by the recommendation big model, the prediction of the recommendation big model based on the recommendation description instruction can be obtained. Then, based on the prediction, the corresponding prediction probability P is generated, and step S202 can be executed.

[0107] The large recommendation model that performs recommendation prediction processing on the recommendation description instructions in the triplet training data is the large model that has completed fine-tuning training. Based on this fine-tuning training (or adjustment training), the large recommendation model can acquire initial recommendation capabilities. Below, we will combine... Figure 3b The fine-tuning training process of the large recommendation model is described, wherein the fine-tuning training of the large recommendation model by computer equipment specifically includes the following steps s10 to s12.

[0108] s10, obtain the recommendation description instruction x and the positive sample yw associated with the recommendation description instruction x.

[0109] The computer device can acquire recommendation description instructions and associated positive samples using the same method described above for acquiring recommendation description instructions and positive samples in the triplet training data. The positive samples acquired by the computer device can be represented by M sequentially arranged tokens; M ≥ 1 and are positive integers.

[0110] s11 calls the pre-trained large model to identify the recommendation description instruction, and combines the t words of the positive sample to perform data prediction, and obtains the likelihood probability of the (t+1)th word output by the pre-trained large model.

[0111] After acquiring the recommendation description instruction x and the positive sample yw, the computer device can input the recommendation description instruction x and the positive sample yw into a pre-trained large model. The pre-trained large model then processes the recommendation description instruction x and the positive sample yw for recognition, and performs data prediction based on the M lexical units contained in the recommendation description instruction x and the positive sample yw. In one embodiment, when the pre-trained large model performs data prediction based on the recommendation description instruction x and the positive sample yw, it predicts the (t+1)th lexical unit of the positive sample based on the recommendation description instruction x and the first t lexical units used to represent the positive sample yw, where 1 ≤ t ≤ M and are positive integers.

[0112] In other words, the pre-trained large model (i.e. the recommendation large model before fine-tuning) can first predict the first token (i.e. the predicted token1) based on the recommendation description instruction x and the positive sample yw. Then, based on the predicted token1, the predicted token2 is obtained. Further based on the predicted token1 and the predicted token2, the predicted token3 is obtained, and so on, to complete the prediction of the positive sample yw.

[0113] By using a pre-trained large model to predict the next token of a positive sample after recognizing and processing the recommended description instructions and positive samples, the likelihood probability of the prediction model can be established, and the model can be fine-tuned based on the obtained likelihood probability.

[0114] s12, based on the likelihood probability, fine-tunes the pre-trained large model to obtain the fine-tuned recommendation large model.

[0115] Likelihood probability can be used to describe the probability of real data occurring under different model parameters of the pre-trained large model. That is, after recognizing the recommendation description instruction x and the positive sample yw, the pre-trained large model can predict the probability of obtaining the (t+1)th real token based on the first t tokens of the positive sample yw. Therefore, when the computer device fine-tunes the model based on the determined likelihood probability, it can use the model parameters corresponding to the maximum probability of real data occurrence as the fine-tuned model. In other words, the fine-tuning process of the large model by the computer device is the process of maximizing the likelihood probability by adjusting the model parameters.

[0116] The model parameters of the pre-trained large model include pre-training parameters (denoted as Φ0) and fine-tuning parameters (denoted as Θ). The fine-tuning process of the pre-trained large model includes adjusting the fine-tuning parameters Θ to obtain the adjusted parameters Θ'. Therefore, the model parameters of the fine-tuned large model obtained by the computer device are the pre-training parameters Φ0 and the adjusted parameters Θ', and the model parameters of the fine-tuned large model are used as the initial model parameters of the recommendation large model trained based on the loss value.

[0117] The process of fine-tuning the pre-trained large model is the supervised fine-tuning SFT process that incorporates user preferences into the training. The model parameters of the fine-tuned large model (i.e., the recommendation model) can be denoted as π. SFT Therefore, we have π SFT =Φ0+Θ'.

[0118] In one embodiment, the large model can adopt the model architecture of LLM4Rec (a large model framework) or LLaRA (another large model framework). After training the corresponding model architecture, further fine-tuning is performed to enable the model to learn the ability to recommend data.

[0119] As can be seen from the aforementioned fine-tuning process for large models, computer equipment can employ parameter-efficient fine-tuning techniques for fine-tuning large models. One example of parameter-efficient fine-tuning is the LoRA (Low-Rank Adaptation) method. This involves transforming the model parameters of the large model to be fine-tuned into a pre-trained parameter Φ0 that retains its full rank and a low-rank fine-tuning parameter Θ. Since low-rank data can approximate or effectively represent the original dimensional data through a lower-dimensional space while preserving the model's important information and structure, the model adjustment for large models can be transformed from directly adjusting the model parameters to adjusting the lower-dimensional (or matrix) model parameters (i.e., the fine-tuning parameter Θ), thereby effectively improving the efficiency of fine-tuning large models.

[0120] Below, in conjunction with Figure 3c This paper provides an overall explanation of the process of fine-tuning the model of computer equipment using the LoRA method.

[0121] ① Input the pre-trained large model according to the specified data format (recommendation description instruction x, positive sample yw), where the positive sample yw can be used as the label data y of recommendation description instruction x;

[0122] ② Based on the recognition and processing of the pre-trained large model (recommendation description instruction x, positive sample yw), the next token of the positive sample is predicted in turn, and the likelihood probability is constructed according to the prediction results. Then, the training objective of the computer device for the pre-trained recommendation large model can be as shown in Equation 1.

[0123]

[0124] Where x is the recommendation description instruction, y is the positive sample yw, |y| is the total number of terms representing the positive sample yw (as mentioned above, M), and y t Let t be the t-th token predicted under the constraint of the recommendation description instruction x, based on the first t-1 tokens of the positive samples yw. Z is the sample set for obtaining the recommendation description instruction and positive samples (i.e., constructing the supervised corpus for the recommendation domain), Φ0 is the model parameters of the pre-trained large model (i.e., the pre-trained parameters that do not participate in the adjustment), and Θ is the LoRA parameter (i.e., the fine-tuning parameter Θ).

[0125] ③ The recommendation model is trained by maximizing the log value of the likelihood probability of the next token in the autoregressive model. During this maximization process, only the fine-tuning parameter Θ is adjusted to effectively improve the model's fine-tuning efficiency.

[0126] S202, the recommendation prediction process based on the large recommendation model for the triplet training data, to obtain the prediction probability; the prediction probability is used to represent the probability that the large recommendation model will reject negative samples and favor positive samples according to the recommendation description instructions.

[0127] S203, Obtain the deterministic parameters of the large recommendation model when performing recommendation prediction processing based on the triplet training data; the deterministic parameters are used to indicate the uncertainty of the large recommendation model in rejecting negative samples and favoring positive samples according to the recommendation description instructions.

[0128] In steps S202 and S203, after fine-tuning the pre-trained large model, the fine-tuned recommendation model can be used to identify the triplet training data and perform recommendation prediction processing. Based on the triplet training data, the recommendation model can be further optimized and adjusted through comparison. Based on this recommendation prediction process, the computer device can obtain the prediction probability P and deterministic parameters. In one embodiment, the predicted probability P is used to indicate the probability that when a computer device performs recommendation processing based on triple training data, it prefers to select a positive sample yw for recommendation processing rather than a negative sample (denoted as yl) based on the recommendation description instruction x. That is, the predicted probability P is used to indicate the probability that the recommendation model makes recommendations according to the user preferences indicated by the triple training data. It can be understood that the process of training the recommendation model is to continuously adjust the model parameters of the recommendation model so that the output predicted probability P reaches its maximum value. When the predicted probability P output by the recommendation model reaches its maximum value, it means that the recommendation model has learned the user preferences reflected by the triple training data (that is, the user preference of choosing positive samples rather than negative samples).

[0129] In one embodiment, the predicted probability P obtained by identifying and processing the training samples based on the recommendation big model can be shown in Equation 2.

[0130]

[0131] Among them, y w Represents a positive sample, y l y represents a negative sample, > indicates a strict preference, and y represents a negative sample. w >y lThis indicates a bias towards positive samples compared to negative samples. π θ To recommend model parameters for large models, π ref To recommend initial model parameters for a large recommended model, since these initial parameters are the fine-tuned parameters of the large language model, π ref The value of can be the aforementioned π. SFT σ and β are coefficients. As shown in Equation 2, a KL divergence term with β is used to constrain the deviation of the large recommendation model from the SFT model. This ensures that when further training the fine-tuned large recommendation model, it does not deviate from the existing recommendation capabilities of the fine-tuned model. In other words, the KL divergence term with β can be used to constrain the training of the fine-tuned large recommendation model. This training is a further enhancement training based on the existing recommendation capabilities of the large recommendation model, making the trained large recommendation model more aligned with user preferences in terms of recommendation capabilities.

[0132] The dataset used by computer devices to select positive and negative samples is user-oriented. However, users' actual interest in each data point is not simply categorized as interested (or truly accepting) and uninterested (easily rejecting), but rather as follows: easily rejectable, difficult to reject, falsely rejected, and truly accepted (i.e., genuinely interested). Therefore, when performing negative sampling, computer devices may select falsely rejected data (data that the user is genuinely interested in but has not interacted with). When this falsely rejected data is used as negative samples to train a large recommendation model, the resulting model's recommendations will deviate from the user's actual preferences.

[0133] Besides the issue of incorrectly rejected data being sampled as negative samples during negative sampling, leading to a discrepancy between the recommended preferences of the trained recommendation model and the user's actual preferences, the training of a model using a hard sampling strategy to obtain negative samples can also cause this discrepancy. The hard sampling strategy involves selecting data of varying difficulty as positive and negative samples. Therefore, negative samples obtained using this strategy are inevitably more similar to positive samples. Using such data as negative samples can result in the inclusion of data that the user is genuinely interested in but has not interacted with, thus biasing the model training results.

[0134] For example, if the positive sample selected by the computer device is a white floral maxi dress, then when data 1 is a white floral mini dress and data 2 is a sports dress, data 1 is obviously more likely to cause confusion than data 2. Therefore, after selecting data 1 as a negative sample, the recommendation model trained with this positive sample may be a model with user preference bias.

[0135] Therefore, to avoid such deviations, the embodiments of this application employ a smoothing factor. This measures the uncertainty in recommendation prediction based on positive and negative samples caused by the computer device selecting incorrectly rejected data during negative sampling. In other words, it measures the deterministic parameters acquired by the computer device during the recommendation prediction process. This parameter represents the uncertainty of the recommendation model in choosing between positive and negative samples (yw) after recognizing the recommendation description instruction. In other words, it represents the uncertainty in the computer device's prediction probability. By incorporating this deterministic parameter into the loss value, the uncertainty in indicating user preferences caused by positive and negative samples can be eliminated during the optimization training of the recommendation model based on this loss value, thereby effectively improving the training quality of the recommendation model.

[0136] The computer device obtains deterministic parameters during the recommendation-based prediction process. In this case, we can first obtain the first probability sw of the recommendation prediction process, which indicates that the large recommendation model is biased towards selecting positive samples yw, and the second probability sl of selecting negative samples yl. Then, we can use the Bradley-Terry method (a statistical model) to perform probability statistical calculations based on the first probability sw and the second probability sl, and use the results of the statistical calculations as deterministic parameters.

[0137] When performing probability statistics based on the first probability and the second probability, the computer device first performs exponential operations on the first probability sw and the second probability sl respectively, and then sums the results of the exponential operations to obtain the probability sum. Then, it calculates the ratio between the exponential operation result of the second probability and the probability sum, and uses the obtained ratio as the statistical operation result. The statistical operation result obtained by the computer device can be shown in Equation 3.

[0138]

[0139] Among them, e sw e is the result of exponential operation based on the first probability sw. sl e is the result of exponential operation based on the second probability sl. sw +e sl Then it is the sum of probabilities.

[0140] When the computer device obtains the first probability sw of positive samples being biased and the second probability sl of negative samples being biased, it can call a finely tuned recommendation model to achieve this, or it can use a regular recommendation model to determine it. This is not limited in the embodiments of this application.

[0141] The process of obtaining deterministic parameters for a large recommendation model during recommendation prediction using computer equipment reveals that these parameters are dynamically adjusted based on the probability of positive and negative samples being recommended. In other words, these parameters are dynamically adjusted based on the difficulty of distinguishing between positive and negative samples, rather than being static hyperparameters. This ensures that the deterministic parameters determined by the computer equipment... It can effectively reflect the uncertainty of positive and negative samples when performing recommendation processing, thereby improving the robustness of the trained recommendation model.

[0142] The computer device obtains the predicted probability P and deterministic parameters. Then, the loss value can be determined, and the determined loss value can be used to further adjust the model parameters of the fine-tuned recommendation model.

[0143] S204. Based on the predicted probability and deterministic parameters, determine the loss value for the large recommendation model, and use the loss value to train the large recommendation model to obtain the trained large recommendation model; the trained large recommendation model is used to output target recommendation data according to the recommendation instructions.

[0144] In one embodiment, when determining the loss value for a large recommendation model based on the prediction probability and deterministic parameters, the computer device may first determine, based on the prediction probability and deterministic parameters, the first loss value of the large recommendation model in the recommendation prediction process, which is to reject the negative sample yl and favor the positive sample yw according to the recommendation description instruction x, and the second loss value of the large recommendation model in the process, which is to reject the positive sample yw and favor the negative sample yl according to the recommendation description instruction x.

[0145] When a computer device determines the first loss value of a large recommendation model during the recommendation prediction process, based on the predicted probability and deterministic parameters, if the model rejects negative sample yl and favors positive sample yw according to the recommendation description instruction x, it can first determine the certainty of the large recommendation model's decision to reject negative sample yl and favor positive sample yw according to the recommendation description instruction x. However, due to the deterministic parameters... This indicates the uncertainty of a large recommendation model in rejecting negative samples yl and favoring positive samples yw based on the recommendation description instruction x. Therefore, the determinism of a computer device in rejecting negative samples yl and favoring positive samples yw based on the recommendation description instruction x can be denoted as: Then, based on the obtained certainty... Given the predicted probability P, we determine the first loss value of the recommendation model during the recommendation prediction process, which, based on the recommendation description instruction x, biases towards selecting positive samples yw instead of negative samples yl. This first loss value can be represented by Equation 4.

[0146]

[0147] In one embodiment, when the computer device determines a second loss value that, according to the recommendation description instruction x, rejects the positive sample yw and favors the negative sample yl, it can determine a reference probability, based on the predicted probability P, that the large recommendation model, according to the recommendation description instruction x, rejects the positive sample yw and favors the negative sample yl. Here, since the predicted probability P represents the probability that the large recommendation model, according to the recommendation description instruction x, rejects the negative sample yl and favors the positive sample yw, the reference probability is (1-P, or...). Then, the computer device can further adjust based on uncertainty. Based on the reference probability, the second loss value of the recommendation model is determined, which rejects positive samples yw and favors negative samples yl according to the recommendation description instruction x.

[0148] The obtained second loss value can be represented by Equation 5.

[0149]

[0150] After the computer device determines the loss value and the second loss value based on the prediction probability and deterministic parameters, the sum of the loss value and the second loss value can be used as the loss value for the recommendation model. The resulting loss value is shown in Equation 6.

[0151]

[0152] Where θ represents the model parameters of the large recommendation model, meaning that in the process of model optimization based on the loss value, all parameters θ of the large language model LLM are directly optimized and trained. PL The triplet dataset is composed of training data for triplet pairs.

[0153] As shown in Equation 6, when a computer device optimizes and trains a large recommendation model based on the loss value, it aims to increase the probability of predicting the selected output (i.e., increasing the probability of positive samples being selected) and decrease the probability of predicting the rejected output (i.e., decreasing the probability of negative samples being selected) by adjusting the model parameters θ, thereby obtaining a well-trained large recommendation model. The trained large recommendation model can then output target recommendation data based on input instructions, and the output target recommendation data meets the user's preference requirements.

[0154] In one embodiment, introducing deterministic parameters into the loss value is used to improve the robustness of large recommendation models, while the smoothing coefficient (i.e., the deterministic parameter) is used to improve the loss. To improve the robustness of the recommendation model in preference optimization, it can also be equivalent to using an auxiliary model to predict β. Since β can control the offset of the preference model relative to the SFT stage, different β values ​​can be used at the instance level to control the impact of negative samples with different difficulties / uncertainties.

[0155] In this embodiment, when training a large recommendation model, the computer device can acquire triplet data containing recommendation description instructions and positive and negative samples as triplet training data. Then, the large recommendation model can be invoked to process this triplet training data to obtain the prediction probability that the large recommendation model, based on the constraint of the recommendation description instructions, prefers to select positive samples for output recommendations rather than negative samples. The computer device can also acquire the deterministic parameters of the large recommendation model when making recommendation predictions based on the triplet training data. Based on the obtained prediction probability and deterministic parameters, the computer device can determine the loss value for the large recommendation model and train the model according to this loss value. Since the deterministic parameters obtained by the computer device are used to indicate the uncertainty of the recommendation model in outputting recommendations by rejecting negative samples and preferring positive samples, the uncertainty of the triple training data is introduced into the loss value. Therefore, training the recommendation model based on this loss value can reduce this uncertainty. Based on the reduction of the uncertainty of the triple training data, the trained recommendation model can effectively reflect the user's preference needs, that is, achieve an accurate understanding of the user's preferences. This can effectively improve the training effectiveness of the computer device on the recommendation model, thereby improving the robustness of the trained recommendation model.

[0156] The following describes in detail the process by which the computer device acquires triplet training data in the embodiments of this application. The subsequent description mainly focuses on the process of negative sampling to obtain negative samples from the triplet training data. For the recommendation description instructions and positive samples included in the triplet training data, please refer to the description of the foregoing embodiments. The methods for negative sampling and obtaining negative samples mainly include the following three types.

[0157] (a) Generation of training data for triples of main preferences.

[0158] After the computer equipment completes the fine-tuning of the recommendation model, i.e., after confirming the completion of the SFT training phase, the generation of triplet training data for primary preferences can begin. This generated triplet training data for primary preferences is used to universally adjust the recommendation capabilities of the fine-tuned model. In other words, even when the fine-tuned model lacks explicit system design preferences, obtaining the triplet training data for primary preferences allows for further capability training, enabling the model to better learn user preferences.

[0159] In one embodiment, the triplet training data acquired by the computer device is triplet data representing user preferences. Therefore, the computer device can first acquire triplet training data based on the preference learning of the LLM recommender, thereby converting user-item interaction data into preference format to obtain the triplet training data. The obtained preference-formatted triplet training data can be shown in Equation 7.

[0160]

[0161] Among them, D PL The training dataset for triples in the preference format (or triplet format), x (i) This represents the i-th recommendation description instruction in the triplet training dataset. Let yw represent the positive sample in the triplet training dataset associated with the i-th recommendation description instruction. Let Yi represent the positive sample yi associated with the i-th recommendation description instruction in the triplet training dataset, and N represent the total number of triples in the triplet training dataset, where N is greater than or equal to 1 and is a positive integer. Therefore, in this embodiment, the finely tuned large model (i.e., the recommendation large model) is invoked to sequentially identify the triplet training data in the triplet training dataset in the format of each triplet, thereby introducing user preferences into the recommendation large model.

[0162] As shown in Equation 7, the following three key data are required to construct the preference triples:

[0163] (1) Recommendation description instruction x used to input recommendations (i) :x (i) This includes a task description for sequence recommendation, the user's interaction history with items, and candidate items that the recommendation system can select, as well as a range that can be used to include negative sampling. For example, the recommendation description instruction could be: "Based on the user's viewing history, recommend the next movie for the user to watch from the options. Viewing history can be denoted as [HISTORY_SEQUENCE], and options can be denoted as [CANDIDATE_ITEMS]."

[0164] (2) The output of the fine-tuned recommendation model is the positive sample yw: this is the movie that the user actually interacts with, such as movie A.

[0165] (3) The recommended model that is fine-tuned rejects the output as a negative sample yl: used to represent a negative sample item, such as movie B.

[0166] Because the training of the recommendation model proposed in this application requires injecting preferences into the rejection sampling step within the preference learning framework, and in the negative sampling theory related to the recommendation model, the smaller the value of K in the TopK evaluation metric (an indicator for evaluating the capabilities of the recommendation model), the more difficult the negative samples need to be sampled. The difficulty of negative samples can be measured by the semantic difference (or feature difference) between positive and negative samples; that is, the smaller the semantic difference between positive and negative samples, the more difficult the negative samples. Therefore, when making finer-grained adjustments to address the deficiencies of the recommendation model after the SFT stage, the results of erroneous recommendation predictions based on a certain recommendation description instruction from the SFT-adjusted recommendation model (and the fine-tuned recommendation model) can be sampled as negative samples.

[0167] When using the TopK evaluation metric to assess the capabilities of a large recommendation model, the value of K influences the performance of this metric. Specifically, the value of K determines the length of the predicted recommendations output by the large recommendation model. For example, K=10 indicates that the evaluation of the large recommendation model primarily focuses on whether the first 10 predicted recommendations meet user preferences. Therefore, it can be understood that a smaller K value requires more accurate recommendation predictions from the large recommendation model to better satisfy user preferences. To achieve more accurate predictions, the model needs to be trained and optimized with more difficult negative samples, ensuring that the evaluation based on a smaller K value meets expectations.

[0168] Therefore, when the recommendation model's erroneous recommendation prediction based on a certain recommendation description instruction is sampled as a negative sample in the triplet training data to obtain triplet training data, the computer device can first call the recommendation model to recognize and process the recommendation description instruction, obtaining the recommendation data output by the recommendation model; then, the recommendation data can be matched with positive samples, and if the matching fails, the recommendation data can be used as a negative sample of the recommendation description instruction; then, the computer device can use the data triple consisting of the recommendation description instruction, positive sample, and negative sample as the triplet training data of the recommendation model.

[0169] (ii) Generation of triplet training data based on specific model requirements.

[0170] The specific requirements of a large recommendation model refer to the designated system design preferences for that model. For example, the system design preference might be to recommend category A applications to users, rather than category B or C applications. Therefore, when generating triplet training data based on the feature requirements of the large recommendation model, the computer device can first acquire a candidate dataset containing positive samples and at least one candidate negative sample. Then, the semantic features of each candidate negative sample in the candidate dataset can be extracted, along with the semantic features of the recommendation description instruction. This allows for the determination of the semantic similarity between the semantic features of each candidate negative sample and the semantic features of the recommendation description instruction. Based on this semantic similarity, a negative sample is selected from at least one candidate negative sample.

[0171] In other words, computer devices can perform negative sampling based on the semantic similarity between semantic features in samples from a candidate dataset. Below, we will explain the process of negative sampling based on the similarity between semantic features, taking into account both mitigating semantic illusion and reducing popularity bias in the recommended system design preferences.

[0172] ① Reduce semantic illusion.

[0173] When a language-based recommender infers user preferences based on the semantic relevance between different data in the candidate dataset, spurious relevance caused by semantics is introduced. In this case, the computer device can use the average of the cosine distance of the semantic representation between each data in the user interaction sequence as the semantic similarity, and thus select the data with the highest semantic similarity as the data to be rejected.

[0174] In other words, when determining the semantic similarity between the semantic features of each candidate negative sample and the semantic features of the recommendation description instruction, the computer device can first obtain at least one historical positive sample associated with the recommendation description instruction and extract the semantic features of each historical positive sample; then, the semantic features of any candidate negative sample can be compared with the semantic features of each historical positive sample to obtain the semantic similarity between any candidate negative sample and each historical positive sample; furthermore, the mean of each semantic similarity of any candidate negative sample can be calculated to obtain the mean similarity of any candidate negative sample, and the mean similarity is used as the semantic similarity between any candidate negative sample and the recommendation description instruction.

[0175] ② Reduce the tendency of the epidemic.

[0176] If the recommendation system is designed to favor recommending niche data to promote data equity or to encourage publishing on UGC (User Generated Content) platforms, then negative samples can be sampled based on the popularity of the data.

[0177] In other words, when selecting a negative sample from at least one candidate negative sample based on semantic similarity, the computer device can select the candidate negative sample with the highest semantic similarity based on the semantic similarity between each candidate negative sample and the recommended description instruction; and use the candidate negative sample with the highest semantic similarity as the selected negative sample. Alternatively, the computer device can also select one or more candidate negative samples whose corresponding semantic similarity satisfies a certain similarity threshold, and use any one of the selected one or more candidate negative samples as the final selected negative sample.

[0178] (III) Generation of triplet training data based on model application scenarios.

[0179] The computer device can also collect corresponding negative samples for the large recommendation model based on the final recommendation requirements of the large recommendation model. In specific implementation, the computer device can first obtain the recommendation requirements and determine the candidate data associated with the recommendation requirements; the recommendation requirements are used to indicate the data types that the trained large recommendation model should refuse to select; the candidate data associated with the recommendation requirements is processed for data selection, and the selected candidate data is used as negative samples.

[0180] Here, the recommendation requirement is used to indicate user preferences. Specifically, if the user preference indicated by the recommendation requirement is to reduce sports data, then the data type of the candidate data associated with that recommendation requirement will be sports-related candidate data. Therefore, the selection strategy adopted by the computer device when selecting and processing the candidate data can be: random selection, selection according to specific selection rules, and obtaining corresponding negative samples.

[0181] After training the large recommendation model using triplet training data, the trained large recommendation model can be used to output target recommendation data according to the input instructions.

[0182] The process by which the trained recommendation model outputs the corresponding target recommendation data based on the recommendation instructions can be described as follows: Figure 4 As shown, when outputting target recommendation data, the computer device first obtains recommendation instructions and calls the trained recommendation model to perform data recommendation processing on the instructions, obtaining one or more recommendation data related to the instructions. Then, based on at least one recommendation data related to the instructions, the target recommendation data is output. The recommendation instructions can be user-input instructions describing user preferences, or they can be randomly generated by the computer device or generated based on the current recommendation scenario.

[0183] In one implementation, the computer device may also directly output at least one piece of recommended data related to the recommendation instruction as target recommended data; in another implementation, the computer device may also select any one of the at least one piece of recommended data as target recommended data for output.

[0184] Practice has shown that the fine-tuned recommendation model trained by the training method of this application embodiment has the following performance improvement.

[0185] 1. The recommendation model of this invention, after preference alignment, improves upon existing recommendation models (such as GRU4Rec (another recommendation model), Caser (yet another recommendation model), and SASRec (yet another recommendation model)) and the recommendation model of this invention (a large model based on SFT fine-tuning) on ​​three datasets: Movielens (a recommendation system dataset), Goodreads (a dataset containing reading data), and Steam (a game dataset) for five metrics: HR@1 (a recommendation evaluation metric), HR@5 (another recommendation evaluation metric), NDCG@5 (a ranking-based recommendation evaluation metric), HR@10 (yet another recommendation evaluation metric), and NDCG@10 (another ranking-based recommendation evaluation metric). The experimental results are shown in Tables 1 to 3 below.

[0186] Table 1

[0187]

[0188] Table 2

[0189]

[0190] Table 3

[0191]

[0192] 2. Under the preference alignment framework of this application embodiment, the experimental results of different negative sampling methods on the Movielens, Goodreads, and Steam datasets for five metrics (HR@1, HR@5, NDCG@5, HR@10, and NDCG@10) can be found in Tables 4 to 6 below.

[0193] Table 4

[0194]

[0195] Table 5

[0196]

[0197] Table 6

[0198]

[0199] 3. Under the preference alignment framework of this application embodiment, the experimental effect of semantic hard negative sampling on the semantic bias index (effectively reducing semantic bias compared to the base model SFT) can be seen in Table 7 below.

[0200] Table 7

[0201] Model Movielens Goodreads Steam SFT 8.50％ 10.99％ 11.43％ The application (semantic) 3.30％ 8.09％ 7.65％

[0202] This application's embodiments construct a personalized preference optimization framework for a large-scale model recommendation system. It achieves specified system design preferences by setting different negative sampling methods and proposes a loss function containing an instance-level smoothing factor to optimize personalized preferences. This allows the language-based large-scale model-enhanced recommendation system to achieve specified system design preferences through different negative sampling methods. The trained recommendation model obtained in this way can be applied in recall, coarse ranking, fine ranking, and re-ranking scenarios. Specifically, in the recall stage, it can be used to recall recommended items for user interaction; in the coarse / fine ranking stage, it can be used to rank the recalled content, especially in the fine ranking stage, which can model the comparative relationships between items at a finer granular level; and in the re-ranking stage, it can be used to align the recommendation results to the system designer's specified preferences, achieving fine-tuning of the recommendation results.

[0203] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a recommended data processing device provided in an embodiment of this application. For example... Figure 5 As shown, the recommended data processing device may include an acquisition unit 501 and a processing unit 502.

[0204] The acquisition unit 501 is used to acquire triplet training data constructed from the recommendation description instruction, positive samples and negative samples associated with the recommendation description instruction;

[0205] The acquisition unit is further configured to acquire a prediction probability based on the recommendation prediction processing of the triplet training data by the recommendation big model; the prediction probability is used to represent the probability that the recommendation big model, according to the recommendation description instruction, rejects the negative sample and favors the positive sample.

[0206] The acquisition unit is further configured to acquire the deterministic parameters corresponding to the recommendation model when performing recommendation prediction processing based on the triplet training data; the deterministic parameters are used to indicate the uncertainty of the recommendation model rejecting the negative sample and favoring the positive sample according to the recommendation description instruction;

[0207] Processing unit 502 is configured to determine the loss value for the recommendation model based on the predicted probability and the deterministic parameter, and train the recommendation model using the loss value to obtain the trained recommendation model; the trained recommendation model is used to output target recommendation data according to the recommendation instruction.

[0208] In one embodiment, the recommended large model is a large model that has undergone fine-tuning training;

[0209] The acquisition unit is further configured to acquire a recommendation description instruction and a positive sample associated with the recommendation description instruction; wherein the positive sample consists of M sequentially arranged word units; M≥1 and is a positive integer;

[0210] The processing unit 502 is further configured to call a pre-trained large model to identify the recommendation description instruction, and combine the t words of the positive sample to perform data prediction, so as to obtain the likelihood probability of the (t+1)th word output by the pre-trained large model; where 1≤t≤M and is a positive integer;

[0211] Based on the likelihood probability, the pre-trained large model is fine-tuned to obtain the recommended large model.

[0212] In one embodiment, the model parameters of the pre-trained large model include pre-training parameters and fine-tuning parameters;

[0213] The fine-tuning process for the pre-trained large model includes adjusting the fine-tuning parameters and obtaining the adjusted parameters.

[0214] The model parameters of the fine-tuned large model are the pre-training parameters and the adjustment parameters, and the model parameters of the fine-tuned large model are used as the initial model parameters of the recommended large model.

[0215] In one embodiment, the processing unit 502 is further configured to call the recommendation big model to identify and process the recommendation description instruction, and obtain the recommendation data output by the recommendation big model;

[0216] The processing unit 502 is further configured to perform data matching processing between the recommended data and the positive sample, and when the matching fails, use the recommended data as a negative sample of the recommended description instruction;

[0217] The processing unit 502 is further configured to use the data triplet consisting of the recommendation description instruction, the positive sample, and the negative sample as the triplet training data of the recommendation big model.

[0218] In one embodiment, the processing unit 501 is further configured to obtain a candidate dataset, the candidate dataset including the positive sample and at least one candidate negative sample;

[0219] The processing unit 502 is further configured to extract the semantic features of each candidate negative sample in the candidate dataset, and extract the semantic features of the recommendation description instruction;

[0220] The processing unit 502 is further configured to determine the semantic similarity between the semantic features of each candidate negative sample and the semantic features of the recommended description instruction, and select a negative sample from the at least one candidate negative sample based on the semantic similarity.

[0221] In one embodiment, the processing unit 502 is specifically used for:

[0222] Obtain at least one historical positive sample associated with the recommendation description instruction, and extract the semantic features of each historical positive sample;

[0223] The semantic similarity between any candidate negative sample and the semantic similarity between each historical positive sample is calculated by comparing the semantic features of any candidate negative sample with the semantic features of each historical positive sample.

[0224] The mean of the semantic similarities of any candidate negative sample is calculated to obtain the mean similarity of the candidate negative sample, and the mean similarity is used as the semantic similarity between the candidate negative sample and the recommendation description instruction.

[0225] In one embodiment, the processing unit 502 is specifically used for:

[0226] Based on the semantic similarity between each candidate negative sample and the recommended description instruction, the candidate negative sample with the highest semantic similarity is selected.

[0227] The candidate negative sample corresponding to the highest semantic similarity is selected as the selected negative sample.

[0228] In one embodiment, the acquisition unit 501 is further configured to acquire recommendation requirements and determine alternative data associated with the recommendation requirements; the recommendation requirements are used to indicate the data types that the trained recommendation model should reject.

[0229] The processing unit 502 is further configured to perform data selection processing on the alternative data associated with the recommendation requirements, and to use the selected alternative data as negative samples.

[0230] In one embodiment, the processing unit 502 is specifically used for:

[0231] The first probability that the recommendation model will favor the positive sample and the second probability that it will favor the negative sample when performing recommendation prediction processing based on the triplet training data are obtained.

[0232] Probability statistics are performed based on the first probability and the second probability, and the results of the statistical calculations are used as deterministic parameters.

[0233] In one embodiment, the processing unit 502 is specifically used for:

[0234] Perform exponential operations on the first probability and the second probability respectively, and sum the results of the exponential operations to obtain the probability sum;

[0235] Calculate the ratio between the exponential result of the second probability and the sum of the probabilities, and use the obtained ratio as the statistical result.

[0236] In one embodiment, the processing unit 502 is specifically used for:

[0237] Based on the predicted probability and the deterministic parameter, the first loss value of the recommendation model in the recommendation prediction process, which is to reject the negative sample and favor the positive sample according to the recommendation description instruction, is determined, and the second loss value is to reject the positive sample and favor the negative sample according to the recommendation description instruction is determined.

[0238] The sum of the first loss value and the second loss value is used as the loss value for the recommendation model.

[0239] In one embodiment, the processing unit 502 is specifically used for:

[0240] Based on the deterministic parameter, the deterministic of whether the large recommendation model will reject the negative sample and favor the positive sample according to the recommendation description instruction is determined.

[0241] Based on the certainty and the prediction probability, the first loss value is determined for the recommendation model during the recommendation prediction process, in accordance with the recommendation description instruction, to reject the negative sample and favor the positive sample.

[0242] In one embodiment, the processing unit 502 is specifically used for:

[0243] Based on the predicted probability, determine the reference probability that the recommendation model, according to the recommendation description instruction, rejects the positive sample and favors the negative sample;

[0244] Based on the uncertainty and the reference probability, a second loss value is determined for the recommendation model to reject the positive sample and favor the negative sample according to the recommendation description instruction.

[0245] In one embodiment, the processing unit 502 is specifically used for:

[0246] Obtain the recommendation instruction, and call the trained recommendation model to perform data recommendation processing on the recommendation instruction to obtain one or more recommendation data related to the input instruction;

[0247] Based on at least one piece of recommendation data associated with the recommendation instruction, output target recommendation data.

[0248] In this embodiment, when training the recommendation model, the acquisition unit 501 can acquire triplet data containing recommendation description instructions and positive and negative samples as triplet training data. Then, the processing unit 502 can call the recommendation model to perform recognition processing on the triplet training data, thereby obtaining the prediction probability that the recommendation model, based on the constraint of the recommendation description instructions, prefers to select positive samples for output recommendations rather than negative samples. Then, the acquisition unit 501 can also acquire the deterministic parameters of the recommendation model when making recommendation predictions based on the triplet training data. Based on the obtained prediction probability and deterministic parameters, the processing unit 502 can determine the loss value for the recommendation model and train the recommendation model according to the loss value. Since the obtained deterministic parameters are used to indicate the uncertainty of the recommendation model in outputting recommendations based on rejecting negative samples and preferring positive samples, the uncertainty of the triple training data is introduced into the loss value. Therefore, training the recommendation model based on this loss value can reduce this uncertainty. Based on the reduction of the uncertainty of the triple training data, the trained recommendation model can effectively reflect the user's preferences and needs, thus achieving an accurate understanding of user preferences. This can effectively improve the training effectiveness of the recommendation model on the computer and enhance the robustness of the trained recommendation model.

[0249] According to one embodiment of this application, Figure 5The modules in the recommended data processing device shown can be individually or entirely combined into one or more units, or some of the units can be further divided into multiple functionally smaller sub-units to achieve the same operation without affecting the technical effects of the embodiments of this application. The above modules are based on logical function division. In practical applications, the function of one module can also be implemented by multiple units, or the function of multiple modules can be implemented by one unit. In other embodiments of this application, the recommended data processing device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0250] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0251] According to one embodiment of this application, a computer program capable of executing the steps involved in the corresponding methods shown in the various embodiments of this application can be run on a general-purpose computer device (which may include processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM)) to construct a system as described in the embodiments of this application. Figure 5 The recommended data processing apparatus shown herein. The computer program described above can be recorded on a computer-readable recording medium, and can be loaded into and run thereon on the computer device via the computer-readable recording medium.

[0252] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 6As shown, the computer device 1000 may include a processor 1001, a network interface 1004, and a memory 1005. In some embodiments, the computer device 1000 may also include a user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the aforementioned processor 1001. Figure 6 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program.

[0253] exist Figure 6 In the computer device 1000 shown, the network interface 1004 provides network communication functionality; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0254] Obtain training data consisting of triplets constructed from recommendation description instructions, positive samples associated with the recommendation description instructions, and negative samples;

[0255] Based on the recommendation prediction process of the large recommendation model on the training data of the triplet, the prediction probability is obtained; the prediction probability is used to represent the probability that the large recommendation model rejects the negative sample and favors the positive sample according to the recommendation description instruction.

[0256] Obtain the deterministic parameters corresponding to the recommendation model when performing recommendation prediction processing based on the triplet training data; the deterministic parameters are used to indicate the uncertainty of the recommendation model in rejecting the negative sample and favoring the positive sample according to the recommendation description instruction;

[0257] Based on the predicted probability and the deterministic parameter, a loss value is determined for the large recommendation model, and the large recommendation model is trained using the loss value to obtain a trained large recommendation model; the trained large recommendation model is used to output target recommendation data according to the recommendation instruction.

[0258] In one embodiment, the recommended large model is a large model that has undergone fine-tuning training; the processor 1001 is also used to implement:

[0259] Obtain the recommendation description instruction and the positive sample associated with the recommendation description instruction; wherein the positive sample consists of M sequentially arranged word units; M≥1 and is a positive integer;

[0260] The pre-trained large model is invoked to identify the recommendation description instruction, and the data prediction is performed by combining the t words of the positive sample to obtain the likelihood probability of the (t+1)th word output by the pre-trained large model; where 1≤t≤M and are positive integers;

[0261] Based on the likelihood probability, the pre-trained large model is fine-tuned to obtain the recommended large model.

[0262] In one embodiment, the model parameters of the pre-trained large model include pre-training parameters and fine-tuning parameters;

[0263] The fine-tuning process for the pre-trained large model includes adjusting the fine-tuning parameters and obtaining the adjusted parameters.

[0264] The model parameters of the fine-tuned large model are the pre-training parameters and the adjustment parameters, and the model parameters of the fine-tuned large model are used as the initial model parameters of the recommended large model.

[0265] In one embodiment, the processor 1001 is further configured to implement:

[0266] The recommendation model is invoked to identify and process the recommendation description instruction, thereby obtaining the recommendation data output by the recommendation model.

[0267] The recommended data is matched with the positive sample, and if the matching fails, the recommended data is used as the negative sample of the recommended description instruction.

[0268] The data triples consisting of the recommendation description instruction, the positive sample, and the negative sample are used as the triple training data for the recommendation model.

[0269] In one embodiment, the processor 1001 is further configured to implement:

[0270] Obtain a candidate dataset, which includes the positive sample and at least one candidate negative sample;

[0271] Extract the semantic features of each candidate negative sample in the candidate dataset, and extract the semantic features of the recommendation description instruction;

[0272] Determine the semantic similarity between the semantic features of each candidate negative sample and the semantic features of the recommended description instruction, and select a negative sample from the at least one candidate negative sample based on the semantic similarity.

[0273] In one embodiment, the processor 1001 is further configured to implement:

[0274] Obtain at least one historical positive sample associated with the recommendation description instruction, and extract the semantic features of each historical positive sample;

[0275] The semantic similarity between any candidate negative sample and the semantic similarity between each historical positive sample is calculated by comparing the semantic features of any candidate negative sample with the semantic features of each historical positive sample.

[0276] The mean of the semantic similarities of any candidate negative sample is calculated to obtain the mean similarity of the candidate negative sample, and the mean similarity is used as the semantic similarity between the candidate negative sample and the recommendation description instruction.

[0277] In one embodiment, the processor 1001 is further configured to implement:

[0278] Based on the semantic similarity between each candidate negative sample and the recommended description instruction, the candidate negative sample with the highest semantic similarity is selected.

[0279] The candidate negative sample corresponding to the highest semantic similarity is selected as the selected negative sample.

[0280] In one embodiment, the processor 1001 is further configured to implement:

[0281] Obtain recommendation requirements and determine alternative data associated with the recommendation requirements; the recommendation requirements are used to indicate the data types that the trained recommendation model should refuse to select.

[0282] The candidate data associated with the recommendation requirement is processed for data selection, and the selected candidate data is used as negative samples.

[0283] In one embodiment, the processor 1001 is further configured to implement:

[0284] The first probability that the recommendation model will favor the positive sample and the second probability that it will favor the negative sample when performing recommendation prediction processing based on the triplet training data are obtained.

[0285] Probability statistics are performed based on the first probability and the second probability, and the results of the statistical calculations are used as deterministic parameters.

[0286] In one embodiment, the processor 1001 is further configured to implement:

[0287] Perform exponential operations on the first probability and the second probability respectively, and sum the results of the exponential operations to obtain the probability sum;

[0288] Calculate the ratio between the exponential result of the second probability and the sum of the probabilities, and use the obtained ratio as the statistical result.

[0289] In one embodiment, the processor 1001 is further configured to implement:

[0290] Based on the predicted probability and the deterministic parameter, the first loss value of the recommendation model in the recommendation prediction process, which is to reject the negative sample and favor the positive sample according to the recommendation description instruction, is determined, and the second loss value is to reject the positive sample and favor the negative sample according to the recommendation description instruction is determined.

[0291] The sum of the first loss value and the second loss value is used as the loss value for the recommendation model.

[0292] In one embodiment, the processor 1001 is further configured to implement:

[0293] Based on the deterministic parameter, the deterministic of whether the large recommendation model will reject the negative sample and favor the positive sample according to the recommendation description instruction is determined.

[0294] Based on the certainty and the prediction probability, the first loss value is determined for the recommendation model during the recommendation prediction process, in accordance with the recommendation description instruction, to reject the negative sample and favor the positive sample.

[0295] In one embodiment, the processor 1001 is further configured to implement:

[0296] Based on the predicted probability, determine the reference probability that the recommendation model, according to the recommendation description instruction, rejects the positive sample and favors the negative sample;

[0297] Based on the uncertainty and the reference probability, a second loss value is determined for the recommendation model to reject the positive sample and favor the negative sample according to the recommendation description instruction.

[0298] In one embodiment, the processor 1001 is further configured to implement:

[0299] Obtain the recommendation instruction, and call the trained recommendation model to perform data recommendation processing on the recommendation instruction to obtain one or more recommendation data related to the input instruction;

[0300] Based on at least one piece of recommendation data associated with the recommendation instruction, output target recommendation data.

[0301] It should be understood that the computer device 1000 described in the embodiments of this application can execute the above-described recommended data processing methods in the various embodiments of this application, and can also execute the methods described above. Figure 5 The description of the recommended data processing apparatus in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated.

[0302] Furthermore, it should be noted that this application also provides a computer-readable storage medium storing a computer program. When a processor executes the computer program, it can perform the data processing methods described in the various embodiments of this application; therefore, these methods will not be repeated here. Additionally, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the embodiments of the computer storage medium involved in this application, please refer to the description of the method embodiments of this application.

[0303] As an example, the aforementioned computer program can be deployed and executed on a single computer device, or deployed and executed on multiple computer devices located in one location, or executed on multiple computer devices distributed across multiple locations and interconnected via a communication network. These multiple computer devices distributed across multiple locations and interconnected via a communication network can form a blockchain network.

[0304] The aforementioned computer-readable storage medium can be an internal storage unit of the computer device, such as a hard drive or memory. It can also be an external storage device, such as a plug-in hard drive, smart media card (SMC), secure digital card (SD) card, or flash card. Furthermore, the computer-readable storage medium can include both internal and external storage units of the computer device. This computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. It can also be used to temporarily store data that has been output or will be output.

[0305] This application provides a computer program product comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the data processing methods described in the embodiments of this application; therefore, these descriptions will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the embodiments of the computer-readable storage medium involved in this application, please refer to the description of the method embodiments of this application.

[0306] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0307] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0308] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method for processing recommendation data, characterized in that, include: Obtain training data consisting of triplets constructed from recommendation description instructions, positive samples associated with the recommendation description instructions, and negative samples; Based on the recommendation prediction process of the large recommendation model on the training data of the triplet, the prediction probability is obtained; the prediction probability is used to represent the probability that the large recommendation model rejects the negative sample and favors the positive sample according to the recommendation description instruction. Obtain the deterministic parameters corresponding to the recommendation model when performing recommendation prediction processing based on the triplet training data; the deterministic parameters are used to indicate the uncertainty of the recommendation model in rejecting the negative sample and favoring the positive sample according to the recommendation description instruction; Based on the predicted probability and the deterministic parameter, a loss value is determined for the large recommendation model, and the large recommendation model is trained using the loss value to obtain a trained large recommendation model; the trained large recommendation model is used to output target recommendation data according to the recommendation instruction.

2. The method as described in claim 1, characterized in that, The recommended large model is a large model that has undergone fine-tuning training; the method also includes: Obtain the recommendation description instruction and the positive sample associated with the recommendation description instruction; wherein the positive sample consists of M sequentially arranged word units; M≥1 and is a positive integer; The pre-trained large model is invoked to identify the recommendation description instruction, and the data prediction is performed by combining the t words of the positive sample to obtain the likelihood probability of the (t+1)th word output by the pre-trained large model; where 1≤t≤M and are positive integers; Based on the likelihood probability, the pre-trained large model is fine-tuned to obtain the recommended large model.

3. The method as described in claim 2, characterized in that, The model parameters of the pre-trained large model include pre-training parameters and fine-tuning parameters; The fine-tuning process for the pre-trained large model includes adjusting the fine-tuning parameters and obtaining the adjusted parameters. The model parameters of the fine-tuned large model are the pre-training parameters and the adjustment parameters, and the model parameters of the fine-tuned large model are used as the initial model parameters of the recommended large model.

4. The method as described in claim 2, characterized in that, The method further includes: The recommendation model is invoked to identify and process the recommendation description instruction, thereby obtaining the recommendation data output by the recommendation model. The recommended data is matched with the positive sample, and if the matching fails, the recommended data is used as the negative sample of the recommended description instruction. The data triples consisting of the recommendation description instruction, the positive sample, and the negative sample are used as the triple training data for the recommendation model.

5. The method as described in claim 1 or 2, characterized in that, The method further includes: Obtain a candidate dataset, which includes the positive sample and at least one candidate negative sample; Extract the semantic features of each candidate negative sample in the candidate dataset, and extract the semantic features of the recommendation description instruction; Determine the semantic similarity between the semantic features of each candidate negative sample and the semantic features of the recommended description instruction, and select a negative sample from the at least one candidate negative sample based on the semantic similarity.

6. The method as described in claim 5, characterized in that, Determining the semantic similarity between the semantic features of each candidate negative sample and the semantic features of the recommendation description instruction includes: Obtain at least one historical positive sample associated with the recommendation description instruction, and extract the semantic features of each historical positive sample; The semantic similarity between any candidate negative sample and the semantic similarity between each historical positive sample is calculated by comparing the semantic features of any candidate negative sample with the semantic features of each historical positive sample. The mean of the semantic similarities of any candidate negative sample is calculated to obtain the mean similarity of the candidate negative sample, and the mean similarity is used as the semantic similarity between the candidate negative sample and the recommendation description instruction.

7. The method as described in claim 5, characterized in that, The step of selecting negative samples from the at least one candidate negative samples based on semantic similarity includes: Based on the semantic similarity between each candidate negative sample and the recommended description instruction, the candidate negative sample with the highest semantic similarity is selected. The candidate negative sample corresponding to the highest semantic similarity is selected as the selected negative sample.

8. The method as described in claim 1 or 2, characterized in that, The method further includes: Obtain recommendation requirements and determine alternative data associated with the recommendation requirements; the recommendation requirements are used to indicate the data types that the trained recommendation model should refuse to select. The candidate data associated with the recommendation requirement is processed for data selection, and the selected candidate data is used as negative samples.

9. The method as described in claim 1, characterized in that, The process of obtaining the determined parameters corresponding to the recommendation model when performing recommendation prediction processing based on the triplet training data includes: The first probability that the recommendation model will favor the positive sample and the second probability that it will favor the negative sample when performing recommendation prediction processing based on the triplet training data are obtained. Probability statistics are performed based on the first probability and the second probability, and the results of the statistical calculations are used as deterministic parameters.

10. The method as described in claim 9, characterized in that, The step of performing probability statistical calculations based on the first probability and the second probability includes: Perform exponential operations on the first probability and the second probability respectively, and sum the results of the exponential operations to obtain the probability sum; Calculate the ratio between the exponential result of the second probability and the sum of the probabilities, and use the obtained ratio as the statistical result.

11. The method as described in claim 1, characterized in that, Determining the loss value for the recommendation model based on the predicted probability and the deterministic parameter includes: Based on the predicted probability and the deterministic parameter, the first loss value of the recommendation model in the recommendation prediction process, which is to reject the negative sample and favor the positive sample according to the recommendation description instruction, is determined, and the second loss value is to reject the positive sample and favor the negative sample according to the recommendation description instruction is determined. The sum of the first loss value and the second loss value is used as the loss value for the recommendation model.

12. The method as described in claim 11, characterized in that, The step of determining, based on the predicted probability and the deterministic parameter, the first loss value by which the recommendation model, in the recommendation prediction process, rejects the negative sample and favors the positive sample according to the recommendation description instruction includes: Based on the deterministic parameter, the deterministic of whether the large recommendation model will reject the negative sample and favor the positive sample according to the recommendation description instruction is determined. Based on the certainty and the prediction probability, the first loss value is determined for the recommendation model during the recommendation prediction process, in accordance with the recommendation description instruction, to reject the negative sample and favor the positive sample.

13. The method as described in claim 11, characterized in that, The determination of the second loss value that, according to the recommendation description instruction, rejects the positive sample and favors the negative sample includes: Based on the predicted probability, determine the reference probability that the recommendation model, according to the recommendation description instruction, rejects the positive sample and favors the negative sample; Based on the uncertainty and the reference probability, a second loss value is determined for the recommendation model to reject the positive sample and favor the negative sample according to the recommendation description instruction.

14. The method as described in claim 1, characterized in that, The method further includes: Obtain the recommendation instruction, and call the trained recommendation model to perform data recommendation processing on the recommendation instruction to obtain one or more recommendation data related to the input instruction; Based on at least one piece of recommendation data associated with the recommendation instruction, output target recommendation data.

15. A recommendation data processing apparatus, characterized in that, include: The acquisition unit is used to acquire triplet training data constructed from the recommendation description instruction, positive samples and negative samples associated with the recommendation description instruction; The acquisition unit is further configured to acquire a prediction probability based on the recommendation prediction processing of the triplet training data by the large recommendation model; the prediction probability represents the probability that the large recommendation model, according to the recommendation description instruction, rejects the negative sample and favors the positive sample. The acquisition unit is further configured to acquire the deterministic parameters corresponding to the recommendation model when performing recommendation prediction processing based on the triplet training data; the deterministic parameters are used to indicate the uncertainty of the recommendation model rejecting the negative sample and favoring the positive sample according to the recommendation description instruction; The processing unit is configured to determine the loss value for the large recommendation model based on the predicted probability and the deterministic parameter, and to train the large recommendation model using the loss value to obtain the trained large recommendation model; the trained large recommendation model is used to output target recommendation data according to the recommendation instruction.

16. A computer device, characterized in that, The system includes a processor, an input device, an output device, and a memory, wherein the processor, the input device, the output device, and the memory are interconnected, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to execute the method as described in any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1 to 14.

18. A computer program product, characterized in that, The computer program product includes a computer program, which includes program instructions that, when invoked by a processor, cause the processor to perform the method as described in any one of claims 1 to 14.

Images