Model training-based resource recommendation method and device, equipment, medium and product

Abstract

The application discloses a resource recommendation method and device based on model training, equipment, medium and product, and the method comprises the steps that a sample object interacts with a media resource sequence and a target token sequence are acquired; a recommendation model is called to infer the generation probability of each token in the target token sequence according to the media resource sequence; the target token sequence is used to construct T prefix sequences with different lengths, and the prefix sequence with a length of m comprises the first m tokens in the target token sequence; the generation loss of each prefix sequence is perceived based on the generation probability of each token, and the recommendation model is optimized based on the generation loss of the T prefix sequences, and the optimized recommendation model is used to infer the token sequence of a media resource to be recommended to perform media resource recommendation, so that the accuracy of media resource recommendation is improved.

Description

Technical Field

[0001] This application relates to the field of Internet technology, specifically to the field of computer technology, and in particular to a resource recommendation method, apparatus, device, medium, and product based on model training. Background Technology

[0002] In recent years, generative recommendation systems have attracted widespread attention because they can uniformly model recommendation tasks as sequence generation problems. These systems typically encode each media resource interacted with by the user as a prefix (prompt) and guide the recommendation model to autoregressively generate a token sequence of the media resources to be recommended, thereby making media resource recommendations based on this token sequence.

[0003] Currently, in order to efficiently retrieve high-quality media resources in a vast media resource space, the Beam Search strategy has become the de facto standard decoding strategy. This strategy requires the recommendation model to generate multiple prefix sequences in each step of the reasoning process, score these prefix sequences, prune the prefix sequences with lower scores, and retain the K prefix sequences with the highest scores for the next step of reasoning, so as to finally generate a complete token sequence of the media resources to be recommended, and output a media recommendation list based on these token sequences.

[0004] It is evident that in the actual inference stage (application stage) of the recommendation model, bundle search dynamically prunes the data. However, existing technologies typically infer the token sequence of the target media resource during training, then calculate the generation loss of the entire token sequence based on the inference result, and optimize the recommendation model based on this generation loss. This training logic causes the recommendation model to focus only on the generation probability of the final complete sequence (i.e., the entire token sequence) during training, completely ignoring the reality of dynamic pruning using the bundle search strategy during the inference stage. This results in an inconsistency between training and inference in existing recommendation models, which will affect the accuracy of media resource recommendations. Summary of the Invention

[0005] This application provides a resource recommendation method, apparatus, device, medium, and product based on model training, which can solve the problem of inconsistency between training and inference in recommendation models, thereby improving the accuracy of media resource recommendation.

[0006] On the one hand, embodiments of this application provide a resource recommendation method based on model training, the method comprising: Obtain the sequence of media resources that the sample object has interacted with, and the target token sequence; the target token sequence is used to identify the target media resource that the sample object interacted with after the interaction time corresponding to the media resource sequence, and the target token sequence includes T tokens, where T is an integer greater than 1; The recommendation model is invoked to infer the generation probability of each token in the target token sequence based on the media resource sequence; Using the target token sequence, construct T prefix sequences of different lengths; wherein, the prefix sequence of length m includes the first m tokens in the target token sequence, m∈[1,T]; The generation loss of each prefix sequence is perceived based on the generation probability of each token. The generation loss refers to the loss incurred by the recommendation model inferring the corresponding prefix sequence based on the media resource sequence. The recommendation model is optimized based on the generation loss of T prefix sequences. The optimized recommendation model is used to infer the token sequence of the media resources to be recommended, so as to make media resource recommendations.

[0007] On the other hand, embodiments of this application provide a resource recommendation apparatus based on model training, the apparatus comprising: The acquisition unit is used to acquire the media resource sequence that the sample object has interacted with, and the target token sequence; the target token sequence is used to identify the target media resource that the sample object interacted with after the interaction time corresponding to the media resource sequence, and the target token sequence includes T tokens, where T is an integer greater than 1; The processing unit is used to call the recommendation model to infer the generation probability of each token in the target token sequence based on the media resource sequence; The processing unit is further configured to construct T prefix sequences of different lengths using the target token sequence; wherein the prefix sequence of length m includes the first m tokens in the target token sequence, m∈[1,T]; The processing unit is also used to perceive the generation loss of each prefix sequence based on the generation probability of each token. The generation loss refers to the loss generated by the recommendation model inferring the corresponding prefix sequence based on the media resource sequence. The processing unit is further configured to optimize the recommendation model based on the generation loss of T prefix sequences. The optimized recommendation model is used to infer the token sequence of the media resource to be recommended, so as to recommend the media resource.

[0008] In one implementation, the generation loss of each prefix sequence includes: prefix-aware point-to-point loss; When perceiving the generation loss of each prefix sequence based on the generation probability of each token, the processing unit can specifically be used to: Based on the generation probability of each token, the generation sub-loss of the corresponding token is calculated. The generation sub-loss of any token refers to the loss generated by the recommendation model inferring the corresponding token based on the media resource sequence. The generation sub-losses of each token in each prefix sequence are integrated to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence.

[0009] In another implementation, when calculating the generation sub-loss of the corresponding token based on the generation probability of each token, the processing unit may specifically be used to: perform negative log-likelihood calculation on the generation probability of each token to obtain the generation sub-loss of the corresponding token. Accordingly, when integrating the generation sub-losses of each token in each prefix sequence to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence, the processing unit can specifically be used to: perform mean calculation on the generation sub-losses of each token in the same prefix sequence to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence.

[0010] In another implementation, the generation loss for each prefix sequence includes: prefix-aware pairwise loss; When perceiving the generation loss of each prefix sequence based on the generation probability of each token, the processing unit can specifically be used to: The target media resource identified by the target token sequence is taken as a positive sample, and the negative sample corresponding to the positive sample is obtained. The negative sample is a media resource that the sample object has not interacted with. Obtain the token sequence of the negative sample, and call the recommendation model to infer the generation probability of each token in the token sequence of the negative sample based on the media resource sequence; The prefix sequence of length m constructed based on the target token sequence is used as the positive prefix sequence, and the first m tokens from the token sequence of the negative sample are selected as the negative prefix sequence. Based on the generation probability of each token in the positive prefix sequence and the generation probability of each token in the negative prefix sequence, the prefix-aware pairwise loss of the prefix sequence of length m is calculated.

[0011] In another implementation, when calculating the prefix-aware pairwise loss of the prefix sequence of length m based on the generation probabilities of each token in the positive prefix sequence and the generation probabilities of each token in the negative prefix sequence, the processing unit may specifically be used to: The cumulative probability logarithmic score of the positive prefix sequence is calculated using the generation probability of each token in the positive prefix sequence. The cumulative probability logarithmic score of the negative prefix sequence is calculated using the generation probability of each token in the negative prefix sequence. Based on the cumulative probability log score of the negative prefix sequence and the cumulative probability log score of the positive prefix sequence, calculate the prefix-aware pairwise loss of the prefix sequence of length m. The prefix-aware pairwise loss is used to increase the cumulative probability log score difference between the positive prefix sequence and the negative prefix sequence.

[0012] In another implementation, there are N negative samples and N negative prefix sequences, where N is an integer greater than 1. One negative prefix sequence corresponds to one negative sample, and any negative prefix sequence is composed of the first m tokens in the token sequence of the corresponding negative sample. When calculating the prefix-aware pairwise loss of the prefix sequence of length m based on the cumulative log probability score of the negative prefix sequence and the cumulative log probability score of the positive prefix sequence, the processing unit may specifically be used to: The cumulative probability logarithmic score of each negative prefix sequence is compared with the cumulative probability logarithmic score of the positive prefix sequence to obtain the score difference between each negative prefix sequence and the positive prefix sequence. The score difference between each negative prefix sequence and the positive prefix sequence is integrated to obtain the prefix-aware pairwise loss of the prefix sequence of length m.

[0013] In another implementation, when optimizing the recommendation model based on the generation loss of T prefix sequences, the processing unit may specifically be used to: Obtain the weight of each of the T prefix sequences; The generation loss of the T prefix sequences is aggregated using the weight of each prefix sequence to obtain the prefix-aware level loss; The recommendation model is optimized based on the prefix-aware loss.

[0014] In another implementation, the recommendation model is optimized in P rounds using the media resource sequence and the target token sequence, where P is an integer greater than 1. In any round of optimization, after obtaining the generation loss of each prefix sequence, the weight of each prefix sequence is dynamically updated, and the updated weight is proportional to the generation loss of the corresponding prefix sequence.

[0015] In another implementation, the step of obtaining the weight of each of the T prefix sequences is performed during the p-th round of optimization, where p∈[1, P-1]. In the p-th round of optimization, the generation loss of the T prefix sequences is aggregated using the weight of each prefix sequence to obtain the prefix-aware level loss. The processing unit can then be specifically used for: Based on the generation loss of each prefix sequence, a weight scaling factor is calculated for each prefix sequence, wherein the weight scaling factor is positively correlated with the generation loss of the corresponding prefix sequence; The weights of the corresponding prefix sequences are scaled using the weight scaling factor of each prefix sequence to update the weights of the corresponding prefix sequences. The updated weights are used to aggregate the generation loss of the T prefix sequences during the (p+1)th round of optimization.

[0016] In another implementation, when optimizing the recommendation model based on the prefix-aware level loss, the processing unit may specifically be used to: Based on the generation probability of each token in the target token sequence, calculate the cross-entropy loss; The prefix-aware loss and the cross-entropy loss are integrated to obtain the model loss, and the recommendation model is optimized based on the model loss.

[0017] In another aspect, embodiments of this application provide a computer device, the computer device including an input interface and an output interface, the computer device further including: Processor and computer storage media; The processor is adapted to implement one or more instructions, and the computer storage medium stores one or more instructions, which are adapted to be loaded and executed by the processor to perform the aforementioned model-trained resource recommendation method.

[0018] In another aspect, embodiments of this application provide a computer storage medium storing one or more instructions, which are adapted to be loaded and executed by a processor of the aforementioned model-training-based resource recommendation method.

[0019] In another aspect, embodiments of this application provide a computer program product comprising one or more instructions; when one or more instructions in the computer program product are executed by a processor, they implement the aforementioned resource recommendation method based on model training.

[0020] This application embodiment can obtain the media resource sequence that the sample object has interacted with, and the token sequence of the target media resource that the sample object interacted with after the corresponding interaction time of the media resource sequence (i.e., the target token sequence). It then calls the recommendation model to infer the generation probability of each token in the target token sequence based on the media resource sequence, and constructs T prefix sequences of different lengths using the target token sequence. This allows the generation loss of each prefix sequence to be perceived based on the generation probability of each token. This method eliminates the need to separately call the recommendation model to infer the generation loss of each prefix sequence, thus saving processing resources consumed by the recommendation model during inference and ensuring the efficiency of perceiving the generation loss of each prefix sequence. Furthermore, the recommendation model can be optimized based on the generation loss of the T prefix sequences. This refines the optimization granularity of the recommendation model from the entire token sequence to prefix sequences of arbitrary length, enabling the recommendation model to focus on the generation probability of prefix sequences of arbitrary length throughout the optimization process and forcing the recommendation model to maximize the generation probability of each prefix sequence in each inference step. As can be seen, this application introduces prefix-level optimization constraints in the model training stage that are consistent with the bundle search strategy behavior used in the model inference stage. This can ensure that the recommendation model learns the ability to make the token sequence of the correct media resource (such as the target token sequence) survive to the end under the strict screening of the bundle search strategy, thereby solving the problem of inconsistency between training and inference in the recommendation model and improving the accuracy of the recommendation model in recommending media resources. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram illustrating the principle of a resource recommendation method based on model training provided in an embodiment of this application; Figure 2 This is a flowchart illustrating a resource recommendation method based on model training provided in an embodiment of this application; Figure 3 This is a schematic diagram illustrating the construction of multiple prefix sequences based on a target token sequence, provided in an embodiment of this application. Figure 4 This is a flowchart illustrating a resource recommendation method based on model training, provided in another embodiment of this application. Figure 5 This is a schematic diagram of the calculation logic for a prefix-sensing point-to-point loss provided in an embodiment of this application; Figure 6a This is an algorithm flowchart of an adaptive prefix-aware weighted framework provided in an embodiment of this application; Figure 6b This is a schematic diagram illustrating a resource recommendation based on a beam search strategy provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a resource recommendation device based on model training provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0023] To facilitate understanding of the specific implementation of the technical solutions proposed in the embodiments of this application, the key technical terms involved in the embodiments of this application are introduced below: I. Media Resources Media resources refer to resources (content and materials) that exist in the form of media and can be disseminated, displayed, or operated through media channels (such as websites, software applications, social platforms, television, radio, etc.).

[0024] In the embodiments of this application, the media resources mentioned thereafter may also be referred to as items, which may include, but are not limited to, any of the following: information (such as news, articles, advertisements, etc.), multimedia data (such as audio, video, images, etc.), goods (such as real items or virtual resources such as game props), applications (such as mini programs, APP (clients), etc.), live streaming rooms (virtual spaces that can be used for activities such as live video streaming, content sharing, interactive communication, and sales of goods), content sharing accounts (platform accounts that can be used to create, publish, disseminate, and share content), etc.

[0025] II. Recommendation Model A recommendation model is a model used for recommending media resources. Its backbone network can be a language model or a variant thereof. A language model, also known as a large language model (LLM), is an artificial intelligence model built using deep learning techniques. It consists of artificial neural networks with hundreds of millions of parameters and can handle complex natural language processing tasks by understanding and generating natural language text. For example, it can be applied to large language model recommendation systems (LLMRec) to handle media resource recommendation tasks.

[0026] As an optional implementation, the recommendation model mentioned in this application embodiment can be a pre-trained language model. A pre-trained language model refers to a language model trained on a large-scale corpus without manual annotation. It can learn general language representations and knowledge from the large-scale corpus and apply them to multiple downstream tasks, such as natural language understanding, text generation, translation, and classification. Therefore, when the recommendation model mentioned in this application embodiment is a pre-trained language model, it can seamlessly integrate the world knowledge of the pre-trained language model. Thus, in the process of handling media resource recommendation tasks, it can leverage its powerful world knowledge, reasoning, and generalization capabilities to achieve more intelligent and flexible media resource recommendations.

[0027] In this embodiment, the recommendation model is based on the generative recommendation (GR) paradigm for media resource recommendation. The generative recommendation paradigm is a new paradigm that reconstructs the media resource sequence recommendation task into an autoregressive sequence generation problem. In this paradigm, the recommendation model no longer scores and ranks the candidate media resource set, but directly generates a token sequence that represents the media resource to be recommended. Media resource recommendations are then made based on the generated token sequence, thus unifying the recall and ranking processes.

[0028] III. Token Sequence Token sequences, also known as resource semantic identifiers or semantic IDs (SIDs), are the core representation of media resources. Specifically, they are identifiers that carry the semantic information of the media resource. This semantic information expresses the resource's inherent meaning, theme, sentiment, intent, and category—deeper aspects of the media resource. Therefore, the semantic information of a media resource reflects its category; similarly, since the token sequence (SID) carries the semantic information of the media resource, it can indicate the category to which the media resource belongs.

[0029] In this embodiment, any token sequence may include T tokens, where T is an integer greater than 1. Different tokens may belong to different levels. Tokens in any level can be a discrete symbol, such as a numerical sequence (e.g., a sequence consisting of at least one integer value) or a letter, used to represent a category. That is, any level may have multiple categories, and the token of a media resource at a certain level can be used to indicate the category to which the media resource belongs in the corresponding level. Each level corresponds to a semantic granularity, and adjacent levels satisfy the characteristic of increasing semantic granularity. Increasing semantic granularity means that the semantic granularity of the next level is greater than that of the previous level.

[0030] For example, taking media resources as smart products, if there are three levels, the semantic granularity corresponding to the first level can be the semantic granularity of product category. The categories in the first level can include, but are not limited to: smart terminal, smart home, smart home appliance, etc.; the semantic granularity corresponding to the second level can be the semantic granularity of product subcategory. The categories in the second level can include, but are not limited to: smartphone, tablet, smart door lock, smart air conditioner, smart TV, smart sofa, etc.; and the semantic granularity corresponding to the third level can be the semantic granularity of product model. The categories in the third level can include, but are not limited to: mobile phone model A-1, mobile phone model A-2, tablet B-1, tablet B-2, smart door lock C-1, smart door lock C-2, smart air conditioner D-1, smart air conditioner D-2, smart TV E-1, smart TV E-2, smart sofa F-1, smart sofa F-2, etc.

[0031] Based on this, if the token sequence of a media resource is [000, 010, 013], then the first discrete token (i.e. "000") in the token sequence can indicate that the media resource is categorized as a smart terminal at the first level, the second discrete token (i.e. "010") can indicate that the media resource is categorized as a smartphone at the second level, and the third discrete token (i.e. "013") can indicate that the media resource is categorized as "phone model A-2" at the third level.

[0032] IV. Prefix Sequence A prefix sequence, or simply a prefix, refers to a sequence formed by extracting one or more consecutive tokens from a token sequence, starting with the first token. The length of a prefix sequence depends on the number of tokens it contains. For example, if a prefix sequence is formed by extracting one token, it includes the first token and has a length of 1. If it's formed by extracting two tokens, it includes the first two tokens and has a length of 2. If it's formed by extracting three tokens, it includes the first three tokens and has a length of 3, and so on.

[0033] Understandably, the length of the prefix sequence can range from [1, T], where T is the total number of tokens in the token sequence (i.e., the length of the token sequence). When the length of the prefix sequence is greater than or equal to 1 and less than T, it indicates that the prefix sequence includes some tokens in the token sequence, which is equivalent to a subsequence of the token sequence; while when the length of the prefix sequence is equal to T, it indicates that the prefix sequence includes all tokens in the token sequence, which is the same as the token sequence.

[0034] V. Bundle Search Strategy The beam search strategy is a heuristic search and decoding strategy used by recommendation models during the inference phase. At each step of the inference process, the beam search strategy maintains a candidate prefix set of size K, where K represents the beam size (a value greater than 1). For each prefix sequence in this candidate prefix set, the recommendation model can expand (infer) all possible next tokens and add the expanded tokens to the corresponding prefix sequence to obtain multiple new prefix sequences. The model then calculates the score for each new prefix sequence, retaining only the K highest-scoring new prefix sequences as the candidate prefix set for the next inference step. The remaining branches (those whose scores do not enter the top K new prefix sequences) are permanently pruned (discarded).

[0035] For example, the specific process of a recommendation model generating a token sequence using a bundle search strategy can be as follows: In the first step of reasoning, the recommendation model can perform H inferences based on the input data (H is an integer greater than 1) to obtain H tokens1 and calculate the score of each token1. From the H tokens1, the top K tokens1 with the highest scores are selected as prefix sequence 1. A prefix sequence 1 can be represented as [token1]. Thus, the candidate prefix set 1 for the second step of reasoning can be constructed using the K prefix sequences 1. The candidate prefix set 1 includes K prefix sequences 1 (i.e., K [token1]). In the second step of reasoning, the recommendation model can perform H inferences based on the input data and each prefix sequence 1 (i.e., [token1]) in the candidate prefix set 1 to obtain H tokens 2 corresponding to each prefix sequence 1. Each token 2 is then added to the corresponding prefix sequence 1 to obtain K×H new prefix sequences. A new prefix sequence is represented as [token1, token2]. The score of each new prefix sequence can be calculated. The top K new prefix sequences with the highest scores from the K×H new prefix sequences are selected as prefix sequences 2 to construct the candidate prefix set 2 for the third step of reasoning. The candidate prefix set 2 includes K prefix sequences 2 (i.e., K [token1, token2]). At the inference of step 3, the recommendation model can perform H inferences according to the input data and each prefix sequence 2 in the candidate prefix set 2 (i.e., [token1, token2]), obtain H tokens 3 corresponding to each prefix sequence 2, and add each token 3 to the corresponding prefix sequence 2 to obtain K×H new prefix sequences. A new prefix sequence is represented as [token1, token2, token3], and the score of each new prefix sequence can be calculated. Select the top K new prefix sequences with the highest scores from the K×H new prefix sequences as the prefix sequence 3 to construct the candidate prefix set 3 for the inference of step 4. The candidate prefix set 3 includes K prefix sequences 3 (i.e., K [token1, token2, token3]); And so on until the inference of step T is performed to obtain K prefix sequences T. A prefix sequence T is represented as [token1, token2, token3, …, tokenT]; At this time, each prefix sequence T can be used as a token sequence to obtain K token sequences; Among them, a token sequence is [token1, token2, token3, …, tokenT], and a token sequence can represent a media resource to be recommended. Then the K media resources to be recommended represented by these K token sequences can form a media resource recommendation list.

[0036] Based on the above description, it can be seen that the beam search strategy gradually constructs the final media resource recommendation list in a greedy manner by retaining the top K partial sequences (i.e., prefix sequences) with the highest scores at each inference step of the recommendation model. If the score of the true prefix sequence of a certain media resource does not enter the top K at any inference step t, then no matter how high the generation probability (or score) of its prefix sequence in the subsequent inference steps is, it will cause the media resource to be discarded at the inference of step t.

[0037] There is a profound mismatch between this powerful inference mechanism and the mainstream (traditional) training paradigm. The traditional training paradigm generally adopts the Teacher Forcing strategy, which makes the input of the recommendation model at each time step t during the model training phase be the true prefix sequence y_{<t}, and its optimization goal is to minimize the negative log-likelihood loss (L_NLL). The calculation formula of this loss can be seen in Formula 1.1 shown below: Formula 1.1 In the above formula 1.1, x is the input data of the recommendation model, y is the tokenized representation of the target media resource (i.e., the token sequence), y_{<t} represents the first t - 1 tokens in the token sequence y (i.e., the true prefix sequence), θ represents the model parameters of the recommendation model, and log P_θ(y_t|y_{<t},x) represents the probability that the recommendation model generates the t-th token in the token sequence y based on the input data and the first t - 1 tokens in the token sequence y (i.e., the true prefix sequence).

[0038] It can be seen that the teacher-forcing strategy commonly adopted in the traditional training paradigm assumes that when the recommendation model generates a token at each step (the t-th step), the recommendation model can obtain the true prefix sequence (the first t - 1 tokens of the token sequence of the target media resource that the recommendation model needs to generate). Thus, the recommendation model is optimized by minimizing the negative log-likelihood loss calculated by the above formula 1.1. This makes the recommendation model only focus on the ability to generate the correct token sequence (i.e., the token sequence of the target media resource) under given perfect conditions during the optimization process, that is, only focus on the generation probability (likelihood probability) of the correct token sequence, completely ignoring the reality that the beam search strategy used in the inference stage will perform dynamic pruning, which will lead to a fundamental training-inference inconsistency in the recommendation model.

[0039] Among them, the training-inference inconsistency refers to the fundamental misalignment of the optimization objectives and constraints faced by the recommendation model in the training stage and the inference stage. Specifically, in the training stage, the recommendation model always makes predictions based on the true Ground-Truth prefix sequence y^{<t}, and its loss function focuses on the sum of token-level negative log-likelihoods, which allows the high scores of subsequent tokens to compensate for the low scores of previous tokens as long as the total score of the entire token sequence is high in the end; however, the beam search strategy used in the inference stage follows the "wooden bucket effect". If the score of the prefix sequence generated in any step of the inference process does not enter the Top-K (the top K), it will cause the prefix sequence to be discarded and unable to continue to expand and generate the corresponding token sequence based on this prefix sequence. It can be seen that the success of the beam search strategy depends on the ranking of each prefix sequence of the media resource to be recommended within the Top-K at the corresponding inference step, while the model only focuses on the final score of the entire token sequence during training, which will lead to a fundamental inconsistency between the training objective of the recommendation model and the inference objective of the beam search strategy, that is, there is a large gap between the training objective and the inference objective of the recommendation model.

[0040] To bridge this gap, recent research has proposed Direct Preference Optimization (DPO) methods and their variants in the recommendation field, such as the S-DPO method (a direct optimization method based on complete sequence preferences). The core idea of ​​these methods is to bypass complex reinforcement learning processes and directly align model outputs with human preferences using implicit preference signals from offline data through a closed-form objective function. The loss function of the S-DPO method can be expressed as Equation 1.2 below: L_S-DPO=-logσ(β×(log P_θ(y+|x)-log P_θ(y-|x)-log P_ref(y+|x)+log P_ref(y-|x))) Formula 1.2 In Equation 1.2 above, y+ and y- represent positive and negative samples, respectively, P_ref is a fixed reference model (usually a pre-trained model), β is the temperature coefficient; P_θ(y+|x) represents the probability that the recommendation model generates a positive sample based on the input data x, P_θ(y-|x) represents the probability that the recommendation model generates a negative sample based on the input data x, P_ref(y+|x) represents the probability that the reference model generates a positive sample based on the input data x, and P_ref(y-|x) represents the probability that the reference model generates a negative sample based on the input data x.

[0041] As can be seen, the core idea of ​​the above direct optimization methods based on complete sequence preferences (such as the S-DPO method) is to add a pairwise ranking loss for complete positive and negative sample sequence pairs on the basis of standard maximum likelihood estimation (MLE) in order to improve the overall ranking ability of the recommendation model.

[0042] While the S-DPO method described above is effective, its optimization perspective remains at the level of the complete token sequence of media resources. It only cares about the relative order of the final generated complete token sequence y, thus causing its optimization objective to act at the level of the complete token sequence and failing to consider the internal working characteristics of the bundle search strategy. Consequently, it fails to delve into the prefix-level decision-making process within the bundle search, resulting in the following drawbacks of the above method: (1) Granularity Mismatch. The optimization objective of the above methods (including the S-DPO method) is applied to the complete token sequence y. However, the success of the bundle search strategy is not solely determined by the total score of the final token sequence, but depends on the score of each intermediate prefix sequence. Even if the complete token sequence of a media resource y+ has a high score, if its early prefix sequences (e.g., the prefix sequence y+^{<2} formed by the first 1 token, or the prefix sequence y+^{<3} formed by the first 2 tokens) fail to enter the Top-K in the corresponding inference steps, the media resource y+ will be permanently discarded in the early stages of inference (e.g., steps 2, 3, etc.) and will not be able to participate in the final ranking competition. Therefore, since the above methods lack attention to prefix-level dynamics, they cannot guarantee that the generation path of highly relevant media resources can be preserved throughout the entire process under the strict screening of the bundle search strategy, thus causing the granularity mismatch problem.

[0043] (2) Objective Misalignment. There is a significant misalignment between the training objective of the methods mentioned above (both MLE and S-DPO) and the actual inference objective of the bundle search strategy. The implicit inference objective of the bundle search strategy is to preserve the relevance of the final media resource corresponding to the prefix sequence in the candidate prefix set at each inference step. This is a path-dependent, multi-stage decision problem. However, the objective functions of MLE and S-DPO are static and single-stage, assuming that all possible prefix sequences are reachable during training. This is seriously inconsistent with the reality that the bundle search strategy only explores exponentially small subspaces during inference, thus creating an objective misalignment problem. This objective misalignment can easily lead to a situation where the distribution learned by the recommendation model appears reasonable globally, but performs poorly in the specific execution path of the bundle search strategy.

[0044] In summary, the fundamental limitation of the above methods lies in the fact that the optimization perspective does not consider the inherent mechanism of the bundle search strategy, and fails to explicitly integrate the inherent structure and constraints of the bundle search decoder into the training process. Therefore, they cannot fundamentally solve the training-inference inconsistency problem of recommendation models introduced by the bundle search strategy.

[0045] To fundamentally address the training-inference inconsistency problem in recommendation models introduced by the bundle search strategy, this application proposes a resource recommendation method based on model training. This method uses a prefix-aware optimization (PAO) mechanism to introduce prefix-level optimization constraints consistent with the behavior of the bundle search strategy during the training phase of the recommendation model. This ensures that the PAO recommendation model not only learns the ability to generate correct token sequences during training, but also learns the ability to ensure that correct token sequences survive the rigorous screening of the bundle search strategy. This solves the problem of inconsistency between training and inference in recommendation models, thereby improving the accuracy of media resource recommendations.

[0046] The aforementioned prefix-aware optimization mechanism is the core optimization mechanism used in the training (optimization) process of the recommendation model. It can refine the optimization granularity of the recommendation model from the entire token sequence to prefix sequences of arbitrary length. By explicitly introducing a loss term L_m for each prefix sequence (i.e., a prefix sequence of length m, where m belongs to {1, ..., T}) into the training objective of the recommendation model, the recommendation model can pay attention to the generation probability of prefix sequences of arbitrary length throughout the optimization process and force the recommendation model to maximize the generation probability of each real prefix sequence in each inference step.

[0047] See Figure 1 As shown, the resource recommendation method based on model training proposed in this application generally includes a model training stage and a model inference stage. Wherein: During the model training phase, the sequence of media resources that the sample object has interacted with can be obtained, as well as the token sequence of the target media resources that the sample object interacts with after the corresponding interaction time of the media resource sequence (i.e., the target token sequence). The recommendation model can be called to infer the generation probability of each token in the target token sequence based on the media resource sequence. T prefix sequences of different lengths are constructed using the target token sequence. The prefix sequence of length m includes the first m tokens in the target token sequence, where m∈[1,T]. Thus, the generation loss of each prefix sequence can be perceived based on the generation probability of each token, and the recommendation model can be optimized based on the generation loss of the T prefix sequences, thereby improving the recommendation model's ability to perceive prefix sequences.

[0048] During the model inference phase, token sequences of all media resources that the target object (the object to be recommended media resources) has historically interacted with can be obtained. These token sequences are then input into the optimized recommendation model, which uses a bundle search strategy to perform T-step inference based on these input token sequences. In each step of inference, K prefix sequences are selected to proceed to the next step. After T-step inference, K new token sequences (i.e., the K prefix sequences selected in the T-th inference step) can be generated for the target object, and the media resources corresponding to these K new token sequences can be recommended to the target object, thereby improving the accuracy of media resource recommendations.

[0049] In practical implementation, the model training and model inference phases of the above-mentioned resource recommendation method based on model training can be executed by the same computer device or by different computer devices, without limitation. For example, computer device 1 can execute the model training phase of the above method to obtain an optimized recommendation model, and this optimized recommendation model can be deployed on computer device 2, with computer device 2 responsible for executing the model inference phase of the above method. Here, any computer device mentioned (such as computer device 1 or computer device 2) can include at least one of a terminal and a service; it is understood that when any computer device contains only a terminal or a server, it indicates that the corresponding processing phase (such as the model training phase or the model inference phase) of that computer device can be executed independently by the terminal or the server, while when any computer device contains both a terminal and a server, it indicates that the corresponding processing phase (such as the model training phase or the model inference phase) of that computer device can be executed jointly by the terminal and the server.

[0050] The terminals mentioned above can be smartphones, computers (such as tablets, laptops, desktop computers, etc.), smart wearable devices (such as smartwatches, smart glasses), smart voice interaction devices, smart home appliances (such as smart TVs), vehicle terminals, or aircraft, etc. Furthermore, the servers mentioned above can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, etc. Further, terminals and servers can be located within or outside the blockchain network, without limitation; even further, terminals and servers can upload any data stored internally to the blockchain network for storage to prevent internal data tampering and improve data security.

[0051] It should be noted that the model-trained resource recommendation method proposed in this application can be deployed as a highly robust and accurate intelligent generative recommendation engine. This engine can be seamlessly integrated into service platforms in various resource recommendation scenarios (such as e-commerce platforms, streaming media platforms, news information platforms, etc.) to recommend (provide) personalized media resource recommendations to users.

[0052] For example, the resource recommendation scenarios mentioned above may include, but are not limited to: information recommendation scenarios, video recommendation scenarios, live streaming recommendation scenarios, mini-program recommendation scenarios, product recommendation scenarios, etc. Specifically, in the information recommendation scenario, the media resources mentioned in this application embodiment may be information (such as articles, news, advertisements, etc.); in the video recommendation scenario, the media resources mentioned in this application embodiment may be videos (such as short videos, movies, etc.); in the live streaming recommendation scenario, the media resources mentioned in this application embodiment may be live streaming rooms; in the mini-program recommendation scenario, the media resources mentioned in this application embodiment may be mini-programs; in the product recommendation scenario, the media resources mentioned in this application embodiment may be products, and so on.

[0053] In addition, it is worth emphasizing that in the embodiments of this application, if user information and other related data (such as media resources that the object has interacted with) are involved, when any method embodiment proposed in the embodiments of this application is applied to a specific product or technology, such related data is collected with the user's permission or consent, and the collection, use and processing of the related data comply with the relevant laws, regulations and standards of the relevant regions.

[0054] Based on the above description, the following section uses a computer device as the execution subject as an example, combined with... Figure 2 The flowchart shown illustrates the specific implementation of the model-training-based resource recommendation method proposed in this application. Please refer to... Figure 2 As shown, the resource recommendation method based on model training proposed in this application can be roughly divided into the following steps S201-S205: S201, Obtain the sequence of media resources that the sample object has interacted with, and the target token sequence, which includes T tokens, where T is an integer greater than 1.

[0055] The sample object is the object (user) involved in the model training phase and who has interacted with multiple media resources.

[0056] The sequence of media resources interacted with by the sample object includes at least one media resource that the sample object has interacted with. Each media resource corresponds to an interaction time, which refers to the time when the corresponding media resource was interacted with by the sample object. For example, if the sample object interacts with a media resource at 10:58, then the interaction time of that media resource is 10:58. In the sequence of media resources interacted with by the sample object, each media resource is arranged in order from earliest to latest according to its interaction time. That is, the earlier a media resource is arranged in the sequence, the earlier its interaction time.

[0057] The target token sequence is used to identify the target media resource that the sample object interacts with after the interaction time corresponding to the media resource sequence. In other words, the target token sequence is a token sequence of the target media resource, and the target media resource is the media resource that the sample object interacts with after each media resource in the media resource sequence. That is, the interaction time of the target media resource is later than the interaction time corresponding to the media resource sequence. The interaction time corresponding to the media resource sequence mentioned here refers to the interaction time of the last media resource in the media resource sequence.

[0058] It should be noted that the meaning of object interaction with media resources (such as individual media resources in a media resource sequence, or the target media resource) can be understood as: the object performing interactive behavior on the corresponding media resource. Therefore, any media resource that the sample object has interacted with mentioned above refers to a media resource on which the sample object has performed interactive behavior. For example, if media resource A has been interacted with by the sample object, then media resource A is a media resource that the sample object has interacted with.

[0059] Interactive behavior refers to actions that perform at least one object operation on a media resource. Interactive behavior can vary depending on the media resource. For example, when the media resource is news, articles, or advertisements, interactive behavior can include clicking, browsing, liking, commenting, and sharing. When the media resource is multimedia data such as audio, video, or images, interactive behavior can include clicking, playing, commenting, and searching. When the media resource is a product, interactive behavior can include clicking, searching, browsing, and purchasing. When the media resource is an application such as a mini-program, interactive behavior can include clicking and downloading. When the media resource is a live stream, interactive behavior can include clicking and watching.

[0060] In a specific implementation, the computer device can obtain the historical interaction sequence of the sample object. This historical interaction sequence may include R media resources for which the sample object has historically performed interactive behaviors, where R is an integer greater than 1. It should be noted that the meaning of the historical interaction sequence containing R media resources mentioned in this application embodiment can specifically be that the historical interaction sequence contains all the content of the R media resources, or that the historical interaction sequence contains the identifiers (such as IDs) of the R media resources; this is not limited. Furthermore, the order of the R media resources in the historical interaction sequence can be the chronological order in which the sample object performs interactive behaviors on the corresponding media resources. Further, the computer device can select r consecutively arranged media resources from the historical interaction sequence of the sample object, construct a media resource sequence using the selected r media resources, and use the (r+1)th media resource in the historical interaction sequence of the sample object as the target media resource, thereby obtaining the token sequence of the target media resource as the target token sequence, where r∈[1, R-1].

[0061] It should be noted that all media resources and the target media resource in the aforementioned media resource sequence originate from the resource recommendation system. Furthermore, this embodiment can pre-define the resource recommendation system using item tokenization. Item tokenization refers to the process of mapping the IDs (identifiers) of a massive number of items (media resources) in the resource recommendation system into discrete token sequences based on Vector Quantization (VQ). Therefore, by pre-defining the resource recommendation system with item tokenization, the token sequences of each media resource (including the target media resource) in the resource recommendation system can be obtained in advance. In this case, when the computer device executes step S201, it can directly obtain the token sequence of the target media resource from the pre-obtained token sequences of each media resource, thereby improving the efficiency of obtaining the target token sequence. Of course, in other embodiments, the computer device can also obtain the target media resource in real time during step S201 and map the target media resource into a sequence composed of multiple discrete tokens using quantization technology, thereby using this sequence as the target token sequence.

[0062] S202, the recommendation model is invoked to infer the generation probability of each token in the target token sequence based on the media resource sequence.

[0063] In a specific implementation, the token sequence of each media resource in the media resource sequence can be obtained, and the token sequence of each media resource can be arranged according to the order of each media resource in the media resource sequence to obtain the input data X of the recommendation model. Then, the input data X is input into the recommendation model to call the recommendation model to perform T-step inference based on the input data X to obtain the generation probability of each token in the target token sequence.

[0064] Specifically, the recommendation model is configured with a token vocabulary containing a large number of tokens. Each token in a token sequence of any media resource originates from this token vocabulary. Based on this, the specific implementation of calling the recommendation model to perform T-step inference based on the input data X to obtain the generation probability of each token in the target token sequence can be roughly as follows: The recommendation model performs the first step of inference based on the input data X. Specifically, the recommendation model scores each token in the token vocabulary based on the input data X, obtaining the score (logits) of each token in the token vocabulary during the first step of inference. Based on the scores of each token in the token vocabulary during the first step of inference, the generation probability of the corresponding token in the first step of inference can be calculated. Thus, the generation probability of the first token in the target token sequence can be obtained from the calculated probabilities. The score of any token in any step of inference can be used to indicate the recommendation model's tendency to infer the corresponding token in that inference step. The higher the score, the more the recommendation model tends to infer the corresponding token, thus indicating a higher generation probability of the corresponding token. Therefore, the score of any token can be directly proportional to the generation probability of the corresponding token. The recommendation model is invoked to perform the second step of reasoning based on the input data X and the first token in the target token sequence. Specifically, the recommendation model can score each token in the token vocabulary based on the input data X and the first token in the target token sequence, obtain the score of each token in the token vocabulary in the second step of reasoning, and calculate the generation probability of the corresponding token in the second step of reasoning based on the score of each token in the token vocabulary in the second step of reasoning. Thus, the generation probability of the second token in the target token sequence can be obtained from the calculated probability. The recommendation model is invoked to perform the third step of reasoning based on the input data X and the first two tokens in the target token sequence. Specifically, the recommendation model can score each token in the token vocabulary based on the input data X and the first two tokens in the target token sequence, obtain the score of each token in the token vocabulary in the third step of reasoning, and calculate the generation probability of the corresponding token in the third step of reasoning based on the score of each token in the token vocabulary in the third step of reasoning. Thus, the generation probability of the third token in the target token sequence can be obtained from the calculated probability. This process continues until the recommendation model is invoked to perform the Tth step of reasoning based on the input data X and the first T-1 tokens in the target token sequence, thereby obtaining the generation probability of the Tth token in the target token sequence.

[0065] Based on the above description, the probability of generating the t-th token in the target token sequence is obtained by the recommendation model in the t-th step of reasoning, where t∈[1,T]. Specifically, the probability of generating the t-th token in the target token sequence can be obtained by the recommendation model in the t-th step of reasoning based on the media resource sequence (specifically, the input data X constructed using the media resource sequence) and the first t-1 tokens in the target token sequence. It can be understood that when t=1, the first t-1 tokens in the target token sequence are empty. Therefore, the probability of generating the t-th token can specifically represent the probability that the recommendation model generates (reasons) the t-th token in the t-th step of reasoning.

[0066] S203, using the target token sequence, construct T prefix sequences of different lengths; wherein, the prefix sequence of length m includes the first m tokens in the target token sequence.

[0067] In a specific implementation, the computer device can poll each token in the target token sequence in turn. When a token is polled, a prefix sequence is constructed using the currently polled token and each token in the target token sequence that precedes the currently polled token. The device continues to poll each token in the target token sequence until all T tokens in the target token sequence have been polled. At this point, T prefix sequences of different lengths can be obtained.

[0068] Among T prefix sequences of different lengths, the prefix sequence of length m includes the first m tokens of the target token sequence, where m ∈ [1, T], meaning the value of m can be taken from 1 to T. Based on this, the T prefix sequences of different lengths can specifically include: a prefix sequence of length 1 (i.e., a sequence composed of the first 1 token of the target token sequence), a prefix sequence of length 2 (i.e., a sequence composed of the first 2 tokens of the target token sequence), a prefix sequence of length 3 (a sequence composed of the first 3 tokens of the target token sequence), ..., a prefix sequence of length T-1 (a sequence composed of the first T-1 tokens of the target token sequence), and a prefix sequence of length T (a sequence composed of T tokens of the target token sequence).

[0069] For example, see Figure 3As shown, assuming the target token sequence is [a1, b2, c3, d4], meaning the target token sequence includes four tokens a1, b2, c3, and d4 in sequence, then we can construct a prefix sequence of length 1 using the first token ("a1"), a prefix sequence of length 2 using the first two tokens ("a1" and "b2"), a prefix sequence of length 3 using the first three tokens ("a1", "b2", and "c3"), and a prefix sequence of length 4 using the first four tokens ("a1", "b2", "c3", and "d4"), thus obtaining four prefix sequences of different lengths.

[0070] Understandably, the tokens in each of the aforementioned prefix sequences all originate from the target token sequence, which is the token sequence of the target media resources that the sample object has actually interacted with. Therefore, each prefix sequence constructed based on the target token sequence is a real prefix sequence.

[0071] S204, based on the generation probability of each token, perceive the generation loss of each prefix sequence. The generation loss refers to the loss generated by the recommendation model inferring the corresponding prefix sequence based on the media resource sequence.

[0072] In one specific implementation, the generation loss of each prefix sequence may include: Prefix-aware Pointwise Loss; the so-called prefix-aware pointwise loss refers to the loss value obtained by sensing the generation sub-loss of each token in the prefix sequence with a single token as the sensing unit and integrating the generation sub-losses of each token. The generation sub-loss of any token refers to the loss generated by the recommendation model inferring the corresponding token based on the media resource sequence. Accordingly, in this specific implementation, step S204 may include: calculating the generation sub-loss of each token based on its generation probability. The generation sub-loss of any token can be inversely proportional to its generation probability; that is, the higher the generation probability, the smaller the generation sub-loss. Furthermore, the generation sub-losses of each token in each prefix sequence can be integrated to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence. This allows the prefix-aware point-to-point loss to reflect the probability distribution of the true prefix sequence. When optimizing the recommendation model based on this prefix-aware point-to-point loss, the recommendation model can fit the probability distribution of the true prefix sequence to pass the threshold screening of the bundle search strategy.

[0073] The specific implementation of calculating the generation sub-loss of each token based on the generation probability of each token can include: performing a negative log-likelihood calculation on the generation probability of each token to obtain the generation sub-loss of the corresponding token. In this case, the specific implementation of integrating the generation sub-losses of each token in each prefix sequence to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence can include: averaging the generation sub-losses of each token in the same prefix sequence to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence.

[0074] As can be seen, the prefix-aware point-to-point loss calculated in this way is a prefix optimization objective based on maximum likelihood estimation. For a prefix sequence of length m, the average of the sum of the negative log-likelihoods (i.e., loss sub-losses) of each of its constituent tokens can be calculated using the generation probability of each token to ensure that the absolute probability value of the prefix sequence is high enough. This focuses on making the recommendation model fit the probability distribution of the real prefix sequence so as to pass the threshold screening of the beam search.

[0075] It is understood that the above merely illustrates the calculation and integration methods of the generation sub-loss of each token, and the embodiments of this application do not limit it. For example, in other embodiments, the method of calculating the generation sub-loss of the corresponding token based on the generation probability of each token may also include: calculating the difference between the base value (such as the value 1) and the generation probability of each token to obtain the generation sub-loss of the corresponding token. For example, if the generation probability of a token is 0.6, then the generation sub-loss of the token can be calculated as 1-0.6=0.4. As another example, in other embodiments, the method of integrating the generation sub-loss of each token in each prefix sequence to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence may include: summing the generation sub-loss of each token in the same prefix sequence to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence.

[0076] In another specific implementation, the generation loss of each prefix sequence may include: Prefix-aware Pairwise Loss; the so-called prefix-aware pairwise loss refers to: constructing at least one prefix sequence pair using positive prefix sequences (prefix sequences of target media resources that the sample object has interacted with) and at least one negative prefix sequence (prefix sequences of media resources that the sample object has not interacted with) of the same length, and using a single prefix sequence pair as the perception unit, perceiving the score difference (or generation probability difference) between the positive and negative prefix sequences in each prefix sequence pair, and calculating the loss based on the perception results.

[0077] Accordingly, in this specific implementation, step S204 may include: taking the target media resource identified by the target token sequence as a positive sample, and obtaining the corresponding negative sample, where the negative sample is a media resource that the sample object has not interacted with. Further, the token sequence of the negative sample can be obtained, and the recommendation model can be invoked to infer the generation probability of each token in the token sequence of the negative sample based on the media resource sequence. Even further, a prefix sequence of length m constructed based on the target token sequence can be used as a positive prefix sequence, and the first m tokens from the token sequence of the negative sample can be selected as a negative prefix sequence. Thus, based on the generation probability of each token in the positive prefix sequence and the generation probability of each token in the negative prefix sequence, the prefix-aware pairwise loss of the prefix sequence of length m can be calculated.

[0078] The specific implementation of calculating the prefix-aware pairwise loss of a prefix sequence of length m based on the generation probabilities of each token in the positive prefix sequence and the generation probabilities of each token in the negative prefix sequence can include: calculating the cumulative logarithmic score of the positive prefix sequence using the generation probabilities of each token in the positive prefix sequence. Specifically, the generation probabilities of each token in the positive prefix sequence can be logarithmically calculated to obtain the logarithmic score of each token's probability. Then, the logarithmic scores of each token in the positive prefix sequence can be summed to obtain the cumulative logarithmic score of the positive prefix sequence. Similarly, the cumulative logarithmic score of the negative prefix sequence can be calculated using the generation probabilities of each token in the negative prefix sequence. Specifically, the generation probabilities of each token in the negative prefix sequence can be logarithmically calculated to obtain the logarithmic score of each token's probability. Then, the logarithmic scores of each token in the negative prefix sequence can be summed to obtain the cumulative logarithmic score of the negative prefix sequence. Furthermore, based on the cumulative probability log score of the negative prefix sequence and the cumulative probability log score of the positive prefix sequence, a prefix-aware pairwise loss of length m can be calculated. This prefix-aware pairwise loss is used to increase the cumulative probability log score difference between the positive and negative prefix sequences.

[0079] As can be seen, the prefix-aware pairwise loss calculated in this way is a prefix optimization objective based on relative ranking. For a prefix sequence of length m, a negative prefix sequence of the same length can be constructed, and the recommendation model is optimized by maximizing the difference between the scores of the positive and negative prefix sequences. This allows the recommendation model to not only learn to infer positive prefix sequences but also to learn to widen the gap between positive and negative prefix sequences, thereby improving the recommendation model's inference ability for positive prefix sequences. Since the prefix-aware pairwise loss proposed in this application embodiment implies the score relationship between positive and negative prefix sequences, and this relationship can reflect the competitive relationship between positive and negative prefix sequences, the prefix-aware pairwise loss proposed in this application embodiment can show the competitive relationship between prefix sequences in the bundle search strategy, thereby solving the problem of training-inference inconsistency in the recommendation model caused by the bundle search strategy, and thus optimizing the lower bound of the recommendation task evaluation metric (such as Recall@K (a metric based on recall rate)).

[0080] S205. Based on the generation loss of T prefix sequences, the recommendation model is optimized. The optimized recommendation model is used to infer the token sequence of the media resources to be recommended, so as to make media resource recommendations.

[0081] In practical implementation, the generation losses of the T prefix sequences can be summed or averaged to obtain the prefix-aware level loss, which can then be used to optimize the recommendation model. Alternatively, the weight of each of the T prefix sequences can be obtained, and the generation losses of the T prefix sequences can be aggregated using each weight to obtain the prefix-aware level loss. This can then be used to optimize the recommendation model. This approach allows for flexible adjustment of the optimization resources of the recommendation model during optimization by controlling the weights of each prefix sequence. For example, higher weights can be assigned to prefix sequences that are easily discarded prematurely, allowing the recommendation model to focus its optimization resources on these sequences. This enables the recommendation model to continuously strengthen the execution accuracy of the inference steps involving these prefix sequences during optimization, thereby improving the robustness of the entire generation path of the target token sequence under the bundle search strategy.

[0082] This application embodiment can obtain the media resource sequence that the sample object has interacted with, and the token sequence of the target media resource that the sample object interacted with after the corresponding interaction time of the media resource sequence (i.e., the target token sequence). It then calls the recommendation model to infer the generation probability of each token in the target token sequence based on the media resource sequence, and constructs T prefix sequences of different lengths using the target token sequence. This allows the generation loss of each prefix sequence to be perceived based on the generation probability of each token. This method eliminates the need to separately call the recommendation model to infer the generation loss of each prefix sequence, thus saving processing resources consumed by the recommendation model during inference and ensuring the efficiency of perceiving the generation loss of each prefix sequence. Furthermore, the recommendation model can be optimized based on the generation loss of the T prefix sequences. This refines the optimization granularity of the recommendation model from the entire token sequence to prefix sequences of arbitrary length, enabling the recommendation model to focus on the generation probability of prefix sequences of arbitrary length throughout the optimization process and forcing the recommendation model to maximize the generation probability of each prefix sequence in each inference step. As can be seen, this application introduces prefix-level optimization constraints in the model training stage that are consistent with the bundle search strategy behavior used in the model inference stage. This can ensure that the recommendation model learns the ability to make the token sequence of the correct media resource (such as the target token sequence) survive to the end under the strict screening of the bundle search strategy, thereby solving the problem of inconsistency between training and inference in the recommendation model and improving the accuracy of the recommendation model in recommending media resources.

[0083] Based on the above description, this application proposes another resource recommendation method based on model training, which aims to solve the fundamental problem of "training-inference inconsistency" in generative recommendation systems. As mentioned above, this problem stems from the inherent conflict between the greedy pruning characteristics of the mainstream decoding strategy—the bundle search strategy—and the standard training objective (such as maximizing likelihood). Standard training employs a teacher-forced strategy, which assumes that the recommendation model can obtain the true prefix history at each generation step, thus focusing only on maximizing the likelihood of the final complete token sequence. However, in the inference process of the bundle search strategy, candidate prefix sequences are strictly filtered according to their cumulative scores at each inference step, and only prefix sequences with scores in the Top-K are retained. This leads to a globally optimal relevant media resource being permanently discarded in the early decoding steps of the bundle search strategy simply because the scores of its initial few tokens (i.e., prefix sequences) fail to enter the Top-K, thus failing to appear in the final media recommendation list.

[0084] To address this issue, the core idea of ​​the resource recommendation method proposed in this application is to elevate the training objective of the recommendation model from a single "final result alignment" to "full-process alignment." By explicitly introducing prefix-level optimization constraints consistent with the reasoning behavior of the bundle search strategy during the training phase, it ensures that the recommendation model learns not only the ability to generate correct media resources, but also the ability to ensure that the generation path of correct media resources (media resources that can be interacted with by objects) can "survive" to the end under the stringent screening of the bundle search strategy.

[0085] Specifically, the key technical points of the resource recommendation method based on model training proposed in this application embodiment can include at least the following: Key Point 1: Formal construction of prefix-level optimization objectives and unified training paradigm.

[0086] This application formally defines a prefix-level weighted objective. Let the true token sequence of the target media resource be y_i = {y_i^1, y_i^2, ..., y_i^T}, with length T; a prefix sequence of any length m (1 ≤ m ≤ T) can be represented as y_i^{≤m} = {y_i^1, ..., y_i^{m}}; based on this, a prefix-level optimization objective function is further constructed, which aggregates the loss terms (generation loss) for all possible prefix lengths. The expression of this function can be found in Equation 2.1 below: Formula 2.1 Where L_m represents the generation loss corresponding to the prefix sequence of length m, w_m≥0, which represents the weight of the prefix sequence of length m, and can be used to control the importance of the prefix sequence of length m, and L_prefix represents the prefix-level perceptual loss.

[0087] In practical implementation, the recommendation model can be optimized based solely on prefix-level perceptual loss, or it can be optimized by combining prefix-level perceptual loss and cross-entropy loss. In this case, the training objective (i.e., model loss) of the recommendation model is designed as a weighted sum of cross-entropy loss L_CE and prefix-level perceptual loss L_prefix, as shown in Equation 2.2 below: L_unified = L_CE + β × L_prefix (Formula 2.2) Here, L_unified represents the model loss of the recommendation model, β≥0. It is a hyperparameter used to balance the impact of the token-level likelihood objective (cross-entropy loss) and the novel prefix-level perceptual loss on the recommendation model. This unified training paradigm ensures that the recommendation model not only optimizes the accuracy of generating each token during training, but is also explicitly guided to maintain high probabilities or high rankings for all valid prefix sequences (especially those leading to the correct media resources (i.e., the media resources that should be recommended)). This fundamentally bridges the gap between the ideal state of the recommendation model during training and the realistic constraints during inference.

[0088] Key Point Two: Design of Prefix-Aware Point-to-Point Loss Function and Point-to-Point Paired Loss Function for Bundle Search Characteristics.

[0089] Given that the beam search strategy is essentially a stepwise filtering and sorting process, this application's embodiments design the following two complementary prefix-aware loss functions to capture this characteristic: (1) Prefix-aware point-to-point loss function: This loss function follows the idea of ​​cross-entropy, but applies it to each prefix sequence. For a prefix sequence of length m, the loss is calculated as the average negative log-likelihood of all tokens in the prefix sequence, as shown in Equation 2.3 below: Formula 2.3 In Equation 2.3 above, L_point(m) represents the prefix-aware point-to-point loss of a prefix sequence of length m, s_t^i=log(exp(z_t^i) / Σ{j∈V}exp(z_t^j)), z_t^i is the score (logits) output by the recommendation model for the t-th token, V is the token vocabulary, z_t^j is the score output by the recommendation model for the j-th token in the token vocabulary, exp(z_t^i) / Σ{j∈V}exp(z_t^j) can be represented as the generation probability of the t-th token obtained by normalizing the score of the t-th token by the recommendation model, then s_t^i can be represented as the logarithm of the generation probability of the t-th token by the recommendation model, and -s_t^i represents the generation sub-loss of the t-th token.

[0090] As can be seen, the prefix-aware point-to-point loss function proposed in this application extends the standard cross-entropy loss. It focuses on the average token generation probability of a prefix sequence of a specific length m, and by averaging the token probability across the prefix sequence, the optimization objective of the recommendation model can be extended from the final sequence to all possible prefix sequences. This loss function is computationally efficient, requiring no negative sampling, but it focuses more on data distribution cloning rather than modeling relative ranking relationships.

[0091] (2) Prefix-aware pairwise loss function: To more directly model the relative ranking relationship, this application introduces pairwise comparison. For a positive sample y+ (i.e., media resources that the sample object has interacted with) and its prefix sequence of length m (which can be denoted as the positive prefix sequence y_+^{≤m}), N negative samples (i.e., media resources that the sample object has not interacted with) are sampled from the corpus (media resource library), and the token sequence of each negative sample is truncated to the same length m, resulting in the negative prefix series set {y_{n,-}^{≤m},n∈N}. The cumulative score s_m of a prefix sequence (positive prefix sequence or negative prefix sequence) is defined as the sum of the probability logarithmic scores s_t of each token it contains, i.e., s_m= The prefix-aware pairwise loss function aims to widen the score gap between positive and negative prefix sequences, as shown in Equation 2.4 below: L_pair(m) =-log(σ(-log(Σ_{n∈N}exp(s_m^{n,-}- s_m^{i,+})))) Formula 2.4 In Equation 2.4 above, L_pair(m) represents the prefix-aware pairwise loss of a prefix sequence of length m, and s_m^{i,+} represents the cumulative probability log score of the positive prefix sequence (i.e., the prefix sequence of length m constructed based on the target token sequence), which can be equal to s_m^{n,-} represents the cumulative probability log score of the nth negative prefix sequence, and σ is the sigmoid function (activation function).

[0092] Of the two prefix-aware loss functions mentioned above, the prefix-aware point-to-point loss function focuses on improving the recommendation model's ability to fit the data distribution and is computationally efficient; while the prefix-aware pairwise loss function, by comparing the relative rankings of positive and negative prefix sequences, explicitly optimizes the relative ranking relationship (relative sorting) between prefix sequences, which better aligns with the competitive nature of the bundle search strategy and can more effectively prevent correct prefix sequences from being squeezed out of the Top-K candidate set by incorrect prefix sequences. It is evident that these two prefix-aware loss functions can jointly provide multi-level prefix-level supervision signals for the recommendation model.

[0093] Key Point 3: Adaptive Worst-Prefix Optimization Strategy The success of the bundle search strategy requires that every prefix sequence of the target media resource be retained in the corresponding inference step. Therefore, the strength of the entire generation chain depends on its weakest link (i.e., the prefix sequence with the largest loss). To address this, this application proposes an adaptive worst-case prefix optimization strategy, which is a dynamic and adaptive weight allocation strategy. Given that the success of the bundle search strategy depends on the worst-performing step (the weakest link effect), this strategy dynamically updates the weights of each prefix sequence based on the generation loss of each prefix sequence in the current training batch and utilizes an online learning algorithm based on KL divergence constraints. The aim is to allow the recommendation model to automatically cluster on the currently poorly performing prefix sequences that are prone to pruning during training, thereby improving the robustness of the recommendation model.

[0094] As can be seen, the core of the adaptive worst-case prefix optimization strategy proposed in this application is to focus on the prefix sequence that is currently the most difficult to optimize (i.e., has the largest loss). In order to avoid the training instability caused by directly using the hard-maximum value, this application adopts a soft weighting scheme based on KL divergence regularization. After each training step (optimization step), the weights of each prefix sequence are dynamically updated according to the generation loss of each prefix sequence.

[0095] Specifically, the soft weighting scheme based on KL divergence regularization introduces an adaptive weight vector w∈Δ^T (a T-dimensional probabilistic simplex, representing a T-dimensional vector where each weight is non-negative and the sum of all weights equals 1), and updates the weights by solving the optimization problem shown in Equation 2.5 below: Formula 2.5 In Equation 2.5 above, KL(w||w_p) represents the KL divergence between the weight vector w and the previous weight vector w_p (i.e., the vector formed by the weights of the T prefix sequences in the p-th round of optimization). η is a hyperparameter controlling the degree of weight change. The meaning of Equation 2.5 is to find a weight vector w∈Δ^T whose value maximizes the value of Equation 2.5, and then update the weight vector w_t using the found weight vector w. The closed-form solution to this optimization problem can be seen in Equation 2.6 below: Formula 2.6 In Equation 2.6 above, w_{t+1}^m represents the weight of the prefix sequence of length m in round P+1, w_m represents the weight of the prefix sequence of length m in round P of optimization, and L_m represents the generation loss of the prefix sequence of length m in round P of optimization. Let w_m represent the scaling factor of the weights, w_j represent the weights of the prefix sequence of length j in the Pth round of optimization, and L_j represent the generation loss of the prefix sequence of length j in the Pth round of optimization. The values ​​of j range from 1 to T. This can represent the scaling factor of the weight w_j. As shown in Equation 2.6, the weight vector w can be updated after each training step based on the generation loss L_m of each current prefix sequence, and Equation 2.6 has a closed-form solution, making it computationally efficient.

[0096] As can be seen, the adaptive worst-case prefix optimization strategy proposed in this application can focus optimization resources on the prefix with the highest current loss (i.e., the worst-case prefix) through a dynamic weight allocation mechanism, thereby systematically improving the robustness of the model under the bundle search constraint. As training progresses, the model will continuously identify and strengthen the "bottleneck" steps that are most likely to cause items to be discarded prematurely, thereby systematically improving the robustness of the entire generation path under the bundle search.

[0097] Based on the above key points, the following will combine... Figure 4 The flowchart shown illustrates the implementation process of the resource recommendation method based on model training proposed in this application embodiment; in this application embodiment, the method is still illustrated using a computer device as an example. Please refer to... Figure 4 As shown, the resource recommendation method based on model training proposed in this application can generally include the following steps S401-S405: S401, obtain the sequence of media resources that the sample object has interacted with and the sequence of target tokens. The sequence of target tokens includes T tokens.

[0098] S402, the recommendation model is invoked to infer the generation probability of each token in the target token sequence based on the media resource sequence; and T prefix sequences of different lengths are constructed using the target token sequence.

[0099] It should be noted that the specific implementation methods of steps S401-S402 can be found in the relevant descriptions in the foregoing embodiments, and will not be repeated here.

[0100] S403, based on the generation probability of each token, senses the generation loss of each prefix sequence.

[0101] In one specific implementation, for a prefix sequence of length m, the generation probabilities of each of the m tokens in the prefix sequence can be calculated using negative log-likelihood to obtain the generation sub-loss (i.e., negative log-likelihood) of the m tokens. Then, the mean of these generation sub-losses is calculated to obtain the prefix-aware point-to-point loss of the prefix sequence of length m. Figure 5 As shown. Furthermore, this prefix-aware point-to-point loss can be used as the generation loss for a prefix sequence of length m.

[0102] In another specific implementation, for a prefix sequence of length m, it can be used as a positive prefix sequence, and the token sequence of negative samples (media resources that the sample object has not interacted with) can be obtained. Then, the first m tokens are selected from the token sequence of the negative samples as negative prefix sequences. Further, the cumulative probability log score of the positive prefix sequence can be calculated using the generation probability of each token in the positive prefix sequence, and the cumulative probability log score of the negative prefix sequence can be calculated using the generation probability of each token in the negative prefix sequence. Further still, based on the cumulative probability log score of the negative prefix sequence and the cumulative probability log score of the positive prefix sequence, the prefix-aware pairwise loss of the prefix sequence of length m can be calculated, and this prefix-aware point-to-point loss can be used as the generation loss of the prefix sequence of length m.

[0103] It should be noted that the number of negative samples mentioned above can be one or N (N is an integer greater than 1), and the number of negative prefix sequences is consistent with the number of negative samples. That is, when the number of negative samples is 1, the number of negative prefix sequences is also 1, and when the number of negative samples is N, the number of negative prefix sequences is also N. One negative prefix sequence corresponds to one negative sample, and any negative prefix sequence is composed of the first m tokens in the token sequence of the corresponding negative sample.

[0104] When there is only one negative prefix sequence, this negative prefix sequence and the positive prefix sequence can form a prefix sequence pair. In this case, the specific way to calculate the prefix-aware pairwise loss of a prefix sequence of length m based on the cumulative probability log score of the negative prefix sequence and the cumulative probability log score of the positive prefix sequence can be as follows: perform a difference operation on the cumulative probability log score of the negative prefix sequence and the cumulative probability log score of the positive prefix sequence to obtain the score difference between the negative prefix sequence and the positive prefix sequence, and determine the prefix-aware pairwise loss of the prefix sequence of length m (i.e., the positive prefix sequence) based on the score difference. For example, the score difference can be directly used as the prefix-aware pairwise loss of the prefix sequence of length m (i.e., the positive prefix sequence), or the score difference can be linearly processed to obtain the prefix-aware pairwise loss of the prefix sequence of length m (i.e., the positive prefix sequence).

[0105] When there are N negative prefix sequences, these N negative prefix sequences and positive prefix sequences can form N prefix sequence pairs. A prefix sequence pair includes a negative prefix sequence and a positive prefix sequence. In this case, the specific way to calculate the prefix-aware pairwise loss of a prefix sequence of length m based on the cumulative probability logarithmic score of the negative prefix sequence and the cumulative probability logarithmic score of the positive prefix sequence can be as follows: perform a difference operation between the cumulative probability logarithmic score of each negative prefix sequence and the cumulative probability logarithmic score of the positive prefix sequence to obtain the score difference between each negative prefix sequence and the positive prefix sequence. The score differences between each negative prefix sequence and the positive prefix sequence can be integrated to obtain the prefix-aware pairwise loss of a prefix sequence of length m. The specific integration method can be found in the aforementioned formula 2.4, which will not be elaborated here.

[0106] In another specific implementation, for a prefix sequence of length m, the prefix-aware point-to-point loss and the prefix-aware pairwise loss of the prefix sequence can be calculated separately in the manner described above. Then, the calculated prefix-aware point-to-point loss and prefix-aware pairwise loss can be integrated (such as summation, mean, weighted aggregation, etc.) to obtain the generation loss of the prefix sequence of length m.

[0107] S404. Obtain the weight of each of the T prefix sequences, and use the weight of each prefix sequence to aggregate the generation loss of the T prefix sequences to obtain the prefix-aware loss.

[0108] In a specific implementation, the generation loss of T prefix sequences is aggregated using the weight of each prefix sequence to obtain the prefix-aware level loss. This can be achieved by weighting the generation loss of the corresponding prefix sequence with the weight of each prefix sequence to obtain the weighted loss of each prefix sequence. The weighted losses of the T prefix sequences can then be summed to obtain the prefix-aware level loss. Alternatively, the weighted losses of the T prefix sequences can be averaged to obtain the prefix-aware level loss.

[0109] In this embodiment, the recommendation model can undergo P rounds of optimization using the media resources of the sample objects and the target token sequence, where P is an integer greater than 1. Furthermore, during any round of optimization, after obtaining the generation loss of each prefix sequence, the weight of each prefix sequence is dynamically updated, and the updated weight is proportional to the generation loss of the corresponding prefix sequence. Since the updated weight of each prefix sequence is used to aggregate the generation losses of T prefix sequences in the next round of optimization, by constraining the updated weight to be proportional to the generation loss of the corresponding prefix sequence, the recommendation model can focus on prefix sequences with higher generation losses (i.e., prefix sequences that are more likely to be pruned) in the next round of optimization, thereby improving the robustness of the recommendation model.

[0110] Specifically, assuming step S404 is executed during the p-th round of optimization, where p∈[1, P-1], then during the p-th round of optimization, the generation loss of the T prefix sequences is aggregated using the weight of each prefix sequence to obtain the prefix-aware level loss. The computer device can then calculate the weight scaling factor for each prefix sequence based on its generation loss. This weight scaling factor is positively correlated with the generation loss of the corresponding prefix sequence. The calculation formula for this weight scaling factor can be found in the relevant description of Formula 2.6 above, and will not be repeated here. Further, the weights of the corresponding prefix sequences can be scaled using the weight scaling factor of each prefix sequence to update their weights. Specifically, the weights of the corresponding prefix sequences can be scaled using the weight scaling factor of each prefix sequence to obtain the scaled weights of each prefix sequence. The scaled weights of the T prefix sequences can then be normalized, and the normalized weights are used as the updated weights. The updated weights are used to aggregate the generation loss of the T prefix sequences during the p+1-th round of optimization.

[0111] It should be noted that the above-described dynamic weight update logic is an asynchronous update logic. It involves aggregating the generation losses of the corresponding prefix sequences using the weights of the T prefix sequences to obtain the prefix-aware level loss, and then dynamically updating the weights of the T prefix sequences so that the updated weights are applied to the next round of optimization. However, in other embodiments, a synchronous update logic can also be used to dynamically update the weights of each prefix sequence. In this case, after obtaining the weights of the T prefix sequences during the p-th round of optimization, the computer device can first calculate the weight scaling factor for each prefix sequence based on the generation loss of each prefix sequence, and then use the weight scaling factor to scale the weights of the corresponding prefix sequences to update their weights. Finally, the updated weights of each prefix sequence are used to aggregate the generation losses of the T prefix sequences to obtain the prefix-aware level loss.

[0112] Optionally, the recommendation model can also be optimized in one round using the media resources and target token sequence of the sample objects. In this case, the weight of each prefix sequence in the T prefix sequences can be preset based on actual needs or business experience. For example, business experience indicates that the prefix sequence of the target media resource is likely to be discarded in the early inference steps, and the shorter the prefix sequence is, the earlier the inference time step. Therefore, the weight of the prefix sequence can be set based on the length of the prefix sequence, and the weight can be inversely proportional to the length.

[0113] S405 optimizes the recommendation model based on prefix-aware loss.

[0114] In practical implementation, the model parameters of the recommendation model can be optimized directly in the direction of reducing the prefix-aware level loss. Alternatively, the cross-entropy loss can be calculated based on the generation probability of each token in the target token sequence, and the prefix-aware level loss and cross-entropy loss can be integrated to obtain the model loss. Specifically, the prefix-aware level loss and cross-entropy loss can be summed or averaged to obtain the model loss, or they can be integrated as described in Formula 2.2 above to obtain the model loss. Furthermore, the recommendation model can be optimized based on the model loss, specifically by optimizing the model parameters in the direction of reducing the prefix-aware level loss.

[0115] Based on the above description, the resource recommendation method based on model training proposed in this application can effectively solve the following key problems: (1) Achieving full-process alignment at the prefix level: The adaptive prefix-aware weighted framework proposed in this application is the first to sink the optimization objective from the complete sequence granularity to the dynamic prefix granularity. By constructing a unified objective function L_unified that aggregates all intermediate prefix losses to train the optimization recommendation model, it can be ensured that the recommendation model explicitly learns how to maintain the competitiveness of each prefix path leading to positive samples during the training process. This enables the training objective and the reasoning logic of the bundle search strategy to achieve accurate alignment of the entire process and the entire path.

[0116] (2) Ensuring the survivability of highly relevant media resources in the bundle search strategy: The adaptive prefix-aware weighted framework proposed in this application can optimize the absolute probability and relative ranking of prefix sequences by introducing prefix-aware point-to-point loss and prefix-aware pairwise loss. In particular, the prefix-aware pairwise loss can effectively widen the score gap between the prefix sequences of positive samples and the prefix sequences of negative samples by comparing and learning positive and negative prefix sequences at each prefix length m, thereby greatly reducing the risk of highly relevant media resources being eliminated prematurely due to insufficient scores of early prefix sequences.

[0117] (3) Dynamically focusing on and strengthening the weakest link: The adaptive prefix-aware weighted framework proposed in this application innovatively proposes an adaptive worst-case prefix optimization strategy. This strategy uses a soft weighting mechanism w_{t+1,m}∝w_{t,m}×exp(η×L_m) to dynamically allocate more optimization attention to the link (i.e. the link most likely to fail) where the prefix sequence with the largest current loss (i.e. the prefix sequence most likely to be discarded in the bundle search strategy) is located. This enables the recommendation model to systematically identify and strengthen important nodes in its generation path, thereby comprehensively improving the robustness and success rate of the entire recommendation path under the bundle search strategy.

[0118] By addressing the above issues, the embodiments of this application not only significantly improve the actual performance of generative recommendation systems under bundle search strategy decoding (such as Recall@K, NDCG@K, etc.), but more importantly, it establishes a new training paradigm that better suits real-world reasoning scenarios, providing crucial methodological guidance for generative AI in fields requiring complex decoding strategies, such as information retrieval and recommendation systems. Furthermore, at the user experience level, the embodiments of this application can significantly improve the relevance and diversity of media resource recommendation lists, enabling users to more frequently see long-tail media resources in their Top-K recommendation results that truly match their interests but may be described in a non-mainstream way, thus obtaining a more accurate and surprising service experience. From a product performance perspective, the methods and frameworks proposed in the embodiments of this application enable recommendation systems to more reliably output high-quality candidate sets when facing real and complex user queries and contexts. Experiments have shown that recommendation engines deployed with the methods and frameworks proposed in the embodiments of this application can achieve a relative improvement of 9%-12% in key business metrics (such as click-through rate (CTR), conversion rate (CVR), and user dwell time).

[0119] Based on the description of the above method embodiments, this application also proposes a complete, end-to-end adaptive prefix-aware weighted (APW) framework. This framework does not change the model architecture of the recommendation model, but guides the recommendation model to learn a generation pattern consistent with the bundle search reasoning behavior through innovative loss function design and adaptive optimization strategy.

[0120] The adaptive prefix-aware weighted framework proposed in this application will be described in detail below, from the aspects of problem definition, framework architecture, and algorithm flow: I. Problem Definition As mentioned earlier, in generative recommender systems, the recommendation task is refactored into an autoregressive sequence generation problem, whereby the recommender model predicts the token sequence of the media resource to be recommended based on the user's historical interaction sequence. During the training phase, the recommender model can assume that the prefix sequence of the target token sequence is always available, thus employing the cross-entropy (CE) loss function to maximize the likelihood probability of the target token sequence. However, during the inference phase, mainstream decoding strategies (beam search strategies) retain the top-K high-probability prefix sequences while pruning low-probability prefix sequences, leading to a fundamental training-inference inconsistency problem.

[0121] Specifically, the optimization objective during training focuses on the average likelihood of the final token sequence, allowing for low-probability tokens in intermediate steps as long as the total probability of the final sequence is high. However, the beam search strategy inference process imposes strict constraints on the prefix sequences of each intermediate step: each prefix sequence of a media resource must rank within the top-K in every inference step; otherwise, the media resource will be permanently discarded. This difference leads to the recommendation model not being optimized during training, resulting in media resources being prematurely filtered during the beam search stage. Consequently, media resources that are normally globally relevant may be prematurely discarded due to the low probability (low score) of the initial token (prefix sequence).

[0122] To quantify this inconsistency, embodiments of this application define Recall@K as a representative metric. In an ideal full-space ranking, a media resource y is successfully recalled if its total score ranks among the top K in the media resource set. I_full(y) = I(rank_Y(S(y|x))≤K) In the reasoning of the bundle search strategy, the condition for media resource y to be successfully recalled is that the score of each of its prefix sequences ranks within the top-K in the corresponding reasoning step: I_beam(y) =∏_{t=1}^TI(rank_t(S_t(y <t))≤K) Among them, the above-mentioned It is the total score of media resources y. It is the cumulative score (i.e., the cumulative logarithmic score of the probability) of the prefix sequence. Unlike I_full(y), I_beam(y) requires that the condition be met at every intermediate step, and failure of a single step will result in the entire media resource being discarded. This inconsistency prevents the model from learning the ability to maintain the correct candidates under beam search constraints during training.

[0123] II. Framework Architecture Specifically, the adaptive prefix-aware weighted framework proposed in this application embodiment may include the following modules: Module 1: Prefix Awareness Optimization Module The prefix-aware optimization module aims to address the aforementioned training-inference inconsistency problem. It introduces a prefix-level optimization objective, enabling the recommendation model to focus not only on the final complete token sequence during training but also on the prediction quality of each intermediate prefix sequence. By explicitly considering all possible prefix sequences during training, it ensures that the recommendation model can optimize its ability to retain the correct candidates (correct prefix sequences) under the constraints of the bundle search strategy.

[0124] Specifically, this module defines a prefix-aware optimization loss, which aggregates losses for different prefix lengths, enabling the recommendation model to optimize all possible prefix sequences during training, not just the final complete sequence. The formal definition of the prefix-aware optimization loss is:

[0125] Where L_m represents the generation loss corresponding to a prefix sequence of length m, and w_m≥0 controls the contribution of different prefix sequences to the final loss. By combining the prefix-aware optimization loss with the standard cross-entropy loss, a unified training objective is obtained:

[0126] Here, β≥0 is a hyperparameter that controls the strength of prefix-level optimization, ensuring that the recommendation model maintains generation quality while satisfying beam search constraints.

[0127] As can be seen, the prefix-aware module achieves a shift from token-level optimization to prefix-level optimization, making the training objective more consistent with the bundle search inference process. In practical applications, the prefix-aware optimization module effectively alleviates the training-inference inconsistency problem by explicitly optimizing the prediction quality of each prefix sequence during training.

[0128] Module 2: Adaptive Worst Prefix Optimization Module The adaptive worst-case prefix optimization module aims to dynamically determine the weights {w_m} of different prefix sequences, avoiding manual adjustment of a large number of hyperparameters. This module is based on the key observation that in beam search, if any prefix sequence of a media resource fails to enter the top-K at any step, the entire media resource is discarded. Therefore, optimization should prioritize the currently worst-performing prefix sequence.

[0129] Specifically, this module can dynamically update the weights of each prefix sequence using Equation 2.6 above, thereby dynamically allocating greater weights to the prefix sequences with higher current losses. This allows the recommendation model to focus more on prefix sequences that are easily pruned during bundle search during training. This adaptive mechanism avoids the complexity of manually adjusting multiple hyperparameters, enabling the recommendation model to dynamically focus on the most vulnerable prefix sequences.

[0130] In practical applications, customers of this module have iteratively optimized the weights of each prefix sequence, enabling the recommendation model to continuously adjust the degree of importance given to different prefix sequences during training. This ensures that the recommendation model can effectively cope with the constraints of the bundle search strategy and significantly improves recommendation performance.

[0131] III. Algorithm Flow See Figure 6aAs shown, the algorithm flow of the adaptive prefix-aware weighted framework proposed in this application embodiment may include the following two processes: (1) The training process can be divided into an initialization phase and a batch training phase. Wherein: The initialization phase includes: initializing the weight vector w = [1 / T, 1 / T, ..., 1 / T] (uniformly distributed), such that the weight of each prefix sequence is 1 / T in the initial stage. In addition, it can acquire all media resources and interaction behavior logs, and store the attribute description text of the media resources involved in the interaction behavior logs and the attribute description text of all media resources. These attribute description texts are then encoded using a text vectorization service to obtain a token sequence for each media resource. The feature sequences of these media resources can be stored in a feature library for subsequent use.

[0132] The batch training phase may include the following steps: Step 1: Obtain the sequence of media resources that the sample object has interacted with, and the sequence of target tokens for the target media resources that the sample object has interacted with.

[0133] Step 2: Call the recommendation model to infer the generation probability of each token in the target token sequence based on the media resource sequence.

[0134] Step 3: Calculate the cross-entropy loss L_CE based on the generation probability of each token in the target token sequence.

[0135] Step 4: Calculate the prefix-aware level loss L_prefix based on the generation probability of each token in the target token sequence. Specifically, the generation loss of each prefix sequence can be calculated separately based on the generation probability of each token in the target token sequence, and the generation losses of the T prefix sequences can be aggregated using the weights of each prefix sequence to obtain the prefix-aware level loss. The generation loss of any prefix sequence can be a prefix-aware point-to-point loss, a prefix-aware pairwise loss, or a loss obtained by integrating the prefix-aware point-to-point loss and the prefix-aware pairwise loss.

[0136] Step 5: Update the weight vector w: Update it using an adaptive worst-case prefix optimization strategy.

[0137] Step 6: Calculate the uniform model loss L_total = L_CE + β × L_prefix Step 7: Update the model parameters θ of the recommendation model using gradient descent based on the model loss.

[0138] It is understandable that in the training process of the embodiments of the present application, by dynamically adjusting the weights of different prefix sequences, the recommendation model can focus on the most vulnerable prefix sequences in each round of optimization process, effectively alleviating the training-inference inconsistency problem. Moreover, the optimized recommendation model can be deployed in the model inference service for subsequent use.

[0139] (2) Inference process. The input of this inference stage is the token sequence corresponding to the historical interaction sequence of the target object (including the media resources interacted by the target object), and the output is K media resources in B_T and their cumulative scores, which are returned sorted by scores.

[0140] Specifically, the beam set B_0 = {[]} can be initialized first. Then, for t = 1 to T (sequence length), the following steps a - step b can be executed: Step a. For each beam k in B_{t - 1} (a total of K): Calculate the score of the extended candidate token: P(v|y<t,(k),x) = e^{f_θ(v|y<t,(k),x)} / ∑_{j∈V}e^{f_θ(j|y<t,(k),x)}; where y<t,(k) represents the k-th prefix sequence in the prefix sequence set, which includes the first t - 1 tokens, and P(v|y<t,(k),x) represents the score of each token in the token vocabulary.

[0141] Calculate the cumulative score of the extended hypothesis: S(k)t(v) = S(k){t - 1} + logP(v|y<t,(k), x), that is, corresponding to the aforementioned: after adding each token to the corresponding prefix sequence to obtain a new prefix sequence, calculate the cumulative probability logarithmic score of the new prefix sequence.

[0142] Step b. Select the top - K extensions to form B_t = {y<t,(k)⊕v|(k, v)∈TopK({S(k)_t(v)|1≤k≤K, v∈V})}. The meaning of this expression is to select the top K new prefix sequences with the cumulative probability logarithmic scores for the next step of inference.

[0143] It is understandable that when t = T, the K new prefix sequences obtained through step b are the token sequences of the media resources to be recommended. By decoding these K token sequences, K media resources to be recommended can be obtained, and thus these K media resources to be recommended and the corresponding cumulative scores (i.e., the cumulative probability logarithmic scores of the token sequences of the media resources to be recommended) can be output.

[0144] Based on the above, the following combines Figure 6b to detail the specific implementation logic of media resource recommendation achieved through the above steps a - step b: If t=1 (i.e., enter the first step of reasoning), the recommendation model can calculate the score of each token a1-a256 in the token word list according to the token sequence corresponding to the historical interaction sequence of the target object (including the media resources that the target object has interacted with), and select the top-K tokens with the highest scores (such as a2, a6, a50, etc.) from the token word list as the prefix sequence 1; If t=2 (i.e., proceed to step 2 of inference), the recommendation model can calculate the scores of each token b1-256 in the token vocabulary based on the token sequence corresponding to the historical interaction sequence of the target object and each prefix sequence 1 (such as a2, a6, a50, etc.). Then, each token is concatenated with the corresponding prefix sequence 1 to obtain multiple new prefix sequences (such as [a2, b1], [a2, b4], [a6, b4], [a50, b9], [a50, b256], etc.). The cumulative score of the corresponding new prefix sequence can be calculated using the scores of the corresponding tokens (such as b1, b4) concatenated in each new prefix sequence. The top K new prefix sequences with the highest cumulative scores (such as [a2, b1], [a2, b4], [a50, b9], etc.) are selected as prefix sequence 2. Let t=3 (i.e., proceed to step 3 of the reasoning). Following the implementation logic of step 2 of the reasoning above, step 3 of the reasoning can be performed to obtain K new prefix sequences (such as [a2, b4, c7], [a9, b3, c5], [a50, b9, c7], etc.) as prefix sequence 3.

[0145] Let t=4 (i.e., proceed to step 4 of the reasoning). Following the implementation logic of step 2 of the reasoning above, step 4 of the reasoning can be performed, and K new prefix sequences (such as [a2, b4, c7, d5], [a2, b4, c7, d2], [a50, b9, c7, d3], etc.) can be obtained as prefix sequence 4.

[0146] Assuming T equals 4, then the K prefix sequences 4 can all be used as token sequences to decode the K resource identifiers (such as resource identifier-19, resource identifier-536, resource identifier-3261, etc.). The media resources corresponding to these resource identifiers can then be used as media resources to be recommended for media resource recommendation.

[0147] To comprehensively evaluate the effectiveness of this scheme (i.e., the adaptive prefix-aware weighted framework proposed in this application embodiment), this application embodiment evaluates the recommendation performance on a sampled dataset from a real-world industrial scenario. The recommendation performance of this scheme is compared with that of current mainstream algorithms under different evaluation metrics to demonstrate the effectiveness of this scheme. Specifically: Current mainstream algorithms include: Caser, GRU4Rec, SASRes, TIGER, HSTU, LETTER, LCRec, ReaRec, R2ec, and APAO, etc. Evaluation Metrics: This application uses two widely used metrics, Recall@K (R@K) and NDCG@K (N@K), to evaluate the recommended performance of each method; where: R@K is a recall-based metric that measures the average proportion of K recommended results that contain the correct media resources (interacted media resources). The value of K can be, for example, 10, 30, 50, etc. N@K considers the ranking position of media resources among K recommendation results (the higher the ranking, the higher the weight) to evaluate the ranking quality of the recommendation results. The value of K can be, for example, 10, 30, 50, etc.

[0148] In evaluation scenario 1 (article content subscription scenario), the evaluation results of this solution and current mainstream algorithms under the above evaluation metrics are shown in Table 1 below: Table 1

[0149] In evaluation scenario 2 (short video live streaming room subscription scenario), the evaluation results of the recommendation effect (recommendation performance) of this solution and the current mainstream algorithms under the above evaluation indicators are shown in Table 2 below: Table 2

[0150] Based on the descriptions of the above method embodiments, this application also discloses a resource recommendation device based on model training; the resource recommendation device based on model training can be a computer program (including one or more instructions) running on a computer device, and the resource recommendation device based on model training can execute each step in any of the above method flows. Please refer to... Figure 7 The model-trained resource recommendation device can operate the following units: The acquisition unit 701 is used to acquire the media resource sequence that the sample object has interacted with, and the target token sequence; the target token sequence is used to identify the target media resource that the sample object interacted with after the interaction time corresponding to the media resource sequence, and the target token sequence includes T tokens, where T is an integer greater than 1; Processing unit 702 is used to call the recommendation model to infer the generation probability of each token in the target token sequence based on the media resource sequence; The processing unit is further configured to construct T prefix sequences of different lengths using the target token sequence; wherein the prefix sequence of length m includes the first m tokens in the target token sequence, m∈[1,T]; The processing unit 702 is further configured to perceive the generation loss of each prefix sequence based on the generation probability of each token, wherein the generation loss refers to the loss generated by the recommendation model inferring the corresponding prefix sequence based on the media resource sequence; The processing unit 702 is further configured to optimize the recommendation model based on the generation loss of T prefix sequences. The optimized recommendation model is used to infer the token sequence of the media resource to be recommended, so as to recommend the media resource.

[0151] In one implementation, the generation loss of each prefix sequence includes: prefix-aware point-to-point loss; When perceiving the generation loss of each prefix sequence based on the generation probability of each token, the processing unit 702 can be specifically used to: Based on the generation probability of each token, the generation sub-loss of the corresponding token is calculated. The generation sub-loss of any token refers to the loss generated by the recommendation model inferring the corresponding token based on the media resource sequence. The generation sub-losses of each token in each prefix sequence are integrated to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence.

[0152] In another implementation, when calculating the generation sub-loss of the corresponding token based on the generation probability of each token, the processing unit 702 may be specifically used to: perform negative log-likelihood calculation on the generation probability of each token to obtain the generation sub-loss of the corresponding token. When integrating the generator losses of each token in each prefix sequence to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence, the processing unit 702 can specifically be used to: perform mean calculation on the generator losses of each token in the same prefix sequence to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence.

[0153] In another implementation, the generation loss for each prefix sequence includes: prefix-aware pairwise loss; When perceiving the generation loss of each prefix sequence based on the generation probability of each token, the processing unit 702 can be specifically used to: The target media resource identified by the target token sequence is taken as a positive sample, and the negative sample corresponding to the positive sample is obtained. The negative sample is a media resource that the sample object has not interacted with. Obtain the token sequence of the negative sample, and call the recommendation model to infer the generation probability of each token in the token sequence of the negative sample based on the media resource sequence; The prefix sequence of length m constructed based on the target token sequence is used as the positive prefix sequence, and the first m tokens from the token sequence of the negative sample are selected as the negative prefix sequence. Based on the generation probability of each token in the positive prefix sequence and the generation probability of each token in the negative prefix sequence, the prefix-aware pairwise loss of the prefix sequence of length m is calculated.

[0154] In another implementation, when calculating the prefix-aware pairwise loss of the prefix sequence of length m based on the generation probabilities of each token in the positive prefix sequence and the generation probabilities of each token in the negative prefix sequence, the processing unit 702 may specifically be used to: The cumulative probability logarithmic score of the positive prefix sequence is calculated using the generation probability of each token in the positive prefix sequence. The cumulative probability logarithmic score of the negative prefix sequence is calculated using the generation probability of each token in the negative prefix sequence. Based on the cumulative probability log score of the negative prefix sequence and the cumulative probability log score of the positive prefix sequence, calculate the prefix-aware pairwise loss of the prefix sequence of length m. The prefix-aware pairwise loss is used to increase the cumulative probability log score difference between the positive prefix sequence and the negative prefix sequence.

[0155] In another implementation, there are N negative samples and N negative prefix sequences, where N is an integer greater than 1. One negative prefix sequence corresponds to one negative sample, and any negative prefix sequence is composed of the first m tokens in the token sequence of the corresponding negative sample. When calculating the prefix-aware pairwise loss of the prefix sequence of length m based on the cumulative log probability score of the negative prefix sequence and the cumulative log probability score of the positive prefix sequence, the processing unit 702 may specifically be used to: The cumulative probability logarithmic score of each negative prefix sequence is compared with the cumulative probability logarithmic score of the positive prefix sequence to obtain the score difference between each negative prefix sequence and the positive prefix sequence. The score difference between each negative prefix sequence and the positive prefix sequence is integrated to obtain the prefix-aware pairwise loss of the prefix sequence of length m.

[0156] In another implementation, when optimizing the recommendation model based on the generation loss of T prefix sequences, the processing unit 702 may specifically be used to: Obtain the weight of each of the T prefix sequences; The generation loss of the T prefix sequences is aggregated using the weight of each prefix sequence to obtain the prefix-aware level loss; The recommendation model is optimized based on the prefix-aware loss.

[0157] In another implementation, the recommendation model is optimized in P rounds using the media resource sequence and the target token sequence, where P is an integer greater than 1. In any round of optimization, after obtaining the generation loss of each prefix sequence, the weight of each prefix sequence is dynamically updated, and the updated weight is proportional to the generation loss of the corresponding prefix sequence.

[0158] In another implementation, the step of obtaining the weight of each of the T prefix sequences is performed during the p-th round of optimization, where p∈[1, P-1]. In the p-th round of optimization, the generation loss of the T prefix sequences is aggregated using the weight of each prefix sequence to obtain the prefix-aware level loss. The processing unit 702 can then be specifically used for: Based on the generation loss of each prefix sequence, a weight scaling factor is calculated for each prefix sequence, wherein the weight scaling factor is positively correlated with the generation loss of the corresponding prefix sequence; The weights of the corresponding prefix sequences are scaled using the weight scaling factor of each prefix sequence to update the weights of the corresponding prefix sequences. The updated weights are used to aggregate the generation loss of the T prefix sequences during the (p+1)th round of optimization.

[0159] In another implementation, when optimizing the recommendation model based on the prefix-aware level loss, the processing unit 702 may specifically be used to: Based on the generation probability of each token in the target token sequence, calculate the cross-entropy loss; The prefix-aware loss and the cross-entropy loss are integrated to obtain the model loss, and the recommendation model is optimized based on the model loss.

[0160] According to another embodiment of this application, Figure 7The units in the model-trained resource recommendation device shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the model-trained resource recommendation device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0161] According to another embodiment of this application, a computer program (including one or more instructions) capable of performing the steps involved in any of the above methods can be run on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), to construct a system such as... Figure 7 The resource recommendation apparatus based on model training shown herein, as well as various methods for implementing the embodiments of this application, are illustrated. The computer program may be recorded on, for example, a computer-readable storage medium, loaded onto the aforementioned computing device via the computer-readable storage medium, and run therein.

[0162] It is worth noting that, in the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program with a predetermined function, which works together with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can contain a portion of the overall module or unit's functionality.

[0163] This application embodiment can obtain the media resource sequence that the sample object has interacted with, and the token sequence of the target media resource that the sample object interacted with after the corresponding interaction time of the media resource sequence (i.e., the target token sequence). It then calls the recommendation model to infer the generation probability of each token in the target token sequence based on the media resource sequence, and constructs T prefix sequences of different lengths using the target token sequence. This allows the generation loss of each prefix sequence to be perceived based on the generation probability of each token. This method eliminates the need to separately call the recommendation model to infer the generation loss of each prefix sequence, thus saving processing resources consumed by the recommendation model during inference and ensuring the efficiency of perceiving the generation loss of each prefix sequence. Furthermore, the recommendation model can be optimized based on the generation loss of the T prefix sequences. This refines the optimization granularity of the recommendation model from the entire token sequence to prefix sequences of arbitrary length, enabling the recommendation model to focus on the generation probability of prefix sequences of arbitrary length throughout the optimization process and forcing the recommendation model to maximize the generation probability of each prefix sequence in each inference step. As can be seen, this application introduces prefix-level optimization constraints in the model training stage that are consistent with the bundle search strategy behavior used in the model inference stage. This can ensure that the recommendation model learns the ability to make the token sequence of the correct media resource (such as the target token sequence) survive to the end under the strict screening of the bundle search strategy, thereby solving the problem of inconsistency between training and inference in the recommendation model and improving the accuracy of the recommendation model in recommending media resources.

[0164] Based on the description of the above method and apparatus embodiments, this application also provides a computer device. Please refer to... Figure 8 The computer device includes at least a processor 801, an input interface 802, an output interface 803, and a computer storage medium 804. The processor 801, input interface 802, output interface 803, and computer storage medium 804 within the computer device can be connected via a bus or other means. The computer storage medium 804 can be stored in the computer device's memory. The computer storage medium 804 is used to store a computer program, which includes one or more instructions. The processor 801 is used to execute one or more instructions from the computer program stored in the computer storage medium 804. The processor 801 (or CPU (Central Processing Unit)) is the computing and control core of the computer device, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve a corresponding method flow or function.

[0165] In one embodiment, the processor 801 described in this application embodiment can be used to perform a series of resource recommendation methods based on model training, specifically including: acquiring a sequence of media resources interacted by a sample object and a target token sequence; the target token sequence is used to identify the target media resource interacted by the sample object after the interaction time corresponding to the media resource sequence, the target token sequence includes T tokens, where T is an integer greater than 1; calling a recommendation model to infer the generation probability of each token in the target token sequence based on the media resource sequence; using the target token sequence to construct T prefix sequences of different lengths; wherein, the prefix sequence of length m includes the first m tokens in the target token sequence, m∈[1,T]; perceiving the generation loss of each prefix sequence based on the generation probability of each token, the generation loss being the loss generated by the recommendation model inferring the corresponding prefix sequence based on the media resource sequence; optimizing the recommendation model based on the generation loss of the T prefix sequences, the optimized recommendation model being used to infer the token sequence of the media resource to be recommended, so as to recommend media resources, etc.

[0166] This application embodiment also provides a computer storage medium (Memory), which is a memory device in a computer device used to store computer programs and data. It is understood that the computer storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer storage medium provides storage space that stores the operating system of the computer device. Furthermore, the computer program is also stored in this storage space, and the computer program includes one or more instructions suitable for loading and execution by the processor 801. These instructions can be one or more program codes. It should be noted that the computer storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one computer storage medium located remotely from the aforementioned processor.

[0167] In one embodiment, a processor may load and execute one or more instructions stored in a computer storage medium to implement the corresponding steps in the above method embodiments; specifically, one or more instructions in the computer storage medium may be loaded and executed by the processor in the following steps: Obtain the sequence of media resources that the sample object has interacted with, and the target token sequence; the target token sequence is used to identify the target media resource that the sample object interacted with after the interaction time corresponding to the media resource sequence, and the target token sequence includes T tokens, where T is an integer greater than 1; The recommendation model is invoked to infer the generation probability of each token in the target token sequence based on the media resource sequence; Using the target token sequence, construct T prefix sequences of different lengths; wherein, the prefix sequence of length m includes the first m tokens in the target token sequence, m∈[1,T]; The generation loss of each prefix sequence is perceived based on the generation probability of each token. The generation loss refers to the loss incurred by the recommendation model inferring the corresponding prefix sequence based on the media resource sequence. The recommendation model is optimized based on the generation loss of T prefix sequences. The optimized recommendation model is used to infer the token sequence of the media resources to be recommended, so as to make media resource recommendations.

[0168] In one implementation, the generation loss of each prefix sequence includes: prefix-aware point-to-point loss; When perceiving the generation loss of each prefix sequence based on the generation probability of each token, the one or more instructions can be loaded and executed by the processor: Based on the generation probability of each token, the generation sub-loss of the corresponding token is calculated. The generation sub-loss of any token refers to the loss generated by the recommendation model inferring the corresponding token based on the media resource sequence. The generation sub-losses of each token in each prefix sequence are integrated to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence.

[0169] In another implementation, when calculating the generation sub-loss of the corresponding token based on the generation probability of each token, the one or more instructions can be loaded and specifically executed by the processor: perform negative log-likelihood calculation on the generation probability of each token to obtain the generation sub-loss of the corresponding token. When integrating the generation sub-losses of each token in each prefix sequence to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence, one or more instructions can be loaded and specifically executed by the processor: averaging the generation sub-losses of each token in the same prefix sequence to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence.

[0170] In another implementation, the generation loss for each prefix sequence includes: prefix-aware pairwise loss; When perceiving the generation loss of each prefix sequence based on the generation probability of each token, the one or more instructions can be loaded and executed by the processor: The target media resource identified by the target token sequence is taken as a positive sample, and the negative sample corresponding to the positive sample is obtained. The negative sample is a media resource that the sample object has not interacted with. Obtain the token sequence of the negative sample, and call the recommendation model to infer the generation probability of each token in the token sequence of the negative sample based on the media resource sequence; The prefix sequence of length m constructed based on the target token sequence is used as the positive prefix sequence, and the first m tokens from the token sequence of the negative sample are selected as the negative prefix sequence. Based on the generation probability of each token in the positive prefix sequence and the generation probability of each token in the negative prefix sequence, the prefix-aware pairwise loss of the prefix sequence of length m is calculated.

[0171] In another implementation, when calculating the prefix-aware pairwise loss of the prefix sequence of length m based on the generation probabilities of each token in the positive prefix sequence and the generation probabilities of each token in the negative prefix sequence, the one or more instructions can be loaded and executed by the processor: The cumulative probability logarithmic score of the positive prefix sequence is calculated using the generation probability of each token in the positive prefix sequence. The cumulative probability logarithmic score of the negative prefix sequence is calculated using the generation probability of each token in the negative prefix sequence. Based on the cumulative probability log score of the negative prefix sequence and the cumulative probability log score of the positive prefix sequence, calculate the prefix-aware pairwise loss of the prefix sequence of length m. The prefix-aware pairwise loss is used to increase the cumulative probability log score difference between the positive prefix sequence and the negative prefix sequence.

[0172] In another implementation, there are N negative samples and N negative prefix sequences, where N is an integer greater than 1. One negative prefix sequence corresponds to one negative sample, and any negative prefix sequence is composed of the first m tokens in the token sequence of the corresponding negative sample. When calculating the prefix-aware pairwise loss of the prefix sequence of length m based on the cumulative log probability score of the negative prefix sequence and the cumulative log probability score of the positive prefix sequence, one or more instructions may be loaded and executed by the processor: The cumulative probability logarithmic score of each negative prefix sequence is compared with the cumulative probability logarithmic score of the positive prefix sequence to obtain the score difference between each negative prefix sequence and the positive prefix sequence. The score difference between each negative prefix sequence and the positive prefix sequence is integrated to obtain the prefix-aware pairwise loss of the prefix sequence of length m.

[0173] In another implementation, when optimizing the recommendation model based on the generation loss of T prefix sequences, the one or more instructions can be loaded and executed by the processor: Obtain the weight of each of the T prefix sequences; The generation loss of the T prefix sequences is aggregated using the weight of each prefix sequence to obtain the prefix-aware level loss; The recommendation model is optimized based on the prefix-aware loss.

[0174] In another implementation, the recommendation model is optimized in P rounds using the media resource sequence and the target token sequence, where P is an integer greater than 1. In any round of optimization, after obtaining the generation loss of each prefix sequence, the weight of each prefix sequence is dynamically updated, and the updated weight is proportional to the generation loss of the corresponding prefix sequence.

[0175] In another implementation, the step of obtaining the weight of each of the T prefix sequences is performed during the p-th round of optimization, where p∈[1, P-1]. During the p-th round of optimization, the generation loss of the T prefix sequences is aggregated using the weight of each prefix sequence to obtain the prefix-aware level loss. Then, one or more instructions can be loaded and executed by the processor. Based on the generation loss of each prefix sequence, a weight scaling factor is calculated for each prefix sequence, wherein the weight scaling factor is positively correlated with the generation loss of the corresponding prefix sequence; The weights of the corresponding prefix sequences are scaled using the weight scaling factor of each prefix sequence to update the weights of the corresponding prefix sequences. The updated weights are used to aggregate the generation loss of the T prefix sequences during the (p+1)th round of optimization.

[0176] In another implementation, when optimizing the recommendation model based on the prefix-aware level loss, the one or more instructions can be loaded and executed by the processor: Based on the generation probability of each token in the target token sequence, calculate the cross-entropy loss; The prefix-aware loss and the cross-entropy loss are integrated to obtain the model loss, and the recommendation model is optimized based on the model loss.

[0177] This application embodiment can obtain the media resource sequence that the sample object has interacted with, and the token sequence of the target media resource that the sample object interacted with after the corresponding interaction time of the media resource sequence (i.e., the target token sequence). It then calls the recommendation model to infer the generation probability of each token in the target token sequence based on the media resource sequence, and constructs T prefix sequences of different lengths using the target token sequence. This allows the generation loss of each prefix sequence to be perceived based on the generation probability of each token. This method eliminates the need to separately call the recommendation model to infer the generation loss of each prefix sequence, thus saving processing resources consumed by the recommendation model during inference and ensuring the efficiency of perceiving the generation loss of each prefix sequence. Furthermore, the recommendation model can be optimized based on the generation loss of the T prefix sequences. This refines the optimization granularity of the recommendation model from the entire token sequence to prefix sequences of arbitrary length, enabling the recommendation model to focus on the generation probability of prefix sequences of arbitrary length throughout the optimization process and forcing the recommendation model to maximize the generation probability of each prefix sequence in each inference step. As can be seen, this application introduces prefix-level optimization constraints in the model training stage that are consistent with the bundle search strategy behavior used in the model inference stage. This can ensure that the recommendation model learns the ability to make the token sequence of the correct media resource (such as the target token sequence) survive to the end under the strict screening of the bundle search strategy, thereby solving the problem of inconsistency between training and inference in the recommendation model and improving the accuracy of the recommendation model in recommending media resources.

[0178] It should be noted that, according to one aspect of this application, a computer program product or computer program is also provided, comprising one or more instructions stored in a computer storage medium. A processor of a computer device reads one or more instructions from the computer storage medium and executes the one or more instructions, causing the computer device to perform the methods provided in various optional embodiments of the above-described methods. It should be understood that the above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, equivalent variations made according to the claims of this application are still within the scope of this application.

Claims

1. A resource recommendation method based on model training, characterized in that, include: Obtain the sequence of media resources interacted with by the sample object, as well as the target token sequence; The target token sequence is used to identify the target media resource that the sample object interacts with after the interaction time corresponding to the media resource sequence. The target token sequence includes T tokens, where T is an integer greater than 1. The recommendation model is invoked to infer the generation probability of each token in the target token sequence based on the media resource sequence; Using the target token sequence, construct T prefix sequences of different lengths; wherein, the prefix sequence of length m includes the first m tokens in the target token sequence, m∈[1,T]; The generation loss of each prefix sequence is perceived based on the generation probability of each token. The generation loss refers to the loss incurred by the recommendation model inferring the corresponding prefix sequence based on the media resource sequence. The recommendation model is optimized based on the generation loss of T prefix sequences. The optimized recommendation model is used to infer the token sequence of the media resources to be recommended, so as to make media resource recommendations.

2. The method as described in claim 1, characterized in that, The generation loss for each prefix sequence includes: prefix-aware point-to-point loss; The generation loss of each prefix sequence based on the generation probability of each token includes: Based on the generation probability of each token, the generation sub-loss of the corresponding token is calculated. The generation sub-loss of any token refers to the loss generated by the recommendation model inferring the corresponding token based on the media resource sequence. The generation sub-losses of each token in each prefix sequence are integrated to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence.

3. The method as described in claim 2, characterized in that, The step of calculating the generation sub-loss of each token based on the generation probability of each token includes: performing a negative log-likelihood operation on the generation probability of each token to obtain the generation sub-loss of the corresponding token. The step of integrating the generation sub-losses of each token in each prefix sequence to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence includes: averaging the generation sub-losses of each token in the same prefix sequence to obtain the prefix-aware point-to-point loss of the corresponding prefix sequence.

4. The method according to any one of claims 1-3, characterized in that, The generation loss for each prefix sequence includes: prefix-aware pairwise loss; The generation loss of each prefix sequence based on the generation probability of each token includes: The target media resource identified by the target token sequence is taken as a positive sample, and the negative sample corresponding to the positive sample is obtained. The negative sample is a media resource that the sample object has not interacted with. Obtain the token sequence of the negative sample, and call the recommendation model to infer the generation probability of each token in the token sequence of the negative sample based on the media resource sequence; The prefix sequence of length m constructed based on the target token sequence is used as the positive prefix sequence, and the first m tokens from the token sequence of the negative sample are selected as the negative prefix sequence. Based on the generation probability of each token in the positive prefix sequence and the generation probability of each token in the negative prefix sequence, the prefix-aware pairwise loss of the prefix sequence of length m is calculated.

5. The method as described in claim 4, characterized in that, The calculation of the prefix-aware pairwise loss of the prefix sequence of length m based on the generation probabilities of each token in the positive prefix sequence and the generation probabilities of each token in the negative prefix sequence includes: The cumulative probability logarithmic score of the positive prefix sequence is calculated using the generation probability of each token in the positive prefix sequence. The cumulative probability logarithmic score of the negative prefix sequence is calculated using the generation probability of each token in the negative prefix sequence. Based on the cumulative probability log score of the negative prefix sequence and the cumulative probability log score of the positive prefix sequence, calculate the prefix-aware pairwise loss of the prefix sequence of length m. The prefix-aware pairwise loss is used to increase the cumulative probability log score difference between the positive prefix sequence and the negative prefix sequence.

6. The method as described in claim 5, characterized in that, There are N negative samples and N negative prefix sequences, where N is an integer greater than 1. One negative prefix sequence corresponds to one negative sample. Any negative prefix sequence is constructed using the first m tokens in the token sequence of the corresponding negative sample. The calculation of the prefix-aware pairwise loss of the prefix sequence of length m based on the cumulative log probability score of the negative prefix sequence and the cumulative log probability score of the positive prefix sequence includes: The cumulative probability logarithmic score of each negative prefix sequence is compared with the cumulative probability logarithmic score of the positive prefix sequence to obtain the score difference between each negative prefix sequence and the positive prefix sequence. The score difference between each negative prefix sequence and the positive prefix sequence is integrated to obtain the prefix-aware pairwise loss of the prefix sequence of length m.

7. The method according to any one of claims 1-3, characterized in that, The optimization of the recommendation model based on the generation loss of T prefix sequences includes: Obtain the weight of each of the T prefix sequences; The generation loss of the T prefix sequences is aggregated using the weight of each prefix sequence to obtain the prefix-aware level loss; The recommendation model is optimized based on the prefix-aware loss.

8. The method as described in claim 7, characterized in that, The recommendation model is optimized in P rounds using the media resource sequence and the target token sequence, where P is an integer greater than 1. In any round of optimization, after obtaining the generation loss of each prefix sequence, the weight of each prefix sequence is dynamically updated, and the updated weight is proportional to the generation loss of the corresponding prefix sequence.

9. The method as described in claim 8, characterized in that, The step of obtaining the weight of each of the T prefix sequences is performed during the p-th round of optimization, where p∈[1, P-1]; In the p-th round of optimization, the generation loss of the T prefix sequences is aggregated using the weight of each prefix sequence to obtain the prefix-aware level loss. The method further includes: Based on the generation loss of each prefix sequence, a weight scaling factor is calculated for each prefix sequence, wherein the weight scaling factor is positively correlated with the generation loss of the corresponding prefix sequence; The weights of the corresponding prefix sequences are scaled using the weight scaling factor of each prefix sequence to update the weights of the corresponding prefix sequences. The updated weights are used to aggregate the generation loss of the T prefix sequences during the (p+1)th round of optimization.

10. The method as described in claim 7, characterized in that, The optimization of the recommendation model based on the prefix-aware loss includes: Based on the generation probability of each token in the target token sequence, calculate the cross-entropy loss; The prefix-aware loss and the cross-entropy loss are integrated to obtain the model loss, and the recommendation model is optimized based on the model loss.

11. A resource recommendation device based on model training, characterized in that, include: The acquisition unit is used to acquire the sequence of media resources that the sample object has interacted with, as well as the target token sequence; The target token sequence is used to identify the target media resource that the sample object interacts with after the interaction time corresponding to the media resource sequence. The target token sequence includes T tokens, where T is an integer greater than 1. The processing unit is used to call the recommendation model to infer the generation probability of each token in the target token sequence based on the media resource sequence; The processing unit is further configured to construct T prefix sequences of different lengths using the target token sequence; wherein the prefix sequence of length m includes the first m tokens in the target token sequence, m∈[1,T]; The processing unit is also used to perceive the generation loss of each prefix sequence based on the generation probability of each token. The generation loss refers to the loss generated by the recommendation model inferring the corresponding prefix sequence based on the media resource sequence. The processing unit is further configured to optimize the recommendation model based on the generation loss of T prefix sequences. The optimized recommendation model is used to infer the token sequence of the media resource to be recommended, so as to recommend the media resource.

12. A computer device, comprising an input interface and an output interface, characterized in that, Also includes: Processor and computer storage media; The processor is adapted to implement one or more instructions, the computer storage medium stores one or more instructions, and the one or more instructions are adapted to be loaded by the processor and executed as described in any one of claims 1-10 for the resource recommendation method based on model training.

13. A computer storage medium, characterized in that, The computer storage medium stores one or more instructions, which are adapted to be loaded by a processor and executed as described in any one of claims 1-10, for the resource recommendation method based on model training.

14. A computer program product, characterized in that, The computer program product includes one or more instructions; when one or more instructions in the computer program product are executed by the processor, they implement the resource recommendation method based on model training as described in any one of claims 1-10.

Images