Item recommendation method, device, equipment, computer readable medium and program product

Abstract

Embodiments of the present disclosure disclose an item recommendation method, device, equipment, computer readable medium and program product. A specific embodiment of the method comprises: obtaining a candidate recommended item set corresponding to a target recommendation request and user behavior information corresponding to a target user; inputting the candidate recommended item set and the user behavior information into an encoding layer in a pre-trained rearrangement model to obtain recommended item encoding information, wherein the rearrangement model further comprises: a plurality of parallel decoding heads; inputting the recommended item encoding information into the plurality of parallel decoding heads to output a plurality of recommended items in parallel multiple times; generating a recommended item sequence corresponding to the plurality of recommended items using a multi-mechanism sampling strategy; and performing item recommendation on the recommended item sequence. This embodiment is related to artificial intelligence, and uses the plurality of parallel decoding heads and the multi-mechanism sampling strategy to achieve fast and accurate response of the recommended item sequence on the basis of exploring more item sequence combinations, thereby avoiding recommendation delay.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the fields of computer technology and artificial intelligence, and specifically to methods, apparatus, devices, computer-readable media, and program products for recommending items. Background Technology

[0002] Currently, in modern industrial-grade recommender systems, whether it's the traditional cascading framework (recall -> coarse ranking -> fine ranking -> re-ranking) or the recent next-generation recommender framework based on generative large language models (LLM), re-ranking remains the crucial stage closest to the user and directly determines the final exposure result. Re-ranking plays a core role in achieving business goals (clicks, conversions, dwell time, etc.) while controlling exposure diversity and alleviating information redundancy. The re-ranking of the candidate recommended item sequences output from upstream typically involves: first, a generator produces multiple candidate recommended item sequences; then, an evaluator models the real exposure context and selects the optimal candidate recommended item sequence from these sequences for online exposure.

[0003] However, when using the above method, the following technical problems often arise: Existing re-evaluation methods (e.g., heuristic score-weighted sampling, beam-search, Pointer Network, etc.) are prone to generating homogeneous trajectories, causing the evaluator to select only within a narrow subspace. This leads to the model getting trapped in local optima, impacting the accuracy of the recommendation results. Furthermore, it is difficult to achieve rapid response of recommendation results under industrial-grade latency constraints. Summary of the Invention

[0004] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0005] Some embodiments of this disclosure provide methods, apparatus, devices, computer-readable media, and program products for recommending items to address the technical problems mentioned in the background section above.

[0006] In a first aspect, some embodiments of this disclosure provide an item recommendation method, including: obtaining a set of candidate recommended items corresponding to a target recommendation request and user behavior information corresponding to a target user; inputting the candidate recommended item set and the user behavior information into the encoding layer of a pre-trained reordering model to obtain recommended item encoding information, wherein the reordering model further includes: multiple parallel decoding heads; inputting the recommended item encoding information into the multiple parallel decoding heads to output multiple recommended items in parallel multiple times; generating a recommended item sequence corresponding to the multiple recommended items using a multi-mechanism sampling strategy; and performing item recommendation for the recommended item sequence.

[0007] Optionally, the above-mentioned inputting the candidate recommended item set and the user behavior information into the encoding layer of the pre-trained reordering model to obtain recommended item encoding information includes: using the encoding layer to perform the following generation steps: for each candidate recommended item, extracting the item representation information of the candidate recommended item under the corresponding interest of the target user based on the user behavior information; generating item embedding information of each candidate recommended item in the candidate recommended item set based on the global contextual dependency between the obtained item representation information, and obtaining an item embedding information set as recommended item encoding information.

[0008] Optionally, the above-mentioned input of the recommended item encoding information to the multiple parallel decoding heads to output multiple recommended items in parallel multiple times includes: using the item embedding information set as key-value information, using the corresponding position embedding set of the candidate recommended item set as query information, querying and updating the item embedding information set to obtain the updated item embedding information set as the recommended item encoding information; and inputting the recommended item encoding information to the multiple parallel decoding heads to perform multiple step-by-step parallel decoding to obtain the multiple recommended items.

[0009] Optionally, the above-mentioned multi-mechanism sampling strategy is a strategy for sequence generation based on bundle search technology and heuristic sampling technology; and the above-mentioned generation of recommended item sequences corresponding to the above-mentioned multiple recommended items using the multi-mechanism sampling strategy includes: generating a first candidate recommended item sequence set based on the above-mentioned multiple recommended items using the above-mentioned bundle search technology; generating a second candidate recommended item sequence set based on the above-mentioned multiple recommended items using the above-mentioned heuristic sampling technology; combining the above-mentioned first candidate recommended item sequence set and the above-mentioned second candidate recommended item sequence set to obtain a candidate recommended item sequence set; and selecting the candidate recommended item sequence with the highest sequence generation probability from the above-mentioned candidate recommended item sequence set as the recommended item sequence.

[0010] Optionally, the above-mentioned re-ranking model is trained through the following steps: For the first target training data in the first training dataset, a first training step is performed: The candidate recommended item set and the corresponding user behavior information in the first target training data are input into the initial re-ranking model to obtain multiple recommended items; A candidate recommended item sequence set corresponding to the multiple recommended items is generated using a multi-mechanism sampling strategy; An exploratory reward model is used to generate exploratory reward information corresponding to each candidate recommended item sequence in the candidate recommended item sequence set; A preference loss for the candidate recommended item sequence set is generated based on each exploratory reward information; The initial re-ranking model is determined to have finished training based on the preference loss; In response to the end of training, the initial re-ranking model is determined to be a re-ranking model; In response to the incomplete training, the initial re-ranking model is retrained based on the preference loss to obtain a re-ranked model after training; The first training data re-selected from the first training dataset is used as the first target training data, and the re-ranked model after training is used as the initial re-ranking model, and the above first training step is continued.

[0011] Optionally, generating the preference loss for the candidate recommended item sequence set based on each exploratory reward information includes: sorting each candidate recommended item sequence in the candidate recommended item sequence set according to each exploratory reward information to obtain a sorting result; selecting candidate recommended item sequences at each quantile from each sequence corresponding to the sorting result to obtain a subset of candidate recommended item sequences and a corresponding preference sample label set; and generating a preference loss using a preference loss function based on the subset of candidate recommended item sequences, the preference sample label set, and the sequence generation probability corresponding to each candidate recommended item sequence.

[0012] Optionally, determining whether the initial re-ranking model has finished training based on the aforementioned preference loss includes: determining the diversity loss among the various recommended items; and determining whether the initial re-ranking model has finished training based on the aforementioned diversity loss and the aforementioned preference loss.

[0013] Optionally, the aforementioned exploratory reward model includes a shared encoding layer and a dual-task prediction layer; and the aforementioned method of generating exploratory reward information corresponding to each candidate recommended item sequence in the candidate recommended item sequence set using the exploratory reward model includes: inputting the candidate recommended item sequence into the shared encoding layer to obtain encoding information; inputting the encoding information into the dual-task prediction layer to obtain item instant value information and sequence exploratory value information, which serve as exploratory reward information.

[0014] Optionally, the above-mentioned exploratory reward model is trained through the following steps: For the second target training data in the second training dataset, a second training step is performed: The real exposure log sequence in the second target training data is input into the initial shared encoding layer in the initial exploratory reward model to obtain initial encoding information, wherein the second target training data also includes: cross-request signal labels; The initial encoding information is input into the initial dual-task prediction layer in the initial exploratory reward model to obtain initial item instant value information and initial sequence exploration value information; Based on the initial item instant value information and the cross-request signal labels, an item instant value loss is generated; Based on the above... The initial sequence explores value information and generates a sequence exploration value loss. Based on the immediate value loss of the items and the initial sequence explores value information, it is determined whether the initial exploratory reward model has finished training. In response to the end of training, the initial exploratory reward model is identified as the exploratory reward model. In response to the end of training, the initial exploratory reward model is retrained based on the immediate value loss of the items and the initial sequence explores value information to obtain a post-trained reward model. The training data reselected from the second training dataset is used as the target training data, and the post-trained reward model is used as the initial exploratory reward model. The second training step is then executed again.

[0015] Optionally, generating the sequence exploration value loss based on the initial sequence exploration value information includes: determining the data weights corresponding to the second target training data based on the label types corresponding to the cross-request signal labels, wherein the label types are one of the following: termination label type, invalid exploration label type, potential interest label type, instant matching label type, and deep exploration label type; and generating the sequence exploration value loss based on the data weights and the initial sequence exploration value information.

[0016] Optionally, determining the data weights corresponding to the second target training data based on the label type corresponding to the cross-request signal label includes: determining the data quality corresponding to the real exposure log sequence in response to the label type being a potential interest label type or an invalid exploration label type; and determining the data weights corresponding to the second target training data based on the data quality.

[0017] Secondly, some embodiments of this disclosure provide an item recommendation apparatus, including: an acquisition unit configured to acquire a set of candidate recommended items corresponding to a target recommendation request and user behavior information corresponding to a target user; a first generation unit configured to input the set of candidate recommended items and the user behavior information into an encoding layer in a pre-trained reordering model to obtain recommended item encoding information, wherein the reordering model further includes: multiple parallel decoding heads; a second generation unit configured to input the recommended item encoding information into the multiple parallel decoding heads to output multiple recommended items in parallel multiple times; a third generation unit configured to generate a recommended item sequence corresponding to the multiple recommended items using a multi-mechanism sampling strategy; and an execution unit configured to perform item recommendation for the recommended item sequence.

[0018] Optionally, the first generation unit can be configured to: utilize the above-mentioned encoding layer to perform the following generation steps: for each candidate recommended item, extract the item representation information of the candidate recommended item under the corresponding interest of the target user based on the above-mentioned user behavior information; based on the global context dependency between the obtained item representation information, generate the item embedding information of each candidate recommended item in the above-mentioned candidate recommended item set, and obtain the item embedding information set as the recommended item encoding information.

[0019] Optionally, the second generation unit can be configured to: use the above-mentioned item embedding information set as key-value information, use the above-mentioned candidate recommended item set corresponding position embedding set as query information, perform query embedding update on the above-mentioned item embedding information set to obtain the updated item embedding information set as recommended item encoding information; input the above-mentioned recommended item encoding information into the above-mentioned multiple parallel decoding heads to perform multiple stepwise parallel decodings to obtain the above-mentioned multiple recommended items.

[0020] Optionally, the above-mentioned multi-mechanism sampling strategy is a sequence generation strategy based on bundle search technology and heuristic sampling technology; and the third generation unit can be configured to: generate a first candidate recommended item sequence set based on the above-mentioned multiple recommended items using the above-mentioned bundle search technology; generate a second candidate recommended item sequence set based on the above-mentioned multiple recommended items using the above-mentioned heuristic sampling technology; combine the above-mentioned first candidate recommended item sequence set and the above-mentioned second candidate recommended item sequence set to obtain a candidate recommended item sequence set; and select the candidate recommended item sequence with the highest sequence generation probability from the above-mentioned candidate recommended item sequence set as the recommended item sequence.

[0021] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, such that when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0022] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0023] Fifthly, some embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0024] The above embodiments of this disclosure have the following beneficial effects: By utilizing multiple parallel decoding heads and multi-mechanism sampling strategies, the item recommendation methods of some embodiments of this disclosure achieve a fast and accurate response to recommended item sequences based on exploring more combinations of item sequences, avoiding recommendation delays. Specifically, the reasons for the inaccuracy and response delay of related item recommendations are that existing reordering methods (e.g., heuristic score-weighted sampling, beam-search, Pointer Network, etc.) are prone to generating homogeneous trajectories, causing the evaluator to select only in a narrow subspace, the model gets trapped in local optima, and the accuracy of the recommendation results is affected. In addition, it is difficult to achieve a fast response to recommendation results under industrial-grade latency constraints. Based on this, the item recommendation methods of some embodiments of this disclosure first obtain the candidate recommended item set corresponding to the target recommendation request and the user behavior information corresponding to the target user, so as to accurately reorder the candidate recommended items based on the candidate recommended item set and user behavior information output from the upstream, taking into account the semantic content of user behavior. Then, the aforementioned candidate recommended item set and user behavior information are input into the encoding layer of the pre-trained reordering model to obtain the recommended item encoding information. The reordering model also includes multiple parallel decoding heads. Here, the encoding layer enables accurate encoding of each candidate recommended item based on the semantic content of user behavior, facilitating accurate decoding of recommended items and ensuring the accuracy of recommended item generation. Next, the recommended item encoding information is input into the multiple parallel decoding heads to output multiple recommended items in parallel. Here, the use of multiple parallel decoding heads significantly reduces the single-batch generation time through parallel mechanisms, meeting the industrial-grade TP99 constraint and achieving rapid response of item recommendation results. Furthermore, a multi-mechanism sampling strategy is used to generate recommended item sequences corresponding to the multiple recommended items, which can significantly improve exploration efficiency, avoid local optima, and improve the diversity and accuracy of recommended item sequences. Finally, item recommendation is performed on the recommended item sequences. In summary, by utilizing multiple parallel decoding heads and a multi-mechanism sampling strategy, a fast and accurate response of recommended item sequences is achieved by exploring more combinations of item sequences, avoiding recommendation delays. Attached Figure Description

[0025] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0026] Figure 1 This is a schematic diagram illustrating an application scenario of an item recommendation method according to some embodiments of the present disclosure; Figure 2 This is a flowchart of some embodiments of the item recommendation method according to this disclosure; Figure 3 These are flowcharts of other embodiments of the item recommendation method according to this disclosure; Figure 4 These are schematic diagrams illustrating the structure of some embodiments of the item recommendation device according to this disclosure; Figure 5 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0027] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0028] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0029] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0030] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0031] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0032] Before performing any of the operations involving the collection, storage, or use of user personal information (such as user behavior information) disclosed in this disclosure, the relevant organizations or individuals shall fulfill their obligations, including conducting personal information security impact assessments, informing personal information subjects, and obtaining prior authorization and consent from personal information subjects.

[0033] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0034] Figure 1This is a schematic diagram illustrating an application scenario of an item recommendation method according to some embodiments of the present disclosure.

[0035] exist Figure 1 In this application scenario, firstly, the electronic device 101 can obtain the candidate recommended item set 102 corresponding to the target recommendation request and the user behavior information 103 corresponding to the target user. In this application scenario, the user behavior information 103 can be "user basic information, user behavior records". Then, the electronic device 101 can input the aforementioned candidate recommended item set 102 and the aforementioned user behavior information 103 into the encoding layer of the pre-trained reordering model 104 to obtain recommended item encoding information 105. The reordering model also includes multiple parallel decoding heads. In this application scenario, the recommended item encoding information can be an item encoding vector. The multiple parallel decoding heads can include: decoding head 1, decoding head 2, and decoding head 3. Next, the electronic device 101 can input the aforementioned recommended item encoding information into the aforementioned multiple parallel decoding heads to output multiple recommended items 105 in parallel multiple times. Furthermore, the electronic device 101 can utilize a multi-mechanism sampling strategy to generate a recommended item sequence 106 corresponding to the aforementioned multiple recommended items 105. In this application scenario, the recommended item sequence 106 can be "{recommended item 1, recommended item 3, recommended item 2}". Finally, the electronic device 101 can perform item recommendations for the aforementioned recommended item sequence 106.

[0036] It should be noted that the aforementioned electronic device 101 can be either hardware or software. When the electronic device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the electronic device is software, it can be installed in the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0037] It should be understood that Figure 1 The number of electronic devices shown is merely illustrative. Any number of electronic devices can be used depending on the implementation requirements.

[0038] Continue to refer to Figure 2 The diagram illustrates a flow 200 of some embodiments of an item recommendation method according to the present disclosure. This item recommendation method includes the following steps: Step 201: Obtain the set of candidate recommended items corresponding to the target recommendation request and the user behavior information corresponding to the target user.

[0039] In some embodiments, the entity executing the above-described item recommendation method (e.g.) Figure 1The electronic device 101 shown can acquire the candidate recommended item set corresponding to the target recommendation request and the user behavior information corresponding to the target user through a wired or wireless connection. The target recommendation request can be a recommendation request for items directed to the target user. In practice, the target recommendation request can have a specific format. The target user can be the object of the item recommendation. For example, the target user can be a user who has just opened the shopping application or a user browsing a shopping page. The candidate recommended items can be items used as candidate recommendations. The candidate recommended item set can be the items selected as candidates by the upstream system after recall, coarse ranking, and fine ranking operations. In practice, the candidate recommended items in the candidate recommended item set have a priority order for recommendation. The user behavior information can be the various behaviors corresponding to the target user. In practice, user behavior information can include: the user's basic personal information and the behavior records corresponding to each behavior. The basic personal information can include: the user's name, age, and occupation. The various behaviors can include: click behavior, browsing behavior, favorite behavior, and add-to-cart behavior. Behavioral records can be the historical performance of a target user in various behaviors.

[0040] Step 202: Input the above candidate recommended item set and the above user behavior information into the encoding layer of the pre-trained rearrangement model to obtain the recommended item encoding information.

[0041] In some embodiments, the execution entity can input the candidate recommended item set and the user behavior information into the encoding layer of a pre-trained reordering model to obtain recommended item encoding information. The reordering model can be a model that filters and reorders items from the candidate recommended item set. In practice, the reordering model may include an encoding layer and multiple parallel decoding heads. The input to the reordering model is the candidate recommended item set output by the upstream system, and the corresponding output is the reordered item information. Specifically, the reordered item information can be the subsequently recommended items or a sequence of recommended items. Here, recommended items can be items recommended to the target user for viewing. The recommended item sequence can be a sequence of items recommended to the target user sequentially. That is, items are recommended sequentially according to the recommendation order corresponding to the recommended item sequence. In practice, when the reordering model does not include a sampling layer corresponding to a multi-mechanism sampling strategy, the output of the corresponding reordering model can be the subsequently recommended items. When the reordering model includes a sampling layer corresponding to a multi-mechanism sampling strategy, the output of the corresponding reordering model can be a sequence of recommended items. The encoding layer can be a network layer used to encode input information. In practice, the encoding layer can be used to encode each candidate recommended item in the candidate recommended item set based on user behavior information. In practice, the output of the encoding layer can be recommended item encoding information. Recommended item encoding information can include the individual item encoding information corresponding to each candidate recommended item. Item encoding information can represent the item semantic content of the corresponding candidate recommended item under the influence of the behavioral semantic content of user behavior information. In practice, item encoding information can be in vector form. Recommended item encoding information can be a collection of vectors. For example, the encoding layer can be the encoding module in the Transformer architecture. Multiple parallel decoding heads can be a network layer built in parallel with multiple decoding heads. In practice, the number of decoding heads can be preset based on the specific scenario of the recommendation scenario. For example, the number of decoding heads can be "6", that is, 6 decoding heads are built in parallel. In practice, the network layer architecture corresponding to each decoding head is the same. For example, the decoding head can be an FFN (Feed-Forward Network) layer. Multiple parallel decoding heads can be decoding heads that output recommended items in multiple stages. For example, if the number of multiple parallel decoding heads is "6", one recommended item can be output in the first stage, three recommended items in the second stage, and six recommended items in the third stage.

[0042] In some optional implementations of certain embodiments, the execution entity may input the candidate recommended item set and the user behavior information into the encoding layer of a pre-trained reordering model to obtain recommended item encoding information, including the following steps: Using the above encoding layer, perform the following generation steps: Sub-step 1: For each candidate recommended item, based on the aforementioned user behavior information, extract the item representation information of the candidate recommended item under the corresponding interests of the target user. The item representation information can be adaptively embedded semantic content based on the target user's corresponding user interests. In practice, the item representation information can be in vector form.

[0043] As an example, the aforementioned implementing entity can use Deep Interest Network (DIN) to perform attention weighting on candidate recommended items according to the user behavior sequence corresponding to user behavior information, extract item embeddings that are adaptive to user interests, and obtain item representation information.

[0044] Sub-step 2 involves generating item embedding information for each candidate recommended item in the aforementioned candidate recommended item set based on the global contextual dependencies between the obtained item representation information. This item embedding information set serves as the recommended item encoding information. The global contextual dependency can be the global item association semantic content corresponding to each item representation information. Item embedding information can be item adjustment information resulting from semantically adjusting the item semantic representation content using context-aware content (i.e., global contextual dependencies). In practice, item embedding information can be in vector form.

[0045] As an example, firstly, multi-head self-attention is used to model the global contextual dependencies between the item representations. Then, based on the global contextual dependencies, semantic content adjustment is performed on the item representations corresponding to each candidate recommended item to obtain item embedding information. For example, the global contextual dependencies and item representations can be concatenated and input into a convolutional block to achieve semantic content adjustment based on context-aware content, thus obtaining item embedding information.

[0046] In some alternative implementations of certain embodiments, the above rearrangement model is trained through the following steps: The first step is to perform the first training step on the first target training data in the first training dataset: Sub-step 1 involves inputting the candidate recommended item set and corresponding user behavior information from the aforementioned first target training data into the initial re-ranking model to obtain multiple recommended items. The initial re-ranking model can be a re-ranking model that has not yet finished training. In practice, the initial re-ranking model can be a re-ranking model that is currently being trained or one that has not yet started training. The first target training data can be the data used to train the initial re-ranking model in this round. In practice, the first target training data can be at least one set of data. In practice, the first target training data can be randomly selected from the first training dataset.

[0047] Sub-step 2 involves using a multi-mechanism sampling strategy to generate a candidate recommended item sequence set corresponding to the aforementioned multiple recommended items. This candidate recommended item sequence can be a pending recommendation sequence for item recommendation. In practice, the candidate recommended item sequence set can include multiple sequences. Specific implementation details are omitted here.

[0048] Sub-step 3 involves using an exploratory reward model to generate exploratory reward information for each candidate recommended item sequence in the aforementioned candidate recommended item sequence set. The exploratory reward model can be any model that generates the exploratory reward information corresponding to the recommended sequence. In practice, the exploratory reward model can be a pre-trained reward model. That is, during the training of the reordering model, the exploratory reward model is only used to generate the exploratory reward information corresponding to the recommended sequence and is not trained along with the reordering model. The exploratory reward information can be reward information indicating the magnitude of the exploratory value corresponding to the candidate recommended item sequence. Here, exploratory value can be whether the user has an exploratory desire for the candidate recommended item sequence. That is, whether the user is interested in each recommended item in the candidate recommended item sequence. The reward information can be a number used to measure the magnitude of the exploratory value. The higher the reward value corresponding to the exploratory reward information, the higher the user's exploratory desire for the candidate recommended item sequence, and the more interested they are. For example, the exploratory reward model can be a reward model based on the Transformer architecture.

[0049] As an example, the aforementioned implementing entity can input each candidate recommended item sequence into the exploratory reward model to obtain exploratory reward information.

[0050] Sub-step 4 involves generating a preference loss for the aforementioned set of candidate recommended item sequences based on each exploratory reward information. This preference loss can represent the degree of preference difference for candidate recommended item sequences with high exploratory reward information. A lower preference loss indicates that the trained reordering model prefers to output candidate recommended item sequences with high exploratory reward information.

[0051] As an example, firstly, the aforementioned execution entity can sort the candidate recommended item sequences according to the exploratory reward information, in descending order of reward information, to obtain the sorting results. Then, a comparison sample set is constructed between candidate recommended item sequences corresponding to high-value reward information and those corresponding to low-value reward information. Finally, based on the comparison sample set, a loss function is used to generate a preference loss for the high-value reward information of the aforementioned candidate recommended item sequence set. This preference loss function can be a conventional comparison loss function.

[0052] Sub-step 5: Based on the above preference loss, determine whether the initial rearrangement model has finished training.

[0053] As an example, first, it is determined whether the individual losses in the loss sequence corresponding to the preference loss have converged. The loss sequence can be a sequence including the preference loss and all previously generated preference losses. The individual losses in the loss sequence are ordered according to training time. Then, in response to determining convergence, the initial reordered model training is determined to have ended. In response to determining non-convergence, the initial reordered model training is determined not to have ended.

[0054] Sub-step 6: In response to the end of training, the initial rearranged model is determined as the rearranged model.

[0055] The second step, in response to the incomplete training, involves retraining the initial rearranged model based on the aforementioned preference loss, resulting in a retrained rearranged model. This retraining can involve updating the model parameters in the initial rearranged model. The retrained rearranged model is the model with updated parameters. Specifically, this parameter updating can be performed via backpropagation.

[0056] The third step involves reselecting the first training data from the first training dataset as the first target training data, using the re-ranked model after training as the initial re-ranked model, and continuing to execute the first training step described above. Note that the selected first training data is not any previously selected training data.

[0057] In some optional implementations of certain embodiments, the aforementioned execution entity may generate a preference loss for the aforementioned candidate recommended item sequence set based on various exploratory reward information, including the following steps: The first step is to sort the candidate recommended item sequences in the aforementioned candidate recommended item sequence set according to the exploratory reward information. The sorting result can be the sequence sorting result corresponding to the candidate recommended item sequences. For example, if the candidate recommended item sequences include: Sequence 1, Sequence 2, Sequence 3, and Sequence 4, then the sorting result could be {Sequence 2, Sequence 1, Sequence 4, Sequence 3}.

[0058] As an example, the candidate recommended item sequences in the above candidate recommended item sequence set can be sorted in descending order of exploratory reward information to obtain the sorting result.

[0059] The second step involves filtering candidate recommended item sequences at each quantile from the sequences corresponding to the sorting results, thus obtaining a subset of candidate recommended item sequences. Each quantile can be one of the sequence quantiles set in the sorting results. Each candidate recommended item in the candidate recommended item sequence is an item whose corresponding sequence position falls under the corresponding quantile in each sequence.

[0060] As an example, first, set the equidistant quantile interval. For example, the equidistant quantile interval could be 3. Then, based on the equidistant quantile interval, filter out the candidate recommended item sequences at each equidistant quantile from the sequences corresponding to the above sorting results, thus obtaining a subset of candidate recommended item sequences.

[0061] The third step involves generating a preference loss based on the subset of candidate recommended item sequences and the sequence generation probability corresponding to each candidate recommended item sequence, using the preference loss function. The sequence generation probability characterizes the accuracy of the sequence corresponding to the candidate recommended item sequence. The sequence generation probability can be a value between 0 and 1. A higher value indicates a more accurate candidate recommended item sequence. It should be noted that the sequence generation probability here refers to the probability generated during the generation of the candidate recommended item sequences. The preference loss function can be any loss function used to generate the preference loss. For example, the preference loss function could be the ORPO (Odds Ratio Preference Optimization) loss function.

[0062] As an example, firstly, based on the aforementioned preference sample label set, the advantage ratio corresponding to each candidate recommended item sequence in the aforementioned subset of candidate recommended item sequences is determined (the specific implementation method will not be elaborated here). Then, the obtained advantage ratios are input into the ORPO loss function to obtain the preference loss.

[0063] Optionally, based on the above subset of candidate recommended item sequences and the sequence generation probability corresponding to each candidate recommended item sequence, a preference loss function is used, including: The first step is to generate an advantage ratio set corresponding to the subset of candidate recommended item sequences based on the sequence generation probability set. There is a one-to-one correspondence between the candidate recommended items in the subset and the advantage ratio in the advantage ratio set. The advantage ratio represents the probability that the model generates a candidate recommended item sequence relative to the probability that it does not generate a candidate recommended item sequence.

[0064] As an example, for each candidate recommended item sequence, firstly, the sequence generation probability corresponding to the candidate recommended item sequence is determined, which is taken as the target sequence generation probability. Then, the target sequence generation probability is subtracted from the value 1 to obtain the subtraction probability. Finally, the target sequence generation probability is divided by the subtraction probability to obtain the advantage ratio.

[0065] The second step involves generating a cumulative advantage ratio set and a sample weight set corresponding to the aforementioned subset of candidate recommended item sequences, based on the aforementioned advantage ratio set. The cumulative advantage ratio can be the sum of the advantage ratios of all worse samples ranked after the candidate recommended item sequence. Each candidate recommended item sequence in the subset has a corresponding cumulative advantage ratio and sample weight. The sample weight characterizes the importance of the candidate recommended item sequence. A higher sample weight indicates that the sample data corresponding to the candidate recommended item sequence is more important.

[0066] As an example, for each candidate recommended item sequence, firstly, the advantage ratio corresponding to the candidate recommended item sequence can be divided by the temperature parameter (which controls the sharpness of the soft weight distribution; when the temperature parameter is greater than 0, all weights approach 1), resulting in a division. Then, the division result is substituted into the sigmoid function to obtain the sample weights. Next, all candidate recommended item sequences following the above candidate recommended item sequence are determined. Then, the advantage ratios corresponding to each of the above candidate recommended item sequences are summed, resulting in a summed value. Finally, the advantage ratio corresponding to each candidate recommended item sequence is divided by the summed value to obtain the cumulative advantage ratio.

[0067] The third step is to generate the preference loss based on the cumulative advantage ratio set and sample weight set mentioned above.

[0068] As an example, the cumulative advantage ratio set and sample weight set are substituted into the ORPO loss function to obtain the preference loss.

[0069] The advantages of this design are: (1) The preference list covers a wider reward range than binary comparison alone, allowing for more precise adjustment of the optimization trajectory. (2) Setting sample weights allows important sequences to make a greater contribution to optimization and reliably suppress noise. (3) It is compatible with the importance sampling idea in sampling, ensuring that the advantage ratio of the best sequence significantly exceeds the cumulative advantage ratio of the worse sequences.

[0070] In some optional implementations of certain embodiments, the aforementioned execution entity may determine whether the initial rearrangement model has finished training based on the aforementioned preference loss, including the following steps: The first step is to determine the diversity loss among the various recommended items. This diversity loss measures the high similarity of output content when multiple decoders output. Here, multiple parallel decoders will output the recommended item set in parallel in subsequent stages. Parallel output from multiple decoders may result in the output of multiple similar recommended items. To avoid outputting similar recommended items and increase item diversity, a diversity loss is set to prevent the decoding of identical recommended items.

[0071] Here, by minimizing the diversity loss, semantic redundancy and representation collapse between decoding headers can be explicitly suppressed, ensuring the diversity of parallel generated items. Online A / B testing shows that the larger the number of decoding stages, the more significant the gain in diversity loss.

[0072] As an example, firstly, based on the decoding stages corresponding to the decoding heads, multiple recommended items can be divided into multiple recommended item groups. Each recommended item in a group represents the output of multiple decoding heads within the same stage. Then, for each recommended item group, the cosine similarity between every two recommended items in that group is determined, resulting in a cosine similarity set. Next, the cosine similarities in the cosine similarity set are processed using the squared mean, yielding a squared mean result, which serves as the diversity loss.

[0073] The second step is to determine whether the initial rearrangement model has finished training, based on the aforementioned diversity loss and preference loss.

[0074] As an example, firstly, the diversity loss and preference loss are weighted to obtain the first weighted loss. Then, in response to the convergence of the loss sequence corresponding to the first weighted loss, the initial reordering model training is considered complete. In response to the failure of the loss sequence corresponding to the first weighted loss to converge, the initial reordering model training is considered incomplete.

[0075] Optionally, based on the aforementioned diversity loss and preference loss, determining whether the initial reordering model has finished training includes: The first step is to determine the sequence loss between the subset of candidate recommended item sequences and the actual exposure sequence. The sequence loss can be a loss value used to measure the accuracy of the candidate recommended item sequence.

[0076] As an example, a subset of candidate recommended item sequences and the actual exposure sequence can be substituted into the autoregressive cross-entropy loss function to obtain the sequence loss.

[0077] The second step is to weight the diversity loss, preference loss, and sequence loss to obtain the second weighted loss.

[0078] The third step involves determining whether the initial rearrangement model has finished training based on the second weighted loss. Details will not be elaborated further.

[0079] In some optional implementations of certain embodiments, the exploratory reward model described above includes a shared encoding layer and a dual-task prediction layer. The shared encoding layer can be an encoding layer shared between the training and application phases. During training, the input to the shared encoding layer is a sequence of real exposure logs. During application, the output is a set of candidate recommended item sequences. The shared encoding layer can be a network module used to encode the input information. In practice, the shared encoding layer can include an item embedding layer, a position embedding summation layer, and a self-attention context layer. The item embedding layer can use a DeepInterest Network (DIN) to perform attention-weighted summation on candidate items according to the user behavior sequence, extracting user interest-adaptive item embeddings. The position embedding summation layer can add a sinusoidal position embedding element-wise to the item embedding at each position in the sequence to obtain a position-aware item representation. The self-attention context layer can perform multi-head self-attention on the position-aware item sequence, outputting a context-aware embedding. The dual-task prediction layer can be a prediction model that outputs the results of two tasks. In practice, the dual-task prediction layer can use a multi-gate hybrid expert network (MMoE) and fully connected layers. The Multi-Mechanical Expert Network (MMoE) is used to output the results corresponding to the item-level immediate value head. Fully connected layers are used to output the results corresponding to the sequence-level exploratory value head. The outputs of both tasks include: the output results corresponding to the item-level immediate value head and the output results corresponding to the sequence-level exploratory value head. The output results corresponding to the item-level immediate value head are value information at the item level under the current situation. The output results corresponding to the sequence-level exploratory value head are value information at the sequence level under the current situation. The output results corresponding to the item-level immediate value head may include click-through rate and purchase rate. The output results corresponding to the sequence-level exploratory value head may include sequence-level exploratory value information. In practice, sequence-level exploratory value information can be in numerical form or in label form. Sequence-level exploratory value information can be used to measure whether the overall sequence of the output candidate item recommendation sequence has high exploratory value. For sequence-level exploratory value information in numerical form, the higher the value, the higher the overall exploratory value of the candidate item recommendation sequence.

[0080] Optionally, the aforementioned implementing entity may utilize an exploratory reward model to generate exploratory reward information corresponding to each candidate recommended item sequence in the aforementioned candidate recommended item sequence set, including the following steps: The first step is to input the above-mentioned candidate recommended item sequence into the above-mentioned shared coding layer to obtain the coding information.

[0081] The second step is to input the above-mentioned encoded information into the dual-task prediction layer to obtain the item's immediate value information and sequence exploration value information, which serve as exploratory reward information.

[0082] In some alternative implementations of certain embodiments, the above-described exploratory reward model is trained through the following steps: First, for the second target training data in the second training dataset, perform the second training step: Sub-step 1 involves inputting the real exposure log sequence from the second target training data into the initial shared encoding layer of the initial exploratory reward model to obtain initial encoding information. The second target training data also includes cross-request signal labels. The second target training data can be second training data selected from the second training dataset for subsequent training. The real exposure log sequence can be a sequence of the real exposure logs corresponding to the target user in chronological order. The initial exploratory reward model can be an exploratory reward model that has not yet finished training. The initial shared encoding layer can be a shared encoding layer that has not yet finished training. The cross-request signal labels can be information generated when processing the current request, not only relying on the information inherent in the current request but also referencing, extracting, or associating with information from "other requests" (historical requests, concurrent requests, or global requests). That is, the cross-request signal labels can represent the target user's request status for multiple requests to the item. In other words, the cross-request signal labels can represent the target user's behavioral labels for the item at various chronological times. For example, a cross-request signal label could be a label indicating that the item corresponding to the target real exposure log was clicked and processed in the first instance, and there is no subsequent click. For example, a cross-request signal tag could be a tag for an item in the target's real exposure log that was not clicked or pulled down in the first instance, and for which there is no next click.

[0083] Sub-step 2 involves inputting the aforementioned initial encoded information into the initial dual-task prediction layer of the initial exploratory reward model to obtain the initial item instant value information and the initial sequence exploration value information. Details will not be elaborated further.

[0084] Sub-step 3: Based on the initial item instant value information and the cross-request signal tags mentioned above, generate the item instant value loss. The item instant value loss can represent the difference in click-through rate and purchase rate.

[0085] As an example, based on the initial item instant value information and the cross-request signal label mentioned above, the item instant value loss can be generated using the Binary Cross-Entropy (BCE) loss function.

[0086] Sub-step 4: Based on the initial sequence exploration value information mentioned above, generate the sequence exploration value loss. The sequence exploration value loss characterizes the value difference in sequence exploration value.

[0087] As an example, based on the initial sequence exploration value information mentioned above, the sequence exploration value loss is generated using the binary crossover loss function.

[0088] Sub-step 5: Based on the immediate value loss of the items and the initial sequence exploration value information, determine whether the initial exploratory reward model has finished training.

[0089] As an example, the immediate value loss of the aforementioned items and the exploration value information of the initial sequence can be weighted to obtain a third weighted loss. Then, in response to the convergence of the loss sequence corresponding to the third weighted loss, it is determined that the initial exploratory reward model training has ended. In response to the failure of the loss sequence corresponding to the third weighted loss to converge, it is determined that the initial exploratory reward model training has not ended.

[0090] Sub-step 6: In response to the end of training, the above initial exploratory reward model is determined as the exploratory reward model.

[0091] The second step, in response to the fact that training is not yet complete, involves retraining the initial exploratory reward model based on the immediate value loss of the items and the initial sequence exploration value information, resulting in a post-trained reward model. Details will not be elaborated further.

[0092] The third step involves reselecting training data from the second training dataset as the target training data, using the above-mentioned post-training reward model as the initial exploratory reward model, and continuing to execute the second training step.

[0093] In some optional implementations of certain embodiments, generating the sequence exploration value loss based on the initial sequence exploration value information includes: The first step is to determine the data weights corresponding to the training data for the second objective based on the label types corresponding to the cross-request signal labels mentioned above. The label types are one of the following: termination label type, invalid exploration label type, potential interest label type, instant matching label type, and deep exploration label type. The termination label type can be a label indicating the termination of item exploration. Here, the termination label type can be a label type where there is no click or pull-down behavior at the first time, and no click behavior at the next click time at the second time (i.e., after the first time). The invalid exploration label type can be a label indicating that item exploration for exposed items is invalid. Here, the invalid exploration label type can be a label type where there is no click behavior and a pull-down behavior at the first time, but no click behavior at the next click time at the second time (i.e., after the first time). The potential interest label type can be a label indicating potential interest in exposed items and item exploration. Here, the potential interest label type can be a label type where there is no click behavior and a pull-down behavior at the first time, and a click behavior at the next click time at the second time (i.e., after the first time). The "Instant Match" tag type corresponds to tags that indicate immediate interest in exploring exposed items. This type of tag is one that triggers a click immediately without any dropdown action. The "In-Depth Exploration" tag type corresponds to tags that indicate in-depth interest in exploring exposed items. This type of tag is one that triggers both a click and a dropdown action immediately.

[0094] Here, samples corresponding to the termination label type and the invalid exploration label type are negative samples (represented by a label of 0). Samples corresponding to the latent interest label type, the immediate matching label type, and the deep exploration label type are positive samples (represented by a label of 1). Data weights can represent the importance of the samples corresponding to the second-target training data. In practice, different label types correspond to different levels of importance in the second-target training data. Data importance can be a value between 0 and 1. The higher the value, the more important the sample data corresponding to the second-target training data.

[0095] As an example, the sample weights under the tag types corresponding to the aforementioned cross-request signal tags can be queried from the sample weight mapping table and used as data weights. The sample weight mapping table represents the mapping relationship between tag types and sample weights. The sample weight mapping table can be set based on historical experience.

[0096] The second step is to generate the sequence exploration value loss based on the data weights and the initial sequence exploration value information mentioned above.

[0097] As an example, firstly, the initial sequence exploration value information is input into the binary cross-entropy loss function to obtain the initial loss value. Then, the data weights are multiplied by the initial loss value to obtain the sequence exploration value loss.

[0098] In some optional implementations of certain embodiments, determining the data weights corresponding to the second target training data based on the label type corresponding to the cross-request signal label includes: The first step, in response to whether the aforementioned tag type is a potential interest tag type or an invalid exploration tag type, is to determine the data quality corresponding to the aforementioned real exposure log sequence. Data quality can be the sequence quality status corresponding to the real exposure log sequence. Data quality can be in numerical form or in tag form. Data quality can be measured in various ways. For example, data quality can be measured by the average number of clicks or average number of purchases corresponding to the real exposure log sequence, or by the sequence accuracy output by the upstream fine-tuning model.

[0099] Here, the potential interest tag type can represent types with potential exploration value. When the data quality corresponding to the actual exposure log sequence may be low, increasing the learning of samples corresponding to the potential interest tag type can make the model focus more on recommending items with potential exploration value. In addition, the invalid exploration tag type refers to the tag type of invalid interest exploration. During item recommendation, to avoid being incorrectly penalized as negative samples in supply-constrained scenarios, the recommendation and learning of items corresponding to invalid exploration tag types can be reduced.

[0100] The second step is to determine the data weights corresponding to the training data for the second objective based on the aforementioned data quality.

[0101] As an example, in response to the above tag type being a potential interest tag type, firstly, the sum of the data quality and the value 1 is logarithmically processed to obtain the processed result. Then, the processed result is multiplied by the value 3 to obtain the multiplied result. Finally, the multiplied result is subtracted from the value 4 to obtain the data weight. In response to the above tag type being an invalid exploration tag type, when the data quality is less than the target value, the data weight is set to 0 (i.e., appropriate processing is performed).

[0102] Here, sample data for potential interest tag types can further reflect users' motivation to continue exploring and is a core signal source with high exploration value.

[0103] Step 203: Input the above-mentioned recommended item encoding information into the above-mentioned multiple parallel decoding heads to output multiple recommended items in parallel.

[0104] In some embodiments, the execution entity may input the recommended item encoding information into the plurality of parallel decoding heads to output multiple recommended items in parallel multiple times. This parallel output may be performed in multiple stages, sequentially outputting multiple recommended items.

[0105] It's important to note that the batch decoding method with multiple parallel outputs can involve initially outputting a small number of recommended items, with the number of recommended items increasing as the stage progresses. For example, in a multi-stage scenario including a first, second, and third stage, the number of recommended items output in the first stage is less than that in the second stage, and vice versa. For instance, the first stage might output one recommended item (i.e., decode one recommended item), the second stage three recommended items (i.e., decode three recommended items), and the third stage six recommended items (i.e., decode six recommended items). This batch decoding method maintains high-fidelity context modeling for the most important head positions and maximizes parallelism for the tail positions.

[0106] It should be noted that the number of items corresponding to multiple recommended items can be preset. For example, the number of items can be 10.

[0107] As an example, firstly, a first decoding head is selected from multiple parallel decoding heads to perform decoding in the first stage. Then, the recommended item encoding information is input into the first decoding head, resulting in one recommended item. Next, three second decoding heads are selected from multiple parallel decoding heads to perform decoding in the second stage. Then, the recommended item encoding information is input into the three second decoding heads, resulting in three recommended items. Subsequently, six second decoding heads are selected from multiple parallel decoding heads to perform decoding in the third stage. Next, the recommended item encoding information is input into the six second decoding heads, resulting in six recommended items. Finally, the recommended items obtained from each stage are determined as multiple recommended items.

[0108] In some optional implementations of certain embodiments, the execution entity may input the recommended item encoding information into the plurality of parallel decoding heads to output multiple recommended items in parallel multiple times, including the following steps: The first step involves using the aforementioned item embedding information set as key-value information and the corresponding location embedding set of the candidate recommended item set as query information. The item embedding information set is then updated using this query embedding to obtain the updated item embedding information set, which serves as the recommended item encoding information. The location embeddings in the location embedding set can be initially predefined. <start>Embedded.

[0109] As an example, the item embedding information set is used as key-value information, and the already generated location is embedded (initially predefined). <start>The query embedding is used as the query information, and a sinusoidal position embedding is superimposed to maintain position awareness. New query embeddings are synthesized step by step using a position autoregressive method until the target length is reached, resulting in an updated set of item embedding information, which serves as the recommended item encoding information.

[0110] The second step involves inputting the recommended item encoding information into the multiple parallel decoding heads to perform multiple progressive parallel decodings, thereby obtaining the multiple recommended items.

[0111] Step 204: Using a multi-mechanism sampling strategy, generate a sequence of recommended items corresponding to the above-mentioned multiple recommended items.

[0112] In some embodiments, the aforementioned executing entity can utilize a multi-mechanism sampling strategy to generate a sequence of recommended items corresponding to the multiple recommended items. The multi-mechanism sampling strategy can be a measurement based on a multi-mechanism sampling method to generate diverse recommended item sequences. That is, based on the multi-mechanism sampling strategy, multiple candidate recommended item sequences can be generated for the multiple recommended items. The candidate recommended item sequences can be item sequences for which further determination is needed to decide whether to recommend them. For example, the multi-mechanism sampling strategy can be to randomly combine the multiple recommended items and select the recommended item sequence with the highest score from the multiple combined item sequences. As another example, the multi-mechanism sampling strategy can be to determine the weight of each recommended item at each sequence position in the recommended item sequence, and then select the recommended item with the highest weight at each sequence position to obtain the recommended item sequence.

[0113] Step 205: Perform item recommendations for the above recommended item sequence.

[0114] In some embodiments, the aforementioned execution entity may perform item recommendations for the aforementioned recommended item sequence.

[0115] As an example, the aforementioned execution entity can recommend items sequentially according to the order of the recommended item sequence, and then recommend them to the target user's terminal for display on the corresponding recommendation page.

[0116] The above embodiments of this disclosure have the following beneficial effects: By utilizing multiple parallel decoding heads and multi-mechanism sampling strategies, the item recommendation methods of some embodiments of this disclosure achieve a fast and accurate response to recommended item sequences based on exploring more combinations of item sequences, avoiding recommendation delays. Specifically, the reasons for the inaccuracy and response delay of related item recommendations are that existing reordering methods (e.g., heuristic score-weighted sampling, beam-search, Pointer Network, etc.) are prone to generating homogeneous trajectories, causing the evaluator to select only in a narrow subspace, the model gets trapped in local optima, and the accuracy of the recommendation results is affected. In addition, it is difficult to achieve a fast response to recommendation results under industrial-grade latency constraints. Based on this, the item recommendation methods of some embodiments of this disclosure first obtain the candidate recommended item set corresponding to the target recommendation request and the user behavior information corresponding to the target user, so as to accurately reorder the candidate recommended items based on the candidate recommended item set and user behavior information output from the upstream, taking into account the semantic content of user behavior. Then, the aforementioned candidate recommended item set and user behavior information are input into the encoding layer of the pre-trained reordering model to obtain the recommended item encoding information. The reordering model also includes multiple parallel decoding heads. Here, the encoding layer enables accurate encoding of each candidate recommended item based on the semantic content of user behavior, facilitating accurate decoding of recommended items and ensuring the accuracy of recommended item generation. Next, the recommended item encoding information is input into the multiple parallel decoding heads to output multiple recommended items in parallel. Here, the use of multiple parallel decoding heads significantly reduces the single-batch generation time through parallel mechanisms, meeting the industrial-grade TP99 constraint and achieving rapid response of item recommendation results. Furthermore, a multi-mechanism sampling strategy is used to generate recommended item sequences corresponding to the multiple recommended items, which can significantly improve exploration efficiency, avoid local optima, and improve the diversity and accuracy of recommended item sequences. Finally, item recommendation is performed on the recommended item sequences. In summary, by utilizing multiple parallel decoding heads and a multi-mechanism sampling strategy, a fast and accurate response of recommended item sequences is achieved by exploring more combinations of item sequences, avoiding recommendation delays.

[0117] Further reference Figure 3 The diagram illustrates a flow 300 of another embodiment of the item recommendation method according to this disclosure. This item recommendation method includes the following steps: Step 301: Obtain the set of candidate recommended items corresponding to the target recommendation request and the user behavior information corresponding to the target user.

[0118] Step 302: Input the above candidate recommended item set and the above user behavior information into the encoding layer of the pre-trained rearrangement model to obtain the recommended item encoding information.

[0119] Step 303: Input the above-mentioned recommended item encoding information into the above-mentioned multiple parallel decoding heads to output multiple recommended items in parallel.

[0120] Step 304: Based on the above multiple recommended items, generate a first candidate recommended item sequence set using the above bundle search technique.

[0121] In some embodiments, the execution entity (e.g. Figure 1 The electronic device 101 shown can generate a first candidate recommended item sequence set based on the above-mentioned multiple recommended items and using the above-mentioned bundle search technology.

[0122] Here, the entire beam is divided into g groups, each group independently maintaining multiple beams with different starting points. At each step, a Top-b expansion is performed on the beams of that group. Simultaneously, to avoid duplicate items within the same sequence, the selected positions are masked. Gumbel noise is superimposed on the logits to encourage exploration, thereby generating the first candidate recommended item sequence set.

[0123] Step 305: Based on the above multiple recommended items, generate a second candidate recommended item sequence set using the above heuristic sampling technique.

[0124] In some embodiments, the execution entity may generate a second candidate recommended item sequence set based on the multiple recommended items and using the heuristic sampling technique described above.

[0125] Specifically, firstly, the multi-objective scores (e.g., CTR, CVR) corresponding to multiple recommended items are dynamically weighted and fused. Then, random perturbations are injected to obtain a candidate recommended item sequence set. Simultaneously, real online exposure sequences are also incorporated into the candidate recommended item sequence set, ensuring the generator always encounters anchor points of the real distribution during training, resulting in a second candidate recommended item sequence set.

[0126] Step 306: Combine the first candidate recommended item sequence set and the second candidate recommended item sequence set to obtain a candidate recommended item sequence set.

[0127] In some embodiments, the executing entity may combine the first candidate recommended item sequence set and the second candidate recommended item sequence set to obtain a candidate recommended item sequence set.

[0128] As an example, the aforementioned executing entity can perform sequence fusion and deduplication on the first candidate recommended item sequence set and the second candidate recommended item sequence set to obtain a candidate recommended item sequence set.

[0129] Step 307: Select the candidate recommended item sequence with the highest sequence generation probability from the above candidate recommended item sequence set, and use it as the recommended item sequence.

[0130] In some embodiments, the execution entity may select the candidate recommended item sequence with the highest sequence generation probability from the candidate recommended item sequence set and use it as the recommended item sequence.

[0131] Step 308: Perform item recommendations for the above recommended item sequence.

[0132] In some embodiments, the specific implementation of steps 301-303 and 308 and their resulting technical effects can be found in [reference needed]. Figure 2 Steps 201-203 and 205 in the corresponding embodiments will not be repeated here.

[0133] from Figure 3 It can be seen from this that, with Figure 2 Compared to the description of some corresponding embodiments, Figure 3 In some corresponding embodiments, the item recommendation method process 300 utilizes bundle search technology and heuristic sampling technology to achieve diversity in sequence space exploration, avoid premature convergence to a single mode, and improve the accuracy and diversity of subsequent item recommendations.

[0134] Further reference Figure 4 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of an item recommendation device, which are similar to... Figure 2 Corresponding to the method embodiments shown, the recommended device can be specifically applied to various electronic devices.

[0135] like Figure 4 As shown, an item recommendation device 400 includes: an acquisition unit 401, a first generation unit 402, a second generation unit 403, a third generation unit 404, and an execution unit 405. The acquisition unit 401 is configured to acquire a set of candidate recommended items corresponding to a target recommendation request and user behavior information corresponding to the target user; the first generation unit 402 is configured to input the candidate recommended item set and the user behavior information into the encoding layer of a pre-trained rearrangement model to obtain recommended item encoding information, wherein the rearrangement model further includes multiple parallel decoding heads; the second generation unit 403 is configured to input the recommended item encoding information into the multiple parallel decoding heads to output multiple recommended items in parallel multiple times; the third generation unit 404 is configured to generate a recommended item sequence corresponding to the multiple recommended items using a multi-mechanism sampling strategy; and the execution unit 405 is configured to perform item recommendation for the recommended item sequence.

[0136] In some optional implementations of some embodiments, the first generation unit 402 may be further configured to: utilize the above-mentioned encoding layer to perform the following generation steps: for each candidate recommended item, extract the item representation information of the candidate recommended item under the corresponding interest of the target user based on the above-mentioned user behavior information; based on the global context dependency between the obtained item representation information, generate the item embedding information of each candidate recommended item in the above-mentioned candidate recommended item set, and obtain the item embedding information set as the recommended item encoding information.

[0137] In some optional implementations of certain embodiments, the second generation unit 403 may be further configured to: use the above-mentioned item embedding information set as key value information, use the above-mentioned candidate recommended item set corresponding position embedding set as query information, perform query embedding update on the above-mentioned item embedding information set to obtain the updated item embedding information set as recommended item encoding information; input the above-mentioned recommended item encoding information to the above-mentioned multiple parallel decoding heads to perform multiple stepwise parallel decodings to obtain the above-mentioned multiple recommended items.

[0138] In some optional implementations of certain embodiments, the above-mentioned multi-mechanism sampling strategy is a strategy for sequence generation based on bundle search technology and heuristic sampling technology; and the third generation unit 404 can be further configured to: generate a first candidate recommended item sequence set based on the above-mentioned multiple recommended items using the above-mentioned bundle search technology; generate a second candidate recommended item sequence set based on the above-mentioned multiple recommended items using the above-mentioned heuristic sampling technology; combine the above-mentioned first candidate recommended item sequence set and the above-mentioned second candidate recommended item sequence set to obtain a candidate recommended item sequence set; and select the candidate recommended item sequence with the highest sequence generation probability from the above-mentioned candidate recommended item sequence set as the recommended item sequence.

[0139] It is understandable that the units described in the recommended device 400 and the reference Figure 2 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the item recommendation device 400 and the units contained therein, and will not be repeated here.

[0140] The following is for reference. Figure 5 It illustrates electronic devices suitable for implementing some embodiments of this disclosure (e.g., Figure 1 A schematic diagram of the structure of electronic device 101)500. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0141] like Figure 5 As shown, the electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory 502 or a program loaded from a storage device 508 into a random access memory 503. The random access memory 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, the read-only memory 502, and the random access memory 503 are interconnected via a bus 504. An input / output interface 505 is also connected to the bus 504.

[0142] Typically, the following devices can be connected to the input / output interface 505: input devices 505 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 508 including, for example, magnetic tape, hard disk, etc.; and communication devices 509. Communication devices 509 allow electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 5 Each box shown can represent a device or multiple devices as needed.

[0143] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a read-only memory 502. When the computer program is executed by the processing device 501, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0144] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0145] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0146] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire a set of candidate recommended items corresponding to the target recommendation request and user behavior information corresponding to the target user; input the set of candidate recommended items and the user behavior information into the encoding layer of a pre-trained reordering model to obtain recommended item encoding information, wherein the reordering model further includes: multiple parallel decoding heads; input the recommended item encoding information into the multiple parallel decoding heads to output multiple recommended items in parallel multiple times; generate a recommended item sequence corresponding to the multiple recommended items using a multi-mechanism sampling strategy; and perform item recommendation for the recommended item sequence.

[0147] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0148] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0149] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including an acquisition unit, a first generation unit, a second generation unit, a third generation unit, and an execution unit. The names of these units do not necessarily limit the unit itself; for example, the acquisition unit may also be described as "a unit that acquires the set of candidate recommended items corresponding to the target recommendation request and the user behavior information corresponding to the target user."

[0150] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0151] Some embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements any of the above-described item recommendation methods.

[0152] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.< / start> < / start>

Claims

1. A method for recommending items, including: Obtain the set of candidate recommended items corresponding to the target recommendation request and the user behavior information corresponding to the target user; The candidate recommended item set and the user behavior information are input into the encoding layer of the pre-trained reordering model to obtain the recommended item encoding information. The reordering model also includes multiple parallel decoding heads. The recommended item encoding information is input into the multiple parallel decoding heads to output multiple recommended items in parallel. A multi-mechanism sampling strategy is used to generate a sequence of recommended items corresponding to the multiple recommended items; Perform item recommendations for the recommended item sequence.

2. The method according to claim 1, wherein, The step of inputting the candidate recommended item set and the user behavior information into the encoding layer of the pre-trained reordering model to obtain recommended item encoding information includes: Using the encoding layer, the following generation steps are performed: For each candidate recommended item, based on the user behavior information, extract the item representation information of the candidate recommended item under the target user's corresponding interest; Based on the global contextual dependencies between the obtained item representation information, item embedding information is generated for each candidate recommended item in the candidate recommended item set, resulting in an item embedding information set, which serves as the recommended item encoding information.

3. The method according to claim 1, wherein, The step of inputting the recommended item encoding information into the multiple parallel decoding heads to output multiple recommended items in parallel includes: The item embedding information set is used as key-value information, and the corresponding position embedding set of the candidate recommended item set is used as query information. The item embedding information set is then queried, embedded, and updated to obtain the updated item embedding information set, which is used as the recommended item encoding information. The recommended item encoding information is input into the multiple parallel decoding heads to perform multiple stepwise parallel decodings to obtain the multiple recommended items.

4. The method according to claim 1, wherein, The multi-mechanism sampling strategy is a sequence generation strategy based on bundle search technology and heuristic sampling technology. as well as The process of generating a recommended item sequence corresponding to the multiple recommended items using a multi-mechanism sampling strategy includes: Based on the multiple recommended items, the bundle search technique is used to generate a first candidate recommended item sequence set; Based on the multiple recommended items, a second candidate recommended item sequence set is generated using the heuristic sampling technique; The first candidate recommended item sequence set and the second candidate recommended item sequence set are combined to obtain the candidate recommended item sequence set; The candidate recommended item sequence with the highest sequence generation probability is selected from the set of candidate recommended item sequences and used as the recommended item sequence.

5. The method according to claim 1, wherein, The rearrangement model is trained through the following steps: For the first target training data in the first training dataset, perform the first training step: The candidate recommended item set and the corresponding user behavior information in the first target training data are input into the initial re-ranking model to obtain multiple recommended items; A multi-mechanism sampling strategy is used to generate a candidate recommended item sequence set corresponding to the multiple recommended items; Using an exploratory reward model, exploratory reward information is generated for each candidate recommended item sequence in the candidate recommended item sequence set; Based on the exploratory reward information, a preference loss is generated for the candidate recommended item sequence set; Based on the preference loss, determine whether the initial rearrangement model has finished training; In response to the end of training, the initial rearranged model is determined as the rearranged model; In response to the fact that training has not ended, the initial rearranged model is retrained according to the preference loss to obtain the retrained rearranged model; The first training data reselected from the first training dataset is used as the first target training data, and the re-ranked model after training is used as the initial re-ranked model. The first training step is then continued.

6. The method according to claim 5, wherein, The step of generating a preference loss for the candidate recommended item sequence set based on each exploratory reward information includes: Based on the aforementioned exploratory reward information, the candidate recommended item sequences in the candidate recommended item sequence set are sorted to obtain the sorting result; From the sorting results, candidate recommended item sequences at each quantile are selected to obtain a subset of candidate recommended item sequences; Based on the subset of candidate recommended item sequences and the sequence generation probability corresponding to each candidate recommended item sequence, a preference loss is generated using the preference loss function.

7. The method according to claim 6, wherein, The step of generating a sequence probability based on the subset of candidate recommended item sequences and the sequence corresponding to each candidate recommended item sequence, using a preference loss function, includes: Generate the advantage ratio set corresponding to the subset of candidate recommended item sequences based on the sequence generation probability set corresponding to the subset of candidate recommended item sequences; Based on the advantage ratio set, generate the cumulative advantage ratio set and sample weight set corresponding to the subset of candidate recommended item sequences; Based on the cumulative advantage ratio set and the sample weight set, a preference loss is generated.

8. The method according to claim 5, wherein, The step of determining whether the initial rearrangement model has finished training based on the preference loss includes: Determine the diversity loss among the various recommended items; Based on the diversity loss and the preference loss, determine whether the initial rearrangement model has finished training.

9. The method according to claim 5, wherein, The exploratory reward model includes: a shared encoding layer and a dual-task prediction layer; and The step of generating exploratory reward information for each candidate recommended item sequence in the candidate recommended item sequence set using an exploratory reward model includes: The candidate recommended item sequence is input into the shared coding layer to obtain coding information; The encoded information is input into the dual-task prediction layer to obtain the item's immediate value information and sequence exploration value information, which serve as exploratory reward information.

10. The method according to claim 9, wherein, The exploratory reward model is trained through the following steps: For the second target training data in the second training dataset, perform the second training step: The real exposure log sequence in the second target training data is input into the initial shared encoding layer in the initial exploratory reward model to obtain initial encoding information. The second target training data also includes cross-request signal labels. The initial encoded information is input into the initial dual-task prediction layer in the initial exploratory reward model to obtain the initial item instant value information and the initial sequence exploration value information. Based on the initial item instant value information and the cross-request signal tag, an item instant value loss is generated; Based on the initial sequence exploration value information, a sequence exploration value loss is generated; Based on the immediate value loss of the item and the initial sequence exploration value information, determine whether the initial exploratory reward model has finished training; In response to the end of training, the initial exploratory reward model is determined as an exploratory reward model; In response to the fact that the training has not ended, the initial exploratory reward model is retrained based on the immediate value loss of the item and the initial sequence exploration value information to obtain the trained reward model. The training data reselected from the second training dataset is used as the target training data, and the post-trained reward model is used as the initial exploratory reward model to continue the second training step.

11. The method according to claim 10, wherein, The step of generating sequence exploration value loss based on the initial sequence exploration value information includes: Based on the label type corresponding to the cross-request signal label, the data weight corresponding to the second target training data is determined, wherein the label type is one of the following: termination label type, invalid exploration label type, potential interest label type, instant matching label type, and deep exploration label type; Based on the data weights and the initial sequence exploration value information, a sequence exploration value loss is generated.

12. The method according to claim 11, wherein, The step of determining the data weights corresponding to the second target training data based on the label type corresponding to the cross-request signal label includes: In response to the tag type being either a potential interest tag type or an invalid exploration tag type, the data quality corresponding to the real exposure log sequence is determined; based on the data quality, the data weights corresponding to the second target training data are determined.

13. An item recommendation device, comprising: The acquisition unit is configured to acquire the set of candidate recommended items corresponding to the target recommendation request and the user behavior information corresponding to the target user; The first generation unit is configured to input the candidate recommended item set and the user behavior information into the encoding layer of a pre-trained reordering model to obtain recommended item encoding information, wherein the reordering model further includes: multiple parallel decoding heads; The second generation unit is configured to input the recommended item encoding information into the plurality of parallel decoding heads to output multiple recommended items in parallel multiple times; The third generation unit is configured to generate a sequence of recommended items corresponding to the multiple recommended items using a multi-mechanism sampling strategy; The execution unit is configured to perform item recommendations for the recommended item sequence.

14. An electronic device comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-12.

15. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-12.

16. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-12.

Images