Recommendation: Training methods, devices, equipment, and storage media for content processing models

Abstract

This disclosure provides a training method, apparatus, device, and storage medium for a recommendation content processing model, relating to the fields of artificial intelligence technology such as intelligent recommendation. The specific implementation scheme includes: obtaining a recommendation content sequence, wherein the recommendation content sequence includes multiple recommendation contents sorted by recommendation probability; determining the label of each recommendation content in the recommendation content sequence; and training the recommendation content processing model using the recommendation content sequence and the labels of each recommendation content in the recommendation content sequence. The recommendation content processing model includes a feature representation module and a classifier with multiple classification heads, wherein the number of classification heads is greater than or equal to the number of recommendation contents included in the recommendation content sequence.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, specifically to the field of artificial intelligence technology such as intelligent recommendation, and in particular to a training method, apparatus, device and storage medium for a recommendation content processing model. Background Technology

[0002] In the scenario of multi-question recommendation in intelligent dialogue, the recommendation system can predict and generate questions that the user may ask in the next round of dialogue based on the user's questions and the big model's answers in the previous round of dialogue. Finally, it recommends and displays several possible questions to the user in the client, from which the user can select one and click.

[0003] To improve the accuracy of the questions displayed to users, the multiple questions generated by the recommendation system can be deduplicated and sorted. Based on the sorting results, the highest quality questions can be selected to form a sequence of recommended content, which can then be recommended and displayed to users. Summary of the Invention

[0004] This disclosure provides a training method, apparatus, device, and storage medium for a recommendation content processing model.

[0005] According to one aspect of this disclosure, a method for training a recommendation content processing model is provided, comprising:

[0006] Obtain a sequence of recommended content, which includes multiple recommended content items sorted by recommendation probability from highest to lowest;

[0007] Determine the tags for each recommended content in the recommended content sequence;

[0008] The recommendation content processing model is trained using the recommended content sequence and the labels of each recommended content in the recommended content sequence. The recommended content processing model includes a feature representation module and a classifier with multiple classification heads, the number of which is greater than or equal to the number of recommended content included in the recommended content sequence.

[0009] According to another aspect of this disclosure, a method for processing recommended content is provided, applicable to a large-model-based recommendation system, comprising:

[0010] Obtain a sequence of recommended content; the sequence of recommended content includes multiple recommended items sorted by recommendation probability from highest to lowest;

[0011] A pre-trained recommendation content processing model is used to predict the ranking score of each recommended content in the recommendation content sequence; the recommendation content processing model includes a feature representation module and a classifier with multiple classification heads; the number of multiple classification heads is greater than or equal to the number of recommended content included in the recommendation content sequence; the recommendation content processing model is trained using the aspects described above and any possible implementation method.

[0012] Based on the sorting identifier of each recommended content in the recommended content sequence, a pre-configured target threshold, and the sorting score of each recommended content, the recommended content in the recommended content sequence is reordered. According to another aspect of this disclosure, a training apparatus for a recommended content processing model is provided, comprising:

[0013] The acquisition module is used to acquire a sequence of recommended content, which includes multiple recommended content items sorted by recommendation probability.

[0014] A determining module is used to determine the tags of each recommended content in the recommended content sequence;

[0015] The training module is used to train the recommendation content processing model using the recommendation content sequence and the labels of each recommendation content in the recommendation content sequence. The recommendation content processing model includes a feature representation module and a classifier with multiple classification heads, wherein the number of classification heads is greater than or equal to the number of recommendation content included in the recommendation content sequence.

[0016] According to another aspect of this disclosure, a recommendation content processing apparatus is provided, applied in a large-model-based recommendation system, comprising:

[0017] The sequence acquisition module is used to acquire a sequence of recommended content; the sequence of recommended content includes multiple recommended items sorted by recommendation probability.

[0018] A prediction module is used to predict the ranking score of each recommended content in the recommended content sequence using a pre-trained recommendation content processing model; the recommendation content processing model includes a feature representation module and a classifier with multiple classification heads; the number of multiple classification heads is greater than or equal to the number of recommended content included in the recommended content sequence; the recommendation content processing model is trained using the aspects described above and any possible implementation method.

[0019] The reordering module is used to reorder each of the recommended contents in the recommended content sequence based on the sorting identifier of each of the recommended contents in the recommended content sequence, the pre-configured target threshold, and the sorting score of each of the recommended contents.

[0020] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0021] At least one processor; and

[0022] A memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described above and any possible implementations.

[0024] According to yet another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described above and any possible implementation thereof.

[0025] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the aspects and any possible implementations described above.

[0026] According to the technology disclosed herein, a recommendation content processing model including a feature representation module and a classifier including multiple classification heads can be accurately and effectively trained to obtain a recommendation content processing model that can simultaneously perform effective deduplication and ranking of multiple recommendation contents, thereby effectively improving the accuracy of deduplication and ranking of multiple recommendation contents and the recommendation efficiency.

[0027] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0028] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0029] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0030] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure;

[0031] Figure 3 This is an architecture diagram of a recommendation content processing model provided in this public disclosure;

[0032] Figure 4 This is a schematic diagram according to the third embodiment of the present disclosure;

[0033] Figure 5This is a schematic diagram according to the fourth embodiment of the present disclosure;

[0034] Figure 6 This is a schematic diagram according to the fifth embodiment of the present disclosure;

[0035] Figure 7 This is a schematic diagram according to the sixth embodiment of the present disclosure;

[0036] Figure 8 This is a schematic diagram according to the seventh embodiment of the present disclosure;

[0037] Figure 9 This is a schematic diagram according to the eighth embodiment of the present disclosure;

[0038] Figure 10 This is a schematic diagram according to the ninth embodiment of the present disclosure;

[0039] Figure 11 This is a block diagram of an electronic device used to implement the methods of the embodiments of this disclosure. Detailed Implementation

[0040] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0041] Obviously, the described embodiments are only some, not all, of the embodiments disclosed herein. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0042] It should be noted that the terminal devices involved in the embodiments of this disclosure may include, but are not limited to, smart devices such as mobile phones, personal digital assistants (PDAs), wireless handheld devices, and tablet computers; the display devices may include, but are not limited to, personal computers, televisions, and other devices with display functions.

[0043] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0044] In practical applications, semantic embedding can be used to calculate the semantic vector of each question. Then, the similarity of multiple questions generated by the recommendation system is assessed pairwise, and questions with excessively high similarity are penalized during ranking to achieve deduplication and ensure that the recommended content sequence to the user is the deduplicated and ranked result. However, this approach requires pairwise similarity assessment of multiple questions, which is computationally intensive and leads to low efficiency in deduplication and ranking.

[0045] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure; as shown Figure 1 As shown in the figure, this embodiment provides a training method for a recommendation content processing model, which may specifically include the following steps:

[0046] S101. Obtain the recommended content sequence, which includes multiple recommended content items sorted by recommendation probability.

[0047] In this embodiment, the trained recommendation content processing model can be applied to multi-question recommendation scenarios in intelligent dialogue. Specifically, the multiple recommended content items included in the recommendation content sequence can be multiple questions that the user might ask in the next round, predicted and generated by the recommendation system based on the user's questions in the previous round and the large model's answers. Each question corresponds to one recommended content item. Specifically, while predicting and generating each question, the recommendation system outputs the recommendation probability corresponding to that question. The higher the recommendation probability value, the higher the accuracy of the question and the greater the probability that it will be selected by the user. Therefore, the recommendation probability value can also be considered as an indicator of the quality of the corresponding question; the higher the recommendation probability value, the better the quality of the corresponding recommended content. Based on this, the multiple recommended content items can be sorted according to the order of the recommendation probability of each recommended content item to obtain the recommendation content sequence.

[0048] For example, multiple recommended items in a recommended content sequence can be preferably sorted in descending order of recommendation probability. Of course, alternatively, in some scenarios, they can also be sorted in ascending order of recommendation probability.

[0049] S102. Determine the tags for each recommended content in the recommended content sequence;

[0050] In this embodiment, the tags of each recommended content in the recommended content sequence can identify the theoretical score of each recommended content after deduplication and sorting, which is used as the basis for deduplication and sorting of each recommended content.

[0051] S103. The recommendation content processing model is trained using the recommendation content sequence and the labels of each recommendation content in the recommendation content sequence. The recommendation content processing model includes a feature expression module and a classifier with multiple classification heads. The number of classification heads is greater than or equal to the number of recommendation content included in the recommendation content sequence.

[0052] In this embodiment, the recommended content processing model may include two parts: a feature representation module and a classifier with multiple classification heads. The number of classification heads in the classifier is greater than or equal to the number of recommended content items in the recommended content sequence, ensuring that each recommended content item can be predicted for its ranking score using one classification head.

[0053] In this embodiment, the labels of each recommended content in the recommended content sequence are used as supervised data. Multiple recommendations from the recommended content sequence are used to conduct supervised joint training of the feature representation module and the classifier with multiple classification heads in the recommended content processing model. The training method of this embodiment, by using multiple recommended content items sorted by recommendation probability and their labels, accurately and effectively trains the recommended content processing model, including the feature representation module and the classifier with multiple classification heads. This results in a recommended content processing model that can effectively deduplicate and rank multiple recommended content items simultaneously, thereby effectively improving the efficiency and accuracy of deduplication ranking.

[0054] Figure 2 This is a schematic diagram based on the second embodiment of the present disclosure; as shown Figure 2 As shown, the training method of the recommendation content processing model in this embodiment is based on the above... Figure 1 Based on the technical solutions of the illustrated embodiments, the technical solutions of this disclosure will be described in further detail. For example... Figure 2 As shown, the training method for the recommendation content processing model in this embodiment may specifically include the following steps:

[0055] S201. Obtain the recommended content sequence, which includes multiple recommended content items sorted by recommendation probability from highest to lowest;

[0056] For details, please refer to the above. Figure 1 The specific details of step S101 in the illustrated embodiment will not be repeated here.

[0057] S202. For each recommended content in the recommended content sequence, a pre-trained semantic detection model is used to detect whether there is a semantic problem in the recommended content; if there is, proceed to step S203; if not, proceed to step S204.

[0058] S203. Configure a second value for the recommended content; proceed to step S206.

[0059] For example, the second value can be 0 to impose a ranking penalty on the recommended content, meaning it can be placed at the end of the ranking after rearrangement.

[0060] S204. Using a pre-trained semantic relevance detection model, detect whether the recommended content is semantically related to other recommended content in the recommended content sequence that has a higher recommendation probability than the recommended content. If the recommended content is not semantically related to any of the other recommended content in the recommended content sequence that has a higher recommendation probability than the recommended content, proceed to step S205. Otherwise, if there are other recommended content in the recommended content sequence that has a higher recommendation probability than the recommended content that is semantically related to the recommended content, proceed to step S203.

[0061] In practice, the current recommended content and other recommended content in the recommended content sequence with a higher recommendation probability can be input into the semantic relevance detection model. This model can predict and output the semantic relevance between the two. Additionally, a relevance threshold can be pre-configured based on experience. If the semantic relevance between the two is greater than the threshold, they are considered semantically related; otherwise, they are considered semantically unrelated.

[0062] S205. Set the tag configured for the recommended content to the first value; proceed to step S206.

[0063] For example, the first value can be 1 to reward the recommended content for ranking, meaning it can be placed at the top of the ranking after reordering.

[0064] If multiple recommended items in the recommended content sequence are sorted in descending order of probability, when configuring tags for each recommended item, the tags are configured for each recommended item in the order from front to back in the recommended content sequence, following the steps S202-S205 above.

[0065] In this embodiment, the tags configured for each recommended content can serve as the basis for deduplicating and ranking multiple recommended content. For recommended content with semantic issues or semantic relevance to recommended content with higher recommendation probabilities, a lower tag value, such as 0, is configured so that the recommendation content processing model can learn to penalize such content during training. Conversely, for recommended content without semantic issues or semantic relevance to recommended content with higher recommendation probabilities, a higher tag value, such as 1, is configured so that the recommendation content processing model can learn to reward such content during training.

[0066] Alternatively, in one embodiment of this disclosure, tags can also be manually configured for each recommended content based on the principles of steps S202-S205 described above. Correspondingly, determining the tags for each recommended content in the recommended content sequence specifically includes: obtaining the manually configured tags for each recommended content in the recommended content sequence. Using this method, accurate and reasonable tags for the recommended content can also be obtained.

[0067] In this embodiment, by using the above steps S202-S205, it is possible to accurately and reasonably configure tags for each recommended content in the recommended content sequence.

[0068] S206. Using the feature representation module in the recommendation content processing model, based on the interactive attention mechanism, obtain the feature representation of each recommended content in the recommendation content sequence; the feature representation of each recommended content includes the semantic features of the recommended content, as well as the semantic relationship between the recommended content and other recommended content in the recommendation content sequence; execute step S207;

[0069] The semantic relationship between the recommended content and other recommended content in the recommended content sequence can be semantically related or semantically unrelated. Specifically, it can be calculated using vector similarity based on the semantic features of each recommended content within the feature representation model. For example, if the similarity of the semantic features of two recommended contents is greater than or equal to a preset similarity threshold, they are considered semantically related; otherwise, they are considered semantically unrelated.

[0070] S207. Using the classifier heads, calculate the predicted ranking score of the recommended content based on the feature representation of the corresponding input recommended content; proceed to step S208.

[0071] Specifically, the feature representation of each recommended content in step S206 is input into a classification head of the classifier. In this way, each classification head can calculate the predicted ranking score of the recommended content based on the feature representation of the corresponding processed recommended content.

[0072] S208. Based on the labels of each recommended content in the recommended content sequence and the predicted ranking score of each recommended content, adjust the parameters of the feature expression module and the classifier including multiple classification heads.

[0073] The feature representation module in this embodiment can be implemented using a Bidirectional Encoder Representations from Transformers (BERT) model based on the Transformer encoder architecture. The number of multiple classifiers can be set based on experience or requirements, for example, 8, 10, or 12. In this embodiment, the number of classifiers is greater than or equal to the number of recommended content items included in the recommended content sequence. This ensures that each recommended content item has a corresponding classifier to predict its ranking score. When the number of classifiers exceeds the number of recommended content items included in the recommended content sequence, the classifiers for recommended content items that are not assigned a classifier can be considered to have empty input.

[0074] In this embodiment, the suggestion can be simply referred to as a Sug. When using it, the data of all Sugs can be formatted and input into the feature representation module in the same way to obtain the hidden state corresponding to each Sug. The hidden state of each Sug contains the semantic information of the current Sug and the semantic relationship between the current Sug and other Sugs.

[0075] In this embodiment, the classifier's classifier heads are architecturally independent, and each classifier head only focuses on the semantic information of the Sug at the current position and its relationship with other Sugs.

[0076] When predicting ranking scores, the information of each Sug, i.e., its hidden state, is fed into an independent classification head. If the classification head detects semantic duplication between the current Sug and other Sugs preceding the current position in the recommended content sequence, or if the current Sug has semantic problems, such as logical errors or semantic inconsistencies, the classification head will assign a lower score; otherwise, it will assign a higher score. Ultimately, each classification head will predict and output a score between 0 and 1 for the corresponding Sug, which is the predicted ranking score.

[0077] Then, a loss function Loss can be constructed based on the predicted ranking scores of multiple Sugs predicted by multiple classification heads and the labels configured for multiple Sugs. Finally, the parameters of the feature representation module and the classifier including multiple classification heads are adjusted based on the loss function so that the predicted ranking scores of multiple classification heads for multiple Sugs are consistent with the labels.

[0078] By using multiple training data sets and training the recommendation content processing model according to the steps described above in this embodiment, the trained recommendation content processing model can be equipped with the ability to deduplicate and rank multiple recommendation content sets, thereby effectively improving the recommendation accuracy and efficiency.

[0079] For example, Figure 3 This is an architecture diagram of a recommendation content processing model provided in this public disclosure. For example... Figure 3 As shown, a recommended content processing model with a classifier that includes 10 classification heads is used as an example.

[0080] like Figure 3As shown, if the classifier includes 10 classification heads, the corresponding input data for this recommendation content processing model can contain a maximum of 10 Sugs, such as Sug1, Sug2, ..., Sug10. Sug1 needs to be preceded by a Classification (CLS), and different Sugs need to be separated by a Separator (SEP). If the number of Sugs is less than 10, the Sug data corresponding to the last few Sugs in the input layer will be empty, and the separators between Sugs with empty data will also be left blank. For example, for a classifier with 10 classification heads, if the input Sugs only include 7, the input data starting from the position corresponding to the 8th Sug in subsequent input layers will all be empty.

[0081] Following the input layer, a tokenizer layer can be included to segment the multiple input sugs into tokens. The input information is split into a certain number of tokens, with each sug being a specific token as a separator. Each sug, depending on its character length, is converted into 5-25 tokens.

[0082] Then, after embedding through the Embedding Layer, and further encoded by the Transformer encoder Trm based on the interaction attention mechanism, the hidden state of each token is obtained, where Trm represents the structure of the Transformer encoder. The hidden state is a vector, so each delimiter corresponds to a vector, and each Sug is transformed into a different number of vectors.

[0083] The classifier identifies whether each vector is a delimiter or belongs to a specific sug. In the average pooling unit corresponding to the pooling layer, all vectors corresponding to the same sug are processed by average pooling and transformed into a single vector. Each sug's vector includes not only the semantic features of that sug, but also the semantic relevance of that sug to other input sugs.

[0084] Each classifier head is a fully connected neural network that takes a hidden state vector as input and outputs a 1-dimensional vector. Classifier heads are independent of each other and do not pass information to each other. After the average pooling of each Sug by the classifier head, the resulting vector is transformed into a numerical value. This numerical value is then processed by the sigmoid function to obtain a value between 0 and 1 representing the score of the current Sug.

[0085] When there are fewer than ten Sug classification heads, the data at the tail of the input data corresponding to the Sug position is empty. Since there are no characters between the delimiters and no corresponding hidden state, the classification head at that position receives no information and therefore will not participate in the calculation, resulting in no score. Therefore, in the output of a classifier with multiple classification heads, the Sug score corresponding to an empty Sug will always be 0, and these scores can be discarded directly in practical use.

[0086] In practical use, it is necessary to ensure that the number of delimiters in the input data is always 10, so that the classifier can identify the position corresponding to each Sug. At the same time, the length of the input data also needs to be considered; the total length after delimiters and all Sugs are segmented cannot exceed the context length that BERT can process.

[0087] The recommended content sequence in this embodiment is the training data. During the training process, multiple recommended content sequences need to be collected in advance, and the recommended content processing model is trained in accordance with the method of this embodiment.

[0088] For example, in practical applications, a batch of user queries and the model's answers can be randomly selected. An existing multi-question recommendation system can then be used to obtain multiple Sug items corresponding to this round of dialogue. If the number of Sug items is less than 3, this data is discarded. If the number of Sug items is greater than 10, only the top ten, based on their recommendation probability, can be retained as a sequence of recommended content.

[0089] Furthermore, during the training process of the recommendation content processing model in this embodiment, since the labels corresponding to the recommended content in the recommendation content sequence are strongly dependent on the sorting order, data augmentation is required to prevent the recommendation content processing model from outputting similar scores for data before and after shuffling, which would reduce the model's capabilities. A specific augmentation measure could be to copy multiple recommended content data from the recommendation content sequence, randomly shuffle the copied recommended content, and then train the model.

[0090] The training method of the recommendation content processing model in this embodiment, by adopting the above-described approach, can accurately and effectively train the recommendation content processing model, resulting in a recommendation content processing model that can simultaneously and effectively deduplicate and rank multiple recommendation contents. This effectively improves the efficiency and accuracy of deduplication and ranking of multiple recommendation contents, and further enhances the efficiency of content recommendation in content recommendation scenarios.

[0091] Figure 4 This is a schematic diagram based on the third embodiment of this disclosure; as shown Figure 4As shown, this embodiment provides a method for processing recommended content, applied in a large-model-based recommendation system, which may specifically include the following steps:

[0092] S401. Obtain the recommended content sequence; the recommended content sequence includes multiple recommended content items sorted by recommendation probability from highest to lowest.

[0093] In this embodiment, the recommended content sequence is obtained by a large-model-based recommendation system, which predicts multiple candidate questions for the next round of dialogue based on the user's question and the large-model's answer in the previous round of dialogue. Each candidate question can be used as a recommended content item. For example, preferably, the recommended content items in this embodiment can be arranged in descending order of recommendation probability. Of course, in special scenarios, based on scenario requirements, they can also be arranged in ascending order of recommendation probability.

[0094] S402. A pre-trained recommendation content processing model is used to predict the ranking score of each recommendation content in the recommendation content sequence. The recommendation content processing model includes a feature representation module and a classifier with multiple classification heads. The number of multiple classification heads is greater than or equal to the number of recommendation content included in the recommendation content sequence.

[0095] Specifically, the above methods can be adopted. Figures 1-3 The recommended content processing model in the illustrated embodiment predicts the ranking score of each recommended content in the recommended content sequence.

[0096] Referring to the description in the above embodiments, the recommendation content processing model in this embodiment can fully consider whether there are semantic problems in each recommended content and whether there is semantic overlap between each recommended content and other recommended content with higher recommendation probability in the recommended content sequence when performing deduplication and ranking. It can penalize the recommended content with semantic problems or semantic overlap with other recommended content with higher recommendation probability in the recommended content sequence, and predict a lower ranking score. On the other hand, it can reward the recommended content without semantic problems and without semantic overlap with other recommended content with higher recommendation probability in the recommended content sequence, and predict a higher ranking score.

[0097] S403. Based on the sorting identifier of each recommended content in the recommended content sequence, the pre-configured target threshold, and the sorting score of each recommended content, reorder each recommended content in the recommended content sequence.

[0098] In this embodiment, to further improve the accuracy of the sorting, after obtaining the sorting score of each recommended content, the multiple recommended contents are not directly sorted according to their sorting scores. This would completely ignore the original sorting of the recommended contents in the recommended content sequence, which is very unreasonable. Therefore, in this embodiment, a pre-configured target threshold and the sorting identifier of each recommended content in the recommended content sequence can be further referenced to reasonably adjust the sorting of the multiple recommended contents in the recommended content sequence, so as to further effectively improve the result after the reordering.

[0099] The recommendation content processing method in this embodiment uses a pre-trained recommendation content processing model to predict the ranking score of each recommended content in the recommendation content sequence. This fully considers whether there are semantic problems and semantic repetition problems among multiple recommended content items, effectively improving the accuracy of the ranking score of each recommended content item. Furthermore, based on the ranking identifier of each recommended content in the recommendation content sequence, the pre-configured target threshold, and the ranking score of each recommended content item, the recommended content in the recommendation content sequence is re-ranked, effectively improving the rationality and accuracy of the ranking of each recommended content item in the re-ranked recommendation content sequence.

[0100] Figure 5 This is a schematic diagram based on the fourth embodiment of the present disclosure; as shown Figure 5 As shown, the recommended content processing method in this embodiment, in the above... Figure 4 Based on the technical solutions of the illustrated embodiments, the technical solutions of this disclosure will be further described in more detail. For example... Figure 5 As shown, the recommended content processing method in this embodiment may specifically include the following steps:

[0101] S501. Obtain the recommended content sequence; the recommended content sequence includes multiple recommended content items sorted by recommendation probability.

[0102] In practical applications, a large-model-based recommendation system can predict multiple candidate questions for the next round of dialogue based on the user's question and the model's answer in the previous round of dialogue. Each subsequent question is treated as a recommended content item, and these recommended content items are arranged in descending order of recommendation probability to obtain a recommended content sequence. Further, it checks whether the number of recommended content items in the recommended content sequence is less than 3. If so, the number is low, and deduplication and sorting can be performed using vector comparison or other methods. Otherwise, it checks whether the number of recommended content items in the recommended content sequence is greater than the number of classifier heads, such as 10. If so, only the top 10 recommended content items with the highest recommendation probability are retained, and subsequent recommended content items are discarded. In this embodiment, a classifier with 10 classifier heads is used as an example; in practical applications, the number of classifier heads can also be other, and this is not limited here.

[0103] S502. Using the feature expression module in the recommendation content processing model, obtain the feature expression of each recommendation content. The feature expression of each recommendation content includes the semantic features of the recommendation content and the semantic relationship between the recommendation content and other recommendation content in the recommendation content sequence.

[0104] S503. Using the classifier heads, predict the ranking score of the recommended content based on the feature representation of the corresponding processed recommended content.

[0105] For example, specifically, one can adopt Figure 3 The feature representation model and the 10 classification heads used in the recommended content processing model shown above predict the ranking score of each recommended content. For details, please refer to the relevant descriptions in the above embodiments, which will not be repeated here.

[0106] The feature representation module and multiple classification heads in the classifier of the recommendation content processing model in this embodiment can simultaneously predict the ranking scores of multiple recommended items, achieving re-ranking. This overcomes the problem of low re-ranking efficiency caused by the need to calculate the similarity of multiple recommended items pairwise in existing technologies. Therefore, the recommendation content processing model of this embodiment can effectively improve the efficiency and accuracy of deduplication ranking of multiple recommended items.

[0107] S504. Based on the pre-configured target threshold and the ranking score of each recommended content, divide multiple recommended content in the recommended content sequence into a first group with a ranking score greater than or equal to the target threshold and a second group with a ranking score less than the target threshold.

[0108] S505. Sort the recommended content in the first group and the second group according to the sorting identifier in the recommended content sequence to obtain the rearranged recommended content sequence.

[0109] In practice, it can detect whether the ranking score of each recommended content is greater than or equal to the target threshold, and then divide each recommended content in the recommended content sequence into a first group (high group) with a ranking score greater than the target threshold and a second group (low group) with a ranking score less than the target threshold.

[0110] Then, within the first and second groups, each recommended item is sorted according to its sorting identifier in the recommended content sequence, resulting in a rearranged recommended content sequence. This sorting method can be called order-preserving rearrangement, which means reordering the recommended content sequence while maintaining its original order within each group. This scheme, after fully considering semantic and semantic repetition issues, penalizes recommended content with semantic problems and semantic repetition issues by resetting its order; then, it continues to sort according to the original order in the recommended content sequence, further increasing the rationality and accuracy of the reordering.

[0111] For example, the ranking scores of the 10 recommended items predicted by the recommendation content processing model are as follows: Sug1 has a ranking score of 0.8, Sug2 has a ranking score of 0.6, Sug3 has a ranking score of 0.2, Sug4 has a ranking score of 0.7, Sug5 has a ranking score of 0.3, Sug6 has a ranking score of 0.65, Sug7 has a ranking score of 0.5, Sug8 has a ranking score of 0.55, Sug9 has a ranking score of 0.33, and Sug10 has a ranking score of 0.2.

[0112] If the configured target threshold is 0.35, then the first group will include [Sug1, Sug4, Sug6, Sug2, Sug8 and Sug7] in order of sorted scores, and the second group will include [Sug9, Sug5, Sug3 and Sug10].

[0113] Furthermore, if the order identifiers of the 10 recommended items in the recommended content sequence are: Sug1 is 1, Sug2 is 2, Sug3 is 1, Sug4 is 4, Sug5 is 5, Sug6 is 6, Sug7 is 7, Sug8 is 8, Sug9 is 9, and Sug10 is 10,

[0114] The reordered recommended content sequence can then include [Sug1, Sug2, Sug4, Sug6, Sug7, Sug8, Sug3, Sug5, Sug9, and Sug10], which can penalize recommended content with semantic problems and semantic repetition, effectively improving the rationality and accuracy of the ranking.

[0115] S506. Based on the rearranged recommended content sequence, obtain a preset number of target recommended content items that are ranked first.

[0116] S507. Recommend a preset number of target recommended content.

[0117] Steps S506 and S507 apply the recommended content sequence after order-preserving rearrangement. Since steps S502 and S503 use a recommended content processing model to rearrange the recommended content in the recommended content sequence, and steps S504 and S505 perform order-preserving rearrangement, steps S506 and S507 effectively improve the accuracy and efficiency of content recommendation in application scenarios.

[0118] Specifically, in practical applications, a preset number can be configured according to needs, such as 3 or 5, and then the preset number of target content items ranked at the top of the rearranged recommended content sequence can be obtained for recommendation.

[0119] The recommendation content processing method in this embodiment, by employing a recommendation content processing model, can fully consider the semantic and semantic repetition issues of the recommended content, re-obtain the ranking score of each recommended content in the recommended content sequence, and implement ranking penalties for recommended content with semantic or semantic repetition issues, thereby effectively improving the accuracy of the ranking score of each recommended content. Furthermore, by simultaneously referring to the pre-configured target threshold and the ranking identifier in the recommended content sequence, the recommended content in the recommended content sequence is rearranged in order-preserving manner, which can further effectively improve the rationality and accuracy of the rearranged recommended content sequence.

[0120] By adopting the recommendation content processing method of this embodiment, the quality of the generated recommendation content can be improved without reducing the quality of the initial quality ranking effect. This improves the quality of the recommendation content presented to users, reduces the proportion of semantic problems and semantic duplication problems, and does not prolong the cost and time of business operations. It also improves the usability of the recommendation content output to the client and drives the growth of the system's core indicators.

[0121] Figure 6 This is a schematic diagram according to the fifth embodiment of this disclosure; as shown Figure 6 As shown, the recommended content processing method in this embodiment, in the above... Figure 4 or Figure 5 Based on the technical solutions of the illustrated embodiments, the technical solutions of this disclosure will be further described in more detail. For example... Figure 6 As shown, the recommended content processing method in this embodiment, in the above... Figure 4 or Figure 5 Previously, a scheme for configuring a target threshold could also be included, which could specifically include the following steps:

[0122] S601. Using a recommended content processing model, predict the ranking score of each recommended content in each test recommended content sequence in the pre-built test set.

[0123] In this embodiment, each test recommendation sequence includes multiple test recommendation contents. Specifically, the recommendation content processing model described in the above embodiment can be used to rearrange and score each test recommendation content in each test recommendation sequence after fully considering semantic issues and semantic repetition issues, so as to obtain a more reasonable and accurate ranking score.

[0124] It should be noted that the test recommendation content in the test recommendation sequence of this embodiment is in accordance with the above. Figure 1 or Figure 2 The illustrated embodiment is configured with corresponding labels, such as labels with values ​​of 0 or 1.

[0125] S602, Configure multiple candidate thresholds;

[0126] In this embodiment, during testing, the threshold can be tested in increments of 0.01 to 0.99, i.e., the configured candidate thresholds are 0.01, 0.02, 0.03, ..., 0.99, a total of 99.

[0127] S603. Based on each candidate threshold, the sorting identifier of each test recommendation in each test recommendation content sequence, and the sorting score of each test recommendation content, rearrange the order of multiple test recommendation contents in each test recommendation content sequence in the test set.

[0128] In practical implementation, for each candidate threshold, it can be done according to the above... Figure 5 Steps S504 and S505 of the illustrated embodiment rearrange the recommended content sequence. For details, please refer to the above-mentioned relevant records, which will not be repeated here.

[0129] S604. Calculate the percentage of target test recommendation content sequences that meet the preset conditions after the test set is rearranged under each candidate threshold; the target test recommendation content sequences that meet the preset conditions include the first value for the tags of the preset number of test recommendation content items that are ranked first.

[0130] Referring to the description in the above embodiments, the first value can indicate that the test recommendation content does not have semantic problems and does not have semantic overlap with the test recommendation content with a higher recommendation probability in the corresponding test recommendation content sequence. For example, the first value can be 1. The preset quantity can be the number of recommendation contents that the recommendation system ultimately wants to recommend, such as 3 or 5.

[0131] S605. Obtain the candidate threshold corresponding to the maximum percentage and use it as the target threshold.

[0132] Specifically, under each candidate threshold, the proportion of target test recommendation content sequences whose tags are all the first value, such as 1, after rearrangement for the first preset number of test recommendation content is statistically analyzed; the larger the proportion, the higher the accuracy during testing. Therefore, in this embodiment, the candidate threshold with the largest proportion can be selected as the target threshold. This target threshold can be strongly bound to the parameters of the recommendation content processing model to serve as the optimal target threshold associated with the corresponding recommendation content processing model. Based on this target test threshold, the accuracy of the order-preserving rearrangement in steps S504 and S505 can be effectively improved.

[0133] Alternatively, in practical applications, other methods can be used, or a reasonable target threshold can be configured based on human experience, which is not limited here.

[0134] The recommended content processing method in this embodiment analyzes multiple candidate thresholds in the above manner, which can accurately and reasonably obtain the optimal target threshold, providing a foundation for subsequent order-preserving rearrangement.

[0135] Figure 7 This is a schematic diagram according to the sixth embodiment of this disclosure; as shown Figure 7 As shown, this embodiment provides a training device 700 for a recommendation content processing model, including:

[0136] The acquisition module 701 is used to acquire a recommended content sequence, wherein the recommended content sequence includes multiple recommended content items sorted according to their recommendation probability.

[0137] The determining module 702 is used to determine the tags of each of the recommended contents in the recommended content sequence;

[0138] Training module 703 is used to train the recommendation content processing model using the recommendation content sequence and the labels of each recommendation content in the recommendation content sequence. The recommendation content processing model includes a feature representation module and a classifier with multiple classification heads, the number of classification heads being greater than or equal to the number of recommendation content included in the recommendation content sequence.

[0139] The training device 700 for the recommended content processing model in this embodiment achieves the same implementation principle and technical effect as the above-mentioned related method embodiments by using the above-mentioned modules. For details, please refer to the description of the above-mentioned related method embodiments, which will not be repeated here.

[0140] Figure 8 This is a schematic diagram according to the seventh embodiment of the present disclosure; as shown Figure 8 As shown, the training device 800 for the recommendation content processing model in this embodiment, in the above-mentioned... Figure 7Based on the technical solutions of the illustrated embodiments, the technical solutions of this disclosure will be described in further detail. For example... Figure 8 As shown, the training device 800 for the recommendation content processing model in this embodiment includes the above-mentioned... Figure 7 The modules with the same name and function shown are: acquisition module 801, determination module 802, and training module 803.

[0141] The determining module 802 is used for:

[0142] Obtain manually configured tags for each recommended content in the recommended content sequence.

[0143] Further, optionally, in one embodiment of this disclosure, the determining module 802 is configured to:

[0144] For each recommended content in the recommended content sequence, a pre-trained semantic detection model is used to detect whether there are semantic problems in the recommended content;

[0145] When a semantic problem exists in the recommended content, the tag configured for the recommended content is a second numerical value.

[0146] Further, optionally, in one embodiment of this disclosure, the determining module 802 is configured to:

[0147] In response to the absence of semantic problems in the recommended content, a pre-trained semantic relevance detection model is further employed to detect whether the recommended content is semantically related to other recommended content in the recommended content sequence that has a higher recommendation probability than the recommended content.

[0148] In response to the fact that the recommended content has no semantic relationship with other recommended content in the recommended content sequence that has a higher recommendation probability than the recommended content, the tag configured for the recommended content is a first value;

[0149] In response to the existence of other recommended content that is semantically related to the recommended content among other recommended content with a higher recommendation probability than the recommended content in the recommended content sequence, the tag configured for the current recommended content is a second value.

[0150] Further optional, such as Figure 8 As shown, in one embodiment of this disclosure, the training module 803 includes:

[0151] The deduplication and sorting calculation unit 8031 ​​is used for:

[0152] The feature representation module in the recommended content processing model is used to obtain the feature representation of each recommended content in the recommended content sequence based on the interactive attention mechanism; the feature representation of each recommended content includes the semantic features of the recommended content and the semantic relationship between the recommended content and other recommended content in the recommended content sequence.

[0153] Using each classification head in the classifier, and based on the feature representation of the corresponding recommended content, the predicted ranking score of the recommended content is calculated;

[0154] The parameter adjustment unit 8032 is used to adjust the parameters of the feature expression module and the classifier including multiple classification heads based on the labels of each of the recommended contents in the recommended content sequence and the predicted ranking scores of each of the recommended contents.

[0155] The training device 800 for the recommended content processing model in this embodiment achieves the same implementation principle and technical effect as the above-mentioned related method embodiments by using the above-mentioned modules. For details, please refer to the description of the above-mentioned related method embodiments, which will not be repeated here.

[0156] Figure 9 This is a schematic diagram based on the eighth embodiment of the present disclosure; as shown Figure 9 As shown, this embodiment provides a recommendation content processing device 900, applied in a large-model-based recommendation system, including:

[0157] The sequence acquisition module 901 is used to acquire a sequence of recommended content; the sequence of recommended content includes multiple recommended items sorted according to their recommendation probability.

[0158] The prediction module 902 is used to predict the ranking score of each recommended content in the recommended content sequence using a pre-trained recommendation content processing model; the recommendation content processing model includes a feature representation module and a classifier with multiple classification heads; the number of the multiple classification heads is greater than or equal to the number of recommended content included in the recommended content sequence;

[0159] The reordering module 903 is used to reorder each of the recommended contents in the recommended content sequence based on the sorting identifier of each of the recommended contents in the recommended content sequence, the pre-configured target threshold, and the sorting score of each of the recommended contents.

[0160] The recommended content processing device 900 in this embodiment achieves the same implementation principle and technical effect as the above-mentioned related method embodiments by using the above-mentioned modules. For details, please refer to the description of the above-mentioned related method embodiments, which will not be repeated here.

[0161] Figure 10 This is a schematic diagram based on the ninth embodiment of this disclosure; as shown Figure 10 As shown, the recommended content processing device 1000 of this embodiment, in the above-described... Figure 9 Based on the technical solutions of the illustrated embodiments, the technical solutions of this disclosure will be described in further detail. For example... Figure 10 As shown, the recommended content processing device 1000 in this embodiment includes... Figure 9 The functional modules with the same names shown are: sequence acquisition module 1001, prediction module 1002, and rearrangement module 1003.

[0162] like Figure 10 As shown, in the recommended content processing apparatus 1000 of this embodiment, the prediction module 1002 includes:

[0163] The feature expression unit 10021 is used to obtain the feature expression of each of the recommended contents using the feature expression module in the recommended content processing model. The feature expression of each of the recommended contents includes the semantic features of the recommended content and the semantic relationship between the recommended content and other recommended contents in the recommended content sequence.

[0164] The classification calculation unit 10022 is used to predict the ranking score of the recommended content based on the feature expression of the corresponding processed recommended content by using each of the classification heads in the classifier.

[0165] Further optional, such as Figure 10 As shown, in the recommended content processing apparatus 1000 of this embodiment, the rearrangement module 1003 includes:

[0166] Grouping unit 10031 is used to divide the multiple recommended contents in the recommended content sequence into a first group with a ranking score greater than or equal to the target threshold and a second group with a ranking score less than the target threshold, based on the pre-configured target threshold and the ranking score of each recommended content;

[0167] The order-preserving rearrangement unit 10032 is used to sort each of the recommended content in the first group and the second group according to the sorting identifier in the recommended content sequence to obtain the rearranged recommended content sequence.

[0168] Further optional, such as Figure 10 As shown, the recommended content processing device 1000 in this embodiment further includes:

[0169] Content acquisition module 1004 is used to acquire a preset number of target recommended content items that are ranked first based on the rearranged recommended content sequence.

[0170] The recommendation module 1005 is used to recommend the preset number of target recommended contents.

[0171] Further optional, such as Figure 10 As shown, the recommended content processing device 1000 in this embodiment further includes:

[0172] Configuration module 1006 is used to configure the target threshold.

[0173] Further optionally, in one embodiment of this disclosure, the configuration module 1006 is configured to:

[0174] Using the aforementioned recommendation content processing model, the ranking score of each test recommendation content in each test recommendation content sequence in the pre-constructed test set is predicted;

[0175] Configure multiple candidate thresholds;

[0176] Based on each of the candidate thresholds, the sorting identifier of each of the test recommended content in each of the test recommended content sequences, and the sorting score of each of the test recommended content, the order of multiple test recommended content in each of the test recommended content sequences in the test set is rearranged.

[0177] Calculate the proportion of target test recommendation content sequences that meet preset conditions after the test recommendation content sequence in the test set after being rearranged under each of the candidate thresholds; the target test recommendation content sequence that meets the preset conditions includes a preset number of test recommendation content items whose tags are all first values; the first value indicates that the test recommendation content does not have semantic problems and does not have semantic overlap with the test recommendation content items with higher recommendation probabilities in the test recommendation content sequence to which it belongs;

[0178] The candidate threshold corresponding to the maximum value of the percentage is obtained and used as the target threshold.

[0179] The recommended content processing device 1000 in this embodiment achieves the same implementation principle and technical effect as the above-mentioned related method embodiments by using the above-mentioned modules. For details, please refer to the description of the above-mentioned related method embodiments, which will not be repeated here.

[0180] The acquisition, storage, and application of any type of information, such as user personal information, involved in the technical solutions disclosed herein comply with relevant laws and regulations and do not violate public order and good morals.

[0181] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0182] Figure 11A schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0183] like Figure 11 As shown, device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1102 or a computer program loaded into random access memory (RAM) 1103 from storage unit 1108. The RAM 1103 may also store various programs and data required for the operation of device 1100. The computing unit 1101, ROM 1102, and RAM 1103 are interconnected via bus 1104. Input / output (I / O) interface 1105 is also connected to bus 1104.

[0184] Multiple components in device 1100 are connected to I / O interface 1105, including: input unit 1106, such as keyboard, mouse, etc.; output unit 1107, such as various types of monitors, speakers, etc.; storage unit 1108, such as disk, optical disk, etc.; and communication unit 1109, such as network card, modem, wireless transceiver, etc. Communication unit 1109 allows device 1100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0185] The computing unit 1101 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods and processes described above, such as the methods described above in this disclosure. For example, in some embodiments, the methods described above in this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1100 via ROM 1102 and / or communication unit 1109. When the computer program is loaded into RAM 1103 and executed by the computing unit 1101, one or more steps of the methods described above in this disclosure can be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to perform the methods described above in this disclosure by any other suitable means (e.g., by means of firmware).

[0186] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0187] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0188] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0189] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0190] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0191] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0192] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0193] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for training a recommendation content processing model, comprising: Obtain a sequence of recommended content, which includes multiple recommended content items sorted by recommendation probability from highest to lowest; Determine the tags for each recommended content in the recommended content sequence; The recommendation content processing model is trained using the recommended content sequence and the labels of each recommended content in the recommended content sequence. The recommended content processing model includes a feature representation module and a classifier with multiple classification heads, the number of which is greater than or equal to the number of recommended content included in the recommended content sequence.

2. The method according to claim 1, wherein, The step of determining the tags for each recommended content in the recommended content sequence includes: Obtain manually configured tags for each recommended content in the recommended content sequence.

3. The method according to claim 1, wherein, The step of determining the tags for each recommended content in the recommended content sequence includes: For each recommended content in the recommended content sequence, a pre-trained semantic detection model is used to detect whether there are semantic problems in the recommended content; When a semantic problem exists in the recommended content, the tag configured for the recommended content is a second numerical value.

4. The method according to claim 3, wherein, The step of determining the tags for each recommended content in the recommended content sequence further includes: In response to the absence of semantic problems in the recommended content, a pre-trained semantic relevance detection model is further employed to detect whether the recommended content is semantically related to other recommended content in the recommended content sequence that has a higher recommendation probability than the recommended content. In response to the fact that the recommended content has no semantic relationship with other recommended content in the recommended content sequence that has a higher recommendation probability than the recommended content, the tag configured for the recommended content is a first value; In response to the existence of other recommended content that is semantically related to the recommended content among other recommended content with a higher recommendation probability than the recommended content in the recommended content sequence, the tag configured for the current recommended content is a second value.

5. The method according to claim 1, wherein, The step of training the recommendation content processing model using the recommended content sequence and the tags of each recommended content in the recommended content sequence includes: The feature representation module in the recommended content processing model is used to obtain the feature representation of each recommended content in the recommended content sequence based on the interactive attention mechanism; the feature representation of each recommended content includes the semantic features of the recommended content and the semantic relationship between the recommended content and other recommended content in the recommended content sequence. Using each classification head in the classifier, and based on the feature representation of the corresponding recommended content, the predicted ranking score of the recommended content is calculated; Based on the labels of each recommended content in the recommended content sequence and the predicted ranking scores of each recommended content, the parameters of the feature representation module and the classifier including multiple classification heads are adjusted.

6. A method for processing recommended content, applied in a large-model-based recommendation system, comprising: Get the recommended content sequence; The recommended content sequence includes multiple recommended content items sorted by recommendation probability from highest to lowest. A pre-trained recommendation content processing model is used to predict the ranking score of each recommended content in the recommendation content sequence; the recommendation content processing model includes a feature representation module and a classifier with multiple classification heads; the number of multiple classification heads is greater than or equal to the number of recommended content included in the recommendation content sequence; the recommendation content processing model is a recommendation content processing model trained using any one of the methods described in claims 1-5 above; Based on the sorting identifier of each recommended content in the recommended content sequence, the pre-configured target threshold, and the sorting score of each recommended content, the recommended content in the recommended content sequence is reordered.

7. The method according to claim 6, wherein, The method of using a pre-trained recommendation content processing model to predict the ranking score of each recommended content in the recommendation content sequence includes: The feature representation module in the recommended content processing model is used to obtain the feature representation of each recommended content. The feature representation of each recommended content includes the semantic features of the recommended content and the semantic relationship between the recommended content and other recommended content in the recommended content sequence. Using the classification heads in the classifier, the ranking score of the recommended content is predicted based on the feature representation of the corresponding processed recommended content.

8. The method according to claim 6, wherein, The step of reordering the recommended content in the recommended content sequence based on the sorting identifier of each recommended content in the recommended content sequence, a pre-configured target threshold, and the sorting score of each recommended content includes: Based on the pre-configured target threshold and the ranking score of each recommended content, the multiple recommended content in the recommended content sequence are divided into a first group with a ranking score greater than or equal to the target threshold and a second group with a ranking score less than the target threshold; The recommended content in the first group and the second group are sorted according to the sorting identifier in the recommended content sequence to obtain the rearranged recommended content sequence.

9. The method according to any one of claims 6-8, wherein, After reordering the recommended content in the recommended content sequence based on the sorting identifier of each recommended content in the recommended content sequence, the pre-configured target threshold, and the sorting score of each recommended content, the method further includes: Based on the rearranged recommended content sequence, obtain a preset number of target recommended content items that rank highly. The preset number of target recommended content will be recommended.

10. The method according to any one of claims 6-8, wherein, Before reordering the recommended content in the recommended content sequence based on the sorting identifier of each recommended content in the recommended content sequence, the pre-configured target threshold, and the sorting score of each recommended content, the method further includes: Configure the target threshold.

11. The method according to claim 10, wherein, Configuring the target threshold includes: Using the aforementioned recommendation content processing model, the ranking score of each test recommendation content in each test recommendation content sequence in the pre-constructed test set is predicted; Configure multiple candidate thresholds; Based on each of the candidate thresholds, the sorting identifier of each of the test recommended content in each of the test recommended content sequences, and the sorting score of each of the test recommended content, the order of multiple test recommended content in each of the test recommended content sequences in the test set is rearranged. Calculate the proportion of target test recommendation content sequences that meet preset conditions after the test recommendation content sequence in the test set after being rearranged under each of the candidate thresholds; the target test recommendation content sequence that meets the preset conditions includes a preset number of test recommendation content items whose tags are all first values; the first value indicates that the test recommendation content does not have semantic problems and does not have semantic overlap with the test recommendation content items with higher recommendation probabilities in the test recommendation content sequence to which it belongs; The candidate threshold corresponding to the maximum value of the percentage is obtained and used as the target threshold.

12. A training device for a recommendation content processing model, comprising: The acquisition module is used to acquire a sequence of recommended content, which includes multiple recommended content items sorted by recommendation probability. A determining module is used to determine the tags of each recommended content in the recommended content sequence; The training module is used to train the recommendation content processing model using the recommendation content sequence and the labels of each recommendation content in the recommendation content sequence. The recommendation content processing model includes a feature representation module and a classifier with multiple classification heads, wherein the number of classification heads is greater than or equal to the number of recommendation content included in the recommendation content sequence.

13. A recommendation content processing apparatus, applied in a large-model-based recommendation system, comprising: The sequence acquisition module is used to acquire the recommended content sequence; The recommended content sequence includes multiple recommended content items sorted by recommendation probability from highest to lowest. The prediction module is used to predict the ranking score of each recommended content in the recommended content sequence using a pre-trained recommendation content processing model; the recommendation content processing model includes a feature representation module and a classifier with multiple classification heads; the number of multiple classification heads is greater than or equal to the number of recommended content included in the recommended content sequence; the recommendation content processing model is the recommendation content processing model trained as described in claim 12 above. The reordering module is used to reorder each of the recommended contents in the recommended content sequence based on the sorting identifier of each of the recommended contents in the recommended content sequence, the pre-configured target threshold, and the sorting score of each of the recommended contents.

14. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-11.

15. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-11.

16. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-11.

Images