Content sorting methods, model training methods, devices, media and program products

By predicting the completion rate of target object's process behavior, and utilizing multi-task learning and deep interest network models, the problem that click-through rate models cannot reflect deep interests is solved, thereby improving the accuracy of content ranking and user experience.

CN123114409APending Publication Date: 2026-11-17GUANGZHOU TENCENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202510629153.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-15
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing click-through rate (CTR) models cannot accurately reflect the depth of interest of users in candidate content and are easily affected by external factors such as video titles and cover images, resulting in inaccurate content ranking results and affecting recommendation, recall, and search performance.

Method used

By acquiring relevant features of the current candidate content and historical browsing content of the target object, the completion rate of its process behavior (such as video completion rate) is predicted, and the results are ranked accordingly. Multi-task learning models and deep interest network models are used to dynamically adjust feature weights to reduce the influence of external factors.

Benefits of technology

It improves the accuracy of content ranking results, enhances content recommendation, recall, and search performance, and improves user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123114409A_ABST
    Figure CN123114409A_ABST
Patent Text Reader

Abstract

This application provides a content ranking method, model training method, apparatus, medium, and program product, relating to Internet technology. The content ranking method includes: obtaining relevant content features of N current candidate contents and M historical browsing contents for a target object; for each current candidate content, inputting the relevant content features of the current candidate content and the relevant content features of the M historical browsing contents into a target model to predict the completion rate of the target object's process behavior for the current candidate content; obtaining a score for the current candidate content based on the completion rate of the target object's process behavior for the current candidate content; and ranking the N current candidate contents based on their scores. This improves the accuracy of the content ranking results, thereby enhancing the user experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a content sorting method, a model training method, an apparatus, a medium, and a program product. Background Technology

[0002] In scenarios involving the recommendation, retrieval, and search of content such as videos and text, it is often necessary to first sort multiple candidate contents, and then implement content recommendation, retrieval, and search based on the sorting results of these candidate contents. Therefore, it is crucial to accurately sort these candidate contents.

[0003] In related technologies, ranking models are primarily used to predict click-through rates (CTR) and rank multiple candidate content items. These models predict the click-through rate (CTR) of each candidate item, allowing for ranking based on CTR. However, CTR only reflects an initial interest in candidate content and fails to capture deeper engagement. For example, if an user clicks on a video but only watches it for a few seconds before exiting, the CTR doesn't accurately reflect their depth of interest. Furthermore, CTR is susceptible to external factors such as video titles and thumbnails. These two issues result in inaccurate ranking of candidate content, impacting content recommendation, recall, and search performance, ultimately affecting the user experience. Summary of the Invention

[0004] This application provides a content ranking method, model training method, apparatus, medium, and program product, which can improve the accuracy of content ranking results, thereby improving the content recommendation, recall, or search effects, and ultimately enhancing the user experience.

[0005] In a first aspect, embodiments of this application provide a content ranking method, comprising: obtaining relevant content features of N current candidate contents and M historical browsing contents for a target object, wherein N is a positive integer greater than 1 and M is a positive integer; for each of the N current candidate contents, inputting the relevant content features of the current candidate contents and the relevant content features of the M historical browsing contents into a target model to predict the completion rate of the target object's process behavior for the current candidate contents, and obtaining a score for the current candidate contents based on the completion rate of the target object's process behavior for the current candidate contents; and ranking the N current candidate contents for the target object based on their respective scores.

[0006] Secondly, embodiments of this application provide a model training method, comprising: acquiring relevant content features of N current candidate contents and M historical browsing contents for a target object, wherein N is a positive integer greater than 1 and M is a positive integer; for each of the N current candidate contents, inputting the relevant content features of the current candidate contents and the relevant content features of the M historical browsing contents into a target model to predict the completion rate of the target object's process behavior for the current candidate contents, and calculating the loss of the target model based on the predicted completion rate of the target object's process behavior for the current candidate contents and the actual completion rate of the target object's process behavior for the current candidate contents; and training the target model based on the loss of the target model.

[0007] Thirdly, embodiments of this application provide a content sorting apparatus, comprising: an acquisition module, a prediction module, and a sorting module, wherein the acquisition module is used to acquire relevant content features of N current candidate contents and M historical browsing contents for a target object, wherein N is a positive integer greater than 1 and M is a positive integer; the prediction module is used to input the relevant content features of the current candidate contents and the relevant content features of the M historical browsing contents into a target model for each of the N current candidate contents to predict the completion rate of the target object's process behavior for the current candidate contents, and obtain a score for the current candidate contents based on the completion rate of the target object's process behavior for the current candidate contents; the sorting module is used to sort the N current candidate contents for the target object based on the scores of the N current candidate contents.

[0008] In some implementations, the content sorting device further includes: a determining module, configured to: determine at least one of the following, before the prediction module inputs the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the target model to predict the completion rate of the target object's process behavior for the current candidate content: the average completion rate of the target object's process behavior for historical browsing contents of the target type, and the average completion time of the target object's process behavior for historical browsing contents of the target type; wherein, the historical browsing contents of the target type are historical browsing contents of the M historical browsing contents that belong to the same category as the current candidate content; correspondingly, the prediction module is specifically configured to: input at least one of the following, the average completion rate of the target object's process behavior for historical browsing contents of the target type, and the average completion time of the target object's process behavior for historical browsing contents of the target type, the relevant content features of the current candidate content, and the relevant content features of each of the M historical browsing contents, into the target model to predict the completion rate of the target object's process behavior for the current candidate content.

[0009] In some implementations, the prediction module is specifically used to: input at least one of the following: the average completion rate of the target object's process behavior for the target type of historical browsing content, and the average completion time of the target object's process behavior for the target type of historical browsing content, the relevant content features of the current candidate content, and the relevant content features of each of the M historical browsing contents, into the target model, so as to predict at least one of the click-through rate and browsing time of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content.

[0010] In some implementations, the target model includes a multi-task model and a bias tower; the prediction module is specifically used to: input the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the multi-task model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the first process behavior completion rate of the target object for the current candidate content; input at least one of the average process behavior completion rate of the target object for the target type of historical browsing contents, and the average process behavior completion duration of the target object for the target type of historical browsing contents into the bias tower to predict the second process behavior completion rate of the target object for the current candidate content; and obtain the third process behavior completion rate of the target object for the current candidate content based on the first and second process behavior completion rates of the target object for the current candidate content.

[0011] In some implementations, the multi-task model includes: a deep interest network model, an embedding layer, and a multi-gate hybrid expert model; the prediction module is specifically used to: input the relevant content features of the current candidate content into the embedding layer to obtain a first vector; input the relevant content features of M historical browsing contents into the deep interest network model to obtain a second vector; and input the first vector and the second vector into the multi-gate hybrid expert model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the first process behavior of the target object for the current candidate content.

[0012] In some implementations, the acquisition module is further configured to: acquire at least one of the attribute features and interest features of the target object before the prediction module inputs the relevant content features of the current candidate content into the embedding layer to obtain the first vector; correspondingly, the prediction module is specifically configured to: input at least one of the attribute features and interest features of the target object and the relevant content features of the current candidate content into the embedding layer to obtain the first vector.

[0013] In some implementation methods, the prediction module is specifically used to: calculate the weighted sum of the first process behavior completion rate and the second process behavior completion rate of the target object for the current candidate content; if the weighted sum is greater than a fifth preset threshold, then the third process behavior completion rate of the target object for the current candidate content is determined to be 1; if the weighted sum is less than or equal to the fifth preset threshold, then the third process behavior completion rate of the target object for the current candidate content is determined to be 0.

[0014] In some implementations, the content sorting apparatus further includes a processing module configured to: perform at least one of the following before the acquisition module acquires the relevant content features of each of the N current candidate contents and the relevant content features of each of the M historical browsing contents for the target object:

[0015] Determine the current search method for each of the N current candidate contents;

[0016] Obtain the current search terms corresponding to N current candidate contents and the K historical search terms corresponding to M historical browsing contents, and determine the relevance of the current search terms to the K historical search terms respectively, where K is a positive integer;

[0017] Obtain at least one sub-content from each of the current search terms and M historical browsing contents corresponding to the N current candidate contents. Based on the current search terms and at least one sub-content from each of the M historical browsing contents, determine the relevance between the current search terms and the M historical browsing contents respectively.

[0018] Accordingly, the acquisition module is specifically used to: if at least one of the following is determined to meet the personalized content ranking conditions based on the current search method, the relevance of the current search term to K historical search terms, and the relevance of the current search term to M historical browsing contents, then acquire the relevant content features of each of the N current candidate contents and the relevant content features of each of the M historical browsing contents for the target object.

[0019] In some possible implementations, personalized content sorting criteria include at least one of the following:

[0020] The current search method is a guided search method;

[0021] The maximum relevance of the current search term to K historical search terms is greater than a first preset threshold.

[0022] The maximum relevance of the current search term to M historical browsing items is greater than the second preset threshold.

[0023] In some implementations, the processing module is specifically used to: for each of the M historical browsing contents, calculate the relevance between the current search term and at least one sub-content of the historical browsing contents, and obtain the relevance between the current search term and the historical browsing contents based on the relevance between the current search term and at least one sub-content of the historical browsing contents.

[0024] In some implementations, the processing module is configured to: perform at least one of the following before the acquisition module acquires the relevant content features of each of the N current candidate contents and the relevant content features of each of the M historical browsing contents for the target object:

[0025] Determine the current search method for each of the N current candidate contents;

[0026] Get the current search terms corresponding to N current candidate contents and L historical search terms corresponding to P historical browsing contents. Determine the relevance of the current search terms to the L historical search terms respectively, where L is a positive integer and P historical browsing contents include M historical browsing contents, where P is an integer greater than or equal to M.

[0027] Obtain at least one sub-content of each of the current search terms and P historical browsing contents corresponding to the N current candidate contents. Based on the current search terms and at least one sub-content of each of the P historical browsing contents, determine the relevance of the current search terms to the P historical browsing contents respectively.

[0028] Accordingly, the acquisition module is specifically used to: if at least one of the following is determined to meet the personalized content ranking conditions based on the current search method, the relevance of the current search term to L historical search terms, and the relevance of the current search term to P historical browsing content, then select M historical browsing content from the P historical browsing content, and acquire the relevant content features of each of the N current candidate content and the relevant content features of each of the M historical browsing content for the target object.

[0029] In some possible implementations, personalized content sorting criteria include at least one of the following:

[0030] The current search method is a guided search method;

[0031] The maximum relevance of the current search term to L historical search terms is greater than the third preset threshold.

[0032] The maximum relevance of the current search term to P historical browsing items is greater than the fourth preset threshold.

[0033] In some implementations, the processing module is specifically used to: for each of the P historical browsing contents, calculate the relevance between the current search term and at least one sub-content of the historical browsing contents, and obtain the relevance between the current search term and the historical browsing contents based on the relevance between the current search term and at least one sub-content of the historical browsing contents.

[0034] Fourthly, embodiments of this application provide a model training apparatus, comprising: an acquisition module, a prediction module, a calculation module, and a training module; wherein, the acquisition module is used to acquire relevant content features of N current candidate contents and M historical browsing contents for a target object, where N is a positive integer greater than 1 and M is a positive integer; for each of the N current candidate contents, the prediction module is used to input the relevant content features of the current candidate contents and the relevant content features of the M historical browsing contents into the target model to predict the completion rate of the target object's process behavior for the current candidate contents; the calculation module is used to calculate the loss of the target model based on the predicted completion rate of the target object's process behavior for the current candidate contents and the actual completion rate of the target object's process behavior for the current candidate contents; the training module is used to train the target model based on the loss of the target model.

[0035] In some implementations, the prediction module is specifically used to: input the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the target model to predict at least one of the predicted click-through rate and predicted browsing duration of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content; the calculation module is specifically used to: calculate the loss of the target model based on at least one of the predicted click-through rate and predicted browsing duration of the current candidate content, the predicted completion rate of the target object's process behavior for the current candidate content, and at least one of the actual click-through rate and actual browsing duration of the current candidate content, as well as the actual completion rate of the target object's process behavior for the current candidate content.

[0036] In some implementations, the computation module is specifically used to: calculate the loss of the target model with respect to at least one of the predicted click-through rate and predicted browsing duration of the current candidate content, and at least one of the actual click-through rate and actual browsing duration of the current candidate content; calculate the loss of the target model with respect to the process behavior completion rate based on the predicted process behavior completion rate of the target object for the current candidate content, and the actual process behavior completion rate of the target object for the current candidate content; and calculate the loss of the target model with respect to at least one of the loss of the target model with respect to the click-through rate and browsing duration, and the loss of the target model with respect to the process behavior completion rate.

[0037] In some implementations, the computation module is specifically used to: calculate the Kolb-Leibler divergence loss of the target model with respect to the process behavior completion rate, based on the target object's predicted process behavior completion rate for the current candidate content and the target object's actual process behavior completion rate for the current candidate content.

[0038] Fifthly, embodiments of this application provide an electronic device, including: a processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to perform the methods as described in the first to second aspects or their respective implementations.

[0039] In a sixth aspect, embodiments of this application provide a computer-readable storage medium for storing a computer program that causes a computer to perform the methods described in the first to second aspects or their respective implementations.

[0040] In a seventh aspect, embodiments of this application provide a computer program product including computer program instructions that cause a computer to perform the methods as described in the first to second aspects or their respective implementations.

[0041] Eighthly, embodiments of this application provide a computer program that causes a computer to perform the methods described in the first to second aspects or their respective implementations.

[0042] The technical solution provided in this application can predict the completion rate of the target object's process behavior for each current candidate content, such as the video completion rate. On the one hand, the process behavior completion rate can reflect the object's deep interest in the candidate content. On the other hand, the process behavior completion rate is not easily affected by external factors, such as video title and cover. Therefore, it can improve the accuracy of content ranking results, thereby improving the content recommendation, recall or search effect, and thus improving the object's experience. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram of the MMoE model.

[0045] Figure 2 This is a schematic diagram of a system architecture according to an embodiment of this application;

[0046] Figure 3A and Figure 3BThis is a screenshot of the video search interface.

[0047] Figure 4 A flowchart illustrating a content sorting method provided in this application embodiment;

[0048] Figure 5 This is a schematic diagram of data processing for a target model provided in an embodiment of this application;

[0049] Figure 6 This is a schematic diagram of data processing for another target model provided in an embodiment of this application;

[0050] Figure 7 This is a schematic diagram illustrating the data processing of another target model provided in an embodiment of this application;

[0051] Figure 8 This is a schematic diagram illustrating the data processing of another target model provided in an embodiment of this application;

[0052] Figure 9 This is a schematic diagram of data processing for a target model provided in an embodiment of this application;

[0053] Figure 10 This is a schematic diagram of data processing for another target model provided in an embodiment of this application;

[0054] Figure 11 This is a schematic diagram illustrating the data processing of another target model provided in an embodiment of this application;

[0055] Figure 12 This is a schematic diagram illustrating the data processing of another target model provided in an embodiment of this application;

[0056] Figure 13 This is a schematic diagram of data processing for a target model provided in an embodiment of this application;

[0057] Figure 14 This is a schematic diagram of data processing for another target model provided in an embodiment of this application;

[0058] Figure 15 This is a schematic diagram illustrating the data processing of another target model provided in an embodiment of this application;

[0059] Figure 16 This is a schematic diagram illustrating the data processing of another target model provided in an embodiment of this application;

[0060] Figure 17 A flowchart illustrating a model training method provided in this application embodiment;

[0061] Figure 18 A schematic diagram illustrating the sample sources and applications provided in the embodiments of this application;

[0062] Figure 19 A schematic diagram of a content sorting device 1900 provided in an embodiment of this application;

[0063] Figure 20 A schematic diagram of a model training device 2000 provided in an embodiment of this application;

[0064] Figure 21 This is a schematic block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0067] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0068] Before introducing the technical solution of this application, the relevant knowledge of this application will be explained below:

[0069] I. Multi-Task Learning (MTL) and Multi-gate Mixture of Experts (MMoE) Models:

[0070] With the rapid development of big data and artificial intelligence technologies, multi-task learning is increasingly widely used in the field of machine learning. Multi-task learning aims to improve the generalization ability of a model, reduce overfitting, and accelerate the convergence speed of the model by learning multiple related tasks simultaneously.

[0071] The MMOE model is a multi-task learning model. Figure 1 A schematic diagram of the MMoE model is shown below. Figure 1 As shown, the MMoE model mainly consists of three parts:

[0072] 1. Expert Networks: Each expert network is an independent neural network. Different expert networks are used to capture information from the input vector from different dimensions.

[0073] 2. Gating Networks: Each task corresponds to a gating network, which determines which expert networks are more important for the current task. The output of the gating network is a weight vector used to weight the outputs of the expert networks.

[0074] 3. Task-Specific Tower Network: This is the final independent network corresponding to each task. The input of each task-specific independent network is a weighted combination of the outputs of the expert network, and the output is the prediction result for that task.

[0075] The core principle of the MMoE model lies in selecting the appropriate expert network for each task through a gating network. During training, the gating network learns to assign different weights to each task. Simultaneously, the expert networks themselves are shared, allowing multiple tasks to share some common features, thereby improving the model's generalization ability.

[0076] II. The Deep Interest Network (DIN) model is a deep learning model used in recommender systems. The core of DIN lies in dynamically modeling user interests through an attention mechanism. It addresses the problem of overly static object interest representations in traditional recommender models, dynamically adjusting the weights of historical behavioral features of objects based on different candidate content.

[0077] The third layer, the embedding layer, is the core component in deep learning that transforms discrete data (usually categorical variables or symbol sequences) into continuous vector representations. It captures the semantic relationships between data through dense vectors.

[0078] IV. The Kullback-Leibler (KL) divergence is a measure of the difference between two probability distributions. In information theory, it is also called relative entropy, and it is used to express the amount of information lost when we approximate the true distribution (P) with a distribution (Q).

[0079] The technical problem to be solved, the inventive concept, and the system architecture of this application will be described below:

[0080] As mentioned above, in related technologies, ranking multiple candidate contents is mainly done using ranking models to predict click-through rates (CTR). However, on the one hand, CTR only reflects an audience's initial interest in the candidate contents and cannot reflect their deeper interest. On the other hand, CTR is easily affected by external factors such as video titles and thumbnails. Due to these two reasons, the ranking results of candidate contents are not accurate enough, thus affecting content recommendation, recall, or search performance, and consequently impacting the user experience.

[0081] To address the aforementioned technical issues, this application proposes a target model for ranking each current candidate content. This target model primarily predicts the completion rate of a target object's actions regarding each current candidate content, such as video completion rate. On one hand, this completion rate reflects the object's deep interest in the candidate content; on the other hand, it is less susceptible to external factors such as video titles and covers. Therefore, it can improve the accuracy of content ranking results, thereby enhancing content recommendation, recall, or search performance, and ultimately improving the user experience.

[0082] In some embodiments of this application, the system architecture is as follows: Figure 2 As shown.

[0083] Figure 2 This is a schematic diagram of a system architecture according to an embodiment of this application. The system architecture includes user equipment 201, data acquisition equipment 202, training equipment 203, execution equipment 204, database 205, and content library 206.

[0084] The data acquisition device 202 is used to read training samples from the content library 206 and store the read training samples in the database 205. The training samples involved in this embodiment include: relevant content features of each current candidate content for the target object, relevant content features of each of M historical browsing contents, and the actual process behavior completion rate of the target object for the current candidate content.

[0085] The training device 203 trains the target model based on the training samples maintained in the database 205, so that the trained target model can predict the completion rate of the target object's process behavior for the current candidate content, or can predict the completion rate of the target object's process behavior for the current candidate content, and obtain the score of the current candidate content based on the completion rate of the target object's process behavior for the current candidate content.

[0086] Additionally, refer to Figure 2 The execution device 204 is equipped with an I / O interface 207 for data interaction with external devices. For example, it can receive the current search term sent by the user device 201 through the I / O interface. The execution device 204 or its internal computing module 208 can first obtain N current candidate contents and determine M historical browsing contents based on the current search term, where N is a positive integer greater than 1 and M is a positive integer. It then obtains the relevant content features of each of the N current candidate contents and the M historical browsing contents for the target object. Further, for each of the N current candidate contents, the computing module 208 uses a trained target model to process the relevant content features of the current candidate contents and the M historical browsing contents, outputting either the completion rate of the target object's process behavior for that current candidate content, or the score of the current candidate content, which is based on the completion rate of the target object's process behavior for the current candidate content. Further, the execution device 204 or its internal computing module 208 can obtain a ranking result of the N current candidate contents based on their respective scores, and send the ranking result of the N current candidate contents to the user device 201 via an I / O interface, so that the user device 201 can display the N current candidate contents based on the ranking result.

[0087] User equipment 201 may include mobile phones, tablets, laptops, handheld computers, mobile internet devices (MIDs), desktop computers, or other terminal devices with browser installation capabilities.

[0088] The execution device 204 can be a server.

[0089] For example, the server can be a rack server, blade server, tower server, or cabinet server, etc. The server can be a standalone test server or a test server cluster composed of multiple test servers.

[0090] In this embodiment, the execution device 204 is connected to the user equipment 201 via a network. The network can be an intranet, the Internet, Global System for Mobile Communications (GSM), Wideband Code Division Multiple Access (WCDMA), 4G, 5G, Bluetooth, Wi-Fi, or a voice communication network, or other wireless or wired networks.

[0091] It should be noted that, Figure 2 This is merely a schematic diagram of a system architecture provided in an embodiment of this application. The positional relationships between the devices, components, modules, etc., shown in the figure do not constitute any limitation. In some embodiments, the data acquisition device 202, user device 201, training device 203, and execution device 204 may be the same device. The database 205 may be distributed across one server or multiple servers, and the content library 206 may be distributed across one server or multiple servers.

[0092] It should be understood that, Figure 2 This is just one system architecture for content search scenarios. It is worth mentioning that the embodiments of this application are not only applicable to content search scenarios, but also to content recommendation scenarios, content retrieval scenarios, etc., but are not limited thereto.

[0093] Among them, the content search scenario is a scenario where multiple candidate contents are sorted based on the search terms entered by the object, and multiple candidate contents are displayed to the object based on the sorting results. Figure 3A and Figure 3B This is a screenshot of the video search interface, such as... Figure 3A As shown, on a certain video interface, the user can click the search icon to enter... Figure 3B The video interface shown allows users to enter search terms in the search box to view videos they are interested in.

[0094] Content recommendation scenarios are based on an object's historical behavior data, such as videos watched in the past and videos searched in the past. Multiple candidate contents are ranked, and these candidate contents are recommended to the object in turn based on the ranking results.

[0095] Content retrieval is often an intermediate step in content search or recommendation. For example, after a user enters a search term, the server first needs to retrieve a portion of content from a massive amount of data based on that search term, then rank the retrieved content, and finally display this retrieved content to the user based on the ranking results. Alternatively, when recommending content to a user based on their historical behavior data, the server can first retrieve a portion of content from a massive amount of data based on that historical behavior data, then rank the retrieved content, and finally recommend this retrieved content to the user sequentially based on the ranking results.

[0096] Before introducing the embodiments of this application, it should be noted that all information involved in the embodiments of this application, including: the attribute characteristics, interest characteristics, browsing history, etc. of the object, are authorized by the object or fully authorized by all parties, and the collection, use and processing of relevant information comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0097] The technical solution of this application will be described in detail below:

[0098] Figure 4 A flowchart illustrating a content sorting method provided in this application embodiment, the method being executed by an execution device, such as... Figure 2 The execution device 204 in the context can be a server, but is not limited to this, such as... Figure 4 As shown, the method may include:

[0099] S410: Obtain the relevant content features of N current candidate contents and M historical browsing contents for the target object, where N is a positive integer greater than 1 and M is a positive integer;

[0100] It should be understood that the content involved in the embodiments of this application can be unimodal or multimodal. For example, it can be unimodal content such as video, image, and text, or multimodal content such as image + text or text + video. In short, the embodiments of this application do not limit the form of the content.

[0101] It should be understood that in content search scenarios, the current candidate content can be content recalled based on the current search term. In content recommendation scenarios, the current candidate content can be content recalled based on the target audience's historical behavioral data. In content recall scenarios, the current candidate content can be any content from a massive amount of data.

[0102] In some implementations, the execution device can select historical browsing content within a fixed time window from all historical browsing content of the target object, wherein the historical browsing content within the fixed time window is the M historical browsing content. In the content search scenario, the end time of the fixed time window can be the current search time, and the size of the fixed time window can be a preset size, such as 24 hours.

[0103] In some implementations, the execution device can select historical browsing content within a fixed time window from all historical browsing content of the target object. The historical browsing content within the fixed time window consists of the P historical browsing items, and the M historical browsing items are selected from the P historical browsing items. For example, in a content search scenario, any one of the M historical browsing items can be a historical browsing item with a high relevance to the current search term.

[0104] In some possible implementations, for any one of the N current candidate contents, the relevant content features of the current candidate content include at least one of the following: the features of the current candidate content, and the features of the relevant content of the current candidate content.

[0105] In some possible implementations, the features of the current candidate content include at least one of the following, but are not limited to: the title of the current candidate content, cover optical character recognition (OCR), content OCR, tag, entity, and category.

[0106] In some possible implementations, the relevant content of the current candidate content includes, but is not limited to, the current search term corresponding to the current candidate content, wherein the current candidate content is obtained by searching the current search term.

[0107] In some possible implementations, the features of the relevant content of the current candidate content include at least one of the following, but are not limited to: the content features of the relevant content and the intent features of the relevant content. For example, in a video search scenario, the features of the current search term corresponding to the current candidate video include: the content features and intent features of the current search term.

[0108] In some possible implementations, for any one of the M historical browsing contents, the relevant content characteristics of the historical browsing content include at least one of the following: the characteristics of the historical browsing content, and the characteristics of the relevant content of the historical browsing content.

[0109] In some possible implementations, the characteristics of the historical browsing content include at least one of the following, but are not limited to: the title, cover OCR, content OCR, tag, entity, category of the historical browsing content, the time interval between the browsing time of the historical browsing content and the current search time, the position information of the historical browsing content in the sequence of M historical browsing content, and the browsing duration of the historical browsing content.

[0110] In some possible implementations, the relevant content of the browsing history includes, but is not limited to, the historical search terms corresponding to the browsing history. The browsing history is obtained by searching for these historical search terms.

[0111] In some possible implementations, the characteristics of the relevant content of the historical browsing content include at least one of the following, but are not limited to: the content characteristics of the historical browsing content, the intent characteristics of the relevant content, and the time interval between the historical search time corresponding to the relevant content and the current search time. For example, in a video search scenario, the characteristics of the historical search terms corresponding to the historical browsing videos include: the content characteristics of the historical search terms, the intent characteristics, and the time interval between the search time corresponding to the historical search terms and the current search time.

[0112] In some implementations, the execution device can obtain the relevant content features of N current candidate contents and M historical browsing contents for the target object from a local database or logs.

[0113] In some implementations, the execution device can extract features from N current candidate contents and M historical browsing contents to obtain the relevant content features of each of the N current candidate contents and the relevant content features of each of the M historical browsing contents.

[0114] It should be understood that the embodiments of this application do not limit the methods for obtaining the relevant content features of each of the N current candidate contents and the relevant content features of each of the M historical browsing contents.

[0115] As described above, the embodiments of this application can be applied to content search scenarios, content retrieval scenarios, and content recommendation scenarios. In the content search scenario, the execution device can first determine whether personalized content sorting is required. If personalized content sorting is required, then steps S410 to S430 are executed; otherwise, personalized content sorting is not performed. Personalized content sorting refers to a content sorting method tailored to a specific object.

[0116] The following section will explain the personalized content sorting methods in content search scenarios:

[0117] In some implementations, prior to S410, the content sorting method further includes at least one of the following:

[0118] S400-a: Determine the current search method corresponding to the N current candidate contents;

[0119] S400-b: Obtain the current search terms corresponding to N current candidate contents and the K historical search terms corresponding to M historical browsing contents, and determine the relevance of the current search terms to the K historical search terms respectively, where K is a positive integer;

[0120] S400-c: Obtain at least one sub-content of each of the current search terms and M historical browsing contents corresponding to the N current candidate contents, and determine the relevance of the current search terms to the M historical browsing contents based on the current search terms and at least one sub-content of each of the M historical browsing contents;

[0121] Accordingly, S410 may include:

[0122] S410-a: If at least one of the following conditions is met: the relevance of the current search method, the relevance of the current search term to K historical search terms, and the relevance of the current search term to M historical browsing contents, then obtain the relevant content features of the N current candidate contents and the relevant content features of the M historical browsing contents for the target object.

[0123] The following is an explanation of S400-a:

[0124] It should be understood that the N current candidate contents are candidate contents obtained through the current search method.

[0125] In some possible implementations, the current search methods include, but are not limited to: guided search methods and unguided search methods.

[0126] In some possible implementations, the guided search method includes, but is not limited to: comment-jump search, search based on search box recommended search terms, and search based on background text.

[0127] It should be understood that comment-based search refers to a search method based on search terms carried in comments of previously viewed content. For example, if a comment on a target's previously viewed video contains the search term "seagull," the search performed by the target clicking on that search term can be described as a comment-based search.

[0128] It should be understood that the search terms automatically included in the search box are often based on the target audience's historical behavioral data, such as their browsing history and search terms. Therefore, a search method that recommends search terms based on the search box is also a guided search method. For example, if the target audience previously viewed a short video of the song "Seagull" on a video app, the video search box can automatically include the search term "seagull." The search performed by the target audience clicking on this search term can be considered a search method that recommends search terms based on the search box.

[0129] It should be understood that background keywords are usually located below the search box and are often based on the target audience's historical behavioral data, such as their browsing history and search terms. Therefore, background keyword-based search is also a guided search method. For example, if the target audience previously viewed a short video of the song "Seagull" on a video app, the search term "seagull" can be automatically displayed below the video search box. The search performed by the target audience clicking on this search term can be considered a background keyword-based search.

[0130] As the name suggests, non-guided search is a search method other than guided search. In other words, non-guided search refers to a search method in which the target object is searched entirely autonomously by entering search terms without any active intervention or prompts from the system.

[0131] The following explanation addresses S400-b and S400-c:

[0132] It should be understood that the N current candidate items are obtained by searching for the current search term. Each of the N current candidate items corresponds to one current search term.

[0133] It should be understood that the M historical browsing items were obtained by searching for historical search terms. Specifically, the M historical browsing items correspond to K historical search terms.

[0134] It should be understood that in S400-b and S400-c, the execution device can select historical browsing content within a fixed time window from all historical browsing content of the target object, wherein the historical browsing content within the fixed time window is the M historical browsing content.

[0135] It should be understood that the relevance of the current search term to each of the K historical search terms can also be referred to as the similarity between the current search term and each of the K historical search terms.

[0136] It should be understood that the current search term and the K historical search terms can be converted into corresponding embeddings, which are often in vector form and are also called embedding vectors. Based on this, in this embodiment of the application, the execution device can calculate the similarity between the current search term and the K historical search terms according to the vector similarity calculation method.

[0137] In some feasible methods, vector similarity calculation includes, but is not limited to, the following: Cosine Similarity, Euclidean Distance, Manhattan Distance, Chebyshev Distance, Jaccard Similarity Coefficient, Hamming Distance, Pearson Correlation Coefficient, and Kendall's Tau Coefficient.

[0138] In some possible implementations, for any one of the M historical browsing contents, the historical browsing content includes at least one of the following sub-contents, but is not limited to: the title of the historical browsing content, cover OCR, content OCR, and tag.

[0139] In some implementations, the execution device determines the relevance of the current search term to each of the M historical browsing contents based on the current search term and at least one sub-content of each of the M historical browsing contents. This includes: for each of the M historical browsing contents, calculating the relevance of the current search term to at least one sub-content of that historical browsing content, and obtaining the relevance of the current search term to that historical browsing content based on the relevance of the current search term to at least one sub-content of that historical browsing content.

[0140] It should be understood that, for any sub-content within the historical browsing content, the relevance between the current search term and that sub-content is also referred to as the similarity between the current search term and that sub-content. The relevance between the current search term and the historical browsing content is also referred to as the similarity between the current search term and the historical browsing content. The method for calculating similarity can be referred to above, and will not be repeated here in the embodiments of this application.

[0141] In some implementation methods, the execution device obtains the relevance between the current search term and the historical browsing content based on the relevance between the current search term and at least one sub-content in the historical browsing content, including: the execution device performs a weighted summation of the relevance between the current search term and at least one sub-content in the historical browsing content to obtain the relevance between the current search term and the historical browsing content.

[0142] In some implementation methods, the execution device obtains the relevance between the current search term and the historical browsing content based on the relevance between the current search term and at least one sub-content in the historical browsing content, including: the execution device performs a weighted average of the relevance between the current search term and at least one sub-content in the historical browsing content to obtain the relevance between the current search term and the historical browsing content.

[0143] It should be understood that the embodiments of this application do not limit how the relevance between the current search term and the historical browsing content is obtained based on the relevance between the current search term and at least one sub-content in the historical browsing content.

[0144] In some implementations, the execution device determines the relevance of the current search term to each of the M historical browsing contents based on the current search term and at least one sub-content of each of the M historical browsing contents. This includes: for each of the M historical browsing contents, the execution device inputs the current search term and at least one sub-content of that historical browsing content into a neural network model to obtain the relevance of the current search term to that historical browsing content.

[0145] It should be understood that the embodiments of this application do not limit the calculation method of the relevance between the current search term and M historical browsing contents.

[0146] The following is a detailed explanation of S410-a:

[0147] In some possible implementations, personalized content sorting criteria include at least one of the following, but are not limited to:

[0148] The current search method is a guided search method;

[0149] The maximum relevance of the current search term to K historical search terms is greater than a first preset threshold.

[0150] The maximum relevance of the current search term to M historical browsing items is greater than the second preset threshold.

[0151] In some possible implementations, personalized content sorting criteria include at least one of the following, but are not limited to:

[0152] The current search method is a guided search method;

[0153] The average relevance of the current search term to K historical search terms is greater than the corresponding preset threshold.

[0154] The average relevance of the current search term to M historical browsing items is greater than the corresponding preset threshold.

[0155] In some possible implementations, personalized content sorting criteria include at least one of the following, but are not limited to:

[0156] The current search method is a guided search method;

[0157] The minimum relevance of the current search term to K historical search terms is greater than the corresponding preset threshold.

[0158] The minimum relevance of the current search term to M historical browsing items is greater than the corresponding preset threshold.

[0159] It should be understood that the embodiments of this application do not impose restrictions on the sorting conditions for personalized content.

[0160] In some implementations, prior to S410, the content sorting method further includes at least one of the following:

[0161] S400-a': Determines the current search method corresponding to the N current candidate contents;

[0162] S400-b': Obtain the current search terms corresponding to N current candidate contents and the L historical search terms corresponding to P historical browsing contents, and determine the relevance of the current search terms to the L historical search terms respectively, where L is a positive integer, and the P historical browsing contents include: M historical browsing contents, where P is an integer greater than or equal to M;

[0163] S400-c': Obtain at least one sub-content of each of the current search term and P historical browsing content corresponding to the N current candidate contents, and determine the relevance of the current search term to the P historical browsing content based on the current search term and at least one sub-content of each of the P historical browsing content;

[0164] Accordingly, S410 may include:

[0165] S410-a': If at least one of the following is determined to satisfy the personalized content ranking conditions: the relevance of the current search term to L historical search terms, and the relevance of the current search term to P historical browsing content, then select M historical browsing content from the P historical browsing content, and obtain the relevant content features of each of the N current candidate content and the relevant content features of each of the M historical browsing content for the target object.

[0166] It should be understood that the explanation of S400-a' can be found in the explanation of S400-a, and will not be repeated in the embodiments of this application.

[0167] The following explanation addresses S400-b' and S400-c':

[0168] It should be understood that in S400-b',

[0169] In some implementations, the execution device can select historical browsing content within a fixed time window from all historical browsing content of the target object, wherein the historical browsing content within the fixed time window is the P historical browsing content. Further, the execution device can select the M historical browsing content from the P historical browsing content.

[0170] It should be understood that the N current candidate items are obtained by searching for the current search term. Each of the N current candidate items corresponds to one current search term.

[0171] It should be understood that the P pieces of historical browsing content were obtained by searching for historical search terms. Specifically, the P pieces of historical browsing content correspond to L historical search terms.

[0172] It should be understood that in S400-b' and S400-c', the execution device can select historical browsing content within a fixed time window from all historical browsing content of the target object, wherein the historical browsing content within the fixed time window is the P historical browsing content, and the M historical browsing content is the historical browsing content selected from the P historical browsing content.

[0173] It should be understood that the relevance of the current search term to each of the L historical search terms can also be referred to as the similarity between the current search term and each of the L historical search terms.

[0174] It should be understood that the current search term and the L historical search terms can be converted into corresponding embeddings, which are often in vector form and are also called embedding vectors. Based on this, in this embodiment, the execution device can calculate the similarity between the current search term and the L historical search terms according to the vector similarity calculation method.

[0175] It should be understood that the method for calculating vector similarity can be referred to above, and will not be repeated in this embodiment.

[0176] In some possible implementations, for any one of P historical browsing contents, the historical browsing content includes at least one of the following sub-contents, but is not limited to: the title of the historical browsing content, cover OCR, content OCR, and tag.

[0177] In some implementations, the execution device determines the relevance of the current search term to each of the P historical browsing contents based on the current search term and at least one sub-content of each of the P historical browsing contents. This includes: for each of the P historical browsing contents, calculating the relevance of the current search term to at least one sub-content of that historical browsing content, and obtaining the relevance of the current search term to that historical browsing content based on the relevance of the current search term to at least one sub-content of that historical browsing content.

[0178] In some implementations, the execution device determines the relevance between the current search term and each of the P historical browsing contents based on the current search term and at least one sub-content of each of the P historical browsing contents. This includes: for each of the P historical browsing contents, the execution device inputs the current search term and at least one sub-content of that historical browsing content into a neural network model to obtain the relevance between the current search term and that historical browsing content.

[0179] It should be understood that the calculation method for the relevance of the current search term to the historical browsing content can be referred to above, and will not be repeated in this embodiment.

[0180] The following is a detailed explanation of S410-a':

[0181] In some possible implementations, personalized content sorting criteria include at least one of the following, but are not limited to:

[0182] The current search method is a guided search method;

[0183] The maximum relevance of the current search term to L historical search terms is greater than the third preset threshold.

[0184] The maximum relevance of the current search term to P historical browsing items is greater than the fourth preset threshold.

[0185] In some possible implementations, personalized content sorting criteria include at least one of the following, but are not limited to:

[0186] The current search method is a guided search method;

[0187] The average relevance of the current search term to L historical search terms is greater than the corresponding preset threshold.

[0188] The average relevance of the current search term to P historical browsing items is greater than the corresponding preset threshold.

[0189] In some possible implementations, personalized content sorting criteria include at least one of the following, but are not limited to:

[0190] The current search method is a guided search method;

[0191] The minimum relevance of the current search term to L historical search terms is greater than the corresponding preset threshold.

[0192] The minimum relevance between the current search term and P historical browsing items is greater than the corresponding preset threshold.

[0193] It should be understood that the embodiments of this application do not impose restrictions on the sorting conditions for personalized content.

[0194] In some possible implementations, the execution device selects M historical browsing items from P historical browsing items, including: the execution device selects M historical browsing items from P historical browsing items based on the relevance of the current search term to L historical search terms, and / or the relevance of the current search term to P historical browsing items.

[0195] In some possible implementations, the execution device selects M historical browsing contents from P historical browsing contents based on the relevance of the current search term to L historical search terms. This includes: the execution device selecting a first historical search term from the L historical search terms whose relevance to the current search term is greater than a corresponding threshold, and selecting the historical browsing contents corresponding to the first historical search term from the P historical browsing contents as the M historical browsing contents.

[0196] It should be understood that the historical browsing content corresponding to the first historical search term refers to the historical browsing content found through that first historical search term.

[0197] In some possible implementations, the execution device selects M historical browsing contents from P historical browsing contents based on the relevance of the current search term to L historical search terms. This includes: the execution device selecting the second historical search term with the highest relevance to the current search term from the L historical search terms, and selecting the historical browsing contents corresponding to the second historical term from the P historical browsing contents as the M historical browsing contents.

[0198] It should be understood that the historical browsing content corresponding to the second historical search term refers to the historical browsing content found through that second historical search term.

[0199] In some implementations, the execution device selects M historical browsing contents from the P historical browsing contents based on the relevance of the current search term to the P historical browsing contents. This includes: the execution device selecting historical browsing contents from the P historical browsing contents whose relevance to the current search term is greater than a corresponding threshold as the M historical browsing contents.

[0200] In some possible implementations, the execution device selects M historical browsing contents from the P historical browsing contents based on the relevance of the current search term to the P historical browsing contents, including: the execution device selects the historical browsing contents with the highest relevance to the current search term from the P historical browsing contents as the M historical browsing contents.

[0201] In some possible implementations, the execution device, based on the relevance of the current search term to L historical search terms and the relevance of the current search term to P historical browsing content, includes: the execution device selecting a third historical search term from the L historical search terms whose relevance to the current search term is greater than a corresponding threshold, selecting the historical browsing content corresponding to the third historical search term from the P historical browsing content, and selecting the historical browsing content corresponding to the third historical search term whose relevance to the current search term is greater than a corresponding threshold from the historical browsing content, as the M historical browsing content.

[0202] In some implementation methods, the execution device, based on the relevance of the current search term to L historical search terms and the relevance of the current search term to P historical browsing content, includes: the execution device selecting a third historical search term from the L historical search terms whose relevance to the current search term is greater than a corresponding threshold, selecting the historical browsing content corresponding to the third historical search term from the P historical browsing content, and selecting the historical browsing content with the highest relevance to the current search term from the historical browsing content corresponding to the third historical search term, as the M historical browsing content.

[0203] In some possible implementations, the execution device, based on the relevance of the current search term to L historical search terms and the relevance of the current search term to P historical browsing content, includes: the execution device selecting the fourth historical search term with the highest relevance to the current search term from the L historical search terms, selecting the historical browsing content corresponding to the fourth historical search term from the P historical browsing content, and selecting the historical browsing content with a relevance greater than a corresponding threshold from the historical browsing content corresponding to the fourth historical search term, as the M historical browsing content.

[0204] In some possible implementations, the execution device, based on the relevance of the current search term to L historical search terms and the relevance of the current search term to P historical browsing content, includes: the execution device selecting the fourth historical search term with the highest relevance to the current search term from the L historical search terms, selecting the historical browsing content corresponding to the fourth historical search term from the P historical browsing content, and selecting the historical browsing content with the highest relevance to the current search term from the historical browsing content corresponding to the fourth historical search term, as the M historical browsing content.

[0205] As can be seen from the above, the M historical browsing contents are often those that are highly relevant to the current search term among the P historical browsing contents. Based on this, when performing personalized content sorting for the target object based on the M historical browsing contents, the content sorting accuracy can be further improved.

[0206] As mentioned above, for content recommendation scenarios, it can be determined whether personalized content sorting is needed. If personalized content sorting is needed, then it is executed, i.e., S410 to S430. If personalized content sorting is not needed, non-personalized content sorting can be performed. However, personalized content sorting usually requires certain computing and storage resources. Therefore, by determining whether personalized content sorting is needed, both personalized content sorting and resource conservation can be achieved.

[0207] It's worth noting that for content recommendation scenarios, the execution device can directly execute steps S410 to S430 without needing to determine whether personalized content sorting is required. Furthermore, for content retrieval and content recommendation scenarios, the execution device can also directly execute steps S410 to S430 without needing to determine whether personalized content sorting is required.

[0208] S420: For each of the N current candidate contents, input the relevant content features of the current candidate contents and the relevant content features of the M historical browsing contents into the target model to predict the completion rate of the target object's process behavior for the current candidate contents, and obtain the score of the current candidate contents based on the completion rate of the target object's process behavior for the current candidate contents.

[0209] In some implementation methods, the relevant content features of the current candidate content and the relevant content features of each of the M historical browsed content are input into the target model to predict the completion rate of the target object's process behavior towards the current candidate content. This includes: inputting the relevant content features of the current candidate content and the relevant content features of each of the M historical browsed content into the target model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the target object's process behavior towards the current candidate content. Accordingly, based on the completion rate of the target object's process behavior towards the current candidate content, a score for the current candidate content is obtained, including: obtaining a score for the current candidate content based on at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the target object's process behavior towards the current candidate content.

[0210] In other words, the target model can predict the completion rate of the target object's process behavior towards the current candidate content, or it can predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the target object's process behavior towards the current candidate content. If the target model is used to predict the completion rate of the target object's process behavior towards the current candidate content, then the target model includes a single-task model. If the target model is used to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the target object's process behavior towards the current candidate content, then the target model includes a multi-task model. The following mainly uses the multi-task model as an example to illustrate the embodiments of this application. In fact, the difference between the single-task model and the multi-task model lies in the number of tasks processed. Therefore, the content sorting method based on the single-task model can refer to the content sorting method based on the multi-task model. The embodiments of this application will not elaborate on this further.

[0211] It should be understood that the completion rate of the target object's process behavior for the current candidate content refers to the completion rate of the target object's process behavior for the current candidate content. This process behavior includes any of the following, but is not limited to: playing, browsing, and purchasing. For example, in video search, video recall, or video recommendation scenarios, the completion rate of the process behavior for the current candidate video can be the video's completion rate, i.e., the playback completion rate. For example, in a web search scenario, the completion rate of the process behavior for the current candidate webpage can be the webpage's browsing rate, i.e., the browsing completion rate. For example, on an e-commerce platform, the completion rate of the process behavior for the current product can be the product's purchase rate, where purchasing behavior can be considered a process behavior from click to purchase, and this purchase rate can also be called the purchase conversion rate or purchase completion rate.

[0212] In some implementations, the target model is used to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content.

[0213] In some implementations, the target model is used to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content, and to obtain a score for the current candidate content based on at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content.

[0214] In some implementations, the target model includes a multi-task model.

[0215] It should be noted that multi-task models can predict multiple task metrics simultaneously. By sharing underlying features, i.e., expert networks, and interactions between tasks, the model's generalization ability and ranking accuracy can be improved.

[0216] In some implementations, the multitasking model includes: the DIN model, the embedding layer, and the MMoE model.

[0217] In some implementations, the multi-task model includes the DIN model and the MMoE model.

[0218] In some implementations, the MMoE model in the multi-task model can be replaced with a shared-bottom model or a one-gate mixture of experts, but is not limited to these.

[0219] It should be understood that the embodiments of this application do not limit the structure of the multi-task model and the target model.

[0220] In some implementations, where the target model includes a multi-task model, and the multi-task model includes a DIN model, an embedding layer, and an MMoE model, the execution device inputs relevant content features of the current candidate content and relevant content features of each of the M historical browsing contents into the target model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content. This includes: the execution device inputting relevant content features of the current candidate content into the embedding layer to obtain a first vector; inputting relevant content features of each of the M historical browsing contents into the DIN model to obtain a second vector; and inputting the first and second vectors into the MMoE model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content. For details, please refer to [link to relevant documentation]. Figure 5 ,in, Figure 5 Take the output click-through rate, browsing time, and process completion rate as examples.

[0221] In some implementations, before the execution device inputs the relevant content features of the current candidate content into the embedding layer to obtain the first vector, the method further includes: the execution device acquiring at least one of the attribute features and interest features of the target object; correspondingly, the execution device inputting the relevant content features of the current candidate content into the embedding layer to obtain the first vector includes: the execution device inputting at least one of the attribute features and interest features of the target object and the relevant content features of the current candidate content into the embedding layer to obtain the first vector, as detailed in [reference needed]. Figure 6 ,in, Figure 6 Take the output click-through rate, browsing time, and process completion rate as examples.

[0222] In some possible implementations, the attribute characteristics of the target object include at least one of the following, but are not limited to: the target object's identity (ID), age, address, and gender.

[0223] In some possible implementations, the target audience’s interest characteristics include at least one of the following, but are not limited to: the category of content the target audience is interested in, the account, and the tag.

[0224] In some implementations, where the target model includes a multi-task model, and the multi-task model includes a DIN model and an MMoE model, the execution device inputs the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the target model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content. This includes: the execution device inputting the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the DIN model to obtain a third vector; and inputting the third vector into the MMoE model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content. (See also...) Figure 7 ,in, Figure 7 Take the output click-through rate, browsing time, and process completion rate as examples.

[0225] In some implementations, before the execution device inputs the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the DIN model to obtain the third vector, the method further includes: the execution device acquiring at least one of the attribute features and interest features of the target object; correspondingly, the execution device inputting the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the DIN model to obtain the third vector includes: the execution device inputting at least one of the attribute features and interest features of the target object, the relevant content features of the current candidate content, and the relevant content features of each of the M historical browsing contents into the DIN model to obtain the third vector, as detailed in [reference needed]. Figure 8 ,in, Figure 8 Take the output click-through rate, browsing time, and process completion rate as examples.

[0226] It should be understood that shorter current candidate content tends to have a higher completion rate, while longer current candidate content tends to have a lower completion rate. For example, short videos tend to have a higher completion rate, while long videos tend to have a lower completion rate. Therefore, there may be a certain ranking bias when sorting content, such as shorter content being ranked higher and longer content being ranked lower. Based on this, embodiments of this application propose to eliminate or reduce ranking bias by using at least one of the average completion rate of the target object's historical browsing content for the target type and the average completion time of the target object's historical browsing content for the target type. This will be explained below:

[0227] In some implementations, before the execution device inputs relevant content features of the current candidate content and relevant content features of each of the M historical browsing contents into the target model to predict at least one of the click-through rate and browsing duration of the current candidate content, and the completion rate of the target object's process behavior for the current candidate content, the method further includes: the execution device determining at least one of the average completion rate of the target object's process behavior for the target type of historical browsing content, and the average completion duration of the target object's process behavior for the target type of historical browsing content; wherein, the target type of historical browsing content is historical browsing content of the same category as the current candidate content among the M historical browsing contents; correspondingly, the execution device inputs relevant content features of the current candidate content into the target model to predict at least one of the following: click-through rate and browsing duration of the current candidate content, and the completion rate of the target object's process behavior for the current candidate content. The relevant content features and the relevant content features of each of the M historical browsing contents are input into the target model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content. This includes: the execution device inputting at least one of the average completion rate of the target object's process behavior for the target type of historical browsing contents, and the average completion duration of the target object's process behavior for the target type of historical browsing contents, the relevant content features of the current candidate content, and the relevant content features of each of the M historical browsing contents into the target model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content.

[0228] In some implementations, the target model includes a multi-task model.

[0229] In some implementations, the multitasking model includes: the DIN model, the embedding layer, and the MMoE model.

[0230] In some implementations, the multi-task model includes the DIN model and the MMoE model.

[0231] In some implementations, the MMoE model in the multi-task model can be replaced with a shared-bottom model or a one-gate mixture of experts, but is not limited to these.

[0232] In some implementations, where the target model includes a multi-task model, and the multi-task model includes a DIN model, an embedding layer, and an MMoE model, the execution device inputs at least one of the following: the average completion rate of the target object's process behavior for historical browsing content of the target type, and the average completion time of the target object's process behavior for historical browsing content of the target type, along with relevant content features of the current candidate content and relevant content features of each of the M historical browsing contents, into the target model to predict at least one of the click-through rate and browsing time of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content. This includes: the execution device inputting at least one of the following: the average completion rate of the target object's process behavior for historical browsing content of the target type, and the average completion time of the target object's process behavior for historical browsing content of the target type, along with relevant content features of the current candidate content, into the embedding layer to obtain a fourth vector; inputting the relevant content features of each of the M historical browsing contents into the DIN model to obtain a second vector; and inputting the fourth vector and the second vector into the MMoE model to predict at least one of the click-through rate and browsing time of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content. For details, please refer to [link to relevant documentation]. Figure 9 ,in, Figure 9 Take the output click-through rate, browsing time, and process completion rate as examples.

[0233] In some implementations, before the execution device inputs at least one of the following—the average completion rate of the target object's historical browsing behavior for the target type and the average completion time of the target object's historical browsing behavior for the target type and the relevant content features of the current candidate content—into the embedding layer to obtain the fourth vector, the process further includes: the execution device acquiring at least one of the target object's attribute features and interest features; correspondingly, the execution device inputs at least one of the following—the average completion rate of the target object's historical browsing behavior for the target type and the average completion time of the target object's historical browsing behavior for the target type and the relevant content features of the current candidate content—into the embedding layer to obtain the fourth vector, including: the execution device inputting at least one of the target object's attribute features and interest features, the average completion rate of the target object's historical browsing behavior for the target type and the average completion time of the target object's historical browsing behavior for the target type and the relevant content features of the current candidate content—into the embedding layer to obtain the fourth vector. See [link to details]. Figure 10 ,in, Figure 10 Take the output click-through rate, browsing time, and process completion rate as examples.

[0234] In some possible implementations, where the target model includes a multi-task model, and the multi-task model includes a DIN model and an MMoE model, the execution device inputs at least one of the following: the average completion rate of the target object's process behavior for historical browsing content of the target type, and the average completion time of the target object's process behavior for historical browsing content of the target type, along with relevant content features of the current candidate content and relevant content features of each of the M historical browsing contents, into the target model to predict at least one of the click-through rate and browsing time of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content. This includes: the execution device inputting at least one of the following: the average completion rate of the target object's process behavior for historical browsing content of the target type, and the average completion time of the target object's process behavior for historical browsing content of the target type, along with relevant content features of the current candidate content and relevant content features of each of the M historical browsing contents, into the DIN model to obtain a fifth vector; and inputting the fifth vector into the MMoE model to predict at least one of the click-through rate and browsing time of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content. (See also...) Figure 11 ,in, Figure 11 Take the output click-through rate, browsing time, and process completion rate as examples.

[0235] In some implementations, before the execution device inputs at least one of the following into the DIN model—the average completion rate of the target object's process behavior for historical browsing content of the target type, and the average completion time of the target object's process behavior for historical browsing content of the target type, along with the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents—to obtain the fifth vector, the method further includes: the execution device acquiring at least one of the target object's attribute features and interest features; correspondingly, the execution device inputting at least one of the following into the DIN model to obtain the fifth vector includes: the execution device inputting at least one of the following into the DIN model—the average completion rate of the target object's process behavior for historical browsing content of the target type, and the average completion time of the target object's process behavior for historical browsing content of the target type, along with the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents—to obtain the fifth vector. For details, please refer to [link to relevant documentation]. Figure 12 ,in, Figure 12 Take the output click-through rate, browsing time, and process completion rate as examples.

[0236] This application proposes that ranking deviation can be eliminated or reduced using a deviation tower, which will be explained below:

[0237] In some implementations, the target model includes a multi-task model and a deviation tower.

[0238] In some possible implementations, the deviation tower is a neural network model, but is not limited thereto.

[0239] In some possible implementations, where the target model includes a multi-task model and a deviation tower, the execution device inputs at least one of the following: the average completion rate of the target object's process behavior for historical browsing content of the target type, and the average completion time of the target object's process behavior for historical browsing content of the target type; relevant content features of the current candidate content; and relevant content features of each of the M historical browsing contents. This is to predict at least one of the click-through rate and browsing time of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content. This includes: the execution device inputting relevant content features of the current candidate content and relevant content features of each of the M historical browsing contents into the multi-task model to predict at least one of the click-through rate and browsing time of the current candidate content, as well as the first process behavior completion rate of the target object for the current candidate content; inputting at least one of the average completion rate of the target object's process behavior for historical browsing content of the target type, and the average completion time of the target object's process behavior for historical browsing content of the target type into the deviation tower to predict the second process behavior completion rate of the target object for the current candidate content; and obtaining the third process behavior completion rate of the target object for the current candidate content based on the first and second process behavior completion rates of the target object for the current candidate content.

[0240] In some possible implementations, the completion rate of the third process behavior of the target object for the current candidate content may be obtained through the multi-task model or may not be obtained through the multi-task model. This application embodiment does not limit this.

[0241] In some implementations, the multitasking model includes: the DIN model, the embedding layer, and the MMoE model.

[0242] In some implementations, the multi-task model includes the DIN model and the MMoE model.

[0243] In some implementations, the MMoE model in the multi-task model can be replaced with a shared-bottom model or a one-gate mixture of experts, but is not limited to these.

[0244] In some implementable methods, where the target model includes a multi-task model and a deviation tower, and the multi-task model includes a DIN model, an embedding layer, and an MMoE model, the execution device inputs the relevant content features of the current candidate content and the relevant content features of each of the M historical browsed content into the multi-task model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the first process behavior of the target object for the current candidate content. This includes: the execution device inputting the relevant content features of the current candidate content into the embedding layer to obtain a first vector; inputting the relevant content features of each of the M historical browsed content into a deep interest network model to obtain a second vector; and inputting the first vector and the second vector into a multi-gate hybrid expert model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the first process behavior of the target object for the current candidate content. For details, please refer to [link to relevant documentation]. Figure 13 ,in, Figure 13 Take the output click-through rate, browsing time, and process completion rate as examples.

[0245] In some implementations, before the execution device inputs the relevant content features of the current candidate content into the embedding layer to obtain the first vector, the method further includes: the execution device acquiring at least one of the attribute features and interest features of the target object; correspondingly, the execution device inputting the relevant content features of the current candidate content into the embedding layer to obtain the first vector includes: the execution device inputting at least one of the attribute features and interest features of the target object and the relevant content features of the current candidate content into the embedding layer to obtain the first vector. See details [link to specific implementations]. Figure 14 ,in, Figure 14 Take the output click-through rate, browsing time, and process completion rate as examples.

[0246] In some implementable methods, where the target model includes a multi-task model and a deviation tower, and the multi-task model includes a DIN model and an MMoE model, the execution device inputs the relevant content features of the current candidate content and the relevant content features of each of the M historical browsed content into the multi-task model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the first process behavior of the target object for the current candidate content. This includes: the execution device inputting the relevant content features of the current candidate content and the relevant content features of each of the M historical browsed content into a deep interest network model to obtain a third vector; and inputting the third vector into the MMoE model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the first process behavior of the target object for the current candidate content. For details, please refer to [link to relevant documentation]. Figure 15 ,in, Figure 15 Take the output click-through rate, browsing time, and process completion rate as examples.

[0247] In some implementations, before the execution device inputs the relevant content features of the current candidate content and the relevant content features of each of the M historical browsed content items into the deep interest network model to obtain the third vector, the method further includes: the execution device acquiring at least one of the attribute features and interest features of the target object; correspondingly, the execution device inputting the relevant content features of the current candidate content and the relevant content features of each of the M historical browsed content items into the deep interest network model to obtain the third vector includes: the execution device inputting at least one of the attribute features and interest features of the target object, as well as the relevant content features of the current candidate content and the relevant content features of each of the M historical browsed content items into the deep interest network model to obtain the third vector. See details [link to specific implementation]. Figure 16 ,in, Figure 16 Take the output click-through rate, browsing time, and process completion rate as examples.

[0248] This application learns ranking bias by introducing the average completion rate and / or average completion time of the target object's historical browsing content of the current candidate content, and constructing a bias tower. This allows for accurate prediction of the target object's completion rate of the current candidate content, thus reducing the impact of ranking bias. By combining the output of the bias tower with the output of the multi-task model, ranking bias can be effectively eliminated, further improving ranking accuracy.

[0249] In some possible implementations, the execution device obtains the third process behavior completion rate of the target object for the current candidate content based on the first process behavior completion rate and the second process behavior completion rate of the target object for the current candidate content. This includes: the execution device calculating the weighted sum of the first process behavior completion rate and the second process behavior completion rate of the target object for the current candidate content; if the weighted sum is greater than a fifth preset threshold, then the third process behavior completion rate of the target object for the current candidate content is determined to be 1; if the weighted sum is less than or equal to the fifth preset threshold, then the third process behavior completion rate of the target object for the current candidate content is determined to be 0.

[0250] In some possible implementations, the execution device obtains a third process behavior completion rate of the target object for the current candidate content based on the first process behavior completion rate and the second process behavior completion rate of the target object for the current candidate content. This includes: the execution device calculating a weighted sum of the first process behavior completion rate and the second process behavior completion rate of the target object for the current candidate content, as the third process behavior completion rate of the target object for the current candidate content.

[0251] It should be understood that the embodiments of this application do not limit the calculation method of the completion rate of the third process behavior.

[0252] In some implementations, the execution device obtains a score for the current candidate content based on at least one of the click-through rate and browsing duration of the current candidate content, and the completion rate of the target object's process behavior towards the current candidate content. This includes: the execution device determining the scores corresponding to at least one of the click-through rate and browsing duration of the current candidate content and the completion rate of the target object's process behavior towards the current candidate content, and performing a weighted summation of the scores corresponding to at least one of the click-through rate and browsing duration of the current candidate content and the completion rate of the target object's process behavior towards the current candidate content to obtain the score for the current candidate content.

[0253] In some implementations, the execution device obtains a score for the current candidate content based on at least one of the click-through rate and browsing duration of the current candidate content, and the completion rate of the target object's process behavior towards the current candidate content. This includes: the execution device determining the scores corresponding to at least one of the click-through rate and browsing duration of the current candidate content and the completion rate of the target object's process behavior towards the current candidate content, and performing a weighted average of the scores corresponding to at least one of the click-through rate and browsing duration of the current candidate content and the completion rate of the target object's process behavior towards the current candidate content to obtain the score for the current candidate content.

[0254] In some implementations, the execution device can compare either click-through rate (CTR) or browsing duration with a corresponding threshold to obtain a score. For example, assuming a CTR of 0.5 and a threshold of 0.7, the CTR is less than the threshold, therefore the score is 0. As another example, assuming a browsing duration of 2 seconds and a threshold of 1 second, the browsing duration is greater than the threshold, therefore the score is 1.

[0255] It should be understood that if the completion rate of the target object's process behavior for the current candidate content is 0 or 1, the execution device can directly use this completion rate as the score for that item. If the completion rate of the target object's process behavior for the current candidate content is between 0 and 1, the execution device can compare that item with the corresponding threshold to obtain the corresponding score. For example, assuming the completion rate is 0.5 and the corresponding threshold is 0.7, since the completion rate is less than the corresponding threshold, its corresponding score is 0.

[0256] In summary, this application does not limit the scoring method for the current candidate content.

[0257] S430: Based on the scores of the N current candidate contents, sort the N current candidate contents for the target object.

[0258] It should be understood that the content sorting method performed for the target audience is the personalized content sorting method.

[0259] In some implementations, the execution device may send a sorting result of N current candidate contents to the terminal device, so that the terminal device can display N current candidate contents based on the sorting result.

[0260] In some implementations, the execution device can recommend N current candidate contents to the target object in sequence based on the ranking results of the N current candidate contents.

[0261] In some implementations, the execution device can perform content retrieval based on the ranking of N current candidate contents.

[0262] This application provides a content ranking method, comprising: acquiring relevant content features of N current candidate contents and M historical browsing contents for a target object, where N is a positive integer greater than 1 and M is a positive integer; for each of the N current candidate contents, inputting the relevant content features of the current candidate content and the relevant content features of the M historical browsing contents into a target model to predict the completion rate of the target object's process behavior towards the current candidate content, and obtaining a score for the current candidate content based on the completion rate of the target object's process behavior towards the current candidate content; and ranking the N current candidate contents for the target object based on their respective scores. Since the process behavior completion rate can reflect the object's deep interest in the candidate content, and is not easily affected by external factors such as video titles and cover images, the accuracy of the content ranking results can be improved, thereby improving the content recommendation, recall, or search effects, and ultimately enhancing the user experience.

[0263] Figure 17 A flowchart illustrating a model training method provided in this application embodiment. This method can be executed by a training device, for example, Figure 2 The training device 203 in the example can be a server, but is not limited to it, such as... Figure 17 As shown, the method may include:

[0264] S1710: Obtain the relevant content features of N current candidate contents and M historical browsing contents for the target object, where N is a positive integer greater than 1 and M is a positive integer;

[0265] It should be understood that the explanation of S1710 can be found in the explanation of S410, and will not be repeated here.

[0266] S1720: For each of the N current candidate contents, input the relevant content features of the current candidate contents and the relevant content features of the M historical browsing contents into the target model to predict the completion rate of the target object's process behavior for the current candidate contents. Based on the predicted completion rate of the target object's process behavior for the current candidate contents and the actual completion rate of the target object's process behavior for the current candidate contents, calculate the loss of the target model.

[0267] In some implementations, the step of inputting the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the target model to predict the completion rate of the target object's process behavior towards the current candidate content includes: inputting the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the target model to predict at least one of the predicted click-through rate and predicted browsing duration of the current candidate content, as well as the completion rate of the target object's process behavior towards the current candidate content; the step of calculating the loss of the target model based on the predicted completion rate of the target object's process behavior towards the current candidate content and the actual completion rate of the target object's process behavior towards the current candidate content includes: calculating the loss of the target model based on at least one of the predicted click-through rate and predicted browsing duration of the current candidate content, the predicted completion rate of the target object's process behavior towards the current candidate content, the actual click-through rate and actual browsing duration of the current candidate content, as well as the actual completion rate of the target object's process behavior towards the current candidate content.

[0268] It should be understood that, for each of the N current candidate contents, the relevant content features of that current candidate content, the relevant content features of each of the M historical browsed contents, at least one of the actual click-through rate and actual browsing duration of the current candidate content, and the actual completion rate of the target object's process behavior regarding the current candidate content can constitute a training sample. The training device can use supervised training to train the target model, where at least one of the actual click-through rate and actual browsing duration of the current candidate content, and the actual completion rate of the target object's process behavior regarding the current candidate content, can serve as sample labels.

[0269] The following is combined Figure 18 The sources and applications of training samples are illustrated by example:

[0270] For example, Figure 18 The illustrations provided for the sample sources and applications in the embodiments of this application are as follows: Figure 18 As shown, training samples can come from content search systems and content recommendation systems.

[0271] In a content search system, when an object searches for content using search terms, the system can obtain relevant content features of the current candidate content and relevant content features of multiple historical browsing content. These features are stored in key-value pairs. In addition, at least one of the actual click-through rate and actual browsing duration of the current candidate content, as well as the actual process completion rate of the target object for the current candidate content, can be obtained from the search behavior log as sample labels. Finally, the training samples consisting of the relevant content features of the current candidate content, the relevant content features of multiple historical browsing content, at least one of the actual click-through rate and actual browsing duration of the current candidate content, and the actual process completion rate of the target object for the current candidate content are stored in the database.

[0272] In a content recommendation system, an active object table can be retrieved from search behavior logs. Then, for each object in the active object table, relevant content features of the current candidate content and relevant content features of multiple historical browsing contents can be obtained and stored in key-value format. At the same time, at least one of the actual click-through rate and actual browsing duration of the current candidate content, as well as the actual process behavior completion rate of the target object for the current candidate content, can be obtained from the recommendation system backend as sample labels. Finally, the training samples consisting of the relevant content features of the current candidate content, the relevant content features of multiple historical browsing contents, at least one of the actual click-through rate and actual browsing duration of the current candidate content, and the actual process behavior completion rate of the target object for the current candidate content are stored in the database.

[0273] Furthermore, the target model can be trained using training samples from the database.

[0274] In some implementations, the training device calculates the loss of the target model based on at least one of the predicted click-through rate and predicted browsing duration of the current candidate content, and the predicted process behavior completion rate of the target object for the current candidate content, and at least one of the actual click-through rate and actual browsing duration of the current candidate content, and the actual process behavior completion rate of the target object for the current candidate content. This includes: the training device calculating the loss of the target model with respect to at least one of the click-through rate and browsing duration based on at least one of the predicted click-through rate and predicted browsing duration of the current candidate content, and at least one of the actual click-through rate and actual browsing duration of the current candidate content; calculating the loss of the target model with respect to the process behavior completion rate based on the predicted process behavior completion rate of the target object for the current candidate content, and the actual process behavior completion rate of the target object for the current candidate content; and calculating the loss of the target model with respect to at least one of the click-through rate and browsing duration, and the loss of the target model with respect to the process behavior completion rate.

[0275] In some feasible implementations, the training device can calculate KL divergence loss, cross-entropy loss, etc., based on the predicted click-through rate and actual click-through rate of the current candidate content, but is not limited to these.

[0276] In some implementations, the training device can calculate KL divergence loss, cross-entropy loss, etc., based on the predicted and actual browsing durations of the current candidate content, but is not limited to these.

[0277] In some possible implementations, the training device can calculate the KL divergence loss, cross-entropy loss, etc. of the target model with respect to the process behavior completion rate based on the target object's predicted process behavior completion rate for the current candidate content and the target object's actual process behavior completion rate for the current candidate content, but is not limited to these.

[0278] In some implementations, the training device calculates the loss of the target model based on the loss of the target model with respect to at least one of click-through rate and browsing duration, and the loss of the target model with respect to process behavior completion rate, including: the training device performs a weighted summation of the loss of the target model with respect to at least one of click-through rate and browsing duration, and the loss of the target model with respect to process behavior completion rate, to obtain the loss of the target model.

[0279] In some implementations, the training device calculates the loss of the target model based on the loss of the target model with respect to at least one of click-through rate and browsing duration, and the loss of the target model with respect to process behavior completion rate, including: the training device summing the loss of the target model with respect to at least one of click-through rate and browsing duration, and the loss of the target model with respect to process behavior completion rate, to obtain the loss of the target model.

[0280] It should be understood that the embodiments of this application do not limit the method of loss calculation for the target model.

[0281] It should be understood that the results of the target model and its data processing methods can be referred to above, and the embodiments of this application will not be repeated here.

[0282] S1730: Train the target model based on the loss of the target model.

[0283] Furthermore, the training device can train the target model based on the loss of the target model until the number of training iterations reaches a preset number, or the training stopping condition such as loss convergence is met, at which point training stops.

[0284] This application provides a model training method, including: acquiring relevant content features of N current candidate contents and M historical browsing contents for a target object, where N is a positive integer greater than 1 and M is a positive integer; for each of the N current candidate contents, inputting the relevant content features of the current candidate content and the relevant content features of the M historical browsing contents into a target model to predict the completion rate of the target object's process behavior for the current candidate content, and calculating the loss of the target model based on the predicted completion rate of the target object's process behavior for the current candidate content and the actual completion rate of the target object's process behavior for the current candidate content; and training the target model based on the loss of the target model. Since the completion rate of the process behavior is considered during model training, the target model can achieve content ranking based on the completion rate of the process behavior.

[0285] The preferred embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this application, various simple modifications can be made to the technical solutions of this application, and these simple modifications all fall within the protection scope of this application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this application will not describe the various possible combinations separately. Furthermore, various different embodiments of this application can also be arbitrarily combined, as long as they do not violate the spirit of this application, they should also be considered as the content disclosed in this application.

[0286] It should also be understood that, in the various method embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0287] The methods provided in the embodiments of this application have been described above. The content sorting device and model training device provided in the embodiments of this application will be described below.

[0288] Figure 19 A schematic diagram of a content sorting device 1900 provided in an embodiment of this application is shown below. Figure 19As shown, the device 1900 includes: an acquisition module 1910, a prediction module 1920, and a sorting module 1930. The acquisition module 1910 is used to acquire the relevant content features of N current candidate contents and M historical browsing contents for the target object, where N is a positive integer greater than 1 and M is a positive integer. The prediction module 1920 is used to input the relevant content features of the current candidate contents and the relevant content features of the M historical browsing contents into the target model for each of the N current candidate contents to predict the completion rate of the target object's process behavior for the current candidate contents, and obtain the score of the current candidate contents based on the completion rate of the target object's process behavior for the current candidate contents. The sorting module 1930 is used to sort the N current candidate contents for the target object based on the scores of the N current candidate contents.

[0289] In some implementations, the content sorting apparatus further includes: a determining module 1940, configured to: determine at least one of the following, before the prediction module 1920 inputs the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the target model to predict the completion rate of the target object's process behavior for the current candidate content: the average completion rate of the target object's process behavior for historical browsing contents of the target type, and the average completion time of the target object's process behavior for historical browsing contents of the target type; wherein, the historical browsing contents of the target type are historical browsing contents of the M historical browsing contents that belong to the same category as the current candidate content; correspondingly, the prediction module 1920 is specifically configured to: input at least one of the following, the average completion rate of the target object's process behavior for historical browsing contents of the target type, and the average completion time of the target object's process behavior for historical browsing contents of the target type, the relevant content features of the current candidate content, and the relevant content features of each of the M historical browsing contents, into the target model to predict the completion rate of the target object's process behavior for the current candidate content.

[0290] In some implementations, the prediction module 1920 is specifically used to: input at least one of the average completion rate of the target object's process behavior for the target type of historical browsing content, and the average completion time of the target object's process behavior for the target type of historical browsing content, the relevant content features of the current candidate content, and the relevant content features of each of the M historical browsing contents into the target model, so as to predict at least one of the click-through rate and browsing time of the current candidate content and the completion rate of the target object's process behavior for the current candidate content.

[0291] In some implementations, the target model includes a multi-task model and a bias tower; the prediction module 1920 is specifically used to: input the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the multi-task model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the first process behavior completion rate of the target object for the current candidate content; input at least one of the average process behavior completion rate of the target object for the target type of historical browsing contents, and the average process behavior completion duration of the target object for the target type of historical browsing contents into the bias tower to predict the second process behavior completion rate of the target object for the current candidate content; and obtain the third process behavior completion rate of the target object for the current candidate content based on the first and second process behavior completion rates of the target object for the current candidate content.

[0292] In some implementations, the multi-task model includes: a deep interest network model, an embedding layer, and a multi-gate hybrid expert model; the prediction module 1920 is specifically used to: input the relevant content features of the current candidate content into the embedding layer to obtain a first vector; input the relevant content features of each of the M historical browsing contents into the deep interest network model to obtain a second vector; input the first vector and the second vector into the multi-gate hybrid expert model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the completion rate of the first process behavior of the target object for the current candidate content.

[0293] In some implementations, the acquisition module 1910 is further configured to: acquire at least one of the attribute features and interest features of the target object before the prediction module 1920 inputs the relevant content features of the current candidate content into the embedding layer to obtain the first vector; correspondingly, the prediction module 1920 is specifically configured to: input at least one of the attribute features and interest features of the target object and the relevant content features of the current candidate content into the embedding layer to obtain the first vector.

[0294] In some implementations, the prediction module 1920 is specifically used to: calculate the weighted sum of the first process behavior completion rate and the second process behavior completion rate of the target object for the current candidate content; if the weighted sum is greater than a fifth preset threshold, then determine that the third process behavior completion rate of the target object for the current candidate content is 1; if the weighted sum is less than or equal to the fifth preset threshold, then determine that the third process behavior completion rate of the target object for the current candidate content is 0.

[0295] In some implementations, the content sorting apparatus 1900 further includes a processing module 1950, configured to: perform at least one of the following before the acquisition module 1910 acquires the relevant content features of each of the N current candidate contents and each of the M historical browsing contents for the target object:

[0296] Determine the current search method for each of the N current candidate contents;

[0297] Obtain the current search terms corresponding to N current candidate contents and the K historical search terms corresponding to M historical browsing contents, and determine the relevance of the current search terms to the K historical search terms respectively, where K is a positive integer;

[0298] Obtain at least one sub-content from each of the current search terms and M historical browsing contents corresponding to the N current candidate contents. Based on the current search terms and at least one sub-content from each of the M historical browsing contents, determine the relevance between the current search terms and the M historical browsing contents respectively.

[0299] Accordingly, the acquisition module 1910 is specifically used to: if at least one of the following is determined to satisfy the personalized content ranking conditions based on the current search method, the relevance of the current search term to K historical search terms, and the relevance of the current search term to M historical browsing contents, then acquire the relevant content features of each of the N current candidate contents and the relevant content features of each of the M historical browsing contents for the target object.

[0300] In some possible implementations, personalized content sorting criteria include at least one of the following:

[0301] The current search method is a guided search method;

[0302] The maximum relevance of the current search term to K historical search terms is greater than a first preset threshold.

[0303] The maximum relevance of the current search term to M historical browsing items is greater than the second preset threshold.

[0304] In some implementations, the processing module 1950 is specifically used to: for each of the M historical browsing contents, calculate the relevance between the current search term and at least one sub-content of the historical browsing contents, and obtain the relevance between the current search term and the historical browsing contents based on the relevance between the current search term and at least one sub-content of the historical browsing contents.

[0305] In some implementations, the processing module 1950 is configured to: perform at least one of the following before the acquisition module 1910 acquires the relevant content features of each of the N current candidate contents and each of the M historical browsing contents for the target object:

[0306] Determine the current search method for each of the N current candidate contents;

[0307] Get the current search terms corresponding to N current candidate contents and L historical search terms corresponding to P historical browsing contents. Determine the relevance of the current search terms to the L historical search terms respectively, where L is a positive integer and P historical browsing contents include M historical browsing contents, where P is an integer greater than or equal to M.

[0308] Obtain at least one sub-content of each of the current search terms and P historical browsing contents corresponding to the N current candidate contents. Based on the current search terms and at least one sub-content of each of the P historical browsing contents, determine the relevance of the current search terms to the P historical browsing contents respectively.

[0309] Accordingly, the acquisition module 1910 is specifically used to: if at least one of the following is determined to meet the personalized content sorting conditions based on the current search method, the relevance of the current search term to L historical search terms, and the relevance of the current search term to P historical browsing content, then select M historical browsing content from the P historical browsing content, and acquire the relevant content features of each of the N current candidate content and the relevant content features of each of the M historical browsing content for the target object.

[0310] In some possible implementations, personalized content sorting criteria include at least one of the following:

[0311] The current search method is a guided search method;

[0312] The maximum relevance of the current search term to L historical search terms is greater than the third preset threshold.

[0313] The maximum relevance of the current search term to P historical browsing items is greater than the fourth preset threshold.

[0314] In some implementations, the processing module 1950 is specifically used to: for each of the P historical browsing contents, calculate the relevance between the current search term and at least one sub-content of the historical browsing contents, and obtain the relevance between the current search term and the historical browsing contents based on the relevance between the current search term and at least one sub-content of the historical browsing contents.

[0315] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details will not be provided here. Specifically, Figure 19 The device 1900 shown can perform Figure 4 The corresponding method embodiments, and the foregoing and other operations and / or functions of each module in the apparatus 1900 are respectively for implementing Figure 4 For the sake of brevity, the corresponding processes in each method are not described in detail here.

[0316] The apparatus 1900 of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0317] Figure 20 A schematic diagram of a model training device 2000 provided in an embodiment of this application is shown below. Figure 20 As shown, the device 2000 includes: an acquisition module 2010, a prediction module 2020, a calculation module 2030, and a training module 2040; wherein, the acquisition module 2010 is used to acquire the relevant content features of N current candidate contents and M historical browsing contents for the target object, where N is a positive integer greater than 1 and M is a positive integer; for each of the N current candidate contents, the prediction module 2020 is used to input the relevant content features of the current candidate contents and the relevant content features of the M historical browsing contents into the target model to predict the completion rate of the target object's process behavior for the current candidate contents; the calculation module 2030 is used to calculate the loss of the target model based on the predicted completion rate of the target object's process behavior for the current candidate contents and the actual completion rate of the target object's process behavior for the current candidate contents; the training module 2040 is used to train the target model based on the loss of the target model.

[0318] In some implementations, the prediction module 2020 is specifically used to: input the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the target model to predict at least one of the predicted click-through rate and predicted browsing duration of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content; the calculation module is specifically used to: calculate the loss of the target model based on at least one of the predicted click-through rate and predicted browsing duration of the current candidate content, the predicted completion rate of the target object's process behavior for the current candidate content, and at least one of the actual click-through rate and actual browsing duration of the current candidate content, as well as the actual completion rate of the target object's process behavior for the current candidate content.

[0319] In some implementations, the computation module 2030 is specifically used to: calculate the loss of the target model with respect to at least one of the predicted click-through rate and predicted browsing duration of the current candidate content, and at least one of the actual click-through rate and actual browsing duration of the current candidate content; calculate the loss of the target model with respect to the process behavior completion rate based on the predicted process behavior completion rate of the target object for the current candidate content, and the actual process behavior completion rate of the target object for the current candidate content; and calculate the loss of the target model with respect to at least one of the loss of the target model with respect to the click-through rate and browsing duration, and the loss of the target model with respect to the process behavior completion rate.

[0320] In some implementations, the computation module 2030 is specifically used to: calculate the Kolb-Leibler divergence loss of the target model with respect to the process behavior completion rate, based on the target object's predicted process behavior completion rate for the current candidate content and the target object's actual process behavior completion rate for the current candidate content.

[0321] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details will not be provided here. Specifically, Figure 20 The device 2000 shown can perform Figure 17 The corresponding method embodiments, and the foregoing and other operations and / or functions of each module in the device 2000 are respectively for implementing Figure 17 For the sake of brevity, the corresponding processes in each method are not described in detail here.

[0322] The apparatus 2000 of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0323] Figure 21 This is a schematic block diagram of an electronic device provided in an embodiment of this application. The electronic device may be the aforementioned execution device or training device.

[0324] like Figure 21 As shown, the electronic device may include:

[0325] The system includes a memory 2110 for storing computer programs and a processor 2120 for transferring program code to the processor 2120. In other words, the processor 2120 can retrieve and run the computer program from the memory 2110 to implement the methods described in the embodiments of this application.

[0326] For example, the processor 2120 can be used to execute the above-described method embodiments according to instructions in the computer program.

[0327] In some embodiments of this application, the processor 2120 may include, but is not limited to:

[0328] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0329] In some embodiments of this application, the memory 2110 includes, but is not limited to:

[0330] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0331] In some embodiments of this application, the computer program may be divided into one or more modules, which are stored in the memory 2110 and executed by the processor 2120 to perform the method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the electronic device.

[0332] like Figure 21 As shown, the electronic device may further include:

[0333] Transceiver 2130, which can be connected to processor 2120 or memory 2110.

[0334] The processor 2120 can control the transceiver 2130 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 2130 may include a transmitter and a receiver. The transceiver 2130 may further include antennas, and the number of antennas may be one or more.

[0335] It should be understood that the various components in the electronic device are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.

[0336] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.

[0337] When implemented using software, it can be implemented entirely or partially as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0338] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0339] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0340] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0341] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A content sorting method, characterized in that, include: Obtain the relevant content features of N current candidate contents and M historical browsing contents for the target object, where N is a positive integer greater than 1 and M is a positive integer; For each of the N current candidate contents, the relevant content features of the current candidate contents and the relevant content features of the M historical browsing contents are input into the target model to predict the completion rate of the target object's process behavior for the current candidate contents, and the score of the current candidate contents is obtained based on the completion rate of the target object's process behavior for the current candidate contents. Based on the scores of each of the N current candidate contents, the N current candidate contents are sorted for the target object.

2. The method according to claim 1, further comprising, before inputting the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the target model to predict the completion rate of the target object's process behavior towards the current candidate content: Determine at least one of the following: the average completion rate of the target object's process behavior for the target type of historical browsing content, and the average completion time of the target object's process behavior for the target type of historical browsing content; wherein, the target type of historical browsing content is historical browsing content of the same type as the current candidate content among the M historical browsing content; The step of inputting the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the target model to predict the completion rate of the target object's process behavior for the current candidate content includes: The target model is input with at least one of the following: the average completion rate of the target object's process behavior for the target type of historical browsing content, and the average completion time of the target object's process behavior for the target type of historical browsing content, along with the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents, in order to predict the completion rate of the target object's process behavior for the current candidate content.

3. The method according to claim 2, characterized in that, The step of inputting at least one of the average completion rate of the target object's process behavior for the target type of historical browsing content, and the average completion time of the target object's process behavior for the target type of historical browsing content, along with the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents, into the target model to predict the completion rate of the target object's process behavior for the current candidate content includes: The target model is input with at least one of the following: the average completion rate of the target object's process behavior for the target type of historical browsing content, and the average completion time of the target object's process behavior for the target type of historical browsing content; the relevant content features of the current candidate content; and the relevant content features of each of the M historical browsing contents; to predict at least one of the click-through rate and browsing time of the current candidate content, as well as the completion rate of the target object's process behavior for the current candidate content.

4. The method according to claim 3, characterized in that, The target model includes: a multi-task model and a deviation tower; The step of inputting at least one of the average completion rate of the target object's process behavior for the target type of historical browsing content, and the average completion time of the target object's process behavior for the target type of historical browsing content, along with the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents, into the target model to predict at least one of the click-through rate and browsing time of the current candidate content and the completion rate of the target object's process behavior for the current candidate content, includes: The relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents are input into the multi-task model to predict at least one of the click-through rate and browsing duration of the current candidate content and the first process behavior completion rate of the target object for the current candidate content. The average completion rate of the target object's process behavior for the target type of historical browsing content, and the average completion time of the target object's process behavior for the target type of historical browsing content are input into the deviation tower to predict the second process behavior completion rate of the target object for the current candidate content. Based on the first process behavior completion rate and the second process behavior completion rate of the target object for the current candidate content, the third process behavior completion rate of the target object for the current candidate content is obtained.

5. The method according to claim 4, characterized in that, The multi-task model includes: a deep interest network model, an embedding layer, and a multi-gate hybrid expert model; The step of inputting the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the multi-task model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the first process behavior completion rate of the target object for the current candidate content, includes: The relevant content features of the current candidate content are input into the embedding layer to obtain the first vector; The relevant content features of each of the M historical browsing contents are input into the deep interest network model to obtain the second vector; The first vector and the second vector are input into the multi-gate hybrid expert model to predict at least one of the click-through rate and browsing duration of the current candidate content, as well as the first process behavior completion rate of the target object for the current candidate content.

6. The method according to claim 5, characterized in that, Before inputting the relevant content features of the current candidate content into the embedding layer to obtain the first vector, the method further includes: Obtain at least one of the attribute features and interest features of the target object; The step of inputting the relevant content features of the current candidate content into the embedding layer to obtain the first vector includes: The first vector is obtained by inputting at least one of the attribute features and interest features of the target object and the relevant content features of the current candidate content into the embedding layer.

7. The method according to any one of claims 1-6, characterized in that, Before obtaining the relevant content features of each of the N current candidate contents and the relevant content features of each of the M historical browsing contents for the target object, at least one of the following is also included: Determine the current search method corresponding to the N current candidate contents; Obtain the current search terms corresponding to the N current candidate contents and the K historical search terms corresponding to the M historical browsing contents, and determine the relevance of the current search terms to the K historical search terms, where K is a positive integer; Obtain the current search term corresponding to the N current candidate contents and at least one sub-content of each of the M historical browsing contents. Based on the current search term and at least one sub-content of each of the M historical browsing contents, determine the relevance of the current search term to the M historical browsing contents respectively. The acquisition of relevant content features for each of the N current candidate contents and the M historical browsing contents for the target object includes: If at least one of the following is determined to satisfy the personalized content ranking conditions: the relevance of the current search term to the K historical search terms, and the relevance of the current search term to the M historical browsing contents, then the relevant content features of the N current candidate contents and the relevant content features of the M historical browsing contents for the target object are obtained.

8. The method according to claim 7, characterized in that, The personalized content sorting criteria include at least one of the following: The current search method is a guided search method; The maximum value of the relevance between the current search term and the K historical search terms is greater than a first preset threshold; The maximum value of the relevance between the current search term and the M historical browsing contents is greater than the second preset threshold.

9. The method according to claim 7, characterized in that, The step of determining the relevance of the current search term to each of the M historical browsing contents, based on the current search term and at least one sub-content of each of the M historical browsing contents, includes: For each of the M historical browsing contents, calculate the relevance between the current search term and at least one sub-content of the historical browsing contents, and obtain the relevance between the current search term and the historical browsing contents based on the relevance between the current search term and at least one sub-content of the historical browsing contents.

10. The method according to any one of claims 1-6, characterized in that, Before obtaining the relevant content features of each of the N current candidate contents and the relevant content features of each of the M historical browsing contents for the target object, at least one of the following is also included: Determine the current search method corresponding to the N current candidate contents; Obtain the current search terms corresponding to the N current candidate contents and the L historical search terms corresponding to the P historical browsing contents, and determine the relevance of the current search terms to the L historical search terms respectively, where L is a positive integer, and the P historical browsing contents include the M historical browsing contents, where P is an integer greater than or equal to M; Obtain the current search terms corresponding to the N current candidate contents and at least one sub-content of each of the P historical browsing contents. Based on the current search terms and at least one sub-content of each of the P historical browsing contents, determine the relevance of the current search terms to the P historical browsing contents respectively. The acquisition of relevant content features for each of the N current candidate contents and the M historical browsing contents for the target object includes: If at least one of the following is determined to satisfy the personalized content sorting conditions: the relevance of the current search term to the L historical search terms, and the relevance of the current search term to the P historical browsing contents, then the M historical browsing contents are selected from the P historical browsing contents, and the relevant content features of the N current candidate contents and the relevant content features of the M historical browsing contents for the target object are obtained.

11. The method according to claim 10, characterized in that, The personalized content sorting criteria include at least one of the following: The current search method is a guided search method; The maximum value of the relevance between the current search term and the L historical search terms is greater than a third preset threshold; The maximum value among the relevance scores of the current search term and the P historical browsing contents is greater than the fourth preset threshold.

12. The method according to claim 10, characterized in that, The step of determining the relevance of the current search term to each of the P historical browsing contents, based on the current search term and at least one sub-content of each of the P historical browsing contents, includes: For each of the P historical browsing contents, calculate the relevance between the current search term and at least one sub-content of the historical browsing contents, and obtain the relevance between the current search term and the historical browsing contents based on the relevance between the current search term and at least one sub-content of the historical browsing contents.

13. A model training method, characterized in that, include: Obtain the relevant content features of N current candidate contents and M historical browsing contents for the target object, where N is a positive integer greater than 1 and M is a positive integer; For each of the N current candidate contents, the relevant content features of the current candidate contents and the relevant content features of the M historical browsing contents are input into the target model to predict the completion rate of the target object's process behavior for the current candidate contents. Based on the predicted completion rate of the target object's process behavior for the current candidate contents and the actual completion rate of the target object's process behavior for the current candidate contents, the loss of the target model is calculated. The target model is trained based on the loss of the target model.

14. The method according to claim 13, characterized in that, The step of inputting the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the target model to predict the completion rate of the target object's process behavior for the current candidate content includes: The relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents are input into the target model to predict at least one of the predicted click-through rate and predicted browsing duration of the current candidate content, as well as the completion rate of the process behavior of the target object for the current candidate content. The calculation of the loss of the target model based on the prediction process completion rate of the target object for the current candidate content and the actual process completion rate of the target object for the current candidate content includes: The loss of the target model is calculated based on at least one of the predicted click-through rate and predicted browsing duration of the current candidate content, the predicted process behavior completion rate of the target object for the current candidate content, and at least one of the actual click-through rate and actual browsing duration of the current candidate content, and the actual process behavior completion rate of the target object for the current candidate content.

15. The method according to claim 14, characterized in that, The loss of the target model is calculated based on at least one of the predicted click-through rate and predicted browsing duration of the current candidate content, and the target object's predicted behavior completion rate for the current candidate content, and at least one of the actual click-through rate and actual browsing duration of the current candidate content, and the target object's actual behavior completion rate for the current candidate content, including: Based on at least one of the predicted click-through rate and predicted browsing duration of the current candidate content, and at least one of the actual click-through rate and actual browsing duration of the current candidate content, calculate the loss of the target model with respect to at least one of the click-through rate and browsing duration; Based on the target object's predicted process behavior completion rate for the current candidate content, and the target object's actual process behavior completion rate for the current candidate content, the loss of the target model with respect to the process behavior completion rate is calculated. The loss of the target model is calculated based on the loss of the target model with respect to at least one of click-through rate and browsing duration, and the loss of the target model with respect to process behavior completion rate.

16. A content sorting device, characterized in that, include: The acquisition module is used to acquire the relevant content features of N current candidate contents and M historical browsing contents for the target object, where N is a positive integer greater than 1 and M is a positive integer; The prediction module is used to input the relevant content features of the current candidate content and the relevant content features of each of the M historical browsing contents into the target model for each of the N current candidate contents, so as to predict the completion rate of the target object's process behavior for the current candidate content, and obtain the score of the current candidate content based on the completion rate of the target object's process behavior for the current candidate content. The sorting module is used to sort the N current candidate contents based on their respective scores for the target object.

17. A model training device, characterized in that, include: The module consists of an acquisition module, a prediction module, a calculation module, and a training module. The acquisition module is used to acquire the relevant content features of N current candidate contents and M historical browsing contents for the target object, where N is a positive integer greater than 1 and M is a positive integer; For each of the N current candidate contents, the prediction module is used to input the relevant content features of the current candidate contents and the relevant content features of the M historical browsing contents into the target model to predict the completion rate of the target object's process behavior for the current candidate contents. The calculation module is used to calculate the loss of the target model based on the predicted completion rate of the target object's process behavior for the current candidate contents and the actual completion rate of the target object's process behavior for the current candidate contents. The training module is used to train the target model based on the loss of the target model.

18. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store a computer program, the processor being used to invoke and run the computer program stored in the memory to perform the method of any one of claims 1 to 15.

19. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the method as described in any one of claims 1 to 15.

20. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method as described in any one of claims 1 to 15.