Recommended information processing method and device, equipment, storage medium and program product

Abstract

The invention provides a recommendation information processing method and device, equipment, a storage medium and a program product. The method comprises the steps of obtaining a plurality of candidate recommendation sequences, object features, content information features and recommendation information features; performing the following processing on each candidate recommendation sequence: performing feature extraction on the candidate recommendation sequence to obtain a first sequence feature of the candidate recommendation sequence, and splicing the object feature, the content information feature, the recommendation content feature and the first sequence feature into a combined feature; performing prediction processing on the sequence recommendation indexes based on the combined features to obtain the sequence recommendation indexes of the candidate recommendation sequences; performing prediction processing on each information recommendation index in the sequence based on the combined features to obtain an information recommendation index of each piece of information in the candidate recommendation sequence; and determining a target recommendation sequence based on the sequence recommendation index and the information recommendation index. Through the information recommendation method and device, the target recommendation sequence can be determined, and the accuracy of information recommendation processing is improved.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and computer technology, and in particular to a method, device, equipment, storage medium and program product for processing recommendation information. Background Art

[0002] In the information flow recommendation system, the optimization of the insertion position of the recommended information is crucial. It is necessary to comprehensively consider the content sequence and the recommended information sequence output by the content recommendation engine and the information engine to be recommended, and determine the insertion position of the recommended information in the content sequence. In order to achieve the best information insertion effect, evaluating the comprehensive benefits of different information insertion sequences is of decisive significance for improving the accuracy of information recommendation on the platform. The existing technology is based on the sequence evaluation and sorting method of neural network, constructs a deep learning model, inputs relevant data, and predicts the information sequence used to perform recommendation processing. The factors for predicting the recommendation effect in the related technology are single, which affects the accuracy of the recommendation processing.

[0003] There is currently no better recommendation information processing method in the relevant technology that can achieve dynamic and accurate recommendation information evaluation and improve the accuracy of information recommendation processing. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, device, storage medium, and program product for processing recommendation information, which can determine a target recommendation sequence and improve the accuracy of information recommendation processing.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides a method for processing recommendation information, the method comprising:

[0007] Acquire multiple candidate recommendation sequences, wherein the candidate recommendation sequences are sorted results of the content information and the recommended material information;

[0008] Acquire object features, content information features, and recommendation information features, wherein the object features are features of the object information to be recommended;

[0009] For each of the candidate recommendation sequences, the following processing is performed: feature extraction is performed on the candidate recommendation sequence to obtain a first sequence feature of the candidate recommendation sequence, and the object feature, the content information feature, the recommended content feature and the first sequence feature are spliced ​​into a combined feature;

[0010] Based on the combined features, an indicator prediction process is performed on the candidate recommendation sequence as a whole to obtain a sequence recommendation indicator of the candidate recommendation sequence;

[0011] Based on the combined features, an index prediction process is performed on each information in the candidate recommendation sequence to obtain an information recommendation index for each information in the candidate recommendation sequence;

[0012] A target recommendation sequence is determined based on the sequence recommendation index and the information recommendation index, wherein the target recommendation sequence is a candidate recommendation sequence for performing a recommendation process.

[0013] The present application provides a device for processing recommendation information, including:

[0014] The feature extraction module is used to obtain multiple candidate recommendation sequences, wherein the candidate recommendation sequence is the sorting result of the content information and the recommended material information; obtain object features, content information features and recommendation information features, wherein the object features are features of the object information to be recommended; perform the following processing on each of the candidate recommendation sequences: extract features from the candidate recommendation sequence to obtain a first sequence feature of the candidate recommendation sequence, and splice the object features, the content information features, the recommended content features and the first sequence features into a combined feature;

[0015] A sequence determination module is used to perform indicator prediction processing on the candidate recommendation sequence as a whole based on the combined features to obtain a sequence recommendation indicator of the candidate recommendation sequence; perform indicator prediction processing on each information in the candidate recommendation sequence based on the combined features to obtain an information recommendation indicator for each information in the candidate recommendation sequence; and determine a target recommendation sequence based on the sequence recommendation indicator and the information recommendation indicator, wherein the target recommendation sequence is a candidate recommendation sequence for performing recommendation processing.

[0016] An embodiment of the present application provides an electronic device, the electronic device comprising:

[0017] A memory for storing computer executable instructions or computer programs;

[0018] The processor is used to implement the method for processing recommendation information provided in the embodiment of the present application when executing the computer executable instructions or computer program stored in the memory.

[0019] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing a method for processing recommendation information provided in an embodiment of the present application when executed by a processor.

[0020] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, the method for processing recommendation information provided by the embodiment of the present application is implemented.

[0021] The embodiments of the present application have the following beneficial effects:

[0022] By combining object features, content information features, recommendation information features, and sequence features, we can more comprehensively understand and analyze the recommendation sequence. The object is the user. The multi-dimensional feature combination helps capture the complexity of the context of users and information, making the recommendation results more in line with user needs and preferences. Based on the combined features, the sequence recommendation index is predicted, and based on the combined features, each information recommendation index in the sequence is predicted. A comprehensive evaluation is performed on the recommendation indexes of the sequence as a whole and individually to improve the accuracy of the prediction results. Based on the sequence recommendation index and the information recommendation index, the target recommendation sequence can be dynamically and accurately determined, improving the accuracy of information recommendation processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of an application mode of the method for processing recommendation information provided in an embodiment of the present application;

[0024] Figure 2 It is a schematic diagram of the structure of an electronic device of the device provided in an embodiment of the present application;

[0025] Figure 3A This is a first flow chart of the method for processing recommendation information provided in an embodiment of the present application;

[0026] Figure 3B is a second flow chart of the method for processing recommendation information provided in an embodiment of the present application;

[0027] Figure 3C is a third flow chart of the method for processing recommendation information provided in an embodiment of the present application;

[0028] Figure 4 is a fourth flow chart of the method for processing recommendation information provided in an embodiment of the present application;

[0029] Figure 5 It is a schematic diagram of the principle of the method for processing recommendation information provided in the embodiment of the present application;

[0030] Figure 6 It is a structural diagram of the evaluation model provided in the embodiment of the present application.

[0031] It should be pointed out that the above-mentioned "first" and "second" are only used to distinguish different solutions, and do not represent the degree of superiority or inferiority of the solutions or the priority in the implementation process. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0033] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0034] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0035] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0036] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0037] The relevant data collection and processing (for example, recommendation information, information on objects to be recommended) in the embodiments of this application should be strictly in accordance with the requirements of relevant laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject should be obtained, and subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0038] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0039] 1) Supervised learning: A machine learning method that uses a set of labeled data to train the model. During the training process, the model learns how to map the input to the desired output.

[0040] 2) Partially supervised learning: In machine learning, it refers to a training method in which only a portion of the data set used for model training is fully labeled. In the embodiments of the present application, partial supervision is partly supervised learning and partly reinforcement learning.

[0041] 3) Reinforcement Learning: A machine learning method in which an agent learns how to make decisions by interacting with the environment. The agent performs actions and receives rewards from the environment, with the goal of maximizing the long-term cumulative reward.

[0042] 4) Multi-task learning: A paradigm of machine learning. The core idea is to learn multiple related tasks simultaneously in the same model. Different tasks can share the underlying feature representation while maintaining task-specific processing at a higher level, thereby improving learning efficiency and the generalization ability of the model.

[0043] 5) Deep Q-Network (DQN): It is an algorithm based on deep learning and reinforcement learning. It is a variant of Q-learning. It uses deep neural networks to approximate the Q function to solve the convergence problem and computational complexity of Q-learning in tasks with high-dimensional input space. In traditional Q-learning, the Q function (also called the action-value function) represents the expected return that can be obtained by taking an action in a given state. The deep Q network uses a deep neural network to approximate this Q function, allowing the algorithm to handle more complex inputs. The neural network used by the deep Q network is usually a convolutional neural network, which is particularly effective for processing image inputs. For other types of data, fully connected layers or other types of networks may be used.

[0044] 6) Q-learning: It is a classic reinforcement learning algorithm used to solve the control problem in Markov decision-making process. It is based on the idea of ​​value iteration and guides the agent to choose the best action in each state by estimating the value function Q value of each state-action pair.

[0045] 7) Evaluation Network: A network used for real-time decision-making. It estimates the Q-values ​​of all possible actions in the current state and selects the optimal action. It receives the current observation value (state) as input and outputs the Q-value corresponding to each possible action.

[0046] 8) Q-value: It is the expected return that the model predicts after executing each action. It can help the algorithm make decisions and select the action that can get the maximum benefit based on the Q-value.

[0047] 9) Target Network: It is a network used to generate target Q-values, which are used to calculate the loss function of the evaluation network. The purpose is to provide a stable training target to reduce fluctuations in the learning process. The target Q-value is calculated based on the next state and the output of the target network after the current state is given and an action is performed, which helps the learning process to be more stable. The structure of the target network is the same as the evaluation network, but its weights are updated less frequently. Usually, the weights of the target network are copied from the weights of the evaluation network periodically, rather than being updated in each training iteration.

[0048] 10) Customized Gate Control (CGC): It is a model structure for multi-task learning, which includes a set of shared expert networks. The shared expert networks are used by multiple tasks to learn common features and knowledge between tasks. Each task has an expert network responsible for learning the unique features and patterns of the task. Customized gate control uses a gating mechanism to dynamically adjust the information transmitted from the shared layer and the expert tower to customize the output according to the specific needs of each task.

[0049] 11) Temporal Difference Learning: It is a type of algorithm in reinforcement learning that updates the value estimate by comparing the predicted instantaneous return with the actual instantaneous return. It does not require a complete trajectory to learn, which makes Temporal Difference Learning very efficient in reinforcement learning.

[0050] In the related technology, a sequence evaluation and ranking method based on a neural network constructs a deep learning model, inputs data including object features, recommended information attributes, and the insertion sequence of candidate recommended information to be evaluated, and predicts and outputs the recommended information benefits and user experience benefits that may be obtained under a specific insertion sequence. It cannot adapt to the dynamic needs of the recommended information insertion sequence as the population and context change, cannot consider the content and recommended information benefits at the same time, and ignores long-term benefits, resulting in low accuracy of the estimated recommended information and poor user experience.

[0051] The embodiments of the present application provide a method for processing recommendation information, a device for processing recommendation information, an electronic device, a computer-readable storage medium, and a computer program product, which can determine a target recommendation sequence and improve the accuracy of information recommendation processing.

[0052] The following describes an exemplary application of the electronic device provided in the embodiments of the present application. The device provided in the embodiments of the present application can be implemented as various types of terminals such as a laptop computer, a tablet computer, a desktop computer, a set-top box, a smart phone, a smart speaker, a smart watch, a smart TV, and a vehicle-mounted terminal, and can also be implemented as a server. The following describes an exemplary application when the device is implemented as a terminal or a server.

[0053] See also Figure 1 , Figure 1 This is a schematic diagram of an application mode of the method for processing recommendation information provided in an embodiment of the present application, for implementing an application for processing recommendation information, for example, Figure 1 The system involves a server 200, a network 300, a terminal device 400 and a database 500. The terminal device 400 is connected to the server 200 via the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0054] In some embodiments, the user may be a technician in the relevant field, the server 200 is a server for executing a method for processing recommended information, the terminal device 400 is a terminal operated by the user, the terminal device 400 is pre-configured with an application for displaying recommended information, and the database 500 stores user information, content information and recommended information.

[0055] In the example, the user triggers the terminal device to send a recommendation information processing instruction, and the terminal device 400 sends the recommendation information processing instruction to the server 200 through the network 300. The server 200 reads the user information, content information and recommendation information from the database 500, determines multiple candidate recommendation sequences and performs feature extraction, concatenates the extracted object features, content information, recommended content features and first sequence features to obtain combined features, performs prediction processing of sequence recommendation indicators and information recommendation indicators based on the combined features, determines a target recommendation sequence for performing recommendation processing based on the sequence recommendation indicators and the information recommendation indicators, and the server 200 sends the target recommendation sequence to the terminal device 400 through the network 300, and displays the information recommendation results corresponding to the target recommendation sequence on the terminal device 400.

[0056] In some embodiments, the method for processing recommendation information in the embodiments of the present application can also be applied in the following application scenarios:

[0057] 1. In the advertising recommendation scenario of news information: the target recommendation sequence obtained by the method for processing recommendation information of the embodiment of the present application is applied to the advertising recommendation scenario of news information, and the insertion position of the advertisement in the content sequence is accurately determined in the news application to achieve the optimal advertising insertion effect, enhance the content value and conversion value of the platform, and improve the long-term user experience.

[0058] 2. In the personalized information recommendation scenario of social network: The target recommendation sequence obtained by the recommendation information processing method of the embodiment of the present application is applied to the personalized information recommendation scenario of social network, and the display order of information in the dynamic is accurately determined in the social application to achieve the optimal information display effect, improve the content quality and user interaction of the social platform, and improve the long-term activity of users.

[0059] 3. In the recommendation scenario of e-commerce products: The target recommendation sequence obtained by the recommendation information processing method of the embodiment of the present application is applied to the recommendation scenario of e-commerce products, and the display position of the product in the commodity list in the e-commerce application is accurately determined to achieve the optimal product display effect, improve the product sales and user conversion rate of the e-commerce platform, and enhance the user's long-term purchasing intention.

[0060] The embodiments of the present application can be implemented through database technology. In short, a database can be regarded as an electronic file cabinet where electronic files are stored. Users can add, query, update, delete, etc. data in the files. The so-called "database" is a collection of data that is stored together in a certain way, can be shared with multiple users, has as little redundancy as possible, and is independent of the application program.

[0061] A database management system (DBMS) is a computer software system designed for managing databases. It generally has basic functions such as storage, retrieval, security, and backup. Database management systems can be classified according to the database model they support, such as relational, XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters, mobile phones; or according to the query language used, such as Structured Query Language (SQL), XQuery; or according to performance focus, such as maximum scale, maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMS can cross categories, for example, supporting multiple query languages ​​at the same time.

[0062] See also Figure 2 , Figure 2 is a schematic diagram of the structure of an electronic device of the device provided in an embodiment of the present application, and the electronic device may be Figure 1 Server 200 in Figure 2The server 200 shown includes: at least one processor 410, a memory 450, and at least one network interface 420. The various components in the server 200 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Figure 2 Various buses are labeled as bus system 440 .

[0063] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0064] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0065] The memory 450 includes a volatile memory or a nonvolatile memory, and may also include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0066] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0067] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0068] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include: Bluetooth, Wireless Authentication (WiFi), and Universal Serial Bus (USB).

[0069] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2 The processing device 455 of the recommendation information stored in the memory 450 is shown, which can be software in the form of a program and a plug-in, etc., including the following software modules: a feature extraction module 4551, a sequence determination module 4552, and a model training module 4553. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. Figure 2 For the sake of convenience, all the above modules are shown at once, but it should not be considered that the recommendation information processing device 455 excludes the implementation that can only include the feature extraction module 4551 and the sequence determination module 4552. The functions of each module will be explained below.

[0070] In some embodiments, the terminal or server can implement the processing method of the recommendation information provided by the embodiment of the present application by running various computer executable instructions or computer programs. For example, the computer executable instructions can be commands, machine instructions or software instructions at the microprogram level. The computer program can be a native program or software module in the operating system; it can be a local (Native) application (APPlicat ion, APP), that is, a program that needs to be installed in the operating system to run, such as a news APP; it can also be a small program that can be embedded in any APP, that is, a program that can be run only by downloading it to a browser environment. In short, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module or plug-in in any form.

[0071] The method for processing recommendation information provided by the embodiment of the present application will be described in conjunction with the exemplary application and implementation of the terminal provided by the embodiment of the present application.

[0072] The following describes the method for processing recommendation information provided in the embodiment of the present application. As mentioned above, the electronic device that implements the method for processing recommendation information in the embodiment of the present application may be a terminal, a server, or a combination of the two. Therefore, the execution subject of each step will not be repeatedly described below.

[0073] See also Figure 3A , Figure 3A This is a first flow chart of the method for processing recommendation information provided in the embodiment of the present application. Figure 3A The steps shown are explained.

[0074] In step 301, a plurality of candidate recommendation sequences are obtained.

[0075] Here, the candidate recommendation sequence is the sorting result of the content information and the recommended material information.

[0076] For example, based on the browsing behavior of the user to be recommended, all possible positions in the content information sequence where the recommendation information can be inserted are determined, and the recommended material information and multiple content information are sorted according to all possible positions to form a content information sequence containing the recommended material information. The content information sequence containing the recommended material information is used as a candidate recommendation sequence, and the placement order of the content information and the recommendation information in each candidate recommendation sequence is different.

[0077] In some embodiments, step 301 can be implemented by the following method: based on the sequence length of the object information to be recommended and the content information sequence, determining a preset insertion interval of the recommended material information in the content information sequence, wherein the content information sequence includes multiple content information, and the preset insertion interval is an interval composed of positions in the content information sequence for inserting the recommended material information; enumerating the order of all insertion positions of the recommended material information based on the preset insertion interval; and performing multiple insertions of the recommended material information on the content information sequence according to the order of each insertion position to obtain multiple candidate recommendation sequences, wherein the insertion position of each insertion of the recommended material information is different.

[0078] For example, based on the information of the object to be recommended and the sequence length of the content information sequence, the preset insertion interval of the recommended material information in the content information sequence is determined. The preset insertion interval is an interval composed of positions in the content information sequence for inserting the recommended material information, including the sequence position range in which the recommended information can be legally inserted in the content sequence. The preset insertion interval can be set according to the needs of the actual application scenario. For example, there are ten content information in a content sequence, and one advertisement is used as the recommended information. According to the rules for advertisement insertion, it is stipulated that the advertisement can be inserted between the third content information and the sixth content information. Then the absolute position interval from the third content information to the sixth content information is used as the preset insertion interval.

[0079] Without changing the relative order of the sequence of content information and the sequence of recommendation information, the order of all insertion positions of the recommendation material information is enumerated based on the insertion interval, and according to all position results obtained by enumeration and the order of each insertion position, the content information sequence is subjected to multiple insertions of the recommendation material information, that is, the content information and the recommendation material information are sorted, and the multiple sorted results obtained are used as candidate recommendation sequences. The insertion position of each insertion of the recommendation material information is different, and each corresponding candidate recommendation list is also different.

[0080] In some embodiments, see Figure 5 , Figure 5 is a schematic diagram of the principle of the method for processing recommendation information provided in the embodiment of the present application; Figure 5In the figure, content sequence 501 includes multiple content information to be browsed by users, and recommendation information sequence 502 contains recommendation information to be inserted into the content sequence, thereby generating content + recommendation information sequence 5021, content + recommendation information sequence 5022 and content + recommendation information sequence 5023 to represent multiple candidate recommendation sequences obtained.

[0081] In the embodiment of the present application, by enumerating all possible insertion positions and taking into account the combination of various recommendation materials and content information, it is ensured that the possibility of providing multiple recommendation sequences is provided without changing the relative order of the content sequence and the recommendation information, thereby providing a basis for subsequent multi-task prediction and evaluation.

[0082] Continue to see Figure 3A In step 302, object features, content information features, and recommendation information features are obtained.

[0083] Here, the object feature is the feature of the object information to be recommended, and the recommendation information feature is the feature of the recommended material information.

[0084] For example, after obtaining the candidate recommendation sequence, each candidate recommendation sequence needs to be evaluated. The evaluation of the candidate recommendation sequence is implemented through an evaluation model. The input of the evaluation model includes multi-dimensional features related to the recommendation information, including object features, content information features, and recommendation information features. Object features are features of user information and context information of the object to be recommended. User information features include user statistical features (for example, statistical features such as active days and historical clicks), as well as interest features such as historical behavior sequences of content or advertisements. Context information features include current swipes, refresh types, etc. Content information features are basic features such as classification, label, type, and features such as clicks and exposure for each content information in the current content recommendation sequence. Recommendation information features are features such as classification, type, sales method, and unit price of the recommended product in the current scenario.

[0085] In step 303, the following processing is performed for each candidate recommendation sequence: feature extraction is performed on the candidate recommendation sequence to obtain the first sequence feature of the candidate recommendation sequence, and the object feature, content information feature, recommended content feature and first sequence feature are spliced ​​into a combined feature.

[0086] For example, feature extraction is performed on each candidate recommendation sequence to obtain first sequence features, which are the insertion methods of the current recommended material information, including the number of recommended material information, the absolute position and interval of each recommended material information, and the arrangement method of the recommended material information and content information under the candidate recommendation sequence. The object features, content information features, recommended content features, and first sequence features are concatenated into combined features, and the combined features are used as inputs of an evaluation model for evaluating the candidate recommendation sequence.

[0087] In the embodiment of the present application, multiple dimensional features such as object features, content information features, recommended content features and first sequence features are spliced ​​together to obtain combined features, and a comprehensive, multi-dimensional recommendation sequence representation is constructed. As the input of the evaluation model, it can assist the evaluation model in learning comprehensive sequence feature information and make more accurate evaluations and recommendations.

[0088] In step 304, an index prediction process is performed on the candidate recommendation sequence as a whole based on the combined features to obtain a sequence recommendation index of the candidate recommendation sequence.

[0089] For example, multi-task learning is to simultaneously learn multiple related tasks based on combined features in the same model, perform multi-task indicator prediction processing on the candidate recommendation sequence as a whole, output the prediction values ​​under multiple tasks at one time, perform overall prediction for the recommendation sequence under each task, obtain the indicators under each task, and combine them into the overall indicator of the candidate recommendation sequence (sequence recommendation indicator).

[0090] In some embodiments, step 304 can be implemented by the following method: performing nonlinear transformation on the combined features to obtain a first nonlinear feature; performing prediction processing on multiple types of target tasks based on the first nonlinear feature to obtain a first prediction value for each target task; and combining each first prediction value to obtain a sequence recommendation index for a candidate recommendation sequence.

[0091] For example, the multi-task indicator prediction processing of the candidate recommendation sequence as a whole is realized through a deep learning model. The deep learning model is a dual deep Q network. The evaluation model is multi-task learning by introducing the reinforcement learning method. The modeling idea of ​​temporal difference learning is adopted. The current instant benefit and the estimated benefit of the next request are used as the predicted target value in the current state. At the same time, two target networks are used to avoid overestimation and delayed update. Temporal difference learning is a type of algorithm in reinforcement learning. It updates the value estimate by comparing the predicted instant benefit with the actual instant benefit. The dual deep Q network uses two neural networks with the same structure but different parameters: the evaluation network and the target network.

[0092] For example, a nonlinear transformation is performed on the combined features. By performing nonlinear mapping on the input combined features, a more complex feature representation is extracted, that is, the first nonlinear feature is obtained. The nonlinear transformation is implemented through the convolutional layer of the convolutional neural network in the evaluation network in the deep learning model. For each target task, a prediction process is performed based on the first nonlinear feature to obtain the first prediction value of each target task. The prediction process is implemented through a customized gated control component. The first prediction values ​​of multiple types of target tasks are combined to obtain the sequence recommendation index of the candidate recommendation sequence. The sequence recommendation index is an indicator of long-term benefit learning. Long-term benefit learning is to use the estimated output under multiple tasks as the request-level target, that is, the number of clicks on the total content in a request. The first prediction value is a Q value output by the evaluation network. The Q value is the expected return that the model predicts after executing each action. It can help the algorithm make decisions and select the action that can obtain the maximum benefit based on the Q value.

[0093] The customized gating control component is a model structure for multi-task learning, which includes a set of shared expert networks. The shared expert networks are used by multiple tasks to learn common features and knowledge between tasks. Each task has an expert network responsible for learning the features and patterns unique to the task. Customized gating control uses a gating mechanism to dynamically adjust the information transmitted from the shared layer and the expert tower to customize the output according to the specific needs of each task.

[0094] In some embodiments, see Figure 6 , Figure 6 It is a structural diagram of the evaluation model provided in an embodiment of the present application; the evaluation network 601 performs indicator prediction processing on the candidate recommendation sequence as a whole based on the combined features, which is specifically implemented through the customized gating control component 605. For example: in the scenario of advertisement insertion, the four objectives of content exposure 6051, content click 6052, advertisement exposure 6053 and advertisement conversion rate 6054 are used for prediction, and the prediction results corresponding to the four objectives are output. The prediction results of the four objectives are weighted and summed to determine the evaluation score of the corresponding candidate recommendation sequence, that is, the sequence recommendation indicator of the candidate recommendation sequence.

[0095] In the embodiment of the present application, the indicator prediction processing of the multi-task objectives is performed on the candidate recommendation sequence as a whole based on the combined features. Through the methods of multi-task learning and reinforcement learning, an overall multi-dimensional evaluation of the candidate recommendation sequence is achieved, which can take into account long-term benefits and help the model make more comprehensive and long-term decisions in multi-task learning.

[0096] Continue to see Figure 3A In step 305, an index prediction process is performed on each information in the candidate recommendation sequence based on the combined features to obtain an information recommendation index for each information in the candidate recommendation sequence.

[0097] For example, based on the combined features, an indicator prediction process is performed for each information in the candidate recommendation sequence, and a prediction process is performed for each information in the recommendation sequence for each task, including predictions for each content information and each recommended material information, to obtain an information recommendation indicator for each information, which is an indicator at a single point level.

[0098] In some embodiments, step 305 can be implemented by the following method: performing the following processing on each information in the candidate recommendation sequence: extracting features from the combined features according to the information to obtain sub-combination features corresponding to the information, performing nonlinear transformation on the sub-combination features to obtain a second nonlinear feature of the information; for multiple types of target tasks, performing prediction processing based on the second nonlinear feature to obtain a second prediction value for each target task; combining multiple second prediction values ​​to obtain an information recommendation index for the information in the candidate recommendation sequence.

[0099] For example, feature extraction is performed on the combined features according to the information. The feature extraction is implemented through the embedding layer of the evaluation network to obtain the sub-combination features corresponding to the information. The sub-combination features are nonlinearly transformed to obtain the second nonlinear features of the information. The nonlinear transformation is implemented through the convolution layer of the convolutional neural network in the evaluation network in the deep learning model. For each target task, prediction processing is performed based on the second nonlinear feature to obtain the second prediction value of each target task. The second prediction value is a Q value output by the evaluation network. The prediction processing is implemented through a customized gate control component. The second prediction values ​​of multiple types of target tasks are combined to obtain the information recommendation index of the candidate recommendation sequence. The information recommendation index is an index of supervised learning of the evaluation model. The output of the evaluation model is designed as supervised learning. The supervised learning part estimates the single point target under each task, that is, the click-through rate of each content in a request. The click-through rate is the ratio of the number of clicks to the number of impressions.

[0100] In the embodiment of the present application, by predicting the indicators of each information, the performance of each information in the recommendation sequence can be evaluated more finely. By predicting the recommendation indicators of each information, the immediate benefits of the candidate recommendation sequence can be learned, the content that the user may be interested in can be more accurately located, and the targeted nature of the recommendation can be improved.

[0101] In step 306, a target recommendation sequence is determined based on the sequence recommendation index and the information recommendation index.

[0102] Here, the target recommendation sequence is a candidate recommendation sequence for performing the recommendation process.

[0103] For example, the sequence recommendation index is the long-term benefit learning of the evaluation model under multiple tasks, and the information recommendation index is the supervised learning of the evaluation model under multiple tasks. All the information recommendation indicators of supervised learning and the sequence recommendation indicators of long-term benefit learning under each task are linearly converted as the final output of the candidate recommendation sequence under the task, corresponding to the evaluation score of the candidate recommendation sequence, and the candidate recommendation sequence with the highest evaluation score is used as the target recommendation sequence, which is used to recommend information to users according to the sorting position of the content information and the recommended information in the target recommendation sequence.

[0104] In some embodiments, step 306 can be implemented by: for each candidate recommendation sequence, linearly converting the sequence recommendation index and the information recommendation index to obtain the total recommendation index of the candidate recommendation sequence; and selecting at least one candidate recommendation sequence with the highest total recommendation index as the target recommendation sequence.

[0105] For example, for each candidate recommendation sequence, the sequence recommendation index and the information recommendation index are directly added or weighted summed to obtain the total recommendation index of the candidate recommendation sequence. The total recommendation index is an evaluation score for the candidate recommendation sequence, and at least one candidate recommendation sequence with the highest total recommendation index is selected as the target recommendation sequence.

[0106] In some embodiments, feature extraction, splicing, and indicator prediction processing are implemented through deep learning models, see Figure 3B , Figure 3B is a second flow chart of the method for processing recommendation information provided in an embodiment of the present application; Figure 3A Before step 301, Figure 3B Steps 3061 to 3064 in the process train the deep learning model, as described in detail below.

[0107] In step 3061, a training sample set is obtained.

[0108] Here, the training sample set includes a first sample recommendation sequence and an actual recommendation index corresponding to the first sample recommendation sequence, and the actual recommendation index is composed of an actual sequence recommendation index and an actual information recommendation index.

[0109] For example, we collect training samples based on real online data, take one request as the granularity, record the current object features, content information features, and recommended content features as s, record the first sequence features of multiple candidate recommendation sequences obtained as a, record each task in this request as k, k is a positive integer, and assume that the indicator is represented by the number of clicks. The actual sequence recommendation index of each task k is marked as r k , the actual information recommendation index is marked as c kFor example: suppose there are always 10 pieces of content in a request, and the user clicks on the first and third pieces of content in the content sequence, then r = 2, c = [1, 0, 1, 0, 0, 0, 0, 0, 0].

[0110] In step 3062, a deep learning model is called to perform prediction processing based on the first sample recommendation sequence to obtain a first prediction recommendation index.

[0111] For example, N training data are taken from the training samples, and the first sample recommendation sequence (s, a) is input into the deep learning model Q to perform prediction processing on multiple target tasks, and the output results of two parts corresponding to each task k are obtained: information recommendation index (denoted as p k ) and the sequence recommendation index, the sum of the information recommendation index and the sequence recommendation index is used as the final output of the deep learning model Q in the task k, and the first prediction recommendation index is marked as Q k (s,a).

[0112] In step 3063, based on the first predicted recommendation index and the actual recommendation index, the total loss function of the deep learning model is determined.

[0113] For example, the variance of all the first predicted recommendation indicators and the actual recommendation indicator results of the calculation model on each target is used as the loss function of the auxiliary training deep learning model, which is multiplied by the coefficient α and added to other loss functions. The coefficient α is used to control the importance of output variability, balance the model prediction accuracy and output variability, and determine the total loss function of the deep learning model.

[0114] In some embodiments, the training sample set further includes: a second sample recommendation sequence and a sample label value corresponding to the second sample recommendation sequence, wherein the second sample recommendation sequence is a sequence used to perform recommendation processing after the first sample recommendation sequence, see Figure 3C , Figure 3C is a third flow chart of the method for processing recommendation information provided in an embodiment of the present application; Figure 3B Step 3063 in can be Figure 3C Steps 30631 to 30635 in are implemented as described in detail below.

[0115] In step 30631, the deep learning model is called to perform prediction processing based on the second sample recommendation sequence to obtain a second prediction recommendation index.

[0116] For example, the second sample recommendation sequence is the recommendation sequence in the next request. If the user continues to browse, the object features, content information features, and recommended content features of the next request are recorded as s′, and the sample label value done is recorded as 0. Otherwise, if the user stops browsing, done is recorded as 1, and (s, a, r, c, s′, done) is stored as the second sample recommendation sequence. The second sample sequence is determined based on the multi-dimensional features in the next request, and the deep learning model is called for prediction based on the second sample recommendation sequence. s′ and all current candidate recommendation sequences a′ are input into the deep learning model, and the prediction value corresponding to the current task k is calculated to obtain the second prediction recommendation index Q k (s′,a′).

[0117] In step 30632, the mean square error loss of the deep learning model is determined based on the first prediction recommendation indicator and the second prediction recommendation indicator.

[0118] For example, the evaluation model Q is randomly initialized, and the parameters of the evaluation network and the target network are initialized with the same parameters, which are denoted as Q1 and Q2. According to the first prediction recommendation indicator and the second prediction recommendation indicator, the loss function of the long-term benefit of reinforcement learning under multiple objectives is determined, and the mean square error is used for calculation. It can be implemented by formula (2), which is described in detail below.

[0119]

[0120] Among them, the object feature, content information feature, and recommended content feature are denoted as s, the first sequence feature of the obtained multiple candidate recommendation sequences is denoted as a, each task in this request is denoted as k, and the actual sequence recommendation index of each task k is denoted as r k , the actual information recommendation index is marked as c k The object features, content information features, and recommended content features of the next request are recorded as s′, and the flag done is recorded as 0. Otherwise, if browsing is stopped, done is recorded as 1. γ is the immediate reward obtained, ω k is the fusion weight of multiple tasks, N is a positive integer, representing N data taken from the training data set.

[0121] In some embodiments, step 30632 can be implemented by the following method: determining the target prediction recommendation indicator from the second prediction recommendation indicators of multiple second sample recommendation sequences; determining the label value of the second sample recommendation sequence corresponding to the target prediction recommendation indicator, and extracting the actual sequence recommendation indicator from the first prediction recommendation indicator; performing the following processing for each first sample recommendation sequence: determining the first product between the target prediction recommendation indicator and the label value corresponding to the first sample recommendation sequence; determining the sum of the actual sequence recommendation indicator and the first product, and taking the square of the difference between the sum and the first prediction recommendation indicator of the first sample recommendation sequence as the sub-mean square error loss; determining the mean square error loss based on each sub-mean square error loss.

[0122] For example, the target prediction recommendation index Q is determined from the second prediction recommendation indexes of the plurality of second sample recommendation sequences. j (s′,a′ max ), determine the label value done of the second sample recommendation sequence corresponding to the target prediction recommendation index, and extract the actual sequence recommendation index r from the first prediction recommendation index k , for each first sample recommendation sequence, determine the first product γ*done* between the corresponding target prediction recommendation index and the label value Determine the sum of the actual sequence recommendation index and the first product r k +γ*done* The square of the difference between the sum and the first prediction recommendation index is taken as the sub-mean square error loss. Based on the product of each sub-mean square error loss and the weight, the mean square error loss loss1 is determined by taking the average value of N training samples.

[0123] In some embodiments, each second prediction recommendation indicator includes sample prediction values ​​of multiple sample tasks; the target prediction recommendation indicator can be determined by the following method: obtaining the preconfigured weight value of each sample task, and performing weighted calculation based on each preconfigured weight value and the corresponding sample prediction value to obtain the weighted sum of each second sample recommendation sequence; the second prediction recommendation indicator of the second sample recommendation sequence with the highest enhanced summation result is used as the target prediction recommendation indicator.

[0124] For example, we obtain the pre-configured weight value of each sample task, and combine the heuristic principle to derive the fusion weight ω of multiple tasks. k , based on each pre-configured weight value and the corresponding sample prediction value, that is, As the weighted summation result of each second sample recommendation sequence, the second prediction recommendation index of the second sample recommendation sequence with the highest enhanced summation result is used as the target prediction recommendation index.

[0125] In step 30633, the cross entropy loss of the mean square error loss of the deep learning model is determined according to the first predicted recommendation index and the actual recommendation index.

[0126] For example, the cross entropy loss is determined according to the difference between the first predicted recommendation index and the mean square error of the actual recommendation index, that is, the actual sequence recommendation index r of each task k is calculated. k , actual information recommendation index c k The loss is taken as loss2, which is calculated by cross entropy loss and can be implemented by formula (3), as described in detail below.

[0127]

[0128] Among them, CrossEntropy cross entropy, the actual sequence recommendation index of each task k is marked as c k , the actual information recommendation index is marked as p k ,ω k is the fusion weight of multiple tasks, N is a positive integer, representing N data taken from the training data set.

[0129] In some embodiments, the first prediction recommendation indicator includes: the prediction information recommendation indicator of each sample information in the sample recommendation sequence; the actual recommendation indicator includes: the actual information recommendation indicator of each sample information; step 30633 can be implemented by the following method: performing the following processing for each sample information: determining the sub-cross entropy loss based on the difference between the prediction information recommendation indicator and the actual information recommendation indicator; determining the cross entropy loss of the mean square error loss of the deep learning model based on each sub-cross entropy loss.

[0130] For example, based on the difference between the predicted information recommendation index and the actual information recommendation index, the sub-cross entropy loss CrossEntropy(c k ,p k ), weighted calculation is performed based on each sub-cross entropy loss and each pre-configured weight value, that is, ∑ k ω k *CrossEntropy(c k ,p k ) is the weighted summation result of each second sample recommendation sequence, and the cross entropy loss of the mean square error loss of the deep learning model is determined by taking the average value of the N training samples.

[0131] In step 30634, the difference loss of the mean square error loss of the deep learning model is determined based on the variance between each second prediction recommendation indicator.

[0132] For example, the variance between each second prediction recommendation indicator is calculated, and the difference loss of the mean square error loss of the deep learning model is determined as loss3, which can be implemented by formula (4), which is described in detail below.

[0133]

[0134] Among them, Variance is the calculated variance. The second sample recommendation sequence a′ in the next request s′ is input into the deep model Q, and the second prediction recommendation index corresponding to the current task k is calculated, which is recorded as Q k (s′,a′),ω k is the fusion weight of multiple tasks, N is a positive integer, representing N data taken from the training data set.

[0135] In step 30635, the mean square error loss, cross entropy loss, and difference loss are linearly transformed to obtain the total loss function.

[0136] For example, the mean square error loss, cross entropy loss and difference loss are based on the multi-task fusion weight ω k After weighted summation, the three loss functions are used as the final optimization target and updated using the gradient descent method. This can be achieved using formula (5), which is described in detail below.

[0137] loss=loss1+loss2+α*loss3 (5)

[0138] Among them, loss is the total loss function, and the coefficient α is used to control the importance of output variability, and to balance the estimation accuracy of the deep learning model with the output variability. The deep learning model is trained and updated based on the total loss function, and the network parameters Q1 and Q2 in the deep learning model are updated.

[0139] Continue to see Figure 3B In step 3064, the parameters of the deep learning model are updated based on the total loss function to obtain a trained deep learning model.

[0140] For example, the parameters of the deep learning model are updated based on the total loss function to obtain a trained deep learning model, and the combined features obtained by concatenating multi-dimensional features are input into the trained deep learning model for prediction processing to obtain a target recommendation sequence for performing information recommendation. Updating the parameters of the deep learning model can be achieved in the following ways: training the deep learning model based on the total loss function, iteratively updating the parameters of the two sub-networks (evaluation network and target network) of the deep learning model during the training process, assigning the updated network parameters Q1 and Q2 in the trained deep learning model to the two sub-networks (evaluation network and target network), and using the sub-networks after updating the network parameters to perform prediction processing on the combined features under multi-tasks.

[0141] In the embodiment of the present application, by obtaining multiple different candidate recommendation sequences, it is ensured that the possibility of providing multiple recommendation sequences is provided without changing the relative order of the content sequence and the recommended information, thereby providing a basis for subsequent multi-task prediction and evaluation. The splicing and combination of features of multiple dimensions can provide rich information input for the evaluation model, which helps to improve the accuracy of model prediction. By predicting and processing the overall recommendation index of the recommendation sequence and the recommendation index of each information under the objectives of multiple tasks, all single-point targets under each task are summed with the estimated value of long-term benefits as the final output of the model under the task, and reinforcement learning is used to enable the model to learn the long-term benefits of multiple tasks, and supervised learning is introduced to assist reinforcement learning training to improve the accuracy of model estimation. By effectively modeling the conversion rate of recommended information and user experience at the same time, the trained evaluation model can more comprehensively evaluate different advertising insertion sequences and rank them, improve the diversity and accuracy of recommended information, and maximize the conversion rate of platform recommended information and content value.

[0142] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0143] In the construction of the information flow recommendation system, the optimization of the insertion position of the recommended information is crucial. It is necessary to comprehensively consider the content sequence and the recommended information sequence output by the content recommendation engine and the recommended information recommendation engine, and accurately decide the insertion position of the recommended information in the content sequence. In order to achieve the best recommended information insertion effect, evaluating the comprehensive benefits of different recommended information insertion sequences is of decisive significance for improving the content value and conversion value of the platform.

[0144] In the related art, the optimal recommendation information insertion sequence of different users is manually evaluated based on the historical behavior statistics of users on the platform through the evaluation and ranking method of recommendation information insertion by population, which can achieve differentiated processing at the population level and maintain user experience to a certain extent. However, the evaluation result is static, lacks the ability to dynamically adjust, and fails to fully consider the impact of the recommended information insertion position on the recommendation effect of the recommended information. The evaluation and ranking strategy based on the value of recommended information focuses on the price factor of the recommended information. The strategy believes that the higher the unit price given by the advertiser for the recommended information, the higher the position in the content, the higher the profit, thereby maximizing the advertising conversion, and can dynamically evaluate the immediate value of the recommended information insertion sequence, but ignores the potential impact of the recommended information on the user's browsing experience, which will weaken the user loyalty of the platform. In the related art, the sequence evaluation and ranking method based on neural network constructs a deep learning model, inputs data including object features, recommended information attributes, and the recommended information insertion sequence to be evaluated, predicts and outputs the recommended information benefits and user experience benefits that may be obtained under a specific insertion sequence, and finally obtains the comprehensive evaluation benefits of recommended information and content through multi-objective fusion. However, existing models mainly focus on the prediction of immediate benefits and fail to fully consider the long-term impact of recommendation information insertion on users' long-term experience and platform benefits.

[0145] The embodiment of the present application obtains all possible recommendation sequences, takes various types of recommendation information features as input of the ranking model, and designs the output of the model into two parts: supervised learning and long-term benefit learning through a multi-task reinforcement learning ranking model. The supervised part estimates the single-point target under each task, and sums all the single-point targets and the estimated value of the long-term benefit under each task as the final output of the model under the task. Finally, the predicted scores of multiple tasks are output, and the weighted sum of the multi-task estimated values ​​is performed to obtain the total score of the insertion sequence. The insertion sequence with the highest score is selected as the final ranking result to be displayed to the user for recommendation.

[0146] The following is a description with reference to the accompanying drawings. Figure 4 , Figure 4 is a fourth flow chart of the method for processing recommendation information provided in the embodiment of the present application. The execution subject may be a terminal device, a server, or a combination of the two. The embodiment of the present application is based on Figure 1 As an example, the server in the example is used as the execution subject. Figure 4 The steps shown are explained in detail.

[0147] In step 401, a plurality of candidate recommendation sequences are obtained.

[0148] For example, according to the current browsing situation of the user to be recommended, determine the interval in which the recommendation information can be inserted in the content sequence, and enumerate all the positions in the content sequence where the recommendation information can be inserted without changing the relative order of the content sequence and the recommendation information, to obtain multiple candidate recommendation sequences, each of which has a different order of placement of the content sequence and the recommendation information. The insertion interval is the sequence interval where the recommendation information can be legally inserted in the content sequence, for example: there are ten contents and one advertisement in a sequence, and according to the rules for advertisement insertion, it is stipulated that the advertisement can be inserted between the third and sixth contents, and the absolute position interval from the third to the sixth contents corresponds to the insertion interval where the advertisement can be inserted into the content sequence.

[0149] In some embodiments, see Figure 5 , Figure 5 is a schematic diagram of the principle of the method for processing recommendation information provided in the embodiment of the present application; Figure 5 In the example, the content sequence 501 includes multiple content information to be browsed by the user, and the recommendation information sequence 502 contains the recommendation information to be inserted into the content sequence. The candidate recommendation sequence generates multiple candidate recommendation sequences of content + recommendation information sequence 5021, content + recommendation information sequence 5022, and content + recommendation information sequence 5023 by enumerating all the positions in the content sequence where the recommendation information sequence can be inserted. After evaluating and ranking the multiple candidate recommendation sequences, the final target recommendation sequence 503 is determined for displaying the recommendation information to the user. Evaluating and ranking the multiple candidate recommendation sequences is to evaluate the comprehensive benefits of each candidate recommendation sequence through an algorithm model (for example, support vector machine, reinforcement learning or deep learning model, etc.), rank them based on the evaluation score results, and select the ranking result with the highest score to display to the user.

[0150] Continue to see Figure 4 In step 402, feature extraction is performed on the information of the recommended object, the content information, the recommendation information and the candidate recommendation sequence to obtain features of the recommended object, the content features, the recommendation information features and the candidate sequence features.

[0151] For example, the evaluation of the candidate recommendation sequence is realized through the evaluation model. The input of the evaluation model includes multi-dimensional features related to the recommendation information. The feature extraction of the recommended object information (equivalent to the above-mentioned object features), content information, recommendation information and candidate recommendation sequence is performed to obtain the features of the recommended object, content features, recommendation information features and candidate sequence features. The features of the recommended object are user (equivalent to the above-mentioned recommended object information) and context features, including user statistical features (for example: statistical features such as active days and historical clicks), as well as interest features such as the historical behavior sequence of content or advertisements. Context features include the current number of refreshes, refresh types, etc. Content features are basic features such as classification, label, type and statistical features such as clicks and exposure of each content in the current content recommendation sequence. Recommendation information features are features such as classification, type, sales method and unit price of the recommended product in the current scenario. Candidate recommendation sequence features are the insertion method of the current recommendation information, including the number of recommended information, the absolute position and interval of each recommended information, and the arrangement method of recommended information and content in the candidate recommendation sequence.

[0152] Continue to see Figure 4 In step 403, multi-task prediction processing is performed based on the features of the object to be recommended, the content features, the recommended information features and the candidate sequence features to obtain a multi-task prediction result.

[0153] For example, multi-task prediction processing is implemented through an evaluation network (for example, a deep learning network), with multi-task output, and multi-task prediction processing is performed based on the characteristics of the object to be recommended, the content characteristics, the recommendation information characteristics, and the candidate sequence characteristics. The prediction values ​​under multiple tasks are output at one time to obtain multi-task prediction results, and the prediction results of multiple tasks are weighted and summed to determine the comprehensive score of the candidate recommendation sequence.

[0154] For example, in an advertising recommendation scenario, multiple tasks can be advertising conversion rate, advertising exposure, content clicks, and content exposure. In an embodiment of the present application, modeling is performed with the recommendation effect index of the recommendation information and the similarity between the object features as the goal, corresponding to the estimation of multiple goals of the current candidate recommendation sequence in a user request. The input of the evaluation model is constructed based on multiple attribute features and candidate recommendation sequences, and the scores of the candidate recommendation sequences under multiple prediction tasks are output. For example: if there are currently high-priced advertisements, the insertion sequence with the advertisement position at the front tends to be selected to ensure that high-value advertisements are exposed and increase the attention of the platform. For users with low activity or newly registered users, the insertion sequence with the advertisement position at the back is selected to display more recommended information content to users.

[0155] For example, in order to focus on both the immediate and long-term benefits of recommendation information, the evaluation model predicts multiple targets for the candidate recommendation sequence through a dual deep Q network. The evaluation model is made to perform multi-task learning by introducing a reinforcement learning method. The modeling idea of ​​temporal difference learning is adopted, and the current immediate benefit and the estimated benefit of the next request are used as the predicted target value in the current state. At the same time, two target networks are used to avoid overestimation and delayed update. Temporal difference learning is a type of algorithm in reinforcement learning that updates the value estimate by comparing the predicted immediate benefit with the actual immediate benefit. Deep Q network is an algorithm based on deep learning and reinforcement learning. It is a variant of Q learning. It solves the convergence problem and computational complexity of Q learning in tasks with high-dimensional input space by using deep neural networks to approximate the Q function. Dual deep Q network uses two neural networks with the same structure but different parameters: one for prediction (evaluation network) and the other for generating target values ​​(target network). The parameters of the target network are regularly copied from the evaluation network, but the parameters of the target network are not updated frequently to stabilize the learning process. Reinforcement learning is a machine learning method in which an agent learns how to make decisions by interacting with the environment. Multi-task learning is a paradigm of machine learning. The core idea is to learn multiple related tasks simultaneously in the same model. Different tasks can share the underlying feature representation while maintaining task-specific processing at a higher level, thereby improving learning efficiency and the generalization ability of the model.

[0156] In some embodiments, see Figure 6 , Figure 6 It is a structural diagram of the evaluation model provided by the embodiment of the present application; the evaluation network 601 makes real-time decisions based on the current state 6011 and the current action 6012, and uses the current object feature 6041, the content feature 6042 and the recommendation information feature 6043 as the current state input to the evaluation network 601, and uses the candidate recommendation sequence feature 6044 as the current action input to the evaluation network 601. The evaluation network 601 accepts the current input, evaluates the Q value of all possible actions under the current state 6011, and outputs the Q value 6015 of each action. The Q value is the expected return that the model predicts after executing each action, which can help the algorithm make decisions and select the action that can obtain the maximum benefit based on the Q value. The target network 602 generates a target Q value 6025 based on the same input as the evaluation network 601. The target Q value 6025 is used to calculate the loss function 603 of the evaluation network 601, with the purpose of providing a stable training target. The loss function 603 can be represented by formula (1), which is described in detail below.

[0157]

[0158] Among them, Q(S t ,a tθ t ) represents the state S at time step t t and action a t Next, according to the current parameter θ of the evaluation network 601 t The estimated Q value is 6015, γ is the value of performing action a at time step t t The immediate reward obtained after that, y is the discount factor, which is used to convert future rewards into current values, usually between [0, 1]. Indicates that in the next state S t+1 Next, according to the current parameters of the target network 602 The maximum value among the estimated target Q values ​​6025, MSE is the mean square error, and the mean square error (Mean Squared Error) is used to measure the difference between the predicted Q value 6015 and the target Q value 6025. The neural network is trained by minimizing the difference between the predicted Q value 6015 and the target Q value 6025.

[0159] The evaluation network 601 has the same structure as the target network 602. The target Q value 6025 is calculated based on the next state 6021 and the next action 6022 after the current state is given and an action is performed, which helps the learning process to be more stable. After the evaluation network 601 and the target network 602 receive the input features, the feature splicing module 604 splices multiple types of features into combined features, that is, the original features are mapped to the low-dimensional space in the embedding layer 6013 and the embedding layer 6023 to obtain the embedding vector, and the neural network 6014 and the neural network 6024 are used to perform multi-task prediction processing based on the combined features, which is specifically implemented through the customized gate control component 605. For example, in the scenario of advertising insertion, the four goals of content exposure 6051, content click 6052, advertising exposure 6053 and advertising conversion rate 6054 are used for prediction, and the prediction results corresponding to the four goals are output. The prediction results of the four goals are weighted and summed to determine the evaluation score of the corresponding candidate recommendation sequence. The customized gating control component 605 is a model structure for multi-task learning, which outputs through a task-sharing expert network and an expert network corresponding to each task in combination with a gating mechanism. The customized gating control component 605 includes a set of shared expert networks, which are used by multiple tasks to learn common features and knowledge between tasks. Each task has a unique expert network responsible for learning the unique features and patterns of the task. For each task, the gating unit controls the input of the corresponding expert module and the shared module to obtain a weighted output as the prediction result corresponding to each target task.

[0160] In step 404, a target recommendation sequence is determined based on the multi-task prediction results.

[0161] For example, the content sequence in a request contains a large number of contents. The output of the evaluation model is designed to be divided into two parts: supervised learning and long-term benefit learning. The supervised learning part estimates the single-point target under each task, that is, the click-through rate of each content in a request. The click-through rate is the ratio of the number of clicks to the number of impressions. Long-term benefit learning is to use the estimated output of the evaluation model under multiple tasks as the request-level target, that is, the number of clicks on the total content in a request. Sum all the single-point targets and the predicted results of long-term benefits under each task. Each prediction result is the weighted sum of the predicted values ​​of multiple tasks. Determine the final output of the evaluation model under the task as the evaluation score of the corresponding candidate recommendation sequence. The candidate recommendation sequence with the highest evaluation score is used as the target recommendation sequence. The target recommendation sequence is used to recommend information to users.

[0162] In some embodiments, before step 404, in order to make the estimation results as diverse as possible, the variance of the estimation results of all candidate sequences of the model on each target is calculated as an auxiliary training loss function, which is multiplied by the coefficient α and then added to other loss functions. The coefficient α is used to control the importance of output diversity and to balance the model estimation accuracy and output diversity. At the initialization node α, α can be set to a smaller initial value so that the model training pays more attention to the estimation accuracy of other loss functions. As the model is trained, the estimation accuracy continues to improve, and α can be gradually increased so that the model pays more attention to the differences in the estimation of different sequences.

[0163] For example, based on online real data, we collect training samples, take one request as the granularity, record the features of the current user, content, and recommended content as s, record the features of multiple candidate recommendation sequences obtained as a, record each task in this request as k, and record the actual total target value of each task k (equivalent to the above-mentioned real sequence recommendation index) (e.g., number of clicks) as r k , the real single-point target value (equivalent to the real information recommendation index mentioned above) (the number of clicks on a single piece of information) is recorded as c k For example, suppose there are always 10 pieces of content in a request, and the user clicks on the first and third pieces of content in the content sequence, then r = 2, c = [1, 0, 1, 0, 0, 0, 0, 0, 0]. If the user continues to browse, the user, content, and recommended content features of the next request are recorded as s′, and the flag done (the label value above) is recorded as 0. Otherwise, if the browsing is stopped, done is recorded as 1. At this point, (s, a, r, c, s′, done) is stored as a sample for subsequent training of the evaluation model.

[0164] For example, the evaluation model Q is randomly initialized, and the parameters of the evaluation network and the target network are initialized with the same parameters, denoted as Q1 and Q2. N training data are taken from the training sample, and the features (s, a) are input into the model Q for prediction processing of multiple target tasks, and the output results of two parts corresponding to each task k are obtained: single point output (denoted as p k ) and long-term output, the sum of the single-point output and the long-term output is taken as the final output of the model Q in the task k and recorded as Q k (s, a). Based on the same training data, for each data sample, calculate the optimal information recommendation sequence in the next request s′, input s′ and all current candidate recommendation sequences a′ into the model Q, and calculate the prediction value corresponding to the current task k, recorded as Q k (s′, a′), combined with the heuristic principle, the fusion weight of multiple tasks ω is obtained k , select the candidate recommendation sequence with the largest weighted sum fusion score of multi-task output, denoted as

[0165] For example, the evaluation model is trained based on the objective function. The objective function contains three parts of the loss function, including: the long-term benefit loss1 of reinforcement learning under multiple objectives, that is, the loss function corresponding to the true total target value of each task k, which is calculated using the mean square error and can be implemented by formula (2), which is described in detail below.

[0166]

[0167] Here, the features of the current user, content, and recommended content are recorded as s, the features of the multiple candidate recommendation sequences obtained are recorded as a, each task in this request is recorded as k, and the actual total target value of each task k is recorded as r k , the user, content, and recommended content features of the next request are recorded as s′, and the flag done is recorded as 0, otherwise, if the browsing stops, done is recorded as 1, γ is the immediate reward obtained, ω k is the fusion weight of multiple tasks, N is a positive integer, representing N data taken from the training data set.

[0168] For example, the single-point benefit of partial supervised learning is taken as loss2, that is, the loss function corresponding to the single-point target value, which can be calculated by cross entropy loss and implemented by formula (3), as described below.

[0169]

[0170] Among them, CrossENtropy cross entropy, the true single-point target value of each task k is recorded as c k , single point output is denoted as p k ,ω kis the fusion weight of multiple tasks, N is a positive integer, representing N data taken from the training data set.

[0171] For example, the difference loss obtained by calculating the variance of the output is recorded as loss3, which can be implemented by formula (4), which is described in detail below.

[0172]

[0173] Among them, Variance is the calculated variance. All candidate recommendation sequences a′ in the next request s′ are input into the model Q, and the prediction value corresponding to the current task k is calculated, which is recorded as Q k (s′,a′),ω k is the fusion weight of multiple tasks, N is a positive integer, representing N data taken from the training data set.

[0174] For example, for multiple estimation tasks, based on the multi-task fusion weight ω k After weighted summation, the three loss functions are used as the final optimization target and updated using the gradient descent method. This can be achieved using formula (5), which is described in detail below.

[0175] loss=loss1+loss2+α*loss3 (5)

[0176] Among them, loss is the objective function (equivalent to the above total loss function), and the coefficient α is used to control the importance of output variability and balance the model estimation accuracy with output variability. At the initialization node α can be set to a smaller initial value so that the model training pays more attention to the estimation accuracy of other loss functions. The evaluation model is trained and updated based on the objective function, and the trained network parameters Q1 and Q2 are assigned to the evaluation network and target network of the evaluation model Q.

[0177] The method for processing recommendation information provided in the embodiment of the present application has the following beneficial effects:

[0178] By obtaining multiple candidate recommendation sequences, it is ensured that the possibility of providing multiple recommendation sequences is provided without changing the relative order of the content sequence and the recommendation information, which provides a basis for subsequent multi-task prediction and evaluation. By extracting features of the recommendation object information, content information, recommendation information and candidate recommendation sequences, it provides rich information input for the evaluation model, which helps to improve the accuracy of model prediction. Through multi-task learning, the evaluation model can output the prediction values ​​of multiple tasks in one prediction, while taking into account the estimated results under multiple tasks, reducing the negative impact of pursuing a single goal on other goals, and helping the model learn more general information representations to improve the generalization ability of the model. At the same time, it can improve training efficiency and avoid the traditional single-objective optimization that may need to maintain multiple models separately to meet different task requirements. The estimated results consider both the overall goal and the single-point goal, and sum all the single-point goals under each task with the estimated value of the long-term benefit as the final output of the model under the task. Reinforcement learning is used to enable the model to learn the long-term benefits of multiple tasks, and supervised learning is introduced to assist reinforcement learning training to improve the accuracy of model estimation. By effectively modeling both revenue and user experience, the trained evaluation model can more comprehensively evaluate different candidate recommendation sequences and rank them to achieve the reasonable maximization of the platform's recommended information conversion rate and content value.

[0179] The following is a description of an exemplary structure of the recommendation information processing device 455 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules in the processing device 455 of the recommendation information stored in the memory 450 may include: a feature extraction module 4551, which is used to obtain multiple candidate recommendation sequences, wherein the candidate recommendation sequence is the sorting result of content information and recommended material information; obtain object features, content information features and recommendation information features, wherein the object features are the features of the object information to be recommended; perform the following processing for each candidate recommendation sequence: extract features from the candidate recommendation sequence to obtain the first sequence features of the candidate recommendation sequence, and splice the object features, content information features, recommended content features and the first sequence features into combined features; a sequence determination module 4552, which is used to perform indicator prediction processing on the candidate recommendation sequence as a whole based on the combined features to obtain the sequence recommendation index of the candidate recommendation sequence; perform indicator prediction processing on each information in the candidate recommendation sequence based on the combined features to obtain the information recommendation index of each information in the candidate recommendation sequence; determine the target recommendation sequence based on the sequence recommendation index and the information recommendation index, wherein the target recommendation sequence is the candidate recommendation sequence for performing recommendation processing.

[0180] In some embodiments, the feature extraction module 4551 is also used to determine a preset insertion interval of the recommended material information in the content information sequence based on the information of the object to be recommended and the sequence length of the content information sequence, wherein the content information sequence includes multiple content information, and the preset insertion interval is an interval composed of positions in the content information sequence for inserting the recommended material information; enumerate the order of all insertion positions of the recommended material information based on the preset insertion interval; and perform multiple insertions of the recommended material information on the content information sequence according to the order of each insertion position to obtain multiple candidate recommendation sequences, wherein the insertion position of each insertion of the recommended material information is different.

[0181] In some embodiments, the sequence determination module 4552 is also used to perform nonlinear transformation on the combined features to obtain a first nonlinear feature; for multiple types of target tasks, perform prediction processing based on the first nonlinear feature to obtain a first prediction value for each target task; and combine each first prediction value to obtain a sequence recommendation index for the candidate recommendation sequence.

[0182] In some embodiments, the sequence determination module 4552 is also used to perform the following processing for each information in the candidate recommendation sequence: extract features from the combined features based on the information to obtain sub-combination features corresponding to the information, perform nonlinear transformation on the sub-combination features to obtain a second nonlinear feature of the information; for multiple types of target tasks, perform prediction processing based on the second nonlinear feature to obtain a second prediction value for each target task; and combine multiple second prediction values ​​to obtain an information recommendation index for the information in the candidate recommendation sequence.

[0183] In some embodiments, the sequence determination module 4552 is also used to perform linear conversion on the sequence recommendation index and the information recommendation index for each candidate recommendation sequence to obtain a total recommendation index for the candidate recommendation sequence; and select at least one candidate recommendation sequence with the highest total recommendation index as the target recommendation sequence.

[0184] In some embodiments, feature extraction, splicing and indicator prediction processing are implemented through a deep learning model; before obtaining multiple candidate recommendation sequences, the model training module 4553 is also used to obtain a training sample set, wherein the training sample set includes a first sample recommendation sequence and an actual recommendation indicator corresponding to the first sample recommendation sequence, and the actual recommendation indicator is composed of an actual sequence recommendation indicator and an actual information recommendation indicator; based on the first sample recommendation sequence, the deep learning model is called to perform prediction processing to obtain a first predicted recommendation indicator; based on the first predicted recommendation indicator and the actual recommendation indicator, the total loss function of the deep learning model is determined; based on the total loss function, the parameters of the deep learning model are updated to obtain a trained deep learning model.

[0185] In some embodiments, the training sample set also includes: a second sample recommendation sequence and a sample label value corresponding to the second sample recommendation sequence, the second sample recommendation sequence being a sequence used to perform recommendation processing after the first sample recommendation sequence; the model training module 4553 is also used to call the deep learning model for prediction processing based on the second sample recommendation sequence to obtain a second prediction recommendation index; determine the mean square error loss of the deep learning model according to the first prediction recommendation index and the second prediction recommendation index; determine the cross entropy loss of the mean square error loss of the deep learning model according to the first prediction recommendation index and the actual recommendation index; determine the difference loss of the mean square error loss of the deep learning model according to the variance between each second prediction recommendation index; perform linear conversion on the mean square error loss, cross entropy loss and difference loss to obtain a total loss function.

[0186] In some embodiments, the model training module 4553 is also used to determine a target prediction recommendation indicator from the second prediction recommendation indicators of multiple second sample recommendation sequences; determine the label value of the second sample recommendation sequence corresponding to the target prediction recommendation indicator, and extract the actual sequence recommendation indicator from the first prediction recommendation indicator; perform the following processing for each first sample recommendation sequence: determine the first product between the target prediction recommendation indicator and the label value corresponding to the first sample recommendation sequence; determine the sum of the actual sequence recommendation indicator and the first product, and use the square of the difference between the sum and the first prediction recommendation indicator of the first sample recommendation sequence as a sub-mean square error loss; determine the mean square error loss based on each sub-mean square error loss.

[0187] In some embodiments, each second prediction recommendation indicator includes sample prediction values ​​of multiple sample tasks; the model training module 4553 is also used to obtain the preconfigured weight value of each sample task, and perform weighted calculation based on each preconfigured weight value and the corresponding sample prediction value to obtain the weighted sum of each second sample recommendation sequence; the second prediction recommendation indicator of the second sample recommendation sequence with the highest enhanced summation result is used as the target prediction recommendation indicator.

[0188] In some embodiments, the first prediction recommendation indicator includes: the prediction information recommendation indicator of each sample information in the sample recommendation sequence; the actual recommendation indicator includes: the actual information recommendation indicator of each sample information; the model training module 4553 is also used to perform the following processing for each sample information: based on the difference between the prediction information recommendation indicator and the actual information recommendation indicator, determine the sub-cross entropy loss; determine the cross entropy loss of the mean square error loss of the deep learning model based on each sub-cross entropy loss.

[0189] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device executes the method for processing recommendation information described in the embodiment of the present application.

[0190] The present application embodiment provides a computer-readable storage medium in which computer executable instructions or computer programs are stored. When the computer executable instructions or computer programs are executed by a processor, the processor will execute the method for processing recommendation information provided by the present application embodiment, for example, Figure 3A The method for processing the recommendation information is shown.

[0191] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0192] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0193] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0194] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0195] In summary, through the embodiments of the present application, by combining object features, content information features, recommendation information features and sequence features, the recommendation sequence is understood and analyzed more comprehensively, the sequence recommendation index is predicted based on the combined features, and the recommendation index of each information in the sequence is predicted based on the combined features. A comprehensive evaluation is performed on the recommendation indicators of the sequence as a whole and individually, the accuracy of the prediction results is improved, the target recommendation sequence is determined dynamically and accurately, and the accuracy of information recommendation processing is improved.

[0196] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for processing recommendation information, characterized in that: The method comprises: Acquire multiple candidate recommendation sequences, wherein the candidate recommendation sequences are sorted results of the content information and the recommended material information; Acquire object features, content information features, and recommendation information features, wherein the object features are features of the object information to be recommended; For each of the candidate recommendation sequences, the following processing is performed: feature extraction is performed on the candidate recommendation sequence to obtain a first sequence feature of the candidate recommendation sequence, and the object feature, the content information feature, the recommended content feature and the first sequence feature are spliced ​​into a combined feature; Based on the combined features, an indicator prediction process is performed on the candidate recommendation sequence as a whole to obtain a sequence recommendation indicator of the candidate recommendation sequence; Based on the combined features, an index prediction process is performed on each information in the candidate recommendation sequence to obtain an information recommendation index for each information in the candidate recommendation sequence; A target recommendation sequence is determined based on the sequence recommendation index and the information recommendation index, wherein the target recommendation sequence is a candidate recommendation sequence for performing a recommendation process.

2. The method according to claim 1, characterized in that The obtaining of multiple candidate recommendation sequences includes: Based on the information of the object to be recommended and the sequence length of the content information sequence, determining a preset insertion interval of the recommended material information in the content information sequence, wherein the content information sequence includes a plurality of the content information, and the preset insertion interval is an interval consisting of positions in the content information sequence for inserting the recommended material information; Enumerating the order of all insertion positions of the recommended material information based on the preset insertion interval; According to the order of each of the insertion positions, the process of inserting the recommended material information is performed multiple times on the content information sequence to obtain multiple candidate recommendation sequences, wherein the insertion position of the process of inserting the recommended material information is different each time.

3. The method according to claim 1, characterized in that The step of performing an indicator prediction process on the candidate recommendation sequence as a whole based on the combined features to obtain a sequence recommendation indicator of the candidate recommendation sequence includes: Performing nonlinear transformation on the combined feature to obtain a first nonlinear feature; For multiple types of target tasks, prediction processing is performed based on the first nonlinear feature to obtain a first prediction value for each of the target tasks; Each of the first prediction values ​​is combined to obtain a sequence recommendation index of the candidate recommendation sequence.

4. The method according to claim 1, characterized in that: The performing an index prediction process on each information in the candidate recommendation sequence based on the combined features to obtain an information recommendation index for each information in the candidate recommendation sequence includes: The following processing is performed for each information in the candidate recommendation sequence: Extract the combined feature according to the information to obtain a sub-combination feature corresponding to the information, and perform nonlinear transformation on the sub-combination feature to obtain a second nonlinear feature of the information; For multiple types of target tasks, prediction processing is performed based on the second nonlinear feature to obtain a second prediction value for each of the target tasks; A plurality of the second prediction values ​​are combined to obtain an information recommendation index of the information in the candidate recommendation sequence.

5. The method according to claim 1, characterized in that: The determining the target recommendation sequence based on the sequence recommendation index and the information recommendation index includes: For each of the candidate recommendation sequences, linearly transform the sequence recommendation index and the information recommendation index to obtain a total recommendation index of the candidate recommendation sequence; At least one of the candidate recommendation sequences with the highest total recommendation index is selected as the target recommendation sequence.

6. The method according to any one of claims 1 to 5, characterized in that: The feature extraction, the splicing and the indicator prediction processing are implemented by a deep learning model; Before acquiring a plurality of candidate recommendation sequences, the method further includes: Acquire a training sample set, wherein the training sample set includes the first sample recommendation sequence and an actual recommendation index corresponding to the first sample recommendation sequence, wherein the actual recommendation index is composed of an actual sequence recommendation index and an actual information recommendation index; Based on the first sample recommendation sequence, the deep learning model is called to perform prediction processing to obtain a first prediction recommendation index; Determining a total loss function of the deep learning model based on the first predicted recommendation index and the actual recommendation index; The parameters of the deep learning model are updated based on the total loss function to obtain the trained deep learning model.

7. The method according to claim 6, characterized in that The training sample set further includes: a second sample recommendation sequence and a sample label value corresponding to the second sample recommendation sequence, wherein the second sample recommendation sequence is a sequence used to perform recommendation processing after the first sample recommendation sequence; The determining, based on the first predicted recommendation index and the actual recommendation index, a total loss function of the deep learning model includes: Based on the second sample recommendation sequence, the deep learning model is called to perform prediction processing to obtain a second prediction recommendation index; Determine the mean square error loss of the deep learning model according to the first prediction recommendation indicator and the second prediction recommendation indicator; Determine the cross entropy loss of the mean square error loss of the deep learning model according to the first predicted recommendation index and the actual recommendation index; Determining a difference loss of a mean square error loss of the deep learning model according to the variance between each of the second prediction recommendation indicators; The mean square error loss, the cross entropy loss, and the difference loss are linearly transformed to obtain the total loss function.

8. The method according to claim 7, characterized in that The determining the mean square error loss of the deep learning model according to the first prediction recommendation indicator and the second prediction recommendation indicator includes: Determining a target prediction recommendation index from the second prediction recommendation indexes of the plurality of second sample recommendation sequences; Determine a label value of the second sample recommendation sequence corresponding to the target prediction recommendation indicator, and extract the actual sequence recommendation indicator from the first prediction recommendation indicator; The following processing is performed for each of the first sample recommendation sequences: Determine a first product between the target prediction recommendation index and the label value corresponding to the first sample recommendation sequence; determining a sum of the actual sequence recommendation index and the first product, and taking the square of a difference between the sum and the first predicted recommendation index of the first sample recommendation sequence as a sub-mean square error loss; The mean square error loss is determined based on each of the sub-mean square error losses.

9. The method according to claim 8, characterized in that Each of the second prediction recommendation indicators includes sample prediction values ​​of multiple sample tasks; The determining a target prediction recommendation index from the second prediction recommendation indexes of the plurality of second sample recommendation sequences includes: Obtaining a preconfigured weight value for each of the sample tasks, and performing weighted calculation based on each of the preconfigured weight values ​​and a corresponding sample prediction value to obtain a weighted sum result of each of the second sample recommendation sequences; The second prediction recommendation index of the second sample recommendation sequence with the highest enhanced summation result is used as the target prediction recommendation index.

10. The method according to claim 7, characterized in that The first prediction recommendation index includes: the prediction information recommendation index of each sample information in the sample recommendation sequence; the actual recommendation index includes: the actual information recommendation index of each sample information; The determining the cross entropy loss of the mean square error loss of the deep learning model according to the first predicted recommendation index and the actual recommendation index includes: The following processing is performed for each sample information: based on the difference between the predicted information recommendation index and the actual information recommendation index, a sub-cross entropy loss is determined; The cross entropy loss of the mean square error loss of the deep learning model is determined according to each of the sub-cross entropy losses.

11. A device for processing recommendation information, characterized in that: The device comprises: The feature extraction module is used to obtain multiple candidate recommendation sequences, wherein the candidate recommendation sequence is the sorting result of the content information and the recommended material information; obtain object features, content information features and recommendation information features, wherein the object features are features of the object information to be recommended; perform the following processing on each of the candidate recommendation sequences: extract features from the candidate recommendation sequence to obtain a first sequence feature of the candidate recommendation sequence, and splice the object features, the content information features, the recommended content features and the first sequence features into a combined feature; A sequence determination module is used to perform indicator prediction processing on the candidate recommendation sequence as a whole based on the combined features to obtain a sequence recommendation indicator of the candidate recommendation sequence; perform indicator prediction processing on each information in the candidate recommendation sequence based on the combined features to obtain an information recommendation indicator for each information in the candidate recommendation sequence; and determine a target recommendation sequence based on the sequence recommendation indicator and the information recommendation indicator, wherein the target recommendation sequence is a candidate recommendation sequence for performing recommendation processing.

12. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions or computer programs; The processor is configured to implement the method for processing recommendation information according to any one of claims 1 to 10 when executing the computer executable instructions or computer programs stored in the memory.

13. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the method for processing recommendation information according to any one of claims 1 to 10 is implemented.

14. A computer program product comprising computer executable instructions or a computer program, characterized in that When the computer executable instructions or computer program are executed by a processor, the method for processing recommendation information according to any one of claims 1 to 10 is implemented.

Images