A recommendation method, training method, and device

Abstract

The application provides a recommendation method, a training method and a device in the field of artificial intelligence, which are used for setting multiple tower expert networks in a recommendation model, thereby avoiding overfitting problems caused by data sparsity and improving output stability of the model. The method comprises the following steps: obtaining input data; subsequently, taking the input data as input of a recommendation model, and outputting recommendation information, wherein the recommendation model is a multi-task model, the recommendation model comprises a shared feature extraction network, multiple tower expert networks corresponding to each task respectively, and a task-specific feature extraction network, the shared feature extraction network is used for extracting shared features from the input data, the task-specific feature extraction network is used for extracting tower expert shared features of a single task from the input data, the tower expert network is used for performing a corresponding task based on the features extracted by the task-specific feature extraction network and the shared feature extraction network, and the outputs of the multiple tower expert networks corresponding to the multiple tasks respectively are fused after weighting to obtain the recommendation information.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular, to a recommendation method, a training method and an apparatus. BACKGROUND

[0002] In a machine learning system, based on input data and labels, the parameters of a machine learning model are trained by an optimization method such as gradient descent, and when the model parameters converge, the model can be used to complete the prediction of unknown data. Taking click rate prediction in a personalized recommendation system as an example, the input data includes user features, item features and context features, and the output is a recommendation list generated by the user.

[0003] How to predict a personalized recommendation list according to the user's preferences has an important influence on improving the user experience and platform revenue of the recommendation system. In the traditional Internet effect advertisement click bid mode, only the click rate is estimated, and the real appeal of the advertiser is not expressed. More and more advertisers have begun to focus on the effect of deep conversion behavior, that is, the conversion rate of the advertisement needs to be estimated.

[0004] However, the existing multi-task learning technical solutions mostly adopt a hard parameter sharing mechanism, and multi-task learning may cause negative transfer phenomenon, that is, the information sharing between tasks will affect the performance of the network. Therefore, a more flexible parameter sharing mechanism is needed. In addition, data sparsity will make the conversion rate estimation model prone to overfitting. Therefore, how to obtain more accurate estimation results has become a problem to be solved. SUMMARY

[0005] The present application provides a recommendation method, a training method and an apparatus, which are used to set multiple tower expert networks in a recommendation model, thereby avoiding the overfitting problem caused by data sparsity and improving the output stability of the model.

[0006] Therefore, in a first aspect, the present application provides a recommendation method, comprising: obtaining input data, the input data including information of a user; subsequently, taking the input data as input of a recommendation model, and outputting recommendation information for the user;

[0007] The recommendation model is used to perform a plurality of tasks of making recommendations for users, the recommendation model comprises a shared feature extraction network, a plurality of tower expert networks corresponding to each task respectively, and a task-specific feature extraction network corresponding to each task respectively, an output end of the shared feature extraction network is connected to an input end of each tower expert network, and an input end of the plurality of tower expert networks corresponding to each task is also connected to an output end of the task-specific feature extraction network corresponding to each task respectively; the plurality of tower expert networks have different parameters, the shared feature extraction network is used to extract shared features from input data, the shared features are shared by the plurality of tower expert networks corresponding to the plurality of tasks, the task-specific feature extraction network is used to extract tower expert shared features from the input data, the tower expert shared features are shared by the plurality of tower expert networks corresponding to a single task, the plurality of tower expert networks are used to perform the corresponding task based on the features extracted by the task-specific feature extraction network and the shared feature extraction network corresponding to the task, and the outputs of the plurality of tower expert networks corresponding to the plurality of tasks are fused after being weighted to obtain recommendation information.

[0008] Therefore, in the embodiments of the present application, a plurality of tower expert networks with different parameters are set for each task, so that the output accuracy of the recommendation model is improved through the output results of the plurality of tower expert networks. Even in the case of sparse data, more stable output results can be obtained through the output results of the plurality of tower expert structures.

[0009] In a possible implementation, the recommendation model further comprises a tower feature extraction network corresponding to each of the plurality of tower expert networks, the tower feature extraction network is used to extract features related to the task performed by the corresponding tower expert network from the input data, the tower feature extraction networks corresponding to the plurality of tower expert networks have different parameters, and the input of each tower expert network further comprises the features extracted by the corresponding tower feature extraction network.

[0010] Therefore, in the embodiments of the present application, a separate feature extraction network is set for each tower, so that the required features can be extracted for each tower expert network, thereby further improving the accuracy of the output results of the recommendation model.

[0011] In a possible implementation, the recommendation model further comprises a plurality of gate networks, each tower expert network corresponds to a gate network, and the gate network is used to fuse the outputs of the task-specific feature extraction network, the shared feature extraction network, and the tower feature extraction network corresponding to the task, and the fusion result is used as the input of the corresponding tower expert network.

[0012] Therefore, in the embodiments of the present application, the weights of various features input into the tower expert network are controlled through the gate network, so that the required features can be adaptively extracted for different tower expert networks, and the output accuracy of each tower expert network is improved.

[0013] In a possible implementation, before the input data is acquired, the method further includes: iteratively training an initial model to obtain a recommended model, the initial model having the same structure as the recommended model.

[0014] In one iteration of the initial model, the training sample is input into the initial model to obtain a first output result, a first loss value between the label of the training sample and the first output result is obtained, a plurality of second output results output by the plurality of tower expert networks corresponding to the plurality of tasks are obtained, a plurality of second loss values between the first output result and the plurality of second output results are obtained, and the initial model is updated according to the first loss value and the plurality of second loss values to obtain the initial model after the current iteration.

[0015] Therefore, in the training process, the loss value between the overall output result of the recommended model and the output result of the sub-network can be used as a constraint to update each tower expert network, so that the output of each sub-network is closer to the overall output of the recommended model, the model convergence speed is improved, and the model training can be efficiently implemented.

[0016] In a possible implementation, the plurality of tasks include predicting a click rate and predicting conversion information, the click rate is a probability of a user clicking a target object, and the conversion information includes a conversion rate or a conversion time length, the conversion rate is a probability of the user performing a conversion operation on the target object after clicking the target object, and the conversion time length is a time length of the user staying after performing the conversion operation on the target object.

[0017] Therefore, the recommended model provided in the application can be used to perform a plurality of tasks, such as predicting a click rate and predicting conversion information, so that the recommended object suitable for the user can be accurately predicted, and the user experience is improved.

[0018] In a second aspect, the application provides a training method, including: acquiring a training set, the training set including a plurality of samples and a label corresponding to each sample; and iteratively training an initial model by inputting the training set into the initial model to obtain a recommended model.

[0019] The recommended model is used to perform a plurality of tasks of recommending for a user, and includes a shared feature extraction network, a plurality of tower expert networks corresponding to each task, and a task-specific feature extraction network corresponding to each task. The output end of the shared feature extraction network is connected to the input end of each tower expert network, and the input end of each tower expert network corresponding to each task is further connected to the output end of the task-specific feature extraction network corresponding to each task.

[0020] In each iteration process, the sample in the training set is taken as the input of the initial model obtained in the last iteration, a first loss value between the first output result of the model obtained in the last iteration and the label of the input sample is obtained, a second loss value between the second output result of each tower expert network and the first output result is obtained, the model obtained in the last iteration is updated according to the second loss value and the first loss value, and a model of the current iteration is obtained.

[0021] Therefore, in the embodiments of the present application, when updating the overall model, the loss value between the output result of the model and the output result of each tower expert is calculated, the loss value is used as a constraint to update each tower expert, so that the output result of each tower expert is closer to the overall output result of the model, which can accelerate the convergence of the model and realize efficient training of the model.

[0022] In a possible implementation, the recommendation model further includes a tower feature extraction network corresponding to each of the plurality of tower expert networks, the input end of each tower expert network is further connected to the output end of the corresponding tower feature extraction network, the tower feature extraction network is configured to extract features related to the task performed by the corresponding tower expert network from the input data, and the tower feature extraction network parameters corresponding to the plurality of tower expert networks are different.

[0023] Therefore, in the embodiments of the present application, a separate feature extraction network is set for each tower, so that the required features can be extracted for each tower expert network, thereby further improving the accuracy of the output result of the recommendation model.

[0024] In a possible implementation, the recommendation model further includes a plurality of gate networks, each tower expert network corresponds to a gate network, and the gate network is configured to fuse the outputs of the corresponding task-specific feature extraction network, the shared feature extraction network and the tower feature extraction network, and take the fusion result as the input of the corresponding tower expert network.

[0025] Therefore, in the embodiments of the present application, the weights of various features input into the tower expert network are controlled by the gate network, so that the required features can be adaptively extracted for different tower expert networks, and the output accuracy of each tower expert network is improved.

[0026] In a possible implementation, the plurality of tasks include predicting a click rate and predicting conversion information, the click rate is a probability that a user clicks a target object, and the conversion information includes a conversion rate or a conversion duration, the conversion rate is a probability that the user performs a conversion operation on the target object after clicking the target object, and the conversion duration includes a duration that the user stays after performing a conversion operation on the target object after clicking the target object.

[0027] Therefore, the recommendation model provided in the application can be used to perform multiple tasks such as predicting click rate and conversion information, so that the user can be accurately predicted with suitable recommended objects, and the user experience is improved.

[0028] In a third aspect, the application provides a recommendation device, comprising:

[0029] The acquisition module is configured to acquire input data, wherein the input data comprises information of a user.

[0030] The recommendation module is configured to take the input data as an input of the recommendation model, and output recommendation information for the user.

[0031] The recommendation model is configured to perform multiple tasks of recommending for the user, and comprises a shared feature extraction network, multiple tower expert networks corresponding to each task respectively, and task-specific feature extraction networks corresponding to each task respectively. An output end of the shared feature extraction network is connected to an input end of each tower expert network. An input end of each of the multiple tower expert networks is further connected to an output end of the task-specific feature extraction network corresponding to each task. Parameters of the multiple tower expert networks are different. The shared feature extraction network is configured to extract shared features from the input data, and the shared features are shared by the multiple tower expert networks corresponding to the multiple tasks. The task-specific feature extraction network is configured to extract tower expert shared features from the input data, and the tower expert shared features are shared by the multiple tower expert networks corresponding to a single task. The multiple tower expert networks are configured to perform corresponding tasks based on the features extracted by the task-specific feature extraction network corresponding to the task and the shared feature extraction network. The output of the multiple tower expert networks corresponding to the multiple tasks is fused after being weighted to obtain the recommendation information.

[0032] In a possible implementation, the recommendation model further comprises a tower feature extraction network corresponding to each of the multiple tower expert networks. The tower feature extraction network is configured to extract features related to the task performed by the corresponding tower expert network from the input data. Parameters of the tower feature extraction networks corresponding to the multiple tower expert networks are different. The input of each tower expert network further comprises the features extracted by the corresponding tower feature extraction network.

[0033] In a possible implementation, the recommendation model further comprises multiple gate networks, one corresponding to each of the tower expert networks. The gate network is configured to fuse the outputs of the corresponding task-specific feature extraction network, the shared feature extraction network, and the tower feature extraction network, and take the fusion result as the input of the corresponding tower expert network.

[0034] In a possible implementation, the device further comprises a training module configured to iteratively train an initial model to obtain the recommendation model. The structure of the initial model is the same as that of the recommendation model.

[0035] In the iteration training process of the initial model, the training sample is taken as an input of the initial model, and a first output result is output; a first loss value between the label of the training sample and the first output result is obtained; a plurality of second output results output by the tower expert networks corresponding to the plurality of tasks are obtained; a plurality of second loss values between the first output and the plurality of second output results are obtained; and the initial model is updated according to the first loss value and the second loss value, to obtain the initial model after the current iteration.

[0036] In a possible implementation, the plurality of tasks include predicting a click rate and predicting conversion information, the click rate is a probability of a user clicking a target object, and the conversion information includes a conversion rate or a conversion duration, the conversion rate is a probability of the user performing a conversion operation on the target object after clicking the target object, and the conversion duration includes a duration of the user staying after performing the conversion operation on the target object.

[0037] In a fourth aspect, the present application provides a training device, comprising:

[0038] The acquisition module is configured to acquire a training set, the training set comprising a plurality of samples and a label corresponding to each sample.

[0039] The training module is configured to perform iterative training on the initial model by taking the training set as an input of the initial model, to obtain a recommendation model.

[0040] The recommendation model is configured to perform a plurality of tasks of making recommendations for users, and the recommendation model comprises a shared feature extraction network, a plurality of tower expert networks corresponding to each task, and a task-specific feature extraction network corresponding to each task. An output end of the shared feature extraction network is connected to an input end of each tower expert network, and an input end of each tower expert network is further connected to an output end of the task-specific feature extraction network corresponding to each task. In each iteration process, a sample in the training set is taken as an input of the initial model obtained in the last iteration, a first loss value between a first output result of the model obtained in the last iteration and a label of the input sample is obtained, a second loss value between a second output result of each tower expert network and the first output result is obtained, and the model obtained in the last iteration is updated according to the second loss value and the first loss value, to obtain a model of the current iteration.

[0041] In a possible implementation, the recommendation model further comprises a tower feature extraction network corresponding to each tower expert network, the tower feature extraction network is configured to extract features related to a task performed by the corresponding tower expert network from input data, and parameters of the tower feature extraction networks corresponding to the plurality of tower expert networks are different.

[0042] In a possible implementation, the recommendation model further comprises a plurality of gating networks, each tower expert network corresponding to a gating network, and the gating network is configured to fuse outputs of the corresponding task-specific feature extraction network, the shared feature extraction network, and the tower feature extraction network, and take the fusion result as an input of the corresponding tower expert network.

[0043] In a possible implementation, the plurality of tasks comprises predicting a click rate and predicting conversion information, the click rate is a probability of a user clicking a target object, and the conversion information comprises a conversion rate or a conversion duration, the conversion rate is a probability of the user performing a conversion operation on the target object after clicking the target object, and the conversion duration comprises a duration of the user staying after performing the conversion operation on the target object.

[0044] In a fifth aspect, the present application provides a recommendation model, which is configured to perform a plurality of tasks of making recommendations for a user, and the recommendation model comprises a shared feature extraction network, a plurality of tower expert networks corresponding to each task, and a task-specific feature extraction network corresponding to each task, an output end of the shared feature extraction network is connected to an input end of each tower expert network, and an input end of each tower expert network corresponding to each task is further connected to an output end of the task-specific feature extraction network corresponding to each task; parameters of the plurality of tower expert networks are different, the shared feature extraction network is configured to extract features from input data, the task-specific feature extraction network is configured to extract features related to each task from the input data, the plurality of tower expert networks are configured to perform corresponding tasks based on the features extracted by the task-specific feature extraction network and the shared feature extraction network, and an output of the plurality of tower expert networks corresponding to the plurality of tasks is fused after weighting to obtain recommendation information.

[0045] Therefore, in the embodiments of the present application, a plurality of tower expert networks with different parameters are set for each task, so that the output accuracy of the recommendation model is improved through the output results of the plurality of tower expert networks. Even in the case of sparse data, more stable output results can be obtained through the output results of the plurality of tower expert structures.

[0046] In a possible implementation, the recommendation model further comprises a tower feature extraction network corresponding to each tower expert network, the tower feature extraction network is configured to extract features related to a task performed by the corresponding tower expert network from input data, and parameters of the tower feature extraction networks corresponding to the plurality of tower expert networks are different.

[0047] Therefore, in the embodiments of the present application, a separate feature extraction network is set for each tower, so that the required features can be extracted for each tower expert network, thereby further improving the accuracy of the output results of the recommendation model.

[0048] In a possible implementation, the recommendation model further includes a plurality of gating networks, each tower expert network corresponds to a gating network, and the gating network is configured to fuse outputs of the corresponding task-specific feature extraction network, the shared feature extraction network, and the tower feature extraction network, and take the fusion result as input of the corresponding tower expert network.

[0049] Therefore, in the embodiments of the present application, the weights of various features input into the tower expert network are controlled by the gating network, so that the required features can be adaptively extracted for different tower expert networks, and the output accuracy of each tower expert network is improved.

[0050] In a possible implementation, the plurality of tasks include predicting a click rate and predicting conversion information, the click rate is a probability of a user clicking a target object, and the conversion information includes a conversion rate or a conversion duration, the conversion rate is a probability of the user performing a conversion operation on the target object after clicking the target object, and the conversion duration includes a duration of the user staying after performing the conversion operation on the target object.

[0051] Therefore, the recommendation model provided by the present application can be used to perform a plurality of tasks, such as predicting a click rate and predicting conversion information, so that the user can be accurately predicted with an adaptive recommended object, and the user experience is improved.

[0052] In a fifth aspect, an embodiment of the present application provides a recommendation device, including a processor and a memory, wherein the processor and the memory are interconnected through a circuit, and the processor invokes program codes in the memory to perform functions related to processing in the recommendation method shown in any one of the first aspect.

[0053] In a sixth aspect, an embodiment of the present application provides a recommendation device, including a processor and a memory, wherein the processor and the memory are interconnected through a circuit, and the processor invokes program codes in the memory to perform functions related to processing in the training method shown in any one of the second aspect.

[0054] In a seventh aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, wherein the processor and the memory are interconnected through a circuit, and the processor invokes program codes in the memory to perform functions related to processing in the recommendation method shown in any one of the first aspect.

[0055] In an eighth aspect, an embodiment of the present application provides a recommendation device, which can also be referred to as a digital processing chip or a chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface, and the program instructions are executed by the processing unit. The processing unit is configured to perform functions related to processing in the first aspect or any optional implementation of the first aspect.

[0056] In a ninth aspect, an embodiment of the present application provides a training device, which can also be referred to as a digital processing chip or a chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface. The program instructions are executed by the processing unit. The processing unit is configured to perform the processing-related functions in the second aspect or any optional implementation manner of the second aspect.

[0057] In a tenth aspect, an embodiment of the present application provides a computer-readable storage medium including instructions. When the instructions are executed on a computer, the computer performs the method in the first aspect or any optional implementation manner of the second aspect.

[0058] In an eleventh aspect, an embodiment of the present application provides a computer program product including instructions. When the instructions are executed on a computer, the computer performs the method in the first aspect or any optional implementation manner of the second aspect. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 An artificial intelligence subject framework schematic diagram for application of the present application;

[0060] Figure 2 A system architecture schematic diagram provided by the present application;

[0061] Figure 3 Another system architecture schematic diagram provided by the present application;

[0062] Figure 4 An application scenario schematic diagram provided by the present application;

[0063] Figure 5 A recommendation model structure schematic diagram provided by the present application;

[0064] Figure 6 Another recommendation model structure schematic diagram provided by the present application;

[0065] Figure 7 A gating network structure schematic diagram provided by the present application;

[0066] Figure 8 A training method flowchart schematic diagram provided by the present application;

[0067] Figure 9 Another training method flowchart schematic diagram provided by the present application;

[0068] Figure 10 A recommendation method flowchart schematic diagram provided by the present application;

[0069] Figure 11 Another application scenario schematic diagram provided by the present application;

[0070] Figure 12 Another application scenario provided in the present application is shown in the figure;

[0071] Figure 13 A structure diagram of a recommendation device provided in the present application is shown in the figure;

[0072] Figure 14 A structure diagram of a training device provided in the present application is shown in the figure;

[0073] Figure 15 A structure diagram of another recommendation device provided in the present application is shown in the figure;

[0074] Figure 16 A structure diagram of another training device provided in the present application is shown in the figure;

[0075] Figure 17 A structure diagram of a chip provided in the present application is shown in the figure. DETAILED DESCRIPTION

[0076] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0077] The recommendation method provided in the present application can be applied in an artificial intelligence (AI) scene. AI is a theory, method, technology and application system for simulating, extending and expanding human intelligence by using a digital computer or a machine controlled by a digital computer, perceiving an environment, acquiring knowledge and using the knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making. The research in the field of artificial intelligence includes robots, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, AI basic theory, etc.

[0078] First, the overall workflow of the artificial intelligence system is described, please refer to Figure 1 , Figure 1The structure of the artificial intelligence subject framework is shown. The artificial intelligence subject framework is described below from two dimensions of "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis). The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, data undergoes a condensation process of "data-information-knowledge-wisdom". The "IT value chain" reflects the value brought by artificial intelligence to the information technology industry from the underlying infrastructure of human intelligence, information (provision and processing technology implementation) to the industrial ecological process of the system.

[0079] (1) Infrastructure

[0080] The infrastructure provides computing power support for the artificial intelligence system, realizes communication with the external world, and realizes support through the underlying platform. Communication with the outside world through sensors; computing power is provided by intelligent chips (CPU, NPU, GPU, ASIC, FPGA, etc. Hardware acceleration chips); the underlying platform includes distributed computing framework and network related platform guarantee and support, which can include cloud storage and computing, interconnection network, etc. For example, sensors and external communication acquire data, which are provided to intelligent chips in the distributed computing system provided by the underlying platform for calculation.

[0081] (2) Data

[0082] The data on the upper layer of the infrastructure is used to represent the data source in the field of artificial intelligence. Data involves graphics, images, speech, text, and also involves Internet of Things data of traditional devices, including business data of existing systems and sensing data such as force, displacement, liquid level, temperature, humidity, etc.

[0083] (3) Data processing

[0084] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.

[0085] Among them, machine learning and deep learning can symbolize and formalize intelligent information modeling, extraction, preprocessing, training, etc.

[0086] Reasoning refers to the process of simulating human intelligent reasoning methods in a computer or intelligent system, using formalized information to perform machine thinking and solve problems according to reasoning control strategies, and the typical function is search and matching.

[0087] Decision-making refers to the process of decision-making after intelligent information reasoning, which usually provides functions such as classification, sorting, prediction, etc.

[0088] (4) General capabilities

[0089] After the data is processed by the above-mentioned data processing, further based on the result of the data processing, some general capabilities can be formed, which can be an algorithm or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0090] (5) Intelligent product and industry application

[0091] Intelligent product and industry application refers to the product and application of artificial intelligence system in various fields, which is the packaging of the overall solution of artificial intelligence, and realizes the application of intelligent information decision productization. Its application fields mainly include: intelligent terminal, intelligent transportation, intelligent medical treatment, automatic driving, smart city, etc.

[0092] The embodiments of the present application relate to the related applications of neural networks. In order to better understand the scheme of the embodiments of the present application, the related terms and concepts of neural networks that may be involved in the embodiments of the present application are introduced below.

[0093] (1) Neural network

[0094] The neural network can be composed of neural units, and the neural unit can refer to an operation unit with x s as input. The output of the operation unit can be:

[0095]

[0096] Wherein, s=1, 2, ……n, n is a natural number greater than 1, W s is the weight of x s , b is the bias of the neural unit. f is the activation function of the neural unit, which is used to perform nonlinear transformation on the features obtained in the neural network, and convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolution layer. The activation function can be a sigmoid function. The neural network is a network formed by connecting many single neural units, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected with the local receptive field of the previous layer to extract the features of the local receptive field, and the local receptive field can be a region composed of several neural units.

[0097] (2) Deep neural network

[0098] A deep neural network (DNN), also known as a multi-layer neural network, can be understood as a neural network with multiple hidden layers. The neural networks inside a DNN can be divided into three categories according to the positions of different layers: input layer, hidden layer, and output layer. Generally, the first layer is the input layer, the last layer is the output layer, and the layers in between are hidden layers. The layers are fully connected, that is, any neuron in the ith layer is connected to any neuron in the (i+1)th layer.

[0099] Although a DNN looks very complex, it is actually not complex in terms of the work of each layer. In simple terms, it can be expressed as the following linear relationship expression: wherein, is an input vector, is an output vector, is a bias vector, W is a weight matrix (also known as a coefficient), and a() is an activation function. Each layer only obtains an output vector from an input vector through such a simple operation. Due to the large number of layers in a DNN, the number of coefficients W and bias vectors is also large. These parameters in a DNN are defined as follows: taking the coefficient W as an example: assuming that in a three-layer DNN, the linear coefficient of the fourth neuron in the second layer to the second neuron in the third layer is defined as The superscript 3 represents the layer number of the coefficient W, and the subscripts correspond to the output third layer index 2 and the input second layer index 4.

[0100] In summary, the coefficient of the kth neuron in the (L-1)th layer to the jth neuron in the Lth layer is defined as

[0101] It should be noted that the input layer has no W parameter. In a deep neural network, more hidden layers enable the network to better depict complex situations in the real world. In theory, the more parameters a model has, the higher its complexity and the greater its "capacity", which means it can complete more complex learning tasks. Training a deep neural network is a process of learning the weight matrix, and the ultimate goal is to obtain the weight matrix of all layers of the trained deep neural network (the weight matrix formed by the vectors W of many layers).

[0102] (3) Convolutional neural network

[0103] A convolutional neural network (CNN) is a deep neural network with a convolutional structure. The CNN contains a feature extractor composed of convolutional layers and subsampling layers, which can be regarded as filters. A convolutional layer refers to a layer of neurons in the CNN that performs convolutional processing on an input signal. In the convolutional layer of the CNN, a neuron can be connected to only part of the adjacent layer neurons. A convolutional layer usually contains several feature planes, each of which can be composed of a number of rectangularly arranged neural units. The neural units in the same feature plane share weights, and the shared weights are convolution kernels. The shared weights can be understood as being independent of the way and position of feature extraction. The convolution kernel can be formalized in the form of a matrix of random size, and the convolution kernel can obtain reasonable weights through learning in the training process of the CNN. In addition, the shared weights directly reduce the connections between the layers of the CNN and reduce the risk of overfitting.

[0104] (4) Loss function

[0105] In the process of training a deep neural network, because it is desired that the output of the deep neural network is as close as possible to the value that is actually intended to be predicted, the weight vectors of each layer of the neural network can be updated according to the difference between the predicted value of the current network and the value that is actually intended to be predicted (before the first update, a process of initialization is usually performed, that is, the parameters of each layer of the deep neural network are pre-configured), for example, if the predicted value of the network is too high, the weight vector is adjusted to make it predict a lower value, and the adjustment is continuously performed until the deep neural network can predict the value that is actually intended to be predicted or a value very close to the value that is actually intended to be predicted. Therefore, it is necessary to define “how to compare the difference between the predicted value and the target value”, which is a loss function or an objective function, which is an important equation for measuring the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, and then the training of the deep neural network becomes a process of minimizing the loss. The loss function can usually include error square mean square, cross entropy, logarithm, exponential, etc. For example, the error mean square can be used as the loss function, which is defined as The specific loss function can be selected according to the actual application scenario.

[0106] (5) Backpropagation algorithm

[0107] The convolutional neural network can adopt a back propagation (BP) algorithm to correct the size of the parameters in the initial network model in the training process, so that the reconstruction error loss of the model becomes smaller and smaller. Specifically, the forward transmission of the input signal until the output generates an error loss, and the error loss information is propagated backward to update the parameters in the initial model, so as to make the error loss converge. The back propagation algorithm is a back propagation movement dominated by error loss, aiming to obtain optimal model parameters, such as weight matrix.

[0108] In the pre-training stage or the noise processing stage, the BP algorithm can be used to train the model to obtain the trained model.

[0109] (6) Gradient: the derivative vector of the loss function with respect to the parameters.

[0110] (7) Stochastic gradient: in machine learning, the number of samples is large, so the loss function calculated each time is calculated by randomly sampled data, and the corresponding gradient is called stochastic gradient.

[0111] (8) Embedding: refers to the feature representation of the sample, which is usually the penultimate layer of the neural network.

[0112] (9) AutoML: refers to designing a series of high-level control systems to operate machine learning models, so that the model can automatically learn suitable parameters and configurations without human intervention. In the learning model based on deep neural network, automatic learning mainly includes network architecture search and global parameter setting. Among them, network architecture search is used to generate the most suitable neural network architecture for the problem according to the data, which has the characteristics of high training complexity and large performance improvement.

[0113] (10) Recommendation system: the recommendation system uses machine learning algorithms to analyze and learn according to the user's historical click behavior data, and then predicts the user's new request and returns a personalized item recommendation list.

[0114] (11) Click rate (CTR): refers to the probability of a user clicking a certain displayed item in a specific environment.

[0115] (12) Conversion rate (CVR): refers to the probability of a user converting a certain displayed item that has been clicked in a specific environment, for example, if a user clicks on the icon of an APP, conversion refers to downloading, installing, registering, etc.

[0116] (13) Transfer learning: using existing knowledge to assist learning of new knowledge, the core is to find the similarity between existing knowledge and new knowledge.

[0117] (14) Multi-task learning: multiple related tasks are put together to learn, and multiple tasks are learned at the same time.

[0118] (15) Ensemble learning: Ensemble learning methods use multiple learning algorithms to obtain better prediction performance than using any single learning algorithm alone.

[0119] (16) Model convergence: after multiple iterations of the model, the error between the predicted value of the model and the actual value is less than a certain pre-set small value.

[0120] (17) Generalization: refers to the adaptability of a machine learning system to new samples. The purpose of machine learning is to learn the rules hidden in the data, and the trained network can also give appropriate output to data outside the learning set with the same rules. This ability is called generalization.

[0121] (18) Robustness: refers to the ability of a machine learning system to handle errors during execution, and the ability of an algorithm to continue normal operation when encountering input, operation, and other abnormalities.

[0122] The recommendation method provided by the embodiments of the present application can be executed on a server, and can also be executed on a terminal device. The terminal device can be a mobile phone, a tablet personal computer (TPC), a media player, a smart television, a laptop computer (LC), a personal digital assistant (PDA), a personal computer (PC), a camera, a camcorder, a smart watch, a wearable device (WD), or an autonomous vehicle, etc., and the embodiments of the present application are not limited thereto.

[0123] The system architecture provided by the embodiments of the present application is introduced below.

[0124] Referring to Figure 2 , the embodiments of the present application provide a system architecture 200. As shown in the system architecture 200, the data acquisition device 260 can be used to acquire training data. After the training data is acquired by the data acquisition device 260, the training data is stored in the database 230, and the target model / rule 201 is trained by the training device 220 based on the training data maintained in the database 230.

[0125] The following describes how the training device 220 obtains the target model / rule 201 based on the training data. For example, the training device 220 processes multiple frames of sample images, outputs corresponding predicted labels, and calculates the loss between the predicted labels and the original labels of the samples. Based on this loss, the classification network is updated until the predicted labels are close to the original labels of the samples or the difference between the predicted labels and the original labels is less than a threshold, thus completing the training of the target model / rule 201. A detailed description of the training method follows.

[0126] The target model / rule 201 in this embodiment can specifically be a neural network. It should be noted that in practical applications, the training data maintained in the database 230 may not all originate from the data acquisition device 260; it may also be received from other devices. Furthermore, the training device 220 may not necessarily train the target model / rule 201 entirely based on the training data maintained in the database 230; it may also obtain training data from the cloud or other sources for model training. The above description should not be construed as limiting the embodiments of this application.

[0127] The target model / rule 201 trained by training device 220 can be applied to different systems or devices, such as... Figure 2 The execution device 210 shown can be a terminal, such as a mobile phone terminal, tablet computer, laptop computer, augmented reality (AR) / virtual reality (VR) device, in-vehicle terminal, television, etc., or it can be a server or cloud service. Figure 2 In the process, the execution device 210 is equipped with a transceiver 212, which may include an input / output (I / O) interface or other wireless or wired communication interfaces, for data interaction with external devices. Taking the I / O interface as an example, the user can input data to the I / O interface through the client device 240.

[0128] During the preprocessing of input data by the execution device 210, or during the calculation module 212 of the execution device 210 performing calculations and other related processes, the execution device 210 can call data, code, etc. in the data storage system 250 for corresponding processing, or store the data, instructions, etc. obtained from the corresponding processing into the data storage system 250.

[0129] Finally, I / O interface 212 returns the processing result to client device 240, thereby providing it to the user.

[0130] It is worth noting that the training device 220 can generate corresponding target models / rules 201 based on different training data for different objectives or tasks. The corresponding target models / rules 201 can be used to achieve the above objectives or complete the above tasks, thereby providing the user with the required results.

[0131] In the appendix Figure 2 In the scenario shown, the user can manually provide input data, which can be done through the interface provided by transceiver 212. Alternatively, client device 240 can automatically send input data to transceiver 212. If user authorization is required for client device 240 to automatically send input data, the user can set the corresponding permissions in client device 240. The user can view the output results of execution device 210 on client device 240, which can be presented in various ways such as display, sound, or animation. Client device 240 can also act as a data acquisition terminal, collecting the input data and output results of transceiver 212 as new sample data and storing them in database 230. Alternatively, data can be collected directly from transceiver 212 without going through client device 240; the transceiver 212 can directly store the input data and output results of transceiver 212 as new sample data in database 230.

[0132] It is worth noting that, attached Figure 2 This is merely a schematic diagram of a system architecture provided in an embodiment of this application. The positional relationships between the devices, components, modules, etc., shown in the diagram do not constitute any limitation. For example, in Figure 2 In this context, the data storage system 250 is an external memory relative to the execution device 210. In other cases, the data storage system 250 may also be placed within the execution device 210.

[0133] like Figure 2 As shown, the target model / rule 201 is obtained by training the training device 220. The target model / rule 201 can be the recommended model in this application embodiment.

[0134] For example, the system architecture for applying the neural network training method provided in this application can be as follows: Figure 3 As shown. In this system architecture 300, the server cluster 310 is implemented by one or more servers, optionally in conjunction with other computing devices, such as data storage, routers, load balancers, etc. The server cluster 310 can use data in the data storage system 250, or call the program code in the data storage system 250 to implement the steps of the neural network training method provided in this application.

[0135] A user can operate respective user devices (e.g., local device 301 and local device 302) to interact with the server cluster 310. Each local device can represent any computing device, such as a personal computer, a computer workstation, a smartphone, a tablet, a smart camera, a smart car, or other types of cellular phone, a media consumption device, a wearable device, a set-top box, a game console, etc.

[0136] Each user's local device can interact with the server cluster 310 through a communication network of any communication mechanism / standard, which can be a wide area network, a local area network, a point-to-point connection, etc., or any combination thereof. Specifically, the communication network can include a wireless network, a wired network, or a combination of wireless and wired networks, etc. The wireless network includes, but is not limited to, any one or more of a fifth-generation (5G) system, a long term evolution (LTE) system, a global system for mobile communication (GSM) or code division multiple access (CDMA) network, a wideband code division multiple access (WCDMA) network, wireless fidelity (WiFi), Bluetooth, Zigbee, radio frequency identification (RFID), long range (Lora) wireless communication, near field communication (NFC), etc. The wired network can include a fiber optic network, a coaxial cable network, etc.

[0137] In another implementation, one aspect or multiple aspects of the execution device 210 can be implemented by each local device, e.g., the local device 301 can provide local data or feedback computation results for the execution device 210.

[0138] It is noted that all functions of the execution device 210 can also be implemented by the local devices. For example, the local device 301 implements the functions of the execution device 210 and provides services for its own user, or provides services for the user of the local device 302.

[0139] Generally, the machine learning system can include a personalized recommendation system, which can train parameters of a machine learning model based on input data and labels through an optimization method such as gradient descent, and when the model parameters converge, the model can be used to complete prediction of unknown data. Taking click rate prediction in a personalized recommendation system as an example, the input data includes user features, item features, and context features. How to predict a personalized recommendation list according to user preferences has an important influence on improving user experience and platform revenue of the recommendation system.

[0140] For example, the recommendation process can be divided into a training part and an online inference part as shown in Figure 4 In the training part, the training set includes input data and corresponding labels. In the APP recommendation scenario, the training set can include APPs that have been clicked by users and APPs that have been clicked and converted. The training set is input into an initial model, and parameters of a machine learning model are trained through an optimization method such as gradient descent to obtain a recommendation model. In the online inference part, the recommendation model can be deployed in a recommendation platform, such as a server or a terminal. Here, taking the server as an example, the server can be used to output a recommendation list for a user. In the APP recommendation scenario, APP icons recommended for a user are displayed on the home page of the user terminal, or after the user clicks on an APP, icons of recommended APPs related to the APP are displayed.

[0141] In some scenarios, such as in the traditional Internet effect advertisement click bid mode, only click rate estimation is performed, and the real appeal of the advertiser is not expressed. More and more advertisers begin to focus on the effect of deep conversion behavior, that is, the conversion rate of the advertisement needs to be estimated. After the platform exposes an item to a user, the user will select to click if interested. In this process, the recommendation system needs to predict the click probability of the user, that is, whether the user clicks. However, in fact, the user will have further operations on the item after clicking. If the recommended item is a mobile phone application, the user may download, install, and register the application, which is collectively referred to as conversion. For example, advertisers will pay different fees for different promotion effects, and therefore the accuracy of conversion rate prediction has a great influence on the advertisement platform.

[0142] Conversion rate estimation faces the following two challenges: sample selection bias: conversion rate estimation model training is performed on the sample space after clicking, and prediction needs to be performed on the exposure sample space. Data sparsity: the positive sample label of the conversion rate estimation model is conversion, and the negative sample label is click. The number of positive samples is greatly reduced compared with the click rate estimation model.

[0143] Some strategies can alleviate these two problems, such as sampling negative examples from the exposure set to alleviate sample selection bias, oversampling conversion samples to alleviate data sparsity, etc. But these ways cannot fundamentally solve any of the above problems. Click-to-conversion is itself two strongly correlated continuous behaviors, and multi-task learning simultaneously models these two tasks, so that it can train and predict in the entire space, which is the mainstream solution in the industry. However, existing multi-task learning methods mostly use hard parameter sharing mechanism, and multi-task learning may cause negative transfer phenomenon, that is, the information sharing between tasks will affect the performance of the network. In addition, data sparsity will make the conversion rate prediction model prone to overfitting.

[0144] For example, an entire space multi-task model (ESMM) can be used for recommendation, which uses a parameter sharing structure of shared Embedding, and the CTR task and the CVR task use the same features and feature embedding. ESMM uses the supervision information of CTCVR and CTR to train the network, and implicitly learns CVR. The structure of ESMM is designed based on the relationship of "multiplication": pCTCVR = pCVR * pCTR. However, ESMM only shares information at the embedding layer, and the CVR task still faces the problem of data sparsity, and hard parameter sharing is not conducive to parameter learning.

[0145] For another example, a multi-task learning model based on an automatic information transfer framework (AITM) is a multi-task model for conversion rate prediction with sequence dependence, which outputs the input features through a task-shared Embedding to multiple tower networks, respectively. Then, the AIT module uses the vector output by the current task tower and the information transmitted from the previous task to learn how to fuse the information between tasks. The attention mechanism is used to automatically assign weights to the transferred information and the original information. The transferred information is learned through a function, which can be a simple fully connected layer, to learn what information should be transferred between two adjacent tasks. Finally, AITM constrains the output probability to meet the sequence dependence as much as possible by applying a calibrator in the loss function, that is, the probability output by the task after the sequence should be less than the probability output by the task before the sequence. However, AITM only shares information at the embedding layer, and the CVR task still faces the problem of data sparsity, and hard parameter sharing is not conducive to parameter learning. Although the sequence relationship calibrator can play a regular role, pcvr is the value obtained by pCTCVR / pCTR, and division will make the prediction result unstable.

[0146] In addition, there are models such as Progressive Layered Extraction (PLE), a novel multi-task learning model for personalized recommendations. PLE's bottom layer mainly consists of shared experts and task-specific experts, while the upper layer comprises a multi-task tower network. The input to each multi-task tower network is weighted by a gating network. The input to the gating network for each sub-task includes two parts: the task-specific network for that task and the shared expert network. The feature vector input serves as the selector for the gating network. However, there is a sequential relationship between CTR and CVR tasks; only clicked samples are likely to be converted. Neither MMOE nor PLE, which learns independently for each task, captures this relationship, resulting in unsatisfactory CVR prediction performance. PLE only designs a multi-expert network at the bottom layer, resulting in a relatively simple tower structure. When the conversion data is very sparse, conversion rate prediction cannot achieve good results.

[0147] Therefore, this application provides a multi-task learning framework based on hierarchical hybrid experts. By setting up a hybrid expert structure at the bottom layer (feature representation layer) and the tower structure where the task resides (feature interaction layer), it learns feature representations and feature interactions hierarchically, which can fully utilize the correlation between tasks to help the conversion rate prediction task achieve better recommendation results. The recommendation method and neural network training method provided in this application are described in detail below.

[0148] First, for ease of understanding, the structure of the recommendation model provided in this application will be introduced.

[0149] See Figure 5 The structural diagram of a recommendation model provided in this application is as follows.

[0150] This recommendation model can be used to perform multiple tasks. Figure 5 Taking P tasks as an example, these multiple tasks are related tasks for recommending users. Each task corresponds to multiple tower expert networks, and each task also corresponds to one or more task-specific feature extraction networks. In addition, these multiple tasks correspond to one or more shared feature extraction networks. That is, the recommendation model can include multiple tower expert networks corresponding to each task, one or more task-specific feature extraction networks corresponding to each task, and one or more shared feature extraction networks. The parameters of each tower expert network are different. These parameters can include the internal parameters of each operation in the tower expert network, such as the parameters in the convolution kernel, or the weight parameters of the output of each operation.

[0151] A shared feature extraction network can be used to extract features from input data. For ease of distinction, the features it outputs are called shared features, and they are used as inputs shared by each tower expert network. That is, the shared features are used by the tower expert networks corresponding to the multiple tasks.

[0152] Task-specific feature extraction networks can be used to extract features for corresponding tasks. For ease of distinction, the features they output are called tower expert shared features, which are used as inputs shared by multiple tower expert networks in a single task. In other words, the tower expert shared features are used by multiple tower expert networks corresponding to a single task.

[0153] Recommendation models can be used to perform multiple tasks, which can be user-related recommendation tasks or tasks that are related or unrelated. Specifically, in some scenarios, these multiple tasks may include predicting click-through rates, predicting conversion information, etc. The conversion information may include conversion rate or conversion duration, where the conversion rate is the probability that a user will further convert after clicking on an object, and the conversion duration is the length of time a user stays on the object after further conversion.

[0154] Tower expert networks are used to perform corresponding tasks. For example, if the task includes click-through rate (CTR) prediction, the tower expert network can be used to predict the CTR based on the input features and obtain the CTR prediction result; if the task includes conversion rate prediction, the tower expert network can be used to predict the conversion rate based on the input features and obtain the conversion rate prediction result; if the task is object recognition, the tower expert network can be used to perform object recognition based on the input features and obtain information about the recognized object, etc. The specific implementation can be adjusted according to the actual application scenario.

[0155] Optionally, such as Figure 6 As shown, one or more tower feature extraction networks can be set up for each tower expert network. The input of each tower expert network is also connected to the output of the corresponding tower feature extraction network. These one or more tower feature extraction networks are used to extract the required features for each tower. Typically, the parameters of the tower feature extraction network are different for each tower, thus allowing for adaptive feature extraction based on each tower, improving the accuracy of the final output of the recommendation model. For ease of distinction, the features output by the tower feature extraction network will be referred to as tower-specific features.

[0156] Optionally, a gating network can also be arranged at the input end of each tower expert network, which is used to fuse the outputs of the task-specific feature extraction network, the shared feature extraction network and the tower feature extraction network to obtain the input of each tower expert network. Thus, the features extracted by each feature extraction network can be input to each tower expert network in an adaptive proportion, so that the tower expert network can output an accurate task output result.

[0157] In addition, after obtaining the input data, the input data can be converted into a feature representation, such as a feature vector, by an Embedding layer, so that the subsequent feature extraction network can extract the required features from the feature representation.

[0158] For example, the structure of the gating network can be as shown in Figure 7 Specifically, the gating network can include a fully connected layer and a softmax layer. Generally, the task-specific features extracted by the task-specific feature extraction network include multiple dimensions of features required by the tower expert network, so the feature representation of the input data can be taken as the input of the fully connected layer, the fully connected layer is used to help the tower expert network extract the required features, and the output of the fully connected layer is mapped to a weight value recognizable by the subsequent network through the softmax layer. Then, the tower-specific features, shared features and task-specific features are fused according to the weight value output by the softmax layer to obtain fused features, which can be taken as the input of the tower-specific network.

[0159] Therefore, in the recommendation model provided by the present application, multiple tower expert networks are arranged for each task, and the parameters of each tower expert network are different. In the case of data sparsity, the output stability of the model can be improved by multiple output results of multiple tower expert networks, and the overfitting problem can be avoided. Even if there is a problem of data sparsity, the recommendation model provided by the present application can stably output a recommendation result, and the user experience can be improved.

[0160] The steps performed by each module of the recommendation model will be described in detail below in combination with the method provided by the present application.

[0161] Based on the above-described recommendation model, the method provided by the present application can be divided into two parts, i.e., a training part and an online inference part, which will be introduced below.

[0162] I. Training part

[0163] Referring to Figure 8 , a flowchart of a training method provided by the present application is as follows.

[0164] 801, obtaining a training set.

[0165] The training set can be historical input data collected from one or more users, or data received from other servers or clients. In essence, the training set can include multiple samples and the label corresponding to each sample.

[0166] Specifically, the data types in the training set are related to the tasks performed by the recommendation model. The data required for training may differ for different tasks, and can be adjusted according to the actual application scenario. For example, in an app recommendation scenario, the training set may include a large amount of information about apps that users have clicked, such as the app's name, application type, and application style, as well as further actions taken after clicking the app, such as downloading, installing, and registering. As another example, in a music recommendation scenario, the training set may include a large amount of information about music that users have clicked, such as music genre, artist information, as well as further actions taken after clicking the music, such as playing and downloading.

[0167] 802. The initial model is iteratively trained using the training set as input to obtain the recommendation model.

[0168] Once the training set is obtained, the initial model can be iteratively trained using the training set to obtain the trained recommendation model.

[0169] The initial model can be a constructed model or an existing model structure. The structure of this initial model can be found in the preceding text. Figure 5 or Figure 6 This recommendation model can be used to perform multiple tasks, which are related tasks for recommending to users. Each task corresponds to multiple tower expert networks, and each task also corresponds to one or more task-specific feature extraction networks. In addition, the multiple tasks correspond to one or more shared feature extraction networks. That is, the recommendation model may include multiple tower expert networks corresponding to each task, one or more task-specific feature extraction networks corresponding to each task, and one or more shared feature extraction networks. The parameters of each tower expert network are different. These parameters may include the internal parameters of each operation in the tower expert network, such as the parameters in the convolution kernel or the parameters in the pooling operation, or the weight parameters of the output of each operation.

[0170] The recommendation model can be used to perform a plurality of tasks, which can be related to recommending for a user, and can be associated or not associated. Specifically, in some scenarios, the plurality of tasks can include predicting a click rate, predicting conversion information, etc., and the conversion information can include conversion rate or conversion duration, etc. The conversion rate is the probability of further conversion of the object after the user clicks on the object, and the conversion duration is the duration of the user staying after further conversion of the object after clicking on the object.

[0171] In each iteration process, the sample in the training set is input into the initial model obtained in the last iteration, a first loss value between the first output result of the initial model obtained in the last iteration and the label of the input sample is obtained, a second loss value between the second output result of each tower expert network and the first output result is obtained, the model obtained in the last iteration is updated using the first loss value and the second loss value corresponding to each tower expert network, and a model in the current iteration is obtained.

[0172] It can be understood that, in addition to using the loss value between the overall output result of the model and the label to update the overall recommendation model, the loss value between the output result of each tower expert network and the overall output result of the model can also be used as a constraint to update the parameters of each tower expert network, so that the output of each tower expert network is closer to the overall output of the model.

[0173] Therefore, in the embodiments of the present application, when training the recommendation model, in addition to using the loss value between the output result of the model and the sample label to update, the loss value between the output result of each tower expert network and the overall output result of the model can also be used as a constraint to update each tower expert, so that the output of each tower expert is closer to the overall output result of the model, and the output of the model is more accurate, which can accelerate the convergence speed of model training and realize efficient training.

[0174] Exemplarily, the task performed by the recommendation model can include CVR prediction and CTR prediction. For example, the update process of the network corresponding to the CVR prediction task can be as shown in Figure 9 Taking CVR prediction as an example, M towers predict conversion probabilities M prediction results are weighted and fused to obtain Then, the result is used to calibrate the prediction value of each tower. The following loss function can be used to constrain the output of the tower expert network:

[0175] 1) Cross-entropy:

[0176] 2) KL distance:

[0177] That is, the cross-validation entropy and the KL distance are used to constrain the update of the tower expert network, so that the output of the tower expert network is closer to the output of the recommendation model.

[0178] For example, the training data is exposure data, the label of the CTR task is click p(click=1|x), the label of the CTR task p(conversion&click=1|x) is click and conversion, the training space of the CTR task and the CTCVR task is in all exposure data, the training space of the CVR task p(conversion=1|click=1,x) is a click sample, that is, there is p(conversion&click=1|x)=p(conversion=1|click=1,x)*p(click=1|x), and the CVR task can be implicitly modeled

[0179] The label data includes a click label y click , a click and conversion label y click&conversion and a predicted value The classification cross-entropy loss is obtained based on the cross-validation entropy (LogLoss) The expert calibration loss function of the 102 module is used at the same time The final loss function is obtained Then, the gradient descent algorithm is used to complete the joint training and optimization of the parameters of the hierarchical mixed expert module, the embedding table and other modules by using the chain rule. The parameters of the model are continuously adjusted through the loss function of the model, and finally an optimized model is obtained. The loss function can be represented as:

[0180]

[0181]

[0182] In the embodiments of the present application, the prediction value of each expert is calibrated by using the expert ensemble learning result. Compared with any one expert, the result of multi-expert ensemble learning is more robust, which can be used to calibrate the prediction value of each expert, avoid the over-divergence of the learning result of the expert, and improve the stability of model convergence and the generalization performance of the model.

[0183] Optionally, the recommendation model further includes a tower feature extraction network corresponding to each of the plurality of tower expert networks, such as the aforementioned Figure 6As shown in the figure, the tower feature extraction network is used to extract features related to the task performed by the corresponding tower expert network from the input data, and the parameters of the tower feature extraction network corresponding to the plurality of tower expert networks are different. Accordingly, when the loss value between the output result using each tower expert network and the overall output result of the model is used as a constraint, the parameters of each tower expert network are updated accordingly, and the tower feature extraction network corresponding to each tower expert network can also be updated, so that the output of each tower expert network is closer to the overall output result of the model, the output of the model is more accurate, the convergence speed of the model training can be accelerated, and efficient training is realized.

[0184] II. Online inference part

[0185] Referring to Figure 10 The application provides a flowchart of a recommendation method.

[0186] 1001. Obtain input data.

[0187] The input data can be user information collected, or input data received from a client, etc. The input data can include user information, which can include user identity information, positioning information, user input data, or user-generated historical information, etc. The user identity information, such as the user's name, identification, etc., represents the user's identity; the positioning information can include the coordinates of the user's own location, which can be obtained by the user using the client to locate; the user input data can include data of the user's input operation, such as the user opening an application market or music software, or the user clicking on an app or clicking on a music icon, etc.; the user-generated historical information, for example, information of apps that the user has clicked or downloaded, information of music that the user has played or downloaded, etc.

[0188] Optionally, the input data can also include information of the object to be recommended, such as the type of the object to be recommended, the alternative list, etc. For example, the input data can include the type of the object to be recommended for the user, such as the type information of apps, music, etc.; or the input data can directly include the alternative list of the object to be recommended for the user, such as in the scenario of recommending apps for the user, the alternative list can include information of a plurality of apps, for subsequent use of the recommendation model to filter out the app to be recommended for the user from the plurality of apps; or in the scenario of recommending music for the user, the alternative list can include information of a plurality of songs, for subsequent use of the recommendation model to filter out the song to be recommended for the user from the plurality of songs.

[0189] Furthermore, if the recommendation model provided in this application is deployed on a server, and the input data is data sent from the client to the server, the information of the object to be recommended can be sent from the client to the server, or it can be generated by the server based on locally stored data. For example, the server can pre-set a database of objects to be recommended to users. When user information is obtained, the objects in the database can be used as a candidate list. Alternatively, the server can set a corresponding type of object to be recommended for each user. When input data is received, the information of the object to be recommended can be obtained from the locally stored data based on the user's identity information.

[0190] For example, if the recommendation method provided in this application is deployed on a client, the user can directly perform operations on the client to obtain input data. For instance, if the user opens an app store, the client can generate input data based on the user's input operations and send it to the server. If the recommendation method provided in this application is deployed on a server, the user can directly perform input operations through an input device connected to the server, such as clicking on an app or opening a music player. The server generates input data through the data generated by the input device. Alternatively, the user can establish a connection with the server using a client, perform input operations on the client, and the client transmits the data generated by the user to the server, which can then obtain the input data.

[0191] Typically, the input data generated varies depending on the scenario. For example, in an app recommendation scenario, the input data might include data generated from the user opening an app store. In this scenario, the input data could include user identification information, such as the user's name and unique identifier, as well as historical data, such as information about apps clicked in the past, including the number, type, name, or identifier of the apps clicked. As another example, in a music recommendation scenario, the input data might include data generated from the user clicking "next song." This input data could include user information and historical playback data, such as the user's name and identifier, as well as information about the previously played music, such as the song title, artist name, and genre.

[0192] 1002. Use the input data as input to the recommendation model and output recommendation information for the user.

[0193] Once the input data is obtained, it can be used as input to the recommendation model to output recommendation information tailored to the user.

[0194] Generally, the type of the recommendation information can also be different in different scenarios. For example, in an app recommendation scenario, the recommendation information can include information of apps recommended for the user, such as app icons, download links, and the like. For another example, in a music recommendation scenario, the recommendation information can include information of music recommended for the user, such as music titles, singers, playback portals, and the like.

[0195] The recommendation model can be used to perform multiple tasks, which can be related to recommendation for the user, and can be associated or not associated. Specifically, in some scenarios, the multiple tasks can include predicting a click rate, predicting conversion information, and the like, where the conversion information can include a conversion rate or a conversion duration, and the like. The conversion rate is a probability that the user further converts the object after clicking the object, and the conversion duration is a duration that the user stays after further converting the object after clicking the object, such as a duration that the user plays a video or a duration that the user plays music, and the like. For example, in an app recommendation scenario, the click rate and the conversion rate of each app in the candidate list can be output, so that when generating a recommendation list, apps with a click rate and a conversion rate higher than a certain value can be selected from the candidate list as recommended apps. For another example, in a music recommendation scenario, the click rate and the conversion duration of each music in the candidate list can be output, where the conversion duration is a duration that the user stays after clicking and playing the music, and musics with a click rate and a conversion duration ranking in the front can be selected from the candidate list as recommended music for the user.

[0196] Specifically, referring to the foregoing Figure 5 , the shared feature extraction network in the recommendation model can extract shared features from the input data, where the shared features can include features required by each tower expert network in the recommendation model to perform a task, and the shared features are input to the tower expert network. The task-specific feature extraction network in the recommendation model is used to extract task-specific features related to a task from the input data, and the task-specific features are input to the tower expert network corresponding to the task. The task-specific features are features extracted for the corresponding task, and can be used to implement feature extraction for the task. After the tower expert network receives the task-specific features and the shared features for fusion, the tower expert network can perform a corresponding task based on the task-specific features and the shared features, and obtain an output result of the tower expert network. The recommendation model can include multiple tower expert networks, and the output results of the multiple tower expert networks are fused, and the recommendation information for the user is obtained.

[0197] Therefore, in the embodiments of the present application, for the recommendation model performing multiple tasks, multiple tower expert networks are set for each task, and the output stability of the recommendation model is improved through the outputs of the multiple tower expert networks, which can avoid the problem of overfitting caused by data sparsity, thereby improving the output accuracy of the recommendation model. Moreover, the model provided by the present application is a multi-task model, and the efficient parameter sharing scheme can enable the two associated task click rate estimation and conversion rate estimation of the multi-task model to provide assistance to each other, thereby achieving better effects than single-task models, thereby directly affecting platform revenue and user experience. Not only can the number of online deployed models be reduced to reduce model maintenance costs, but also the information contained in the associated tasks can be more effectively mined to achieve better recommendation effects.

[0198] Optionally, referring to the foregoing Figure 6 When multiple tower feature extraction networks are set in the recommendation model, each tower expert network can correspond to one tower feature extraction network, which can be used for feature extraction for the corresponding tower expert network to obtain tower-specific features, and input the tower-specific features to the corresponding tower expert network. Therefore, the input on each tower expert network can include shared features extracted by the shared feature extraction network, task-specific features extracted by the task-specific feature extraction network, and tower-specific features extracted by the tower feature extraction network, so that each tower expert network can use multiple features to perform the corresponding task, thereby outputting more accurate output results. Therefore, in the embodiments of the present application, a feature extraction network is separately set for each tower expert network, thereby more accurately extracting the features required for each tower, so that each tower can use more accurate features to obtain more accurate output results.

[0199] In addition, referring to the foregoing Figure 6 As shown in the recommendation model, after the input data is input into the recommendation model, it is converted into a feature representation recognizable by the feature extraction network after the Embedding layer, or is called a feature vector. Then it is input into the tower feature extraction corresponding to each tower, the task-specific feature extraction network corresponding to each task, and the shared feature extraction network, so as to perform feature extraction for each tower and each task. After the features are extracted by the tower feature extraction corresponding to each tower, the task-specific feature extraction network corresponding to each task, and the shared feature extraction network, the features extracted by these feature extraction networks can be input into the gate network corresponding to each tower expert network, and the features extracted by each feature extraction network are fused through the gate network. Generally, the features required by different tower expert networks can be different, and the gate network set therefor can adaptively fuse the features extracted by each feature extraction network, such as fusion according to different weights, so as to obtain the features required by the tower expert network and input into the tower expert network.

[0200] It can be understood that the input layer transmits the data features, and the corresponding embedding vector expression is taken out from the embedding table through the sparse coding id, and finally the embedding vector expressions of all input features are spliced in sequence to form a feature vector. Each feature extraction network receives a feature vector x0 as input, learns the feature representation required by the task through an expert sharing mechanism in the feature representation layer, and then learns the task-specific feature interaction through the tower expert in the feature interaction layer. Finally, through weighted aggregation, each task fuses the tower expert results of the corresponding tower expert to give the prediction of the task.

[0201] Taking the prediction of click rate CTR and conversion rate CVR as an example, referring to the recommendation model shown in the foregoing Figure 6 The feature vector x0 of the input layer is input into the bottom feature representation layer, which is composed of various feature extraction networks. The feature representation layer includes a shared expert layer (Shared Expert), i.e., the aforementioned shared feature extraction network, a task-specific expert layer (CTR / CVR Task-Expert), i.e., the aforementioned task-specific feature extraction network, and a tower-specific expert layer (CTR / CVR Tower-Specific Expert), i.e., the aforementioned tower feature extraction network. Each expert is composed of multiple sub-networks, and the number of sub-networks, the dimension of sub-networks, and the network structure are all hyperparameters.

[0202] Each task of the feature interaction layer includes several tower expert (Tower Expert) networks, and the input of each tower expert network is weighted controlled by a gate control network. The input of the gate control network of each tower expert network of each task includes three parts: the output of the tower-specific expert layer under the tower, the output of the task-specific expert layer under the task, and the output of the shared expert layer. The feature vector x0 serves as a selector of the gate control network.

[0203] The structure of the gate control network can be a fully connected network or other deep networks. The feature vector x0 is used as a selector to obtain the weight size occupied by different sub-networks, so as to obtain the weighted sum of the gate control network under different tower experts of different tasks. In this way, the tower experts of each feature interaction layer will weight and sum the outputs of the three parts of the expert layer, i.e., the tower-specific expert layer under the tower, the task-specific expert layer under the task, and the shared expert layer, according to the input feature vector x0, so that each tower expert network of each task obtains a unique feature representation. After the tower expert network of each sub-task, the output of the corresponding sub-task tower expert network is obtained. The prediction value of each task is the weighted aggregation of the outputs of the multiple tower expert networks included in the sub-task.

[0204] Assuming the feature vector is x0, the structure type of all expert networks is MLP, and the learning process of a tower expert output of a feature interaction layer of a subtask CVR task in a multi-task framework is taken as an example to formalize the above process, which is shown as follows:

[0205]

[0206]

[0207] G(x0)=softmax(MLP(x0))

[0208] f(x0)=∑G(x0)*E(x0)

[0209]

[0210] wherein represents the output of the bottom expert network, type∈{Tower, Task, Shared}, k represents the output of the kth subnetwork of the bottom expert network layer, represents the set of outputs of all subnetworks of the bottom expert network layer, E(x0) is the set of outputs of all types of bottom expert network layers of the tower expert, G(x0) is the weight assigned by the gating network to the output of each subnetwork of each class expert layer, f(x0) is the feature representation learned by the mixed expert, and then input into the tower expert network of the feature interaction layer to obtain the output of the jth tower expert network of the CVR task The final output of the CVR task is The learning process of the CTR task is similar, and the final output of the CTR task is

[0211] The expert network in the present application can use various networks, such as any deep network, such as a Squeeze-and-Excitation network, an Attention network, etc.

[0212] Therefore, in the embodiments of the present application, hierarchical expert structures are set on the bottom layer (feature representation layer) and the task tower network layer (feature interaction layer) respectively. In order to make full use of the correlation between tasks, the present application designs a mechanism of multi-expert parameter sharing in the feature representation layer. The feature representation layer includes three types of expert networks, namely shared experts, task-specific experts and tower-specific experts. The shared experts share the shared knowledge between tasks, the task-specific experts extract the knowledge required by the task, and the tower-specific experts independently learn knowledge to serve the experts on the tower structure. Each expert has its own function and efficiently extracts information representation. A single network cannot effectively learn the general expression between tasks under the same scale, but after dividing into multiple sub-networks, each sub-network can always learn some relevant unique expressions in a certain task. Therefore, for a prediction task, multiple tower experts are set in the task tower network to further learn feature interaction from different angles, improve the learning ability and generalization ability of the model, and finally achieve better prediction accuracy than traditional methods.

[0213] It can be understood that, in the feature interaction layer, multiple sub-networks (tower experts) are divided, each of which can always learn some relevant unique expressions in a certain task, and a more flexible parameter sharing mechanism is provided in the feature representation layer. The parameters of the tower-specific experts only serve the tower expert, the parameters of the task-specific experts are only shared between the tower experts of the same task, and the parameters of the shared experts are shared by all tower experts, which can efficiently extract information representation. By setting hybrid experts at different levels, the learning ability and generalization ability of the model can be improved, and finally better prediction accuracy can be achieved.

[0214] The foregoing introduces the structure of the recommendation model and the method flow provided by the present application. The application scenarios of the present application will be exemplarily introduced below in combination with the foregoing recommendation model and method.

[0215] Taking the display advertisement scenario in the mobile intelligent assistant service direct as an example, display advertisement sorting needs click rate prediction and conversion rate prediction, and the input includes user features, product features and context features. Then, the click rate prediction and conversion rate prediction tasks are jointly modeled by a multi-task method. The present application improves the information sharing ability of the multi-task learning model by improving the parameter sharing mechanism of the multi-task learning framework, so as to more accurately estimate the click rate and conversion rate and give more accurate recommendations. The hierarchical hybrid expert module enhances the information sharing ability between multi-tasks.

[0216] When training the multi-task prediction model offline, the specific process is as follows:

[0217] The system log is collected and data cleaning is performed, and the obtained original features are input into the model. The original features are one-hot encoded, and then the corresponding embedding vector representation is obtained from the embedding table, and finally the embedding vector representation of all input features is spliced in order to form a feature vector;

[0218] The feature vector is extracted by the three experts mentioned above, and the information representation is obtained from the three experts through the gate network, and then input into the tower network layer (feature interaction layer) to learn feature interaction;

[0219] For each task, the output results of the tower network experts are weighted and aggregated, and input into the activation function to finally obtain the prediction value of the task.

[0220] The prediction value and the true label are used to calculate the classification loss function The calibration loss function of the 102 module is also calculated The sum of the four is the final loss, and the model and the 101 module are updated based on the loss.

[0221] During online inference, the model can be directly loaded for online estimation, i.e., without using the prediction value and the true label to calculate the loss value and update the network.

[0222] For example, the various parts of the advertisement recommendation system can be as shown in Figure 11 The advertisement display ranking scenario is a typical scenario in machine learning applications, and its main structure is as shown in Figure 11 , including display ads, offline logs, offline training, online inference, online ranking, etc.

[0223] The basic operation logic of the display advertisement recommendation system is as follows: the user performs a series of behaviors such as browsing, clicking, commenting, downloading, etc. in the front-end display list, generates behavior data, and stores it in the log. The recommendation system uses data including user behavior logs for offline model training, and generates a prediction model after training convergence. The model is deployed in an online service environment and gives a click rate estimation score P ctr and a conversion rate estimation score P cvr , and then the online ranking module will sort the candidate ads based on the above two scores and business logic, and show the final recommendation list to the user. Finally, the user feedback forms user data. For example, as shown in Figure 12 , the icon of the recommended app can be displayed on the display interface of the user's terminal, so that the user can further click or download the recommended app, etc., so that the user can quickly find the required app and improve the user experience.

[0224] The score P of click rate estimation in the online ranking stage ctr And the score P of conversion rate estimation cvr Multi-task learning can alleviate the influence of sample selection bias and data sparsity on the model effect. An efficient parameter sharing scheme can enable the two related tasks of click rate estimation and conversion rate estimation of the multi-task model to provide assistance to each other, achieve better results than single-task models, and thus directly affect platform revenue and user experience. A good multi-task learning scheme can not only reduce the number of models deployed online and reduce model maintenance costs, but also more effectively mine the information contained in related tasks to achieve better recommendation results.

[0225] Take the public data set Ali-CCP commonly used for conversion rate estimation and the industrial data set collected as an example. The statistical information is shown in Table 1.

[0226] Dataset Number of Users Number of Items Number of Samples Number of Clicks Number of Conversions Public Dataset 0.4M 4.3M 84M 3.4M 18k Industrial Dataset 0.5M 67.6k 736M 3.7M 1.8M

[0227] Table 1

[0228] The offline evaluation index is AUC (Area Under Curve). Experiments are performed on the above two data sets, a single-layer MLP is used as the bottom expert network layer and the skeleton of the gating network, and a multi-layer MLP is used as the skeleton of the tower expert layer. Both the CTR task and the CVR task use two tower expert networks, and the output results are shown in Table 2.

[0229]

[0230] Table 2

[0231] Obviously, from the above table, compared with only using MLP and models such as MMoE, PLE, ESMM, and AITM, the method provided by the present application can achieve a higher AUC.

[0232] Therefore, in some commonly used manners, information can only be extracted from the feature representation layer one-sidedly, and the feature interaction layer is prone to overfitting. The application improves the parameter sharing mechanism by setting a hierarchical mixed expert module. Compared with the commonly used scheme, the application proposes that multiple tower expert structures should also be set in the feature interaction layer. Based on the idea of ensemble learning, a single network cannot effectively learn the general expression between tasks under the same scale, but after dividing into multiple sub-networks, each sub-network can always learn some relevant unique expressions in a task. Therefore, the application sets multiple tower experts in the task tower network for the individual prediction task to further learn the feature interaction from different angles, improves the learning ability and generalization ability of the model, and finally can achieve better prediction accuracy than the traditional method. Meanwhile, in the feature representation layer, in addition to the task-specific expert layer and the shared expert layer, the application proposes a tower-specific expert. The input of each tower expert layer is controlled by a gating network, which accepts the feature representation learned by the tower-specific expert (parameters only serve the tower), the task-specific expert (parameters are only shared between the tower experts of the same task), and the shared expert (parameters are shared by all tower experts) as input, and performs weighted summation. This enables the tower experts in the feature interaction layer to include both personalized information unique to the network and shared information between the same tasks, as well as more generalized information between all tasks when learning feature interaction. This flexible parameter sharing mechanism can efficiently extract information representation, enabling the multi-task learning scheme proposed by the application to fully share the associated information between tasks. In addition, during training, the loss value between the overall output of the model and the output of a single tower is used as a constraint to update the model, which can make the model converge faster, reduce the possibility of learning bias, and improve the generalization performance of the model.

[0233] The foregoing introduces the recommendation model and method flow provided by the application. Next, the device provided by the application is introduced.

[0234] Referring to Figure 13 , a structure diagram of a recommendation device provided by the application is used to execute the steps in the foregoing Figures 10-12 , and the recommendation device comprises:

[0235] The acquisition module 1301 is configured to acquire input data, and the input data comprises information of a user.

[0236] The recommendation module 1302 is configured to take the input data as an input of a recommendation model and output recommendation information for the user.

[0237] The recommendation model is used to perform a plurality of tasks of making recommendations for a user, the recommendation model comprises a shared feature extraction network, a plurality of tower expert networks corresponding to each task respectively, and a task-specific feature extraction network corresponding to each task respectively, an output end of the shared feature extraction network is connected to an input end of each tower expert network, and an input end of the plurality of tower expert networks corresponding to each task is also connected to an output end of the task-specific feature extraction network corresponding to each task respectively; parameters of the plurality of tower expert networks are different, the shared feature extraction network is used to extract shared features from input data, the shared features are shared by the plurality of tower expert networks corresponding to the plurality of tasks, the task-specific feature extraction network is used to extract tower expert shared features from the input data, the tower expert shared features are shared by the plurality of tower expert networks corresponding to a single task, the plurality of tower expert networks are used to perform corresponding tasks based on the features extracted by the task-specific feature extraction network and the shared feature extraction network, and recommendation information is obtained by weighted fusion of outputs of the plurality of tower expert networks corresponding to the plurality of tasks.

[0238] In a possible implementation, the recommendation model further comprises a tower feature extraction network corresponding to each of the plurality of tower expert networks, the tower feature extraction network is used to extract features related to a task performed by the corresponding tower expert network from the input data, and parameters of the tower feature extraction networks corresponding to the plurality of tower expert networks are different.

[0239] In a possible implementation, the recommendation model further comprises a plurality of gating networks, each tower expert network corresponds to a gating network, and the gating network is used to fuse outputs of the corresponding task-specific feature extraction network, the shared feature extraction network, and the tower feature extraction network, and take the fusion result as an input of the corresponding tower expert network.

[0240] In a possible implementation, the recommendation device further comprises:

[0241] The training module 1303 is further configured to perform iterative training on the initial model to obtain the recommendation model, and the initial model has the same structure as the recommendation model.

[0242] In one of the iterative training processes of the initial model, the training sample is taken as an input of the initial model to output a first output result, a first loss value between a label of the training sample and the first output result is obtained, a plurality of second output results output by the plurality of tower expert networks corresponding to the plurality of tasks are obtained, a plurality of second loss values between the first output and the plurality of second output results are obtained, and the initial model is updated according to the first loss value and the second loss values to obtain the initial model after the current iteration.

[0243] In a possible implementation, the plurality of tasks include predicting a click rate and predicting conversion information, the click rate being a probability of a user clicking on a target object, and the conversion information including a conversion rate or a conversion duration, the conversion rate being a probability of the user performing a conversion operation on the target object after clicking on the target object, and the conversion duration including a duration of the user staying after performing the conversion operation on the target object after clicking on the target object.

[0244] Referring to Figure 14 The present application provides a structural diagram of a training device, which can be used to perform the foregoing Figures 8-9 The training device can include:

[0245] The acquisition module 1401 is configured to acquire a training set, the training set including a plurality of samples and a label corresponding to each sample.

[0246] The training module 1402 is configured to perform iterative training on an initial model by taking the training set as an input of the initial model, to obtain a recommendation model.

[0247] The recommendation model is used to perform a plurality of tasks of making recommendations for users, and includes a shared feature extraction network, a plurality of tower expert networks corresponding to each task respectively, and a task-specific feature extraction network corresponding to each task respectively. An output end of the shared feature extraction network is connected to an input end of each tower expert network, and an input end of each of the plurality of tower expert networks is further connected to an output end of the task-specific feature extraction network corresponding to each task respectively. In each iteration process, a sample in the training set is taken as an input of the initial model obtained in the last iteration, a first loss value between a first output result of the model obtained in the last iteration and the label of the input sample is acquired, a second loss value between a second output result of each tower expert network and the first output result is acquired, and the model obtained in the last iteration is updated according to the second loss value and the first loss value, to obtain a model of the current iteration.

[0248] In a possible implementation, the recommendation model further includes a tower feature extraction network corresponding to each of the plurality of tower expert networks one by one, an input end of each tower expert network is further connected to an output end of the corresponding tower feature extraction network, the tower feature extraction network is used to extract features related to the task performed by the corresponding tower expert network from the input data, and the tower feature extraction network parameters corresponding to the plurality of tower expert networks are different.

[0249] In a possible implementation, the recommendation model further includes a plurality of gating networks, each tower expert network corresponding to a gating network, and the gating network is used to fuse outputs of the corresponding task-specific feature extraction network, the shared feature extraction network, and the tower feature extraction network, and take the fusion result as an input of the corresponding tower expert network.

[0250] In a possible implementation, the plurality of tasks include predicting a click rate and predicting conversion information, the click rate being a probability of a user clicking on a target object, and the conversion information including a conversion rate or a conversion duration, the conversion rate being a probability of the user performing a conversion operation on the target object after clicking on the target object, and the conversion duration including a duration of the user staying after performing the conversion operation on the target object after clicking on the target object.

[0251] Referring to Figure 15 , another structural diagram of a recommendation device is provided in the application, and is described as follows.

[0252] The recommendation device can include a processor 1501 and a memory 1502. The processor 1501 and the memory 1502 are interconnected through a line. The memory 1502 stores program instructions and data.

[0253] The memory 1502 stores program instructions and data corresponding to the steps in the foregoing Figures 10-12 .

[0254] The processor 1501 is configured to execute the method steps performed by the recommendation device shown in any of the embodiments in the foregoing Figures 10-12 .

[0255] Optionally, the recommendation device can further include a transceiver 1503 for receiving or sending data.

[0256] The application also provides a computer-readable storage medium, which stores a program. When the program is run on a computer, the computer executes the steps in the method described in the embodiments of the foregoing Figures 10-12 .

[0257] Optionally, the recommendation device shown in the foregoing Figure 15 is a chip.

[0258] Referring to Figure 16 , another structural diagram of a training device is provided in the application, and is described as follows.

[0259] The training device can include a processor 1601 and a memory 1602. The processor 1601 and the memory 1602 are interconnected through a line. The memory 1602 stores program instructions and data.

[0260] The memory 1602 stores program instructions and data corresponding to the steps in the foregoing Figures 8-9 .

[0261] The processor 1601 is configured to execute the method steps performed by the training device shown in the foregoing Figures 8-9 .

[0262] Optionally, the training apparatus can further include a transceiver 1603 for receiving or transmitting data.

[0263] The embodiment of the present application further provides a computer readable storage medium, which stores a program, and when the program is run on a computer, the computer is enabled to perform the steps of the method described in the foregoing Figures 8-9 embodiments.

[0264] Optionally, the training apparatus shown in the foregoing Figure 16 is a chip.

[0265] The embodiment of the present application further provides a recommendation apparatus, which can also be referred to as a digital processing chip or a chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface. The program instructions are executed by the processing unit. The processing unit is used to perform the steps of the method described in the foregoing Figures 10-12 embodiments.

[0266] The embodiment of the present application further provides a training apparatus, which can also be referred to as a digital processing chip or a chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface. The program instructions are executed by the processing unit. The processing unit is used to perform the steps of the method described in the foregoing Figures 8-9 embodiments.

[0267] The embodiment of the present application further provides a digital processing chip. The digital processing chip integrates a circuit for implementing the functions of the processor 1501 and the processor 1601 or the functions of the processor 1501 and the processor 1601 and one or more interfaces. When the digital processing chip integrates a memory, the digital processing chip can complete the steps of the method of any one or more of the foregoing embodiments. When the digital processing chip does not integrate a memory, the digital processing chip can be connected with an external memory through a communication interface. The digital processing chip implements the actions performed by the recommendation apparatus or the training apparatus in the foregoing embodiments according to program codes stored in the external memory.

[0268] The embodiment of the present application further provides a computer program product, which, when run on a computer, enables the computer to perform the steps of the method described in the foregoing Figures 8-12 embodiments.

[0269] The recommendation apparatus or the training apparatus provided by the embodiment of the present application can be a chip. The chip includes a processing unit and a communication unit. The processing unit can be a processor for example. The communication unit can be an input / output interface, a pin or a circuit for example. The processing unit can execute computer execution instructions stored in a storage unit, so that the chip in the server executes the steps of the method described in the foregoing Figures 8-12The method steps described in the embodiments are described. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc., and the storage unit can also be a storage unit outside the chip in the wireless access device, such as a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM), etc.

[0270] Specifically, the aforementioned processing unit or processor can be a central processing unit (CPU), a neural-network processing unit (NPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0271] Exemplarily, refer to Figure 17 , Figure 17 A structural schematic diagram of a chip provided by the embodiments of the present application is shown. The chip can be a neural-network processing unit (NPU) 170, which is mounted on a host CPU (Host CPU) as a coprocessor and is assigned tasks by the Host CPU. The core part of the NPU is an operation circuit 1703, which extracts matrix data in a memory and performs multiplication operation under the control of a controller 1704.

[0272] In some implementations, the operation circuit 1703 internally includes a plurality of processing units (PEs). In some implementations, the operation circuit 1703 is a two-dimensional systolic array. The operation circuit 1703 can also be a one-dimensional systolic array or other electronic circuit capable of performing mathematical operations such as multiplication and addition. In some implementations, the operation circuit 1703 is a general-purpose matrix processor.

[0273] For example, assume there is an input matrix A, a weight matrix B, and an output matrix C. The operation circuit takes the data of matrix B from the weight memory 1702 and caches it on each PE of the operation circuit. The operation circuit takes the data of matrix A from the input memory 1701 and performs matrix operation with matrix B to obtain a partial result or a final result of the matrix, which is stored in the accumulator 1708.

[0274] The unified memory 1706 is used to store input data and output data. The weight data is transferred to the weight memory 1702 through the DMAC 1705. The input data is also transferred to the unified memory 1706 through the DMAC.

[0275] The bus interface unit (BIU) 1710 is used for the interaction between the AXI bus and the DMAC and the instruction fetch buffer (IFB) 1709.

[0276] The bus interface unit (BIU) 1710 is used for the instruction fetch buffer (IFB) 1709 to obtain instructions from the external memory, and is also used for the direct memory access controller (DMAC) 1705 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0277] The DMAC is mainly used to transfer the input data in the external memory DDR to the unified memory 1706, or to transfer the weight data to the weight memory 1702, or to transfer the input data to the input memory 1701.

[0278] The vector calculation unit 1707 includes a plurality of operation processing units, which further process the output of the operation circuit as needed, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolution / full connection layer network calculation in neural networks, such as batch normalization, pixel-level summation, upsampling of feature planes, etc.

[0279] In some implementations, the vector computation unit 1707 can store the processed output vector to the unified memory 1706. For example, the vector computation unit 1707 can apply a linear function and / or a non-linear function to the output of the arithmetic circuit 1703, such as linear interpolation on the feature planes extracted by a convolution layer, and / or accumulate the vector of values to generate activation values. In some implementations, the vector computation unit 1707 generates normalized values, pixel-wise summed values, or both. In some implementations, the processed output vector can be used as activation input to the arithmetic circuit 1703, such as for use in a subsequent layer in a neural network.

[0280] The controller 1704 is connected to an instruction fetch buffer 1709 for storing instructions used by the controller 1704;

[0281] The unified memory 1706, the input memory 1701, the weight memory 1702, and the instruction fetch buffer 1709 are on-chip memories. Off-chip memories are private to the NPU hardware architecture.

[0282] In some implementations, the operations of each layer in the recurrent neural network can be performed by the arithmetic circuit 1703 or the vector computation unit 1707.

[0283] In some implementations, the processor mentioned in any of the above can be a general central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for executing programs for controlling the above Figures 8-12 method.

[0284] It should be noted that the apparatus embodiments described above are merely illustrative, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. In addition, the connection relationship between the modules in the apparatus embodiment provided in the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines.

[0285] Those skilled in the art can clearly understand that the application can be implemented by means of software plus necessary universal hardware, and of course can also be implemented by means of dedicated hardware including special-purpose integrated circuits, special-purpose CPUs, special-purpose memories, special-purpose components, etc. Generally, any function completed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure for implementing the same function can also be various, such as analog circuits, digital circuits, or special-purpose circuits, etc. However, for the present application, software program implementation is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a readable storage medium, such as a floppy disk, a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application.

[0286] In the above embodiments, all or part can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part can be implemented in the form of a computer program product.

[0287] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

[0288] The terms "first", "second", "third", "fourth", and the like in the description and in the claims of the present application, and in the above-described drawings if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of these terms herein is to be construed to cover the embodiments described herein unless otherwise indicated herein and / or unless otherwise indicated by the context of their usage. Furthermore, the terms "comprising", "having", "fronted with" and "including" or any variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, product or apparatus that comprises, has, is fronted with or includes a list of steps or elements can not necessarily be limited to those steps or elements but can include other not expressly listed steps or elements.

Claims

1. A recommendation method, characterized in that, include: Obtain input data, which includes user information; The input data is used as input to the recommendation model, and the model outputs recommendation information for the user. The recommendation model is used to perform multiple tasks for recommending to the user. The recommendation model includes a shared feature extraction network, multiple tower expert networks corresponding to each task, and a task-specific feature extraction network corresponding to each task. The output of the shared feature extraction network is connected to the input of each tower expert network, and the input of each of the multiple tower expert networks corresponding to each task is also connected to the output of the task-specific feature extraction network corresponding to each task. The parameters of the multiple tower expert networks are different. The shared feature extraction network is used to extract shared features from the input data. The shared features are used by the tower expert networks corresponding to the multiple tasks. The task-specific feature extraction network is used to extract tower expert network shared features from the input data. The tower expert network shared features are used by the multiple tower expert networks corresponding to a single task. The multiple tower expert networks are used to execute the corresponding task based on the features extracted by the task-specific feature extraction network and the shared feature extraction network. The outputs of the multiple tower expert networks corresponding to the multiple tasks are weighted and fused to obtain the recommendation information. The multiple tower expert networks are used to execute the corresponding task from different perspectives.

2. The method according to claim 1, characterized in that, The recommendation model also includes a tower feature extraction network corresponding to each of the multiple tower expert networks. The tower feature extraction network is used to extract features related to the task performed by the corresponding tower expert network from the input data. The tower feature extraction network parameters corresponding to the multiple tower expert networks are different. The input of each tower expert network also includes the features extracted by the corresponding tower feature extraction network.

3. The method according to claim 2, characterized in that, The recommendation model also includes multiple gating networks, with each tower expert network corresponding to a gating network. The gating network is used to fuse the outputs of the corresponding task-specific feature extraction network, shared feature extraction network, and tower feature extraction network, and uses the fusion result as the input of the corresponding tower expert network.

4. The method according to any one of claims 1-3, characterized in that, Before acquiring the input data, the method further includes: The initial model is iteratively trained to obtain the recommendation model, wherein the structure of the initial model is the same as that of the recommendation model. During one iteration of training the initial model: The training samples are used as input to the initial model, and the first output result is output. Obtain the first loss value between the label of the training sample and the first output result; Obtain multiple second output results from the tower expert network corresponding to the multiple tasks; Obtain multiple second loss values ​​between the first output and the plurality of second output results; The initial model is updated based on the first loss value and the second loss value to obtain the initial model after the current iteration.

5. The method according to any one of claims 1-3, characterized in that, The multiple tasks include predicting click-through rate and predicting conversion information. The click-through rate is the probability that a user clicks on a target object. The conversion information includes conversion rate or conversion duration. The conversion rate is the probability that the user performs a conversion operation on the target object after clicking on it. The conversion duration includes the duration the user stays on the target object after performing a conversion operation.

6. A training method, characterized in that, include: Obtain a training set, which includes multiple samples and the label corresponding to each sample; The initial model is iteratively trained using the training set as input to obtain the recommendation model; The recommendation model is used to perform multiple tasks for recommending users. The recommendation model includes a shared feature extraction network, multiple tower expert networks corresponding to each task, and a task-specific feature extraction network corresponding to each task. The output of the shared feature extraction network is connected to the input of each tower expert network, and the input of each of the multiple tower expert networks corresponding to each task is also connected to the output of each of the task-specific feature extraction networks. The multiple tower expert networks are used to perform corresponding tasks from different perspectives. In each iteration, the samples in the training set are used as input to the initial model obtained in the previous iteration. The first loss value between the first output result of the model obtained in the previous iteration and the label of the input sample is obtained. The second loss value between the second output result of each tower expert network and the first output result is obtained. The model obtained in the previous iteration is updated according to the second loss value and the first loss value to obtain the model of the current iteration.

7. The method according to claim 6, characterized in that, The recommendation model also includes a tower feature extraction network that corresponds one-to-one with the plurality of tower expert networks. The input of each tower expert network is connected to the output of the corresponding tower feature extraction network. The tower feature extraction network is used to extract features related to the task performed by the corresponding tower expert network from the input data, and the tower feature extraction network parameters corresponding to the plurality of tower expert networks are different.

8. The method according to claim 7, characterized in that, The recommendation model also includes multiple gating networks, with each tower expert network corresponding to a gating network. The gating network is used to fuse the outputs of the corresponding task-specific feature extraction network, shared feature extraction network, and tower feature extraction network, and uses the fusion result as the input of the corresponding tower expert network.

9. The method according to any one of claims 6-8, characterized in that, The multiple tasks include predicting click-through rate and predicting conversion information. The click-through rate is the probability that a user clicks on a target object. The conversion information includes conversion rate or conversion duration. The conversion rate is the probability that the user performs a conversion operation on the target object after clicking on it. The conversion duration includes the duration the user stays on the target object after performing a conversion operation.

10. A recommended device, characterized in that, include: The acquisition module is used to acquire input data, which includes user information; The recommendation module is used to take the input data as input to the recommendation model and output recommendation information for the user. The recommendation model is used to perform multiple tasks for recommending to the user. The recommendation model includes a shared feature extraction network, multiple tower expert networks corresponding to each task, and a task-specific feature extraction network corresponding to each task. The output of the shared feature extraction network is connected to the input of each tower expert network, and the input of each of the multiple tower expert networks corresponding to each task is also connected to the output of the task-specific feature extraction network corresponding to each task. The parameters of the multiple tower expert networks are different. The shared feature extraction network is used to extract shared features from the input data. The shared features are used by the tower expert networks corresponding to the multiple tasks. The task-specific feature extraction network is used to extract tower expert network shared features from the input data. The tower expert network shared features are used by the multiple tower expert networks corresponding to a single task. The multiple tower expert networks are used to execute the corresponding task based on the features extracted by the task-specific feature extraction network and the shared feature extraction network. The outputs of the multiple tower expert networks corresponding to the multiple tasks are weighted and fused to obtain the recommendation information. The multiple tower expert networks are used to execute the corresponding task from different perspectives.

11. The apparatus according to claim 10, characterized in that, The recommendation model also includes a tower feature extraction network corresponding to each of the multiple tower expert networks. The tower feature extraction network is used to extract features related to the task performed by the corresponding tower expert network from the input data. The tower feature extraction network parameters corresponding to the multiple tower expert networks are different. The input of each tower expert network also includes the features extracted by the corresponding tower feature extraction network.

12. The apparatus according to claim 11, characterized in that, The recommendation model also includes multiple gating networks, with each tower expert network corresponding to a gating network. The gating network is used to fuse the outputs of the corresponding task-specific feature extraction network, shared feature extraction network, and tower feature extraction network, and uses the fusion result as the input of the corresponding tower expert network.

13. The apparatus according to any one of claims 10-12, characterized in that, The device further includes: The training module is used to iteratively train the initial model to obtain the recommendation model, wherein the structure of the initial model is the same as that of the recommendation model; During one iteration of training the initial model: The training samples are used as input to the initial model, and the first output result is output. Obtain the first loss value between the label of the training sample and the first output result; Obtain multiple second output results from the tower expert network corresponding to the multiple tasks; Obtain multiple second loss values ​​between the first output and the plurality of second output results; The initial model is updated based on the first loss value and the second loss value to obtain the initial model after the current iteration.

14. The apparatus according to any one of claims 10-12, characterized in that, The multiple tasks include predicting click-through rate and predicting conversion information. The click-through rate is the probability that a user clicks on a target object. The conversion information includes conversion rate or conversion duration. The conversion rate is the probability that the user performs a conversion operation on the target object after clicking on it. The conversion duration includes the duration the user stays on the target object after performing a conversion operation.

15. A training device, characterized in that, include: The acquisition module is used to acquire a training set, which includes multiple samples and the label corresponding to each sample. The training module is used to iteratively train the initial model by taking the training set as input to obtain the recommendation model; The recommendation model is used to perform multiple tasks for recommending users. The recommendation model includes a shared feature extraction network, multiple tower expert networks corresponding to each task, and a task-specific feature extraction network corresponding to each task. The output of the shared feature extraction network is connected to the input of each tower expert network, and the input of each of the multiple tower expert networks corresponding to each task is also connected to the output of each of the task-specific feature extraction networks. The multiple tower expert networks are used to perform corresponding tasks from different perspectives. In each iteration, the samples in the training set are used as input to the initial model obtained in the previous iteration. The first loss value between the first output result of the model obtained in the previous iteration and the label of the input sample is obtained. The second loss value between the second output result of each tower expert network and the first output result is obtained. The model obtained in the previous iteration is updated according to the second loss value and the first loss value to obtain the model of the current iteration.

16. The apparatus according to claim 15, characterized in that, The recommendation model also includes a tower feature extraction network that corresponds one-to-one with each of the multiple tower expert networks. The tower feature extraction network is used to extract features related to the tasks performed by the corresponding tower expert network from the input data, and the tower feature extraction network parameters corresponding to the multiple tower expert networks are different.

17. The apparatus according to claim 16, characterized in that, The recommendation model also includes multiple gating networks, with each tower expert network corresponding to a gating network. The gating network is used to fuse the outputs of the corresponding task-specific feature extraction network, shared feature extraction network, and tower feature extraction network, and uses the fusion result as the input of the corresponding tower expert network.

18. The apparatus according to any one of claims 15-17, characterized in that, The multiple tasks include predicting click-through rate and predicting conversion information. The click-through rate is the probability that a user clicks on a target object. The conversion information includes conversion rate or conversion duration. The conversion rate is the probability that the user performs a conversion operation on the target object after clicking on it. The conversion duration includes the duration the user stays on the target object after performing a conversion operation.

19. A recommendation device, characterized in that, It includes at least one processor and a memory, the at least one processor being coupled to the memory for reading and executing instructions in the memory to perform the method as claimed in any one of claims 1-5.

20. A recommended device, characterized in that, It includes at least one processor and a memory, the at least one processor being coupled to the memory for reading and executing instructions in the memory to perform the method as claimed in any one of claims 6-9.

21. A computer-readable storage medium comprising a program, which, when executed by a processing unit, performs the method as described in any one of claims 1 to 9.

22. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 9.

Images