Training of click rate prediction model, click rate prediction method, system and device

Abstract

The application discloses a click rate prediction model training method, a click rate prediction method, a system and equipment, and belongs to the field of user behavior prediction, and aims to solve the problem of low user click rate prediction accuracy of the current model. The method comprises the following steps: inputting basic features into a to-be-trained click rate prediction model and mapping the basic features into a basic feature vector; inputting the features of a historical interactive material of a target user and the features of a target material into the to-be-trained click rate prediction model to generate a first feature vector; inputting the interactive material feature vector into an auxiliary loss module of the to-be-trained click rate prediction model to calculate an auxiliary loss; inputting the basic feature vector and the first feature vector into a multi-layer perception neural network of the click rate prediction model to calculate a main loss; and adjusting parameters of the click rate prediction model based on the main loss and the auxiliary loss. The auxiliary task is trained synchronously with the main task to fully capture the correlation between the materials, thereby improving the accuracy of predicting the user click rate.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of user behavior prediction, and particularly relates to training of a click rate prediction model, a click rate prediction method, a system and a device. BACKGROUND

[0002] A CTR (Click-Through-Rate) prediction model based on deep learning can find the relationship between the feedback behavior (such as clicking, collecting, purchasing, etc.) of a user to recommended information or search results and various features (including the features of the user himself, the features of the recommended information, and / or the features of the search results, etc.), so as to predict whether a certain user will click on a certain advertisement or other recommended information or search results. Therefore, the CTR prediction model has been widely applied in the field of information recommendation and search and has achieved remarkable results.

[0003] The prediction process of the CTR prediction model is generally that, given a user and a query keyword and the information matched with the query keyword, the model can predict the probability of obtaining clicks after the information is exposed.

[0004] And the training process of the CTR prediction model is generally that, user behavior data of a user to recommended information or search results is collected, a feature sequence serving as a training sample is constructed, and then the CTR prediction model is trained based on the constructed feature sequence.

[0005] One way of training the CTR prediction model is embedding-based end-to-end training. With this training method, the embedding layer is taken as part of the neural network, and the embedding layer is updated at the same time when each layer of parameters is updated by forward propagation. The embedding-based end-to-end training often first divides the feature values of the feature sequence according to a hash function to convert them into categorical features, that is, after calculating the hash values of the feature values in the feature sequence based on the hash function, the feature values with the same hash value are classified into a hash bucket as features of the same category, and then the embedding layer is used to map the features of the same category to the same embedding vector, and then the CTR prediction model is trained based on the embedding vector.

[0006] Taking the scenario of "information recommendation in a push (pop-up) manner" as an example, when the embedding-based end-to-end training method is used to train the CTR prediction model applied to this scenario, a behavior feature sequence of a user to the recommended material is constructed.

[0007] The characteristics of the scene are that the behaviors of the user are often concentrated on hot materials, that is, the user has relatively more click, collection, purchase and other behaviors on the hot materials, which makes the long tail effect more serious from the perspective of the total materials: the behavior feature sequence of a small number of hot materials is relatively rich, and the behavior feature sequence of a large number of non-hot materials is relatively sparse. For the feature sequence, "sparse" means that most of the data in the feature sequence is 0, a small part is not 0, and the number of the small part of data that is not 0 is much smaller than the length of the feature sequence.

[0008] In the above case, the model is difficult to fully learn the relationship between the behavior feature sequence of a large number of non-hot materials and the CTR, thereby resulting in low prediction accuracy of the CTR prediction model finally trained. SUMMARY

[0009] The embodiment of the present application provides a training method of a CTR prediction model, to solve the problem of low prediction accuracy of the CTR prediction model finally trained when the CTR prediction model is trained based on the embedding end-to-end training method, because the model is difficult to fully learn the relationship between the behavior feature sequence of a large number of non-hot materials and the CTR.

[0010] The embodiment of the present application also provides a prediction method of a click rate, to solve the problem of low prediction accuracy of the CTR prediction model finally trained when the CTR prediction model is trained based on the embedding end-to-end training method, because the model is difficult to fully learn the relationship between the behavior feature sequence of a large number of non-hot materials and the CTR.

[0011] The embodiment of the present application also provides a training system of a click rate prediction model, a prediction system of a click rate, an electronic device and a computer readable storage medium.

[0012] The embodiment of the present application adopts the following technical solutions:

[0013] A training method of a click rate prediction model, comprising:

[0014] inputting a basic feature into a to-be-trained click rate prediction model, so that the to-be-trained click rate prediction model performs: mapping the basic feature into a corresponding basic feature vector through an embedding layer; the basic feature includes: basic features of a target user, interaction behavior features of the target user on exposed materials, and exposure mode features of the exposed materials;

[0015] The features of the historical interaction materials of the target user, the features of the target material, and the interaction material features of other users are input into the to-be-trained click rate prediction model, so that the to-be-trained click rate prediction model performs: through an embedding layer, the features of the historical interaction materials of the target user are mapped into interaction material feature vectors of the target user, the features of the target material are mapped into a target material feature vector, and the interaction material features of other users are mapped into interaction material feature vectors of other users; based on the interaction material feature vectors of the target user and the target material feature vector, a first feature vector representing the similarity between the target material and the historical interest of the target user is generated;

[0016] The latest interaction material feature vector corresponding to the latest interaction time in the interaction material feature vectors of the target user and the other interaction material feature vectors except the latest interaction material feature vector are input into an auxiliary loss module of the to-be-trained click rate prediction model, so that the auxiliary loss module performs: based on the latest interaction material feature vector and the other interaction material feature vectors, a second feature vector representing the similarity between the latest interaction material and the other interaction materials is determined as a positive sample; the interaction material feature vectors of other users are used as negative samples, the probability of the target user clicking a material at the latest interaction time is determined based on the positive sample and the negative sample, and an auxiliary loss is calculated according to the probability of clicking the material;

[0017] The spliced feature vector is input into a multi-layer perceptron neural network of the to-be-trained click rate prediction model, so that the multi-layer perceptron neural network performs: a main loss is calculated, a global loss is determined based on the main loss and the auxiliary loss, and the parameters of the to-be-trained click rate prediction model are adjusted according to the global loss;

[0018] The spliced feature vector is obtained by splicing the basic feature vector and the first feature vector.

[0019] A click rate prediction method, comprising:

[0020] The historical interaction material features and the basic features of the to-be-predicted user are obtained.

[0021] The historical interaction material features and the basic features are input into the trained click rate prediction model to obtain the click rate of the to-be-predicted user on the material at the next time;

[0022] The click rate prediction model is trained by the training method of the click rate prediction model.

[0023] A click rate prediction model training system, comprising:

[0024] The base feature input unit is configured to input base features into a click rate prediction model to be trained, so that the click rate prediction model to be trained performs: mapping the base features into corresponding base feature vectors through an embedding layer; the base features include basic features of a target user, interaction behavior features of the target user for exposed materials, and exposure mode features of the exposure;

[0025] The material feature input unit is configured to input features of historical interaction materials of the target user, features of the target material, and interaction material features of other users into the click rate prediction model to be trained, so that the click rate prediction model to be trained performs: mapping the features of the historical interaction materials of the target user into respective interaction material feature vectors of the target user, mapping the features of the target material into a target material feature vector, and mapping the interaction material features of the other users into respective interaction material feature vectors of the other users through an embedding layer; generating a first feature vector representing similarity between the target material and historical interests of the target user based on the respective interaction material feature vectors of the target user and the target material feature vector;

[0026] The auxiliary training unit is configured to input a latest interaction material feature vector corresponding to a latest interaction time in the respective interaction material feature vectors of the target user and other interaction material feature vectors except the latest interaction material feature vector into an auxiliary loss module of the click rate prediction model to be trained, respectively, so that the auxiliary loss module performs: determining a second feature vector representing similarity between the latest interaction material and the other interaction materials as a positive sample based on the latest interaction material feature vector and the other interaction material feature vectors; taking the respective interaction material feature vectors of the other users as negative samples, determining a probability that the target user clicks a material at the latest interaction time based on the positive sample and the negative samples, and calculating an auxiliary loss according to the probability of clicking the material;

[0027] The main task training unit is configured to input the spliced feature vector into a multi-layer perceptron neural network of the click rate prediction model to be trained, so that the multi-layer perceptron neural network performs: calculating a main loss, determining a global loss based on the main loss and the auxiliary loss, and adjusting parameters of the click rate prediction model to be trained according to the global loss;

[0028] The spliced feature vector is obtained by splicing the base feature vector and the first feature vector.

[0029] A click rate prediction system includes:

[0030] The acquisition unit is configured to acquire historical interaction material features and base features of a user to be predicted.

[0031] a prediction unit configured to input the historical interaction material features and the base features into a trained click rate prediction model to obtain a click rate of a user to be predicted on a material at a next time point;

[0032] The click rate prediction model is trained by using the training method of the click rate prediction model.

[0033] An electronic device includes a memory and a processor, wherein:

[0034] The memory is configured to store a computer program.

[0035] The processor is coupled to the memory and is configured to execute the computer program stored in the memory to perform the method.

[0036] A computer readable storage medium storing a computer program, the computer program being executable by a computer to implement the method.

[0037] The above at least one technical solution adopted by the embodiments of the present application can achieve the following beneficial effects:

[0038] When the click rate prediction model is trained, the latest interaction material feature vector and the feature vectors of other interaction materials except the latest interaction material are added to calculate the latest interaction time

[0039] The auxiliary task of the clicked material is trained synchronously with the main task to fully capture the correlation between the materials, better learn the embedding representation of the material, and thus improve the prediction accuracy of the trained click rate prediction model. In the auxiliary task, the similarity of the latest interaction material feature vector and the feature vectors of other interaction materials except the latest interaction material is calculated to capture the correlation between the historical interaction material and the target material, so that the trained click rate prediction model can more accurately predict the click rate of the user. BRIEF DESCRIPTION OF DRAWINGS

[0040] The accompanying drawings used to provide further understanding of the present application, form a part of the present application, the illustrative embodiments of the present application and the description thereof serve to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 is a flowchart of a click rate prediction model training method provided by an embodiment of the present application;

[0042] Figure 2 is a flowchart of an implementation of a click rate prediction model training method provided by an embodiment of the present application;

[0043] Figure 3is a flowchart of a click rate prediction method provided by an embodiment of the present application;

[0044] Figure 4 is a structural diagram of a click rate prediction model training system provided by an embodiment of the present application;

[0045] Figure 5 is a structural diagram of a click rate prediction system provided by an embodiment of the present application;

[0046] Figure 6 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0047] Before introducing the technical solutions disclosed by the embodiments of the present application, first, some terms are explained as follows:

[0048] 1、end-to-end: refers to that the neural network learns or predicts with the original data as input and the final result as output. The end-to-end concept is easier to understand in combination with a non-end-to-end manner: when learning or predicting in the non-end-to-end manner, the input of the neural network is not the original data directly, but the features extracted from the original data.

[0049] The classical machine learning manner is to pre-process the raw data into features according to the prior knowledge of human beings, and then classify the features, i.e., the non-end-to-end manner. Since the classification result depends on the quality of the features, the machine learning experts in the past spent most of their time on designing features.

[0050] With the development of deep learning neural networks, it is proved that the multi-layer neural network can couple any nonlinear function, and through some configurations (such as configuring the parameters of the neurons of some layers), the network can do the feature design work that previously required human intervention, so that the entire input to the final output can be realized without too much human setting from the original data to the final output indicator, which forms the end-to-end manner.

[0051] Taking the end-to-end learning manner of the neural network as an example, the end-to-end learning can save the data labeling before the execution of each independent learning task, and save the human processing resources.

[0052] 2、embedding: i.e., embedding, derived from the natural language processing (NLP) field, called word embedding, mainly using background information to construct a distributed representation of vocabulary, and finally obtaining a vectorized expression of a word, i.e., using an abstract dense vector to represent a word.

[0053] Embedding in mathematics means a mapping, also a function, that is f: X→Y, where the function is injective (that is, we call it a single function, each Y has only a unique X corresponding, and vice versa) and structure-preserving (structure preservation, such as X1 < X2 in the space to which X belongs, then the mapping is Y1 < Y2 in the space to which Y belongs). So for word embedding, it is to map the word word to another space, where the mapping has the characteristics of injective and structure-preserving.

[0054] If the word is regarded as the smallest unit of text, word embedding itself is a special mapping process, that is, a certain word in the text space is mapped (or embedded) to another numerical vector space by a certain method, and the original integer is all changed into real number, which is a method of representing discrete variables with continuous vectors. The reason why it is called embedding is that this representation method often accompanies a dimension reduction, just like a high-dimensional thing is flattened and embedded into another low-dimensional space.

[0055] The embedding vectorization process is usually accompanied by the following changes: high dimension → low dimension; sparse → dense; discrete → continuous; integer → real number.

[0056] After embedding vectorization, the data becomes more suitable for the training and learning of deep neural networks, and is also beneficial to the engineering processing of data in the industry. This is because: high-dimensional sparse data is not very friendly to parameter learning and related calculations of machine learning - high dimension can easily cause "dimension disaster", making it difficult to effectively measure the space distance, and in addition, high dimension often makes the number of parameters very large, increasing the computational complexity and easily leading to overfitting; sparse data is prone to gradient vanishing, which makes it difficult to effectively complete parameter learning - therefore, discrete data that is particularly sparse is more suitable for using embedding instead of traditional One-Hot encoding.

[0057] Meanwhile, the embedding, although a dimension reduction representation, carries semantic information, and this representation is not limited to words, but can be sentences, documents, items, people, etc., and the embedding can well mine the internal correlation between embedded entities, and even the dimension reduction can preserve this potential relationship. When training the embedding vectors of items and users, if the embedding of the two is in the same vector space, we can calculate their similarity through inner product, cosine, Euclidean distance, etc. similarity calculation method, so as to make personalized recommendation through user-item similarity, or find similar items through item-item similarity.

[0058] The technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0059] Those of ordinary skill in the art can know that, with the development of technology and the appearance of new scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0060] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that a process, method, system, product or device containing a series of units does not have to be limited to those units, but can include other units not clearly listed or inherent to these processes, methods, products or devices.

[0061] The training of the click rate prediction model, the click rate prediction method, system and device provided by the embodiments of the present application will be described in detail below in combination with the drawings and specific embodiments and their application scenarios. The click rate prediction method provided by the embodiments of the present application is used to solve the problem that when training a CTR prediction model based on an end-to-end training method, the model is difficult to sufficiently learn the relationship between the behavior feature sequence and the CTR for a large number of non-hot materials, resulting in a low prediction accuracy of the CTR prediction model obtained by training.

[0062] For ease of description, the following describes an embodiment of the method, taking a computer device training a click rate prediction model as an example. It should be understood that the description of the computer device training the click rate prediction model does not limit the method.

[0063] The specific implementation process of the method for training the click rate prediction model provided by the embodiments of the present application is shown in Figure 1 The following describes the steps in combination with the flow of an embodiment of the method. Figure 2 The following describes the steps in combination with the flow of an embodiment of the method.

[0064] Step 11: input the basic features into the click rate prediction model to be trained, so that the click rate prediction model to be trained performs: mapping the basic features into corresponding basic feature vectors through an embedding layer; the basic features include basic features of a target user, interactive behavior features of the target user for an exposed material, and exposure mode features of the exposure.

[0065] The execution process of step 11 is implemented by Figure 2 a basic feature module.

[0066] The basic features of the target user include gender, age, and click rate in the past month, and other feature information.

[0067] The interactive behavior features of the target user for the exposed material include feature information such as the number of views of the material in the past day, the number of interactions, and the interaction rate.

[0068] The exposure mode features of the exposure include feature information such as the time point at which the material is exposed to the user and whether it is a weekend.

[0069] The above basic features such as "in the past month", "in the past day", and "weekend" are examples and do not limit the basic features.

[0070] It should be noted that in the basic features, numerical features are discretized into categorical features, which are then concatenated with other categorical features, and projected from high-dimensional sparse categorical features to low-dimensional dense vectors featureembedding through an embedding_table, i.e., the embeding of the categorical features. The sparse representation of the categorical features has the problems of too large ont-hot vector and distance between vectors, which may make the model difficult to learn effectively, so the embeding of the categorical features is born.

[0071] Step 12: input the features of the historical interaction materials of the target user, the features of the target material, and the interaction material features of other users into the to-be-trained click rate prediction model, so that the to-be-trained click rate prediction model performs: through an embedding layer, map the features of the historical interaction materials of the target user to each interaction material feature vector of the target user, map the features of the target material to a target material feature vector, and map the interaction material features of other users to each interaction material feature vector of other users; generate a first feature vector representing the similarity between the target material and the historical interest of the target user based on each interaction material feature vector of the target user and the target material feature vector.

[0072] The execution process of step 12 is as follows Figure 2 The part of the sequence feature module other than the auxiliary loss module is implemented.

[0073] In an implementation manner, the historical interaction materials of the target user are selected in the vicinity of T according to the interaction time. The features of the historical interaction materials of the target user can be mid vectors of the interaction materials, the features of the target material are mid vectors of the target material, and the mid vector is an ID code of the material.

[0074] It should be noted that for the historical interaction materials of the target user, T interaction materials meeting the requirements are obtained by preprocessing the historical interaction materials. The preprocessing includes deleting invalid interaction behaviors, and the invalid interaction behaviors are determined by a preset condition, such as filtering out interaction behaviors with a browsing time less than a preset time after clicking. The T interaction materials are valid interaction materials after preprocessing.

[0075] The embedding layer is used to map the features of the historical interaction materials of the target user to each interaction material feature vector of the target user, including:

[0076] Step 1211: obtain the side information of each material in the historical interaction materials of the target user, wherein the side information represents the interaction information between the target user and each material in the historical interaction materials; and the side information includes an interaction scene, an interaction time, and an interaction behavior weight.

[0077] Step 1212: input the side information of each material in the historical click sequence into a shallow fully connected network for learning, output the weight value corresponding to each material, and correspondingly use the weight value of each material as the weighted weight of each interaction material feature vector; in an ideal case, the material with a closer interaction time and a stronger interaction behavior can learn a larger weight.

[0078] Step 1213, using the weight value, calculating the weighted sum of each interaction material feature vector as a interest feature vector u representing the user's interest.

[0079] The first feature vector representing the similarity between the target material and the target user's historical interest is generated based on the target user's interaction material feature vector and the target material feature vector, which includes: calculating the inner product of the interest feature vector u and the target material feature vector to obtain the first feature vector u representing the similarity between the target material and the user's historical interest. T v, used to represent the similarity between the target material and the target user's historical interest.

[0080] It should be noted that the embedding layer in steps 11 and 12 is mapped from high-dimensional sparse category features to low-dimensional dense vectors feature embeddings through embedding_table.

[0081] Step 13, inputting the latest interaction material feature vector corresponding to the latest interaction time in the target user's interaction material feature vector and other interaction material feature vectors except the latest interaction material feature vector into the auxiliary loss module of the to-be-trained click rate prediction model, so that the auxiliary loss module executes: based on the latest interaction material feature vector and the other interaction material feature vectors, determining a second feature vector representing the similarity between the latest interaction material and the other interaction materials as a positive sample; taking each interaction material feature vector of other users as a negative sample, based on the positive sample and the negative sample, determining the probability of the target user clicking the material at the latest interaction time, and calculating the auxiliary loss according to the probability of clicking the material.

[0082] The execution process of step 13 is realized through Figure 2 auxiliary loss module.

[0083] T interaction materials are arranged in reverse order according to time sequence, and are divided into a latest interaction material feature vector corresponding to the latest interaction time and other interaction material feature vectors except the latest interaction material feature vector.

[0084] The second feature vector representing the similarity between the latest interaction material and the other interaction materials is determined based on the latest interaction material feature vector and the other interaction material feature vectors, which includes:

[0085] The weighted sum of the other interaction material feature vectors is calculated as the representation vector u of the target user T-1; and obtaining a second feature vector representing the similarity between the latest interaction material and other interaction materials by performing an inner product operation on the representation vector of the target user and the latest interaction material feature vector.

[0086] The negative samples are randomly sampled from the interaction material feature vectors of other users, and the number of randomly sampled samples can be pre-set according to requirements, such as 100.

[0087] The interaction material corresponding to the interaction material feature vector as the negative sample excludes the historical interaction material of the target user.

[0088] The probability of the target user clicking the material at the latest interaction time is determined based on the positive samples and the negative samples, and an auxiliary loss is calculated according to the probability of clicking the material, including:

[0089] The second feature vector as the positive sample and each interaction material feature vector as the negative sample are normalized by using a softmax function, the probability of the target user clicking the material at the latest interaction time is calculated, and an auxiliary loss is calculated based on the probability of clicking the material at the latest interaction time by using a cross-entropy loss function.

[0090] In an implementation manner, the probability p j , specifically:

[0091]

[0092] (1) In the formula, u T-1 is a representation vector of the historical interaction material of the target user, which is obtained by weighted summation of other interaction material feature vectors except the latest interaction material feature vector; K is the total number of positive samples and negative samples involved when calculating the auxiliary loss of sample k, i is a feature vector of material i, and material i represents the corresponding material in the positive sample and the negative sample; v j is a feature vector of material j, and material j represents the corresponding material in the positive sample.

[0093] In an implementation manner, the determination of the auxiliary loss is specifically:

[0094]

[0095] (2) In the formula, N is the number of samples in a data packet in the training set, represents whether material j is the corresponding material of the behavior at the latest interaction time when sample k is calculated, The value of is 1 or 0. P (z = 1 | x, k) represents the probability of clicking the material j at the latest interaction time when the sample k is calculated.

[0096] In step 14, the spliced feature vector is input into the multi-layer perception neural network of the to-be-trained click rate prediction model, so that the multi-layer perception neural network performs: calculating a main loss, determining a global loss based on the main loss and the auxiliary loss; and adjusting parameters of the to-be-trained click rate prediction model according to the global loss; wherein the spliced feature vector is obtained by splicing the base feature vector and the first feature vector.

[0097] In an implementation manner, the output of the base feature module, that is, the base feature vector, and the output of the sequence feature module, that is, the first feature vector, are spliced, and the spliced feature vector is input into the multi-layer perception neural network (MLP) to calculate the main loss.

[0098] The main loss is determined by a cross-entropy loss function, and specifically includes:

[0099] The cross-entropy loss function is used to determine the main loss function, where the main loss function L target is represented as:

[0100]

[0101] (3) In formula (3), N represents the number of samples in a data packet in the training set, z k represents whether the user behavior in the kth sample corresponds to the target material, z k takes a value of 1 or 0, q k represents the probability that the kth sample estimated by the model is a positive sample.

[0102] According to the auxiliary loss and the main loss, a global loss function main loss is obtained, where the global loss function L final is represented as:

[0103] L final = L target + βL aux (4)

[0104] (4) In formula (4), is a hyperparameter used to adjust the influence of the auxiliary task on the training task, L aux is an auxiliary loss function.

[0105] L final is used as the main loss to back-propagate the network parameters, and the gradient of the auxiliary loss aux only trains the parameters related to the user sequence module, and the gradient of the main loss L target acts on all parameters in the model. During the training process, when L finalWhen the decrease in the training time is no longer decreasing with the increase in the training time, we consider that the parameters have converged and obtain a trained model.

[0106] The training method of the click rate prediction model provided in the embodiments of the present application is used to input the basic features into the click rate prediction model to be trained, so that the click rate prediction model to be trained performs: mapping the basic features into corresponding basic feature vectors through an embedding layer; the basic features include basic features of a target user, interaction behavior features of the target user for exposed materials, and exposure mode features of the exposure; the features of the historical interaction materials of the target user, the features of the target material, and the interaction material features of other users are input into the click rate prediction model to be trained, so that the click rate prediction model to be trained performs: mapping the features of the historical interaction materials of the target user into each interaction material feature vector of the target user, mapping the features of the target material into a target material feature vector, and mapping the interaction material features of other users into each interaction material feature vector of other users through an embedding layer; generating a first feature vector representing the similarity between the target material and the historical interest of the target user based on each interaction material feature vector of the target user and the target material feature vector; inputting a latest interaction material feature vector corresponding to the latest interaction time in each interaction material feature vector of the target user and other interaction material feature vectors except the latest interaction material feature vector into an auxiliary loss module of the click rate prediction model to be trained, so that the auxiliary loss module performs: determining a second feature vector representing the similarity between the latest interaction material and other interaction materials as a positive sample based on the latest interaction material feature vector and the other interaction material feature vectors; taking each interaction material feature vector of other users as a negative sample, determining the probability that the target user clicks a material at the latest interaction time based on the positive sample and the negative sample, calculating an auxiliary loss according to the probability of clicking the material; inputting the spliced feature vector into a multi-layer perceptron neural network of the click rate prediction model to be trained, so that the multi-layer perceptron neural network performs: calculating a main loss, determining a global loss based on the main loss and the auxiliary loss; and adjusting the parameters of the click rate prediction model to be trained according to the global loss. When training the click rate prediction model, the auxiliary task of predicting the clicking of the material at the latest interaction time through the latest interaction material feature vector and the other interaction material feature vectors except the latest interaction material is added, and the auxiliary task is trained synchronously with the main task, so as to fully capture the correlation between materials, better learn the embedding representation of the material, and thus improve the accuracy of predicting the click rate of the user; in the auxiliary task, the similarity between the latest interaction material feature vector and the other interaction material feature vectors except the latest interaction material is calculated, the correlation between the historical interaction material and the target material is captured, and the click rate of the user is more accurately predicted.

[0107] The embodiment of the application further provides a click rate prediction method to solve the problem that the prediction accuracy of a CTR prediction model obtained by training is low due to the difficulty of the model in fully learning the relationship between the behavior feature sequence and the CTR of a large number of non-hot materials when the CTR prediction model is trained in an embedding-based end-to-end training manner. The process of the click rate prediction method is shown in Figure 3 , and includes the following steps.

[0108] In step 31, the historical interaction material features and the basic features of a user to be predicted are obtained.

[0109] In step 32, the historical interaction material features and the basic features are input into the trained click rate prediction model to obtain the click rate of the material by the user to be predicted at the next moment.

[0110] The click rate prediction model is obtained by training the training method of the click rate prediction model.

[0111] The click rate prediction method provided by the embodiment of the application fully captures the correlation between materials, better learns the embedding representation of the material, and thus improves the accuracy of predicting the click rate of the user.

[0112] To solve the problem that the prediction accuracy of a CTR prediction model obtained by training is low due to the difficulty of the model in fully learning the relationship between the behavior feature sequence and the CTR of a large number of non-hot materials when the CTR prediction model is trained in an embedding-based end-to-end training manner, the embodiment of the application also provides a training system of a click rate prediction model based on the same inventive concept as the above-mentioned embodiments of the application. The structure of the prediction system is shown in Figure 4 , and includes the following steps.

[0113] The basic feature input unit 41 is configured to input the basic features into the click rate prediction model to be trained, so that the click rate prediction model to be trained performs the following operations: the basic features are mapped into corresponding basic feature vectors by an embedding layer; the basic features include the basic features of a target user, the interaction behavior features of the target user to exposed materials, and the exposure mode features of the exposed materials.

[0114] The material feature input unit 42 is configured to input the features of the historical interaction materials of the target user, the features of the target material, and the interaction material features of other users into the to-be-trained click rate prediction model, so that the to-be-trained click rate prediction model performs: mapping the features of the historical interaction materials of the target user into interaction material feature vectors of the target user, mapping the features of the target material into a target material feature vector, and mapping the interaction material features of other users into interaction material feature vectors of other users through an embedding layer; generating a first feature vector representing the similarity between the target material and the historical interest of the target user based on the interaction material feature vectors of the target user and the target material feature vector;

[0115] The auxiliary training unit 43 is configured to input a latest interaction material feature vector corresponding to the latest interaction time in the interaction material feature vectors of the target user and other interaction material feature vectors except the latest interaction material feature vector into an auxiliary loss module of the to-be-trained click rate prediction model, so that the auxiliary loss module performs: determining a second feature vector representing the similarity between the latest interaction material and other interaction materials based on the latest interaction material feature vector and the other interaction material feature vectors as a positive sample; taking the interaction material feature vectors of other users as negative samples, determining the probability of the target user clicking a material at the latest interaction time based on the positive sample and the negative sample, and calculating an auxiliary loss according to the probability of the clicked material.

[0116] The main task training unit 44 is configured to input the spliced feature vector into a multi-layer perceptron neural network of the to-be-trained click rate prediction model, so that the multi-layer perceptron neural network performs: calculating a main loss, determining a global loss based on the main loss and the auxiliary loss, and adjusting parameters of the to-be-trained click rate prediction model according to the global loss; wherein the spliced feature vector is obtained by splicing the basic feature vector and the first feature vector.

[0117] In the basic feature input unit 41, the basic features of the target user include gender, age, and click rate in the past month, and the like; the interaction behavior features of the target user for the exposed material include the number of views, the number of interactions, and the interaction rate of the material in the past day, and the like; and the exposure mode features of the exposure include the time point at which the material is exposed to the user and whether it is a weekend, and the like.

[0118] In the material feature input unit 42, the historical interaction materials of the target user are selected in the vicinity of T according to the interaction time. The features of the historical interaction materials of the target user are mapped into interaction material feature vectors of the target user, and are specifically used for:

[0119] obtain side information of each material in the historical interaction material of the target user, wherein the side information represents interaction information between the target user and each material in the historical interaction material; and the side information includes an interaction scenario, an interaction time, and an interaction behavior weight.

[0120] The side information of all materials in the historical click sequence is collectively input into a shallow fully connected network for learning, and a weight value corresponding to each material is output. The weight value corresponding to each material is used as a weighted weight of each interaction material feature vector. In an ideal case, the material with a closer interaction time and a stronger interaction behavior can learn a larger weight.

[0121] The weight value is used to calculate a weighted sum pooling of each interaction material feature vector, and an interest feature vector u representing user interest is obtained.

[0122] The interaction material feature vector of the target material and the target material feature vector are used to generate a first feature vector representing the similarity between the target material and the historical interest of the target user, and the first feature vector is used to:

[0123] The inner product of the interest feature vector u and the target material feature vector is calculated to obtain a first feature vector u representing the similarity between the target material and the historical interest of the target user. T v, which is used to represent the similarity between the target material and the historical interest of the target user.

[0124] In the auxiliary training unit 43, the latest interaction material feature vector and the other interaction material feature vector are used to determine a second feature vector representing the similarity between the latest interaction material and the other interaction material, and the second feature vector is used to:

[0125] The target user representation vector u of the other interaction material feature vector is calculated. T-1 The inner product of the target user representation vector and the latest interaction material feature vector is calculated to obtain a second feature vector representing the similarity between the latest interaction material and the other interaction material, and the second feature vector is used as a positive sample.

[0126] The negative sample is obtained by randomly sampling from the other user interaction material feature vector, and the number of randomly sampled samples can be pre-set according to requirements, such as 100.

[0127] The interaction material of the negative sample excludes the interaction material of the target user.

[0128] The probability of the target user clicking the material at the latest interaction time is determined based on the positive sample and the negative sample, and an auxiliary loss is calculated according to the probability of the clicked material, including:

[0129] The second feature vector of the positive sample and each interaction material feature vector of the negative sample are normalized by using a softmax, and a probability of the target user clicking the material at the latest interaction time is calculated; based on the probability of the target user clicking the material at the latest interaction time, an auxiliary loss is calculated by using a cross-entropy loss function.

[0130] When the click rate prediction model is trained, the training system of the click rate prediction model provided in the embodiments of the present application increases the auxiliary task of calculating the latest interaction time clicking material based on the latest interaction material feature vector and the other interaction material feature vectors except the latest interaction material, and synchronously trains the auxiliary task with the main task, so as to sufficiently capture the correlation between materials, better learn the embedding representation of the material, and thus improve the accuracy of predicting the click rate of the user. In the auxiliary task, the similarity of the latest interaction material feature vector and the other interaction material feature vectors except the latest interaction material is calculated, the correlation between the historical interaction material and the target material is captured, and the click rate of the user is more accurately predicted.

[0131] The embodiments of the present application further provide a click rate prediction system to solve the problem that the prediction accuracy of the CTR prediction model obtained by training is low because the model is difficult to sufficiently learn the relationship between the behavior feature sequence of a large number of non-hotspot materials and CTR when the CTR prediction model is trained in an end-to-end training manner based on embedding. The structure of the prediction system is shown in Figure 5 and includes:

[0132] The acquisition unit 51 is configured to acquire the historical interaction material features and the basic features of the user to be predicted.

[0133] The prediction unit 52 is configured to input the historical interaction material features and the basic features into the trained click rate prediction model, and obtain the click rate of the user to be predicted on the material at the next time.

[0134] The click rate prediction model is trained by using the training method of the click rate prediction model.

[0135] The click rate prediction system provided in the embodiments of the present application sufficiently captures the correlation between materials, better learns the embedding representation of the material, and thus improves the accuracy of predicting the click rate of the user, because the click rate prediction model adopted by the click rate prediction system is trained based on the auxiliary task of calculating the latest interaction time clicking material based on the latest interaction material feature vector and the other interaction material feature vectors except the latest interaction material.

[0136] Figure 6Fig. 1 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. At the hardware level, the electronic device 60 includes a processor 62, and optionally further includes an internal bus, a network interface, and a memory 61. The memory 61 can include an internal memory such as a random-access memory (RAM), and can further include a non-volatile memory such as at least one disk memory. Of course, the electronic device can further include other hardware required by a business.

[0137] The processor 62, the network interface, and the memory 61 can be connected to each other through the internal bus, which can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCIe) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the figure to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0138] The memory 61 is configured to store a computer program. Specifically, the program can include program code including computer operation instructions. The memory can include an internal memory and a non-volatile memory, and provide instructions and data to the processor.

[0139] The processor 62 is coupled with the memory 61 to read corresponding computer programs from the non-volatile memory into the memory and then run, and form a data synchronization device at a logical level, for executing the computer programs stored in the memory 31 to perform the following operations: inputting basic features into a to-be-trained click rate prediction model, so that the to-be-trained click rate prediction model performs: mapping the input basic features into corresponding basic feature vectors through an embedding layer; the basic features include: basic features of a user, interaction behavior features of the user on exposed materials, exposure mode features of the exposure; inputting features of historical interaction materials of a target user and features of a target material into the to-be-trained click rate prediction model, so that the to-be-trained click rate prediction model performs: mapping the features of the historical interaction materials of the target user into target user interaction material feature vectors and mapping the features of the target material into a target material feature vector through an embedding layer; generating a first feature vector representing similarity between the target material and historical interests of the target user based on the target user interaction material feature vectors and the target material feature vector; inputting a latest interaction material feature vector corresponding to a latest interaction time in the target user interaction material feature vectors and other interaction material feature vectors except the latest interaction material feature vector into an auxiliary loss module included in the to-be-trained click rate prediction model, respectively, so that the auxiliary loss module performs: determining a second feature vector representing similarity between the latest interaction material and the other interaction materials based on the latest interaction material feature vector and the other interaction material feature vectors as a positive sample; taking each interaction material feature vector of other users as a negative sample, determining a probability that the target user clicks a material at the latest interaction time based on the positive sample and the negative sample, calculating an auxiliary loss according to the probability of clicking the material; inputting the spliced feature vector into a multi-layer perceptron neural network of the to-be-trained click rate prediction model, so that the multi-layer perceptron neural network performs: calculating a main loss, determining a global loss based on the main loss and the auxiliary loss; and adjusting parameters of the to-be-trained click rate prediction model according to a value of the global loss function.

[0140] The above as described in the present application Figure 6The method executed by the electronic device provided in the embodiment can be applied to a processor or implemented by the processor. The processor can be an integrated circuit chip with processing capability. In the implementation process, each step of the above method can be completed by integrated logic circuits or instructions in the form of software in the processor. The processor can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; or a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Each method, step and logic block diagram disclosed in the embodiment of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can be any conventional processor. The steps of the method disclosed in combination with the embodiment of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory, and the processor reads the information in the memory and combines the hardware to complete the steps of the above method.

[0141] Of course, in addition to the software implementation, the electronic device of the present application does not exclude other implementation manners, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but also can be hardware or logic device.

[0142] The embodiment of the present application further provides a computer readable storage medium storing a computer program, the computer readable storage medium storing one or more programs, the one or more programs comprising instructions which, when executed by a portable electronic device comprising a plurality of application programs, enable the portable electronic device to perform the following operations: inputting basic features into a click rate prediction model to be trained, so that the click rate prediction model to be trained performs: mapping the input basic features into corresponding basic feature vectors through an embedding layer; the basic features comprise basic features of a user, interaction behavior features of the user on exposed materials, and exposure mode features of the exposure; inputting features of historical interaction materials of a target user and features of a target material into the click rate prediction model to be trained, so that the click rate prediction model to be trained performs: mapping the features of the historical interaction materials of the target user into each interaction material feature vector of the target user and mapping the features of the target material into a target material feature vector through an embedding layer; generating a first feature vector representing similarity between the target material and historical interests of the target user based on each interaction material feature vector of the target user and the target material feature vector; inputting a latest interaction material feature vector corresponding to a latest interaction time in the each interaction material feature vector of the target user and other interaction material feature vectors except the latest interaction material feature vector into an auxiliary loss module included in the click rate prediction model to be trained, so that the auxiliary loss module performs: determining a second feature vector representing similarity between the latest interaction material and the other interaction materials as a positive sample based on the latest interaction material feature vector and the other interaction material feature vectors; taking each interaction material feature vector of another user as a negative sample, determining a probability that the target user clicks a material at the latest interaction time based on the positive sample and the negative sample, and calculating an auxiliary loss according to the probability that the material is clicked; inputting the spliced feature vector into a multi-layer perceptron neural network of the click rate prediction model to be trained, so that the multi-layer perceptron neural network performs: calculating a main loss, determining a global loss based on the main loss and the auxiliary loss, and adjusting parameters of the click rate prediction model to be trained according to a value of the global loss function.

[0143] Those skilled in the art will understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD ROM, optical storage, etc.) containing computer-usable program code.

[0144] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0145] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0146] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0147] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0148] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, for storing instructions and data used and / or generated by the computing device. The memory can also include non-volatile memory, such as read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or nonvolatile random access memory (RAM) (e.g., fault tolerant RAM), for storing instructions and data used and / or generated by the computing device. The memory is an example of computer-readable media.

[0149] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0150] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but also other elements not explicitly listed, or other elements inherent to such a process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising the element.

[0151] Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, system or computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD ROM, optical storage and the like) containing computer usable program code.

[0152] The above only describes the embodiments of the present application and is not intended to limit the present application. Those skilled in the art can make various changes and modifications to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.

Claims

1. A method for training a click rate prediction model, the method comprising: The method comprises the following steps: inputting the basic features into a to-be-trained click rate prediction model, so that the to-be-trained click rate prediction model performs: mapping the basic features into corresponding basic feature vectors through an embedding layer; the basic features include: basic features of a target user, interaction behavior features of the target user for exposed materials, exposure mode features of the exposure; inputting the features of the historical interaction materials of the target user, the features of the target material and the interaction material features of other users into the to-be-trained click rate prediction model, so that the to-be-trained click rate prediction model performs: mapping the features of the historical interaction materials of the target user into each interaction material feature vector of the target user, mapping the features of the target material into a target material feature vector, and mapping the interaction material features of other users into each interaction material feature vector of other users through an embedding layer; generating a first feature vector representing the similarity between the target material and the historical interest of the target user based on each interaction material feature vector of the target user and the target material feature vector; inputting the latest interaction material feature vector corresponding to the latest interaction time in each interaction material feature vector of the target user and the other interaction material feature vectors except the latest interaction material feature vector into an auxiliary loss module of the to-be-trained click rate prediction model, so that the auxiliary loss module performs: determining a second feature vector representing the similarity between the latest interaction material and the other interaction materials based on the latest interaction material feature vector and the other interaction material feature vectors as a positive sample; taking each interaction material feature vector of other users as a negative sample, determining the probability of the target user clicking a material at the latest interaction time based on the positive sample and the negative sample, and calculating an auxiliary loss according to the probability of the clicked material; inputting the spliced feature vector into a multi-layer perceptron neural network of the to-be-trained click rate prediction model, so that the multi-layer perceptron neural network performs: calculating a main loss, determining a global loss based on the main loss and the auxiliary loss, and adjusting the parameters of the to-be-trained click rate prediction model according to the global loss; wherein the spliced feature vector is obtained by splicing the basic feature vector and the first feature vector.

2. The method of claim 1, wherein, The first feature vector representing the similarity between the target material and the historical interest of the target user is generated based on each interaction material feature vector of the target user and the target material feature vector, and specifically comprises: weighting and summing each interaction material feature vector of the target user to obtain an interest feature vector representing the interest of the user; calculating the inner product of the interest feature vector and the target material feature vector to obtain the first feature vector representing the similarity between the target material and the historical interest of the target user.

3. The method of claim 2, wherein, The weighting and summing of each interaction material feature vector of the target user, wherein the determination of the weight specifically comprises: obtaining marginal information of each of the historical interaction materials, wherein the marginal information represents interaction information of the target user and each of the historical interaction materials; inputting the marginal information of each of the historical interaction materials into a shallow fully connected network to obtain a weight value corresponding to each of the materials, and taking the weight value corresponding to each of the materials as a weight of each interaction material feature vector.

4. The method of claim 1, wherein, determining a probability of the target user clicking a material at the latest interaction time based on the positive sample and the negative sample, and calculating an auxiliary loss according to the probability of clicking the material, specifically including: after normalizing the second feature vector as the positive sample and each interaction material feature vector as the negative sample, calculating the probability of the target user clicking a material at the latest interaction time; wherein the interaction material corresponding to the interaction material feature vector as the negative sample is a material other than the historical interaction material of the target user; based on the probability of clicking the material at the latest interaction time, calculating the auxiliary loss by using a cross-entropy loss function.

5. The method of claim 4, wherein, a probability p that a target user clicks on item j at the latest interaction time j , in particular: In the formula, u T-1 is a representation vector of the historical interaction materials of the target user, obtained by weighted summation of other interaction material feature vectors except the latest interaction material feature vector; K is the total number of positive samples and negative samples involved in calculating the auxiliary loss of sample k; v i is a feature vector of material i, and material i represents the corresponding material in the positive sample and the negative sample; v j is a feature vector of material j, and material j represents the corresponding material in the positive sample.

6. The method of claim 5, wherein, The determination of the auxiliary loss specifically includes: In the formula, N is the number of samples in a data packet in the training set, represents the material corresponding to the behavior of the latest interaction time when calculating sample k, The value of is 1 or 0; represents the probability of clicking on material j at the latest interaction time when calculating sample k.

7. The method of claim 1, wherein, The determination of the global loss specifically includes: A cross-entropy loss function is used to determine a main loss function, wherein the main loss function L target is represented as: In the formula, N is the number of samples in a data packet in the training set, z k represents whether the user behavior in the kth sample corresponds to the target material, z k , and q k represents the probability that the kth sample estimated by the model is a positive sample. According to the auxiliary loss function and the main loss function, a global loss function is obtained, wherein the global loss function L final is represented as: L final = L target + βL aux where β is a hyperparameter, L aux is the auxiliary loss function.

8. A method for predicting a click rate, the method comprising: including: obtaining historical interaction material features and basic features of a to-be-predicted user; inputting the historical interaction material features and the basic features into the trained click rate prediction model to obtain a click rate of a material at a next time by the to-be-predicted user; The click rate prediction model is trained by using the method of any one of claims 1-7. 9.A system for training a click-through rate prediction model, characterized in that, including: a basic feature input unit configured to input a basic feature into a to-be-trained click rate prediction model, so that the to-be-trained click rate prediction model performs: mapping the basic feature into a corresponding basic feature vector through an embedding layer; the basic feature includes: basic features of a target user, interaction behavior features of the target user to exposed materials, and exposure mode features of the exposed materials; a material feature input unit configured to input features of historical interaction materials of a target user, features of a target material, and interaction material features of other users into the to-be-trained click rate prediction model, so that the to-be-trained click rate prediction model performs: mapping the features of the historical interaction materials of the target user into interaction material feature vectors of the target user, mapping the features of the target material into a target material feature vector, and mapping the interaction material features of the other users into interaction material feature vectors of the other users; and generating a first feature vector representing similarity between the target material and historical interests of the target user based on the interaction material feature vectors of the target user and the target material feature vector. The auxiliary training unit is configured to input a latest interaction material feature vector corresponding to a latest interaction time in each interaction material feature vector of the target user and other interaction material feature vectors except the latest interaction material feature vector into an auxiliary loss module of the to-be-trained click rate prediction model, so that the auxiliary loss module performs the following operations: determining a second feature vector representing similarity between the latest interaction material and the other interaction materials as a positive sample based on the latest interaction material feature vector and the other interaction material feature vectors; determining a probability of the target user clicking a material at the latest interaction time based on the positive sample and a negative sample of each interaction material feature vector of other users, and calculating an auxiliary loss according to the probability of the clicked material; The main task training unit is configured to input the spliced feature vector into a multi-layer perception neural network of the to-be-trained click rate prediction model, so that the multi-layer perception neural network performs the following operations: calculating a main loss, determining a global loss based on the main loss and the auxiliary loss, and adjusting parameters of the to-be-trained click rate prediction model according to the global loss. The spliced feature vector is obtained by splicing the base feature vector and the first feature vector. 10.A system for predicting a click rate, characterized by, The method comprises the following steps: An acquisition unit is configured to acquire historical interaction material features and base features of a to-be-predicted user. A prediction unit is configured to input the historical interaction material features and base features into the trained click rate prediction model to obtain a click rate of the to-be-predicted user on a material at a next time. The click rate prediction model is trained by the method in any one of claims 1-7.

11. An electronic device, comprising: The computer program is executed by a computer to implement the steps of the method in any one of claims 1-8. The computer program is executed by a computer to implement the steps of the method in any one of claims 1-8. ​ 12. A computer readable storage medium storing a computer program, characterized in that, ​

Images