Model training, behavior scoring and content mixed arrangement method and device, equipment and medium

Abstract

The embodiment of the invention provides a model training, behavior scoring and content mixed arrangement method and device, equipment and a medium. The method comprises the steps that attribute information, display positions and refreshing modes of materials and real frequency scores of user behaviors are collected online; converting the attribute information corresponding to the material into a first feature embedding vector through a first feature extraction model; inputting the first feature embedding vector into a first non-first material scoring model for prediction to obtain a first non-first prediction frequency score; the first feature embedding vector, the display position and the refreshing mode are combined and input into a first first material scoring model for prediction, and a first first prediction frequency score is obtained; and taking the first non-first prediction frequency score and the first first prediction frequency score as training prediction values, and taking the real frequency score as a training target value for training to obtain a trained first feature extraction model, a first non-first material scoring model and a first first material scoring model.

Description

Technical Field

[0001] The present invention relates to the field of Internet information flow recommendation, and in particular to a method, device, equipment and medium for model training, behavior scoring and content mixing. Background Art

[0002] In the information flow recommendation system, users will give feedback on the output results of the recommendation system, thereby generating related behaviors, such as browsing, clicking, interacting, and complaining. The recommendation system needs to predict the user's interest preferences, improve the quality of user experience, and improve the accuracy of the materials delivered by the recommendation system. Common recommendation goals in the industry include but are not limited to the probability of users clicking on materials (hereinafter referred to as click-through rate), the probability of interactive behavior (hereinafter referred to as interaction rate), and the user's consumption time. The recommendation system builds a deep neural network model (DNN) based on the recommendation goal, uses all user data and all materials as training samples of the model, and designs a classification or regression task at the output end to supervise the training of the parameters of the DNN model. All materials to be recommended are sorted by the trained DNN model.

[0003] In the process of realizing the present invention, the inventors found that in the recommendation business scenario, the attention paid to the displayed materials will be affected by the display position. The user interaction deviation caused by the display position is the position deviation. The position deviation makes it easy for items with position advantages to obtain a relatively high posterior click-through rate. Position deviation is more common in search scenarios and has a greater impact. This impact is reflected in the inconsistency of sample confidences at different positions during model training. Furthermore, the inventors found that the prediction deviation problem of the recommendation model when processing the first material has become one of the technical problems that need to be solved urgently in the prior art. Summary of the invention

[0004] The embodiment of the present invention provides a method, device, equipment and medium for model training, behavior scoring and content mixing to solve the problem of prediction deviation of the recommendation model when processing the first material.

[0005] To achieve the above objectives, in a first aspect, an embodiment of the present invention provides a model training method, comprising:

[0006] Collecting a training data set online, each piece of training data in the training data set includes attribute information, display location, refresh method, and at least one user behavior record of the user on the material corresponding to the material pushed to the user;

[0007] Determining a real frequency score of at least one user behavior of the user with respect to the material according to at least one user behavior record corresponding to the material in the training data;

[0008] The attribute information corresponding to the material in the training data is converted into a first feature embedding vector through a first feature extraction model;

[0009] Inputting the first feature embedding vector into a first non-first item rating model for prediction, and obtaining a non-first prediction frequency score of at least one user behavior of the user on the item;

[0010] Obtaining a first feature concatenation embedding vector according to the display position and refresh mode corresponding to the material in the training data and the first feature embedding vector, inputting the first feature concatenation embedding vector into a first material rating model for prediction, and obtaining a first predicted frequency score of at least one user behavior of the user on the material;

[0011] Determine whether the material is the first material according to the display position and refresh mode corresponding to the material in the training data and the preset first material determination rule;

[0012] Determine a loss function value according to a result of determining whether the material is the first material, a real frequency score of at least one user behavior of the material, a predicted frequency score of a non-first material, and a predicted frequency score of a first material, and adjust model parameters of the first non-first material scoring model, the first first material scoring model, and the first feature extraction model according to the loss function value;

[0013] Until the preset convergence condition is met, the first feature extraction model, the first non-first material scoring model and the first first material scoring model after training are obtained to construct a user behavior scoring prediction model for material sorting.

[0014] In a second aspect, an embodiment of the present invention provides a method for predicting a user behavior score, comprising:

[0015] Obtain the refresh mode corresponding to the target user and the attribute information corresponding to the material to be pushed under the refresh mode;

[0016] The attribute information corresponding to the material to be pushed is converted into a second feature embedding vector through a second feature extraction model;

[0017] Inputting the second feature embedding vector into a second non-first item rating model for prediction, to obtain a non-first predicted frequency score of at least one user behavior of the target user on the item to be pushed;

[0018] According to the refresh mode and the second feature embedding vector, a second feature splicing embedding vector is obtained, and the second feature splicing embedding vector is input into a second first item rating model for prediction to obtain a first predicted frequency score of at least one user behavior of the target user on the item to be pushed;

[0019] Among them, the non-first predicted frequency score of at least one user behavior of the material to be pushed is used to determine the non-first interest score of the target user in the material to be pushed when the material to be pushed is a non-first material; the first predicted frequency score of at least one user behavior of the material to be pushed is used to determine the first interest score of the target user in the material to be pushed when the material to be pushed is a first material, and the non-first interest score and the first interest score of the material to be pushed are used for material sorting; wherein, the model parameters of the second feature extraction model, the second non-first material scoring model and the second first material scoring model are updated through the model parameters of the first feature extraction model, the first non-first material scoring model and the first first material scoring model with the same model structure; the first feature extraction model, the first non-first material scoring model and the first first material scoring model are trained using the training method of the model described in any of the preceding ones.

[0020] In a third aspect, an embodiment of the present invention provides a user content shuffling method, including:

[0021] In response to a recommendation request of a target user, obtaining a refresh mode corresponding to the target user and attribute information corresponding to each of a plurality of materials to be pushed under the refresh mode;

[0022] For each material to be pushed, the refresh mode corresponding to the target user and the attribute information corresponding to the material to be pushed are input into the user behavior score prediction model to obtain the output non-first prediction frequency score and the first prediction frequency score of at least one user behavior corresponding to the material to be pushed; wherein the user behavior score prediction model includes a second feature extraction model, a second non-first material scoring model and a second first material scoring model, the model parameters of the second feature extraction model, the second non-first material scoring model and the second first material scoring model are updated by the model parameters of the first feature extraction model, the first non-first material scoring model and the first first material scoring model with the same model structure, and the first feature extraction model, the first non-first material scoring model and the first first material scoring model are trained using the training method of the model described in any of the preceding ones;

[0023] For each item to be pushed, determine the non-first interest score and the first interest score of the target user for the item to be pushed according to the non-first predicted frequency score and the first predicted frequency score of at least one user behavior corresponding to the item to be pushed;

[0024] Selecting, from the multiple materials to be pushed, a first material to be pushed with the highest interest score as the first material to be pushed to the target user;

[0025] From other materials to be pushed except the first material, a preset number of materials to be pushed are selected as non-first materials according to the non-first interest points;

[0026] The first material and the non-first materials sorted from high to low according to the non-first interest scores constitute mixed content and are pushed to the target user.

[0027] In a fourth aspect, an embodiment of the present invention provides a model training device, comprising:

[0028] A first data collection unit is used to collect a training data set online, wherein each piece of training data in the training data set includes attribute information, display location, refresh mode, and at least one user behavior record of the user on the material corresponding to the material pushed to the user;

[0029] a real score determination unit, configured to determine a real frequency score of at least one user behavior of the user with respect to the material according to at least one user behavior record corresponding to the material in the training data;

[0030] A first feature embedding vector determining unit, configured to convert the attribute information corresponding to the material in the training data into a first feature embedding vector through a first feature extraction model;

[0031] A first non-first prediction frequency score determination unit, configured to input the first feature embedding vector into a first non-first material scoring model for prediction, and obtain a non-first prediction frequency score of at least one user behavior of the user on the material;

[0032] A first first-item predicted frequency score determination unit is configured to obtain a first feature concatenation embedding vector according to a display position and a refresh mode corresponding to the material in the training data and the first feature embedding vector, and input the first feature concatenation embedding vector into a first first-item scoring model for prediction to obtain a first predicted frequency score of at least one user behavior of the user on the material;

[0033] A first item determination unit, configured to determine whether the item is the first item according to the display position and refresh mode corresponding to the item in the training data and a preset first item determination rule;

[0034] a training model parameter updating unit, configured to determine a loss function value according to a result of determining whether the material is the first material, a real frequency score of at least one user behavior of the material, a predicted frequency score of a non-first material, and a predicted frequency score of a first material, and to adjust model parameters of the first non-first material scoring model, the first first material scoring model, and the first feature extraction model according to the loss function value;

[0035] A convergence judgment unit is used to obtain the trained first feature extraction model, the first non-first material scoring model and the first first material scoring model until a preset convergence condition is met, so as to construct a user behavior scoring prediction model for material sorting.

[0036] In a fifth aspect, an embodiment of the present invention provides a user behavior score prediction device, comprising:

[0037] The second data collection unit is used to obtain the refresh mode corresponding to the target user and the attribute information corresponding to the material to be pushed under the refresh mode;

[0038] A second feature embedding vector determining unit, configured to convert the attribute information corresponding to the material to be pushed into a second feature embedding vector through a second feature extraction model;

[0039] A second non-first prediction frequency score determination unit, configured to input the second feature embedding vector into a second non-first material scoring model for prediction, to obtain a non-first prediction frequency score of at least one user behavior of the target user on the material to be pushed;

[0040] A second first-item prediction frequency score determination unit is configured to obtain a second feature splicing embedding vector according to the refresh mode and the second feature embedding vector, input the second feature splicing embedding vector into a second first-item material scoring model for prediction, and obtain a first-item prediction frequency score of at least one user behavior of the target user on the material to be pushed;

[0041] Among them, the non-first predicted frequency score of at least one user behavior of the material to be pushed is used to determine the non-first interest score of the target user in the material to be pushed when the material to be pushed is a non-first material; the first predicted frequency score of at least one user behavior of the material to be pushed is used to determine the first interest score of the target user in the material to be pushed when the material to be pushed is a first material, and the non-first interest score and the first interest score of the material to be pushed are sorted by materials; wherein, the model parameters of the second feature extraction model, the second non-first material scoring model and the second first material scoring model are updated through the model parameters of the first feature extraction model, the first non-first material scoring model and the first first material scoring model with the same model structure; the first feature extraction model, the first non-first material scoring model and the first first material scoring model are trained using the training method of the model described in any of the preceding ones.

[0042] In a sixth aspect, an embodiment of the present invention provides a user content shuffling device, including:

[0043] The third data collection unit is used to respond to the recommendation request of the target user and obtain the refresh mode corresponding to the target user and the attribute information corresponding to each of the multiple materials to be pushed under the refresh mode;

[0044] A frequency score determination unit is used to input the refresh mode corresponding to the target user and the attribute information corresponding to the material to be pushed into a user behavior score prediction model for each material to be pushed, and obtain the output non-first predicted frequency score and first predicted frequency score of at least one user behavior corresponding to the material to be pushed; wherein the user behavior score prediction model includes a second feature extraction model, a second non-first material scoring model and a second first material scoring model, and the model parameters of the second feature extraction model, the second non-first material scoring model and the second first material scoring model are updated by the model parameters of the first feature extraction model, the first non-first material scoring model and the first first material scoring model with the same model structure, and the first feature extraction model, the first non-first material scoring model and the first first material scoring model are trained using the training method of the model described in any of the above;

[0045] an interest score determination unit, for determining, for each item to be pushed, a non-first item interest score and a first item interest score of the target user for the item to be pushed according to a non-first item predicted frequency score and a first item predicted frequency score of at least one user behavior corresponding to the item to be pushed;

[0046] a first item determination unit, configured to select a first item to be pushed with the highest interest score from the plurality of items to be pushed as the first item to be pushed to the target user;

[0047] A non-first item sorting unit is used to select a preset number of items to be pushed as non-first items according to the non-first item interest from other items to be pushed except the first item.

[0048] The content mixing unit is used to push the mixed content consisting of the first material and the non-first materials sorted from high to low according to the non-first interest scores to the target user.

[0049] In the seventh aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, it implements any one of the model training methods described above, or a user behavior rating prediction method, or a user content mixing method.

[0050] In an eighth aspect, an embodiment of the present invention provides an electronic device, comprising: a processor and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement any one of the model training methods described above or a user behavior rating prediction method or a user content mixing method.

[0051] The above technical solution has the following beneficial effects: by distinguishing between the first material and the non-first material, the scoring models for the first material and the non-first material are trained separately, so that the prediction of the first material and the prediction of the non-first material do not need to be concentrated in one model for generalization and compatibility, and the scoring prediction accuracy of the first material and the non-first material can be improved at the same time. Moreover, for the first material, the scoring model of the first material is trained in combination with the refresh method and display position of the material, and the first material is refined to model the score, which can more accurately predict the score of the first material. This avoids the problem of model prediction deviation caused by the different refresh methods and display positions of the first material and the non-first material when the score prediction is made for the first material and the non-first material based on the same model, as well as the large difference in the amount of available training data between the first material and the non-first material. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1is a flow chart of a model training method according to one embodiment of the present invention;

[0054] Figure 2 is a flow chart of a method for predicting user behavior scores according to one embodiment of the present invention;

[0055] Figure 3 is a flow chart of a user content shuffling method according to one embodiment of the present invention;

[0056] Figure 4 It is an architectural diagram of a training device for a model according to one embodiment of the present invention;

[0057] Figure 5 This is an architecture diagram of a user behavior score prediction device according to one embodiment of the present invention;

[0058] Figure 6 This is an architectural diagram of a user content shuffling device according to one embodiment of the present invention;

[0059] Figure 7 It is a structural diagram of a user behavior score prediction model according to one embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] The inventors have discovered that in recommendation business scenarios, the attention paid to displayed materials will be affected by the display position. For example, in list-based recommendations, materials displayed at the front are more likely to attract the user's attention than materials displayed at the back, and are also more likely to obtain user clicks and other behavioral interactions. This user interaction deviation caused by the display position is position deviation. Position deviation makes it easy for items with position advantages to obtain a relatively high posterior click-through rate. Position deviation is more common in search scenarios and has a greater impact. This impact is reflected in the inconsistency of sample confidences at different positions in model training. In the recommendation scenario, the first material refreshed each time is definitely browsed by the user, while other materials may not be browsed; at the same time, the display method of the first material corresponding to different refresh methods of users in the recommendation system is also different. The display of the first material generated when the user first enters the app and pulls down to refresh is different from the display of the first material generated by the user's natural browsing refresh. If we simply add position features to each material in a prediction model, and process the first material and non-first material based on the same model, these characteristics of the first material will be ignored or weakened due to the generalization of the model on the training data, resulting in more significant deviations in the prediction model's prediction for the first material.

[0062] First, as Figure 1 As shown, an embodiment of the present invention provides a model training method, characterized by comprising:

[0063] Step S10, collecting a training data set online, wherein each piece of training data in the training data set includes attribute information, display location, refresh method, and at least one user behavior record of the user on the material corresponding to the material pushed to the user;

[0064] Step S11, determining a real frequency score of at least one user behavior of the user with respect to the material according to at least one user behavior record corresponding to the material in the training data;

[0065] Step S12, converting the attribute information corresponding to the material in the training data into a first feature embedding vector through a first feature extraction model;

[0066] Step S13, inputting the first feature embedding vector into a first non-first item rating model for prediction, and obtaining a non-first prediction frequency score of at least one user behavior of the user on the item;

[0067] Step S14, obtaining a first feature concatenation embedding vector according to the display position and refresh mode corresponding to the material in the training data and the first feature embedding vector, inputting the first feature concatenation embedding vector into the first first material rating model for prediction, and obtaining a first predicted frequency score of at least one user behavior of the user on the material;

[0068] Step S15, judging whether the material is the first material according to the display position and refresh mode corresponding to the material in the training data and the preset first material judgment rule;

[0069] Step S16, determining a loss function value according to a result of determining whether the material is the first material, a real frequency score of at least one user behavior of the material, a predicted frequency score of a non-first material, and a predicted frequency score of a first material, and adjusting model parameters of the first non-first material scoring model, the first first material scoring model, and the first feature extraction model according to the loss function value;

[0070] Step S17, until the preset convergence condition is met, the first feature extraction model, the first non-first item scoring model and the first first item scoring model after training are obtained to construct a user behavior scoring prediction model for item sorting.

[0071] In some embodiments, the attribute information, display location, and refresh method corresponding to each material pushed to each user each time are obtained online, and the obtained attribute information, display location, refresh method, and other information corresponding to the material, and at least one user behavior record of the user on the material are recorded. The obtained attribute information, display location, refresh method, and other information corresponding to the material, and at least one user behavior record of the user on the material are used as training data to train the model parameters of the first feature extraction model, the first non-first material scoring model, and the first first material scoring model online. When obtaining the above training data, the acquisition can be started after a preset time after the material is pushed to the user. Because it is obtained online in real time, when the material is just pushed to the user, the user has no time to operate, there is no related user behavior, and it is necessary to wait for a preset time. The preset time represents the time required for the vast majority of users to complete the related user behavior operations on the material after the material is pushed according to statistics from previous logs. The refresh method includes but is not limited to a method of displaying materials from the first display position, and a display method of adding other materials after the displayed materials, wherein the method of displaying materials from the first display position includes but is not limited to the first refresh and pull-down refresh; the first refresh means the first refresh after the user client application is started, and the pull-down refresh means that the user updates the material display page through the pull-down operation; the display method of adding other materials after the displayed materials includes but is not limited to natural refresh, etc.; natural refresh means that the user slides the display page, triggering the display page to continue to add new display content. User behavior includes but is not limited to clicks, views, interactions, etc. By counting each user behavior such as the number of clicks, viewing time, interaction frequency, etc., the real frequency score of the relevant user behavior can be obtained. The first feature extraction model is used to extract features from the attribute information of the material, thereby converting the attribute information into a first feature embedding vector. Specifically, the attribute information corresponding to the material is vectorized into a first attribute embedding vector, and the first attribute embedding vector is input into the first feature extraction model for feature extraction to obtain a first feature embedding vector. The first feature extraction model can be a neural network model. The first feature embedding vector is input into the first non-first item rating model to obtain the first predicted frequency score corresponding to each user behavior. The first non-first item rating model can be a neural network model. The first feature embedding vector is combined with the display position and refresh mode corresponding to the item to obtain a first feature splicing embedding vector. Preferably, the display position and refresh mode can be vectorized respectively to obtain a display position embedding vector and a refresh mode embedding vector, and then the first feature splicing embedding vector, the display position embedding vector and the refresh mode embedding vector are spliced ​​to obtain the first feature splicing embedding vector.The first feature concatenation embedding vector is input into the first first item scoring model for prediction, and the first predicted frequency score of at least one user behavior of the user on the item is obtained; the first first item scoring model can simultaneously consider the display location, refresh mode and attribute information of the item, and perform refined modeling prediction to obtain a more accurate first predicted frequency score of the user behavior of the first item. The embodiment of the present invention trains the first item and the non-first item separately. During training, it is necessary to determine the loss function value corresponding to the preset loss function according to whether the item is the first item, the real frequency score, the predicted frequency score of the non-first item and the predicted frequency score of the first item, and the preset loss function, and update the model parameters of the first non-first item scoring model, the first first item scoring model and the first feature extraction model according to the gradient descent method. For example, when the current item is the first item, the model parameters of the first first item scoring model and the first feature extraction model need to be updated. When the current item is not the first item, the model parameters of the first non-first item scoring model and the first feature extraction model need to be updated. Whether the convergence condition is met can be judged based on neural network model convergence indicators including but not limited to AUC (Area Under Curve, the area under the ROC curve and the coordinate axis) and MSE (Mean Square Error) indicators, where ROC stands for receiving characteristic curve, and its corresponding Chinese name is receiver operating characteristic curve. Those skilled in the art are aware of the specific usage of relevant convergence indicators for model training.

[0072] The embodiments of the present invention have the following technical effects: by distinguishing between the first material and the non-first material, the scoring models of the first material and the non-first material are trained separately, so that the prediction of the first material and the prediction of the non-first material do not need to be concentrated in one model for generalization and compatibility, and the scoring prediction accuracy of the first material and the non-first material can be improved at the same time. Moreover, for the first material, the scoring model of the first material is trained in combination with the refresh mode and display position of the material, and the first material is refined to model the score, which can more accurately predict the score of the first material. This avoids the problem of model prediction deviation caused by the different refresh modes and display positions of the first material and the non-first material when the score prediction is made for the first material and the non-first material based on the same model, as well as the large difference in the amount of available training data between the first material and the non-first material.

[0073] Further, the determining of a loss function value according to a result of determining whether the material is the first material, a real frequency score of at least one user behavior of the material, a predicted frequency score of a non-first material, and a predicted frequency score of a first material, and adjusting model parameters of the first non-first material scoring model, the first first material scoring model, and the first feature extraction model according to the loss function value includes:

[0074] In the case where it is determined that the material is a non-first material, a loss value of a loss function corresponding to the first non-first material scoring model is determined according to a real frequency score of at least one user behavior of the material and a predicted frequency score of the non-first material, and model parameters of the first non-first material scoring model and the first feature extraction model are adjusted according to the loss value of the loss function corresponding to the first non-first material scoring model; the loss function corresponding to the first non-first material scoring model represents the gap between the predicted frequency score of the non-first material of at least one user behavior of the material and the real frequency score;

[0075] When it is determined that the material is the first material, a loss value of a loss function corresponding to the first material scoring model is determined according to a real frequency score of at least one user behavior of the material and a predicted frequency score of the first material, and model parameters of the first first material scoring model and the first feature extraction model are adjusted according to the loss value of the loss function corresponding to the first first material scoring model; the loss function corresponding to the first first material scoring model represents the gap between the predicted frequency score of the first material of at least one user behavior of the material and the real frequency score.

[0076] In some embodiments, during training (and prediction), the first feature extraction model, the first non-first item scoring model, and the first first item scoring model do not judge whether the item is the first item. Each item in the training data set will pass through the first feature extraction model, the first non-first item scoring model, and the first first item scoring model, and obtain the first item predicted frequency score and the non-first item predicted frequency score. However, when updating the model parameters by gradient decreasing according to the loss function, it will be judged whether the current item is the first item. For non-first items, the loss function corresponding to the first non-first item scoring model is used to calculate the loss function value determined by the real frequency score and the predicted frequency score of the non-first item, and the model parameters of the first non-first item scoring model and the first feature extraction model are updated based on the gradient descent method; for the first item, the loss function corresponding to the first first item scoring model is used to calculate the loss function value determined by the real frequency score and the predicted frequency score of the first item, and the model parameters of the first first item scoring model and the first feature extraction model are updated based on the gradient descent method.

[0077] The embodiment of the present invention has the following technical effects: by respectively updating the first first material scoring model and the first non-first material scoring model through the first material and the non-first material, the first material prediction and the non-first material prediction do not need to be concentrated in one model for generalization and compatibility, and the scoring prediction accuracy of the first material and the non-first material can be improved at the same time.

[0078] Furthermore, the attribute information corresponding to the material includes a plurality of key-value pairs consisting of attribute types and attribute values;

[0079] The method of converting the attribute information corresponding to the material in the training data into a first feature embedding vector by using a first feature extraction model includes: for each group of key-value pairs in the attribute information corresponding to the material, determining the attribute type serial number corresponding to the attribute type in the key-value pair according to a preset attribute type and code mapping relationship table, performing hash calculation on the attribute value in the key-value pair to obtain the hash value corresponding to the attribute value, and concatenating the attribute type serial number corresponding to the attribute type with the hash value corresponding to the attribute value to obtain the concatenated code corresponding to the key-value pair;

[0080] Using the concatenated code corresponding to the key-value pair as an index number, looking up a table in a preset attribute information vector matrix to obtain an embedded vector corresponding to the key-value pair;

[0081] Concatenate the embedded vectors corresponding to all key-value pairs in the attribute information corresponding to the material to obtain a first attribute embedded vector corresponding to the material;

[0082] The first attribute embedding vector is input into a first feature extraction model for feature extraction to obtain a first feature embedding vector. In which, all attribute types that may appear are encoded in advance to obtain a table of attribute type and encoding mapping relationships;

[0083] The concatenated code can be expressed as a natural number, such as an unsigned char type, a uint type, a ulong type, etc. in a computer system.

[0084] In some embodiments, a piece of training data includes material attribute information, user behavior data on the material (at least one user behavior record), material display location (information) and refresh method (information). Material attribute information includes but is not limited to material attributes, user attributes corresponding to the material, user and material matching attributes, etc. The (display) position information of the material refers to the sorting position of the material in the refresh, and the refresh (mode) information refers to the user's refresh mode information this time; when processing data, firstly, the user's own attributes, blog post (material) own attributes, and user and material matching attributes are uniformly hashed and mapped, and a sufficiently large hash space is opened to avoid conflicts. For each attribute value under each attribute type, a unique code is used to correspond; an attribute type may take multiple attribute values, for example, the user's gender is an attribute type, and the attribute value of gender can be male or female. The attribute type is encoded with the attribute type serial number, and each attribute type has a unique code, for example, a 10-bit binary number is used to represent the attribute type serial number (the value range is 0-1024), and the attribute value is hashed into a hash value of a specified number of hash bits, for example, the attribute value is hashed and mapped to a 54-bit binary code; for each pair of attribute type-attribute value pairs, the attribute type serial number and the attribute value encoding are concatenated to obtain a concatenated code, for example, a 64-bit code is obtained, which is used to uniquely represent a combination of "attribute type-attribute value". After the above-mentioned unique hash coding, the 64-bit binary code (concatenated code) is converted into a decimal index number, and then looked up through a large-scale embedding matrix (preset attribute information vector matrix); the elements in the attribute information vector matrix are pre-encoded for all possible attribute type-attribute value pairs according to the above-mentioned method to obtain the corresponding concatenated codes, and the low-dimensional dense vector obtained by vectorizing the concatenated code is used as the embedded vector corresponding to the attribute type-attribute value pair, which effectively reduces the time complexity and space complexity of the model. The vector dimension in the embodiment of the present invention is 16.

[0085] Further, the first feature extraction model includes three independent multi-layer fully connected neural networks;

[0086] Inputting the first attribute embedding vector into a first feature extraction model for feature extraction to obtain a first feature embedding vector, including: inputting the first attribute embedding vector into the three independent multi-layer fully connected neural networks respectively to perform feature extraction in different aspects to obtain three feature extraction vectors, and splicing the three feature extraction vectors to obtain the first feature embedding vector.

[0087] In some embodiments, three independent multi-layer fully connected neural networks are three shared neural networks, which are used to deeply extract the characteristics of users and materials, and splice the corresponding embedded vectors of all attribute types and attribute value pairs of the material to be processed to obtain the embedded splicing vector (first attribute embedding vector) corresponding to the material to be processed; the embedded splicing vector (first attribute embedding vector) corresponding to the material to be processed is respectively input into three shared neural networks (three independent multi-layer fully connected neural networks) to obtain three shared neural network output vectors respectively, and the three shared neural network output vectors are spliced ​​to obtain the material feature vector (first feature embedding vector) corresponding to the material to be processed. Specifically, the user attributes and material attributes of the sample (material) and the user and material matching attribute features are spliced ​​into a vector Vec features =concat(Vec user ,Vec item ,Vec match ), input this vector into three shared neural networks respectively to obtain three shared neural network output vectors (three feature extraction vectors), and concatenate the three feature extraction vectors to obtain the first feature embedding vector, wherein the internal structure of each shared network is a three-layer fully connected neural network. The main function of the three independent fully connected neural networks in parallel is to increase the complexity and diversity of the model, thereby improving the performance and generalization ability of the model. A fully connected neural network is a basic neural network structure in which each node of each layer is connected to all nodes of the previous layer. When three independent fully connected neural networks are connected in parallel, each network can process the input information independently and integrate this information in subsequent layers. This parallel structure can be regarded as increasing the width of the model, allowing the model to consider more features and process more input variables at the same time. Specifically, the functions of this parallel structure include: increasing the complexity of the model: by connecting more fully connected layers in parallel, the model can learn more complex nonlinear relationships, thereby improving the ability to fit the data. Improving generalization ability: multiple independent networks can learn different features of the input data, which can be comprehensively utilized in subsequent layers, thereby improving the model's prediction accuracy for unseen data. Enhance the robustness of the model: Each independent network can be regarded as a sub-model, which can complement each other, reduce the model's dependence on specific training data, and enhance the robustness of the model.

[0088] Furthermore, the first material rating model includes: a first independent tower model corresponding to each user behavior;

[0089] Inputting the first feature concatenated embedding vector into the first first material rating model for prediction to obtain the first predicted frequency score of at least one user behavior of the user on the material, including:

[0090] The first feature concatenated embedding vector is respectively input into the first independent tower model corresponding to each user behavior to obtain the first predicted frequency score of the corresponding user behavior output by each first independent tower model, wherein the first predicted frequency score represents the frequency of the user generating the corresponding user behavior on the material when the material is taken as the first material.

[0091] Furthermore, the first material scoring model includes: a first independent tower model corresponding to each user behavior; the first independent tower model includes: a first independent tower pre-neural network and a first independent tower post-neural network;

[0092] Inputting the first feature concatenated embedding vector into the first first material rating model for prediction to obtain the first predicted frequency score of at least one user behavior of the user on the material, including:

[0093] The first feature concatenation embedding vector is respectively input into the first independent tower front neural network corresponding to each user behavior to obtain the neural network intermediate latent variable corresponding to the corresponding user behavior output by each first independent tower front neural network; the neural network intermediate latent variable corresponding to the corresponding user behavior output by each first independent tower front neural network is input into the corresponding first independent tower back neural network to obtain the first predicted frequency score of the corresponding user behavior output by each first independent tower back neural network. The first independent tower front neural network includes three layers of fully connected neural networks; the first independent tower back neural network includes three layers of fully connected neural networks.

[0094] In some embodiments, each user behavior corresponds to this first independent tower model, and each first independent tower model includes a first independent tower front neural network and a first independent tower rear neural network; during training, a corresponding loss function is configured for the first independent tower model corresponding to each user behavior, and the model parameters of the first independent tower model corresponding to the user behavior, including the first independent tower front neural network and the first independent tower rear neural network, are updated through the loss function and the gradient descent method. The loss functions configured for the first independent tower models corresponding to different user behaviors may be the same or different. According to the values ​​of the loss functions configured for all the first independent tower models, the model parameters of the first feature extraction model will also be updated by applying the gradient descent method. For example, Figure 7As shown, in the main network, the output a2 of the first independent tower post-position (neural) network 4 corresponding to the click rate (click belongs to user behavior) is the predicted value of the click rate of the first material (the first predicted frequency score), the output b2 of the first independent tower post-position (neural) network 5 corresponding to the interaction rate (interaction belongs to user behavior) is the predicted value of the interaction rate of the first material (the first predicted frequency score), and the output c2 of the first independent tower post-position (neural) network 6 corresponding to the consumption time (consumption includes but is not limited to watching, which belongs to user behavior) is the predicted value of the consumption time of the first material (the first predicted frequency score), wherein a2 and b2 are trained by the cross entropy loss function, and the cross entropy loss function is expressed as shown in formula (1):

[0095]

[0096] c2 is trained by using the MSE loss function. The MSE loss function is expressed as shown in formula (2):

[0097]

[0098] in, is the true value of the user behavior, and y is the predicted value of the user behavior.

[0099] Furthermore, the first non-first material scoring model includes: a second independent tower model corresponding to each user behavior;

[0100] Inputting the first feature embedding vector into a first non-first item rating model for prediction to obtain a non-first prediction frequency score of at least one user behavior of the user on the item, including:

[0101] The first feature embedding vector is respectively input into the second independent tower model corresponding to each user behavior to obtain the non-first prediction frequency score of the corresponding user behavior output by each second independent tower model, where the non-first prediction frequency score represents the frequency of the user generating the corresponding user behavior on the material under the condition that the material is the non-first material.

[0102] Furthermore, the first non-first material scoring model includes: a second independent tower model corresponding to each user behavior; the second independent tower model includes: a second independent tower pre-neural network and a second independent tower post-neural network;

[0103] Inputting the first feature embedding vector into a first non-first item rating model for prediction to obtain a non-first prediction frequency score of at least one user behavior of the user on the item, including:

[0104] The first feature embedding vector is respectively input into the second independent tower front neural network corresponding to each user behavior to obtain the neural network intermediate latent variable corresponding to the corresponding user behavior output by each second independent tower front neural network; the neural network intermediate latent variable corresponding to the corresponding user behavior output by each second independent tower front neural network is input into the corresponding second independent tower back neural network to obtain the non-first prediction frequency score of the corresponding user behavior output by each second independent tower back neural network. The second independent tower front neural network includes three layers of fully connected neural networks; the second independent tower back neural network includes three layers of fully connected neural networks.

[0105] In some embodiments, each user behavior corresponds to this second independent tower model, and each second independent tower model includes a second independent tower front neural network and a second independent tower rear neural network; during training, a corresponding loss function is configured for each second independent tower model corresponding to each user behavior, and the model parameters of the second independent tower model corresponding to the user behavior, including the second independent tower front neural network and the second independent tower rear neural network, are updated through the loss function and the gradient descent method. The loss functions configured for the second independent tower models corresponding to different user behaviors may be the same or different. The values ​​of the loss functions configured for all the second independent tower models will also update the model parameters of the first feature extraction model through the gradient descent method. For example, Figure 7 As shown, in the main network, the output a1 of the second independent tower post-position (neural) network 1 corresponding to the click rate (belonging to user behavior) is the predicted value of the click rate of non-first materials (non-first predicted frequency score), the output b1 of the second independent tower post-position (neural) network 2 corresponding to the interaction rate (belonging to user behavior) is the predicted value of the interaction rate of non-first materials (non-first predicted frequency score), and the output c1 of the second independent tower post-position (neural) network 3 corresponding to the consumption time (belonging to user behavior) is the predicted value of the consumption time of non-first materials (non-first predicted frequency score), wherein a1 and b1 are trained by the cross entropy loss function, and the cross entropy loss function is expressed as shown in formula (1). c1 is trained by using the MSE loss function. The MSE loss function is expressed as shown in formula (2).

[0106] Furthermore, the first material judgment rule includes: judging the material whose refresh mode is to display the material from the first position and whose display position is the first position as the first material; otherwise, judging it as a non-first material.

[0107] Refresh methods include but are not limited to a method of displaying materials from the first display position, and a method of adding materials to be displayed after the materials that have been displayed (not a method of displaying materials from the first display position), wherein the method of displaying materials from the first display position includes but is not limited to first refresh and pull-down refresh; first refresh refers to the first refresh after the user client application is started, and pull-down refresh means that the user updates the material display page from the first display position of the display page by pulling down; display methods of adding materials to be displayed after the materials that have been displayed include but are not limited to natural refresh, etc.; natural refresh means that the user slides the display page, triggering the display page to continue to add new content (materials) to be displayed.

[0108] Second, as Figure 2 As shown, an embodiment of the present invention provides a user behavior score prediction method, comprising:

[0109] Step S20, obtaining a refresh mode corresponding to the target user and attribute information corresponding to the material to be pushed under the refresh mode;

[0110] Step S21, converting the attribute information corresponding to the material to be pushed into a second feature embedding vector through a second feature extraction model;

[0111] Step S22, inputting the second feature embedding vector into a second non-first item rating model for prediction, and obtaining a non-first predicted frequency score of at least one user behavior of the target user on the item to be pushed;

[0112] Step S23, obtaining a second feature splicing embedding vector according to the refresh mode and the second feature embedding vector, inputting the second feature splicing embedding vector into a second first item rating model for prediction, and obtaining a first predicted frequency score of at least one user behavior of the target user on the item to be pushed;

[0113] Among them, the non-first predicted frequency score of at least one user behavior of the material to be pushed is used to determine the non-first interest score of the target user in the material to be pushed when the material to be pushed is a non-first material; the first predicted frequency score of at least one user behavior of the material to be pushed is used to determine the first interest score of the target user in the material to be pushed when the material to be pushed is a first material, and the non-first interest score and the first interest score of the material to be pushed are used for material sorting; wherein, the model parameters of the second feature extraction model, the second non-first material scoring model and the second first material scoring model are updated through the model parameters of the first feature extraction model, the first non-first material scoring model and the first first material scoring model with the same model structure; the first feature extraction model, the first non-first material scoring model and the first first material scoring model are trained using the training method of the model described in any of the preceding ones.

[0114] In some embodiments, the inventors found that in the recommendation business scenario, the attention of the displayed materials will be affected by the display position. For example, in list-based recommendations, materials displayed at the front are more likely to attract the user's attention than materials displayed at the back, and are also more likely to obtain user clicks and other behavioral interactions. This user interaction deviation caused by the display position is the position deviation. Position deviation makes it easy for items with position advantages to obtain a relatively high posterior click-through rate. Position deviation is more common in search scenarios and has a greater impact. This impact is reflected in the inconsistency of sample confidences at different positions in model training. In the recommendation scenario, the first material refreshed each time is definitely browsed by the user, while other materials may not be browsed; at the same time, the display method of the first material corresponding to different refresh methods of users in the recommendation system is also different. The display of the first material generated by the user's first entry into the app and the pull-down refresh is different from the display of the first material generated by the user's natural browsing refresh. If a prediction model simply adds position features to each material and processes each material uniformly, these characteristics of the first material will be ignored, resulting in deviations in the prediction model. The embodiment of the present invention obtains attribute information, display position information and refresh mode information of the material to be processed, wherein the attribute information includes but is not limited to: user's own attributes, material's own attributes and user-material matching attributes, etc.; user's own attributes include but are not limited to information such as age, gender, occupation, etc., material's own attributes include but are not limited to blog post type, theme, author information, etc., user and material matching attributes include but are not limited to user's click rate on materials in a specific field, user's click rate on materials published by a specific blogger, etc.; material display position information refers to the sorting position of the material in the refresh, and the materials are divided into the first material and the non-first material through the two pieces of information of display position and refresh mode, wherein the material at position 1 in the first refresh and pull-down refresh entering the app is regarded as the first material, and the material at any position in the normal browsing refresh and the material except the first one in other refreshes are regarded as the non-first material. Distinguish between non-first items and first items, and use different prediction models for prediction respectively. The first item prediction model only predicts the first item, not the non-first item, and is trained only with samples of the first item. Therefore, the model parameters of the first item prediction model will not be affected by the training samples of the non-first items, and there is no need to generalize the features of the non-first items. The first item prediction model can better predict the first item, and uses the features corresponding to the refresh method in the prediction of the first item, avoiding model deviations caused by different refresh methods of the recommendation system for the first item. The embodiment of the present invention uses the model obtained by the aforementioned model training method to predict the score of user behavior, and the embodiment of the present invention can be understood based on the embodiment of the aforementioned model training method. Only some steps are additionally explained below.When a user requests a recommended material, the user triggers a recommendation request in a certain refresh mode. The backend server can obtain the refresh mode through the recommendation request and provide the refresh mode to the embodiment of the present invention. The material to be pushed can come from a material pool, which stores multiple materials that can be sent to each user. The material pool can be composed of materials that meet specific business requirements, such as the latest materials, materials with high click rates, etc. Through the second feature extraction model, the attribute information corresponding to the material to be pushed is converted into a second feature embedding vector. The specific processing process can refer to the corresponding description of the embodiment of the training method of the aforementioned model, in which the attribute information corresponding to the material in the training data is converted into the first feature embedding vector through the first feature extraction model, and will not be repeated here. The specific processing process of inputting the second feature embedding vector into the second non-first material scoring model for prediction, and obtaining the non-first predicted frequency score of at least one user behavior of the target user for the material to be pushed can refer to the embodiment of the training method of the aforementioned model, in which the first feature embedding vector is input into the first non-first material scoring model for prediction, and the non-first predicted frequency score of at least one user behavior of the user for the material is obtained. It will not be repeated here. According to the refresh mode corresponding to the material to be pushed and the second feature embedding vector, a second feature splicing embedding vector is obtained, and the second feature splicing embedding vector is input into the second first material scoring model for prediction, so as to obtain the first predicted frequency score of the target user for at least one user behavior of the material to be pushed. The specific processing process can refer to the embodiment of the training method of the aforementioned model, according to the display position and refresh mode corresponding to the material in the training data, and the first feature embedding vector, a first feature splicing embedding vector is obtained, and the first feature splicing embedding vector is input into the first first material scoring model for prediction, so as to obtain the first predicted frequency score of at least one user behavior of the user for the material. The complexity score is understood and will not be described in detail here. Only the processing of the display position is explained. In the embodiment of the present invention, the refresh method corresponding to the material to be pushed and the second feature embedding vector are used to obtain the second feature splicing embedding vector. The display position is not reflected here because in the business prediction stage, the display position of the current material to be processed has not yet been determined on the user side. The final display position is determined after the prediction according to the prediction result. Therefore, the display position here in the embodiment of the present invention, when determining the first feature splicing embedding vector, processes the display position as a fixed value representing no specific meaning, for example, vectorizes the display position to a vector of all 0s, so that the data input in the prediction stage meets the data format input during training.

[0115] Thirdly, Figure 3 As shown, an embodiment of the present invention provides a user content shuffling method, including:

[0116] Step S30, in response to the recommendation request of the target user, obtaining a refresh mode corresponding to the target user and attribute information corresponding to each of a plurality of materials to be pushed under the refresh mode;

[0117] Step S31, for each material to be pushed, input the refresh mode corresponding to the target user and the attribute information corresponding to the material to be pushed into the user behavior score prediction model, and obtain the output non-first prediction frequency score and first prediction frequency score of at least one user behavior corresponding to the material to be pushed; wherein the user behavior score prediction model includes a second feature extraction model, a second non-first material scoring model and a second first material scoring model, and the model parameters of the second feature extraction model, the second non-first material scoring model and the second first material scoring model are updated by the model parameters of the first feature extraction model, the first non-first material scoring model and the first first material scoring model with the same model structure, and the first feature extraction model, the first non-first material scoring model and the first first material scoring model are trained using the training method of the model described in any of the above;

[0118] Step S32, for each item to be pushed, according to the non-first predicted frequency score and the first predicted frequency score of at least one user behavior corresponding to the item to be pushed, determining the non-first interest score and the first interest score of the target user for the item to be pushed;

[0119] Step S33, selecting the first material to be pushed with the highest interest score from the multiple materials to be pushed as the first material to be pushed to the target user;

[0120] Step S34, selecting a preset number of materials to be pushed as non-first materials according to the interest points of the non-first materials from the other materials to be pushed except the first material;

[0121] Step S35, the first material and the non-first materials sorted from high to low according to the non-first interest scores are combined into mixed content and pushed to the target user.

[0122] In some embodiments, the target user triggers a recommendation request by refreshing in a certain refresh mode on the client, and the recommendation request is submitted to the embodiment of the present invention. The embodiment of the present invention obtains the refresh mode from the recommendation request, and can also obtain the material to be pushed and the attribute information corresponding to the material to be pushed from the pre-prepared material pool. Through the user behavior score prediction method as described above, the non-first predicted frequency score and the first predicted frequency score of at least one user behavior corresponding to the material to be pushed are obtained. Those skilled in the art can determine the non-first interest score of the material to be pushed according to the non-first predicted frequency score of at least one user behavior through a variety of methods, such as by weighted summing the non-first predicted frequency score of each user behavior, etc., to determine the non-first interest score of the material to be pushed, which is not specifically limited in the embodiment of the present invention. The first interest score of the material to be pushed can be determined according to the first predicted frequency score of at least one user behavior through a variety of methods, such as by weighted summing the first predicted frequency score of each user behavior, etc., to determine the first interest score of the material to be pushed, which is not specifically limited in the embodiment of the present invention. Each material to be pushed can get a corresponding non-first interest score and a first interest score. The material to be pushed with the highest first interest score is selected as the first material to be pushed to the target user. From all the remaining materials to be pushed after removing the first material, a preset number of materials to be pushed are selected as non-first materials in descending order of non-first interest scores. The non-first materials are sorted after the first material in descending order of interest scores to form mixed content pushed to the target user. The preset number can be all or part of the materials to be pushed other than the first material, and the specific value can be set as needed. The first material is the material displayed in the first display position.

[0123] Fourthly, Figure 4 As shown, an embodiment of the present invention provides a model training device, comprising:

[0124] A first data collection unit 400 is used to collect a training data set online, wherein each piece of training data in the training data set includes attribute information, display location, refresh mode, and at least one user behavior record of the user on the material corresponding to the material pushed to the user;

[0125] A real score determination unit 401, configured to determine a real frequency score of at least one user behavior of the user with respect to the material according to at least one user behavior record corresponding to the material in the training data;

[0126] A first feature embedding vector determining unit 402, configured to convert the attribute information corresponding to the material in the training data into a first feature embedding vector through a first feature extraction model;

[0127] A first non-first prediction frequency score determination unit 403, configured to input the first feature embedding vector into a first non-first material scoring model for prediction, and obtain a non-first prediction frequency score of at least one user behavior of the user on the material;

[0128] A first first predicted frequency score determination unit 404 is used to obtain a first feature splicing embedding vector according to the display position and refresh mode corresponding to the material in the training data and the first feature embedding vector, and input the first feature splicing embedding vector into the first first material scoring model for prediction to obtain a first predicted frequency score of at least one user behavior of the user on the material;

[0129] The first item determination unit 405 is used to determine whether the item is the first item according to the display position and refresh mode corresponding to the item in the training data and the preset first item determination rule;

[0130] A training model parameter updating unit 406 is used to determine a loss function value according to a judgment result of whether the material is the first material, a real frequency score of at least one user behavior of the material, a non-first material predicted frequency score, and a first material predicted frequency score, and adjust model parameters of the first non-first material scoring model, the first first material scoring model, and the first feature extraction model according to the loss function value;

[0131] The convergence judgment unit 407 is used to obtain the first feature extraction model, the first non-first material scoring model and the first first material scoring model after training until a preset convergence condition is met, so as to construct a user behavior scoring prediction model for material sorting.

[0132] Furthermore, the training model parameter updating unit 406 includes:

[0133] a non-first item model parameter updating module, for determining, when determining that the material is a non-first item, a loss value of a loss function corresponding to the first non-first item scoring model according to a real frequency score of at least one user behavior of the material and a predicted frequency score of the non-first item, and adjusting model parameters of the first non-first item scoring model and the first feature extraction model according to the loss value of the loss function corresponding to the first non-first item scoring model; the loss function corresponding to the first non-first item scoring model represents the gap between the predicted frequency score of the non-first item of at least one user behavior of the material and the real frequency score;

[0134] A first-item model parameter updating module is used to determine the loss value of the loss function corresponding to the first-item scoring model according to the real frequency score of at least one user behavior of the material and the first-item predicted frequency score when it is determined that the material is the first-item, and adjust the model parameters of the first-item scoring model and the first feature extraction model according to the loss value of the loss function corresponding to the first-item scoring model; the loss function corresponding to the first-item scoring model represents the gap between the first-item predicted frequency score of at least one user behavior of the material and the real frequency score.

[0135] Furthermore, the attribute information corresponding to the material includes a plurality of key-value pairs consisting of attribute types and attribute values;

[0136] The first feature embedding vector determining unit 402 includes:

[0137] A splicing coding module, for determining, for each group of key-value pairs in the attribute information corresponding to the material, an attribute type serial number corresponding to the attribute type in the key-value pair according to a preset attribute type and code mapping relationship table, performing hash calculation on the attribute value in the key-value pair to obtain a hash value corresponding to the attribute value, and splicing the attribute type serial number corresponding to the attribute type with the hash value corresponding to the attribute value to obtain a splicing code corresponding to the key-value pair;

[0138] An embedded vector determination module is used to use the concatenated code corresponding to the key-value pair as an index number, look up a table in a preset attribute information vector matrix, and obtain the embedded vector corresponding to the key-value pair;

[0139] An attribute embedding vector determination module, used to concatenate the embedded vectors corresponding to all key-value pairs in the attribute information corresponding to the material to obtain a first attribute embedding vector corresponding to the material;

[0140] The feature embedding vector determination module is used to input the first attribute embedding vector into a first feature extraction model for feature extraction to obtain a first feature embedding vector.

[0141] Among them, all possible attribute types are encoded in advance to obtain an attribute type and encoding mapping relationship table;

[0142] The concatenated code can be expressed as a natural number, such as an unsigned char type, a uint type, a ulong type, etc. in a programming language in a computer system, so that it can be conveniently used as an index value.

[0143] Further, the first feature extraction model includes three independent multi-layer fully connected neural networks;

[0144] The feature embedding vector determination module is specifically used to input the first attribute embedding vector into the three independent multi-layer fully connected neural networks respectively, perform feature extraction in different aspects, obtain three feature extraction vectors, and splice the three feature extraction vectors to obtain the first feature embedding vector.

[0145] Furthermore, the first material rating model includes: a first independent tower model corresponding to each user behavior;

[0146] The first first prediction frequency score determination unit 404 is specifically used to input the first feature splicing embedding vector into the first independent tower model corresponding to each user behavior, and obtain the first prediction frequency score of the corresponding user behavior output by each first independent tower model. The first prediction frequency score represents the frequency of the user's corresponding user behavior on the material under the condition that the material is the first material.

[0147] Furthermore, the first material scoring model includes: a first independent tower model corresponding to each user behavior; the first independent tower model includes: a first independent tower pre-neural network and a first independent tower post-neural network;

[0148] The first first prediction frequency score determination unit 404 is specifically used to:

[0149] The first feature concatenation embedding vector is respectively input into the first independent tower front neural network corresponding to each user behavior to obtain the neural network intermediate latent variable corresponding to the corresponding user behavior output by each first independent tower front neural network; the neural network intermediate latent variable corresponding to the corresponding user behavior output by each first independent tower front neural network is input into the corresponding first independent tower back neural network to obtain the first predicted frequency score of the corresponding user behavior output by each first independent tower back neural network. The first independent tower front neural network includes three layers of fully connected neural networks; the first independent tower back neural network includes three layers of fully connected neural networks.

[0150] Furthermore, the first non-first material scoring model includes: a second independent tower model corresponding to each user behavior;

[0151] The first non-first prediction frequency score determination unit 403 is specifically used to input the first feature embedding vector into the second independent tower model corresponding to each user behavior, and obtain the non-first prediction frequency score of the corresponding user behavior output by each second independent tower model, and the non-first prediction frequency score represents the frequency of the user generating the corresponding user behavior on the material under the condition that the material is the non-first material.

[0152] Furthermore, the first non-first material scoring model includes: a second independent tower model corresponding to each user behavior; the second independent tower model includes: a second independent tower pre-neural network and a second independent tower post-neural network;

[0153] The first non-first prediction frequency score determination unit 403 is specifically used to: input the first feature embedding vector into the second independent tower front neural network corresponding to each user behavior, and obtain the neural network intermediate latent variable corresponding to the corresponding user behavior output by each second independent tower front neural network; input the neural network intermediate latent variable corresponding to the corresponding user behavior output by each second independent tower front neural network into the corresponding second independent tower back neural network, and obtain the non-first prediction frequency score of the corresponding user behavior output by each second independent tower back neural network. The second independent tower front neural network includes a three-layer fully connected neural network; the second independent tower back neural network includes a three-layer fully connected neural network.

[0154] Furthermore, the first material judgment rule includes: judging the material whose refresh mode is to display the material from the first position and whose display position is the first position as the first material; otherwise, judging it as a non-first material.

[0155] The embodiments of the present invention are device embodiments that correspond one-to-one to the embodiments of the training method of the aforementioned model. The embodiments of the present invention can be understood based on the embodiments of the training method of the aforementioned model, and will not be described in detail here.

[0156] Fifthly, Figure 5 As shown, an embodiment of the present invention provides a user behavior score prediction device, comprising:

[0157] The second data collection unit 500 is used to obtain the refresh mode and the attribute information corresponding to the target user's to-be-pushed material;

[0158] A second feature embedding vector determining unit 501 is used to convert the attribute information corresponding to the material to be pushed into a second feature embedding vector through a second feature extraction model;

[0159] A second non-first prediction frequency score determination unit 502 is used to input the second feature embedding vector into a second non-first material scoring model for prediction, and obtain a non-first prediction frequency score of at least one user behavior of the target user on the material to be pushed;

[0160] A second first-item prediction frequency score determination unit 503 is configured to obtain a second feature concatenation embedding vector according to the refresh mode corresponding to the material to be pushed and the second feature embedding vector, input the second feature concatenation embedding vector into a second first-item scoring model for prediction, and obtain a first-item prediction frequency score of at least one user behavior of the target user on the material to be pushed;

[0161] A business model parameter updating unit 504 is used to regularly use the model parameters of the first feature extraction model, the first non-first item scoring model and the first first item scoring model obtained by the model training method according to claim 1 to update the model parameters of the second feature extraction model, the second non-first item scoring model and the second first item scoring model; the model structures of the first feature extraction model, the first non-first item scoring model and the first first item scoring model are respectively the same as the model structures of the second feature extraction model, the second non-first item scoring model and the second first item scoring model;

[0162] Among them, the non-first predicted frequency score of at least one user behavior of the material to be pushed is used to determine the non-first interest score of the target user in the material to be pushed when the material to be pushed is a non-first material; the first predicted frequency score of at least one user behavior of the material to be pushed is used to determine the first interest score of the target user in the material to be pushed when the material to be pushed is a first material, and the non-first interest score and the first interest score are used to sort among multiple materials to be pushed.

[0163] The embodiments of the present invention are device embodiments corresponding one-to-one to the aforementioned user behavior score prediction method embodiments. The embodiments of the present invention can be understood based on the aforementioned user behavior score prediction method embodiments, and will not be described in detail here.

[0164] Sixth, such as Figure 6 As shown, an embodiment of the present invention provides a user content shuffling device, comprising:

[0165] The third data collection unit 600 is used to obtain the refresh mode corresponding to the target user and the attribute information corresponding to each of the multiple materials to be pushed under the refresh mode in response to the recommendation request of the target user;

[0166] The frequency score determination unit 601 is used to input the refresh mode corresponding to the target user and the attribute information corresponding to the material to be pushed into the user behavior score prediction model for each material to be pushed, and obtain the output non-first predicted frequency score and first predicted frequency score of at least one user behavior corresponding to the material to be pushed; wherein the user behavior score prediction model includes a second feature extraction model, a second non-first material scoring model and a second first material scoring model, and the model parameters of the second feature extraction model, the second non-first material scoring model and the second first material scoring model are updated by the model parameters of the first feature extraction model, the first non-first material scoring model and the first first material scoring model with the same model structure, and the first feature extraction model, the first non-first material scoring model and the first first material scoring model are trained using the training method of the model described in any of the above;

[0167] The interest score determination unit 602 is used to determine, for each material to be pushed, the non-first interest score and the first interest score of the target user for the material to be pushed according to the non-first predicted frequency score and the first predicted frequency score of at least one user behavior corresponding to the material to be pushed;

[0168] A first material determining unit 603 is configured to select a first material to be pushed with the highest interest score from the multiple materials to be pushed as the first material to be pushed to the target user;

[0169] A non-first item sorting unit 604 is used to select a preset number of items to be pushed as non-first items according to the non-first item interest from other items to be pushed except the first item.

[0170] The content shuffling unit 605 is used to push the shuffled content consisting of the first material and the non-first materials sorted from high to low according to the non-first interest scores to the target user.

[0171] The embodiment of the present invention is a device embodiment corresponding one-to-one to the aforementioned user content shuffling method embodiment. The embodiment of the present invention can be understood based on the aforementioned user content shuffling method embodiment, and will not be described in detail herein.

[0172] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any one of the aforementioned methods for predicting user behavior ratings for material sorting.

[0173] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising: a processor and a memory for storing instructions executable by the processor;

[0174] Wherein, the processor is configured to execute the instructions to implement any one of the methods for predicting user behavior scores for material sorting as described above.

[0175] As for the device, readable storage medium and electronic device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0176] The above technical solution of the embodiment of the present invention is described in detail below in conjunction with specific application examples. For technical details not introduced during the implementation process, please refer to the relevant description in the previous text.

[0177] The embodiment of the present invention proposes a recommendation method and device, which is also a recommendation method and device based on the popularity correction of a multi-objective model. On the basis of a multi-objective deep neural network, an additional neural network is designed for the first item to perform refined modeling and capture its characteristics. For a recommendation flow system, a first item with a good recommendation effect can guide users to browse and consume more in the future, which is of great significance to the entire recommendation system.

[0178] Data acquisition and data preprocessing: The training data set is used to train the neural network model on the server and is the most important part of the preliminary work. A piece of data contains user attributes, material attributes, user and material matching attributes as features, and user behavior data on materials. It also includes the location information of the material and the corresponding refresh information. The location information of the material refers to the sorting position of the material in the refresh, and the refresh information refers to the user's refresh method information this time. We divide the training into the first material and the non-first material through these two pieces of information. Among them, the material with position 1 in the first refresh and pull-down refresh of entering the app is regarded as the first material. For materials at any position in the normal browsing refresh and other refreshes except the first material, they are regarded as non-first materials. User attributes include but are not limited to age, gender, occupation and other information, material attributes include but are not limited to blog type, theme, author information, etc., and user and material matching attributes include but are not limited to the user's click-through rate on materials in a specific field and the user's click-through rate on materials published by a specific blogger. When processing data, first, the user's own attribute features, blog post's own attributes, and user and material matching attribute features are uniformly hash-mapped, and a sufficiently large hash space is opened to avoid conflicts. For each eigenvalue under each feature, a unique code is used, and the feature number to which the eigenvalue belongs is represented by a 10-bit binary number (the value range is 0-2^10). Each eigenvalue under the feature will be hash-mapped to a 54-bit binary code. The two sets of codes are spliced ​​to obtain a 64-bit code, which is used to uniquely represent a "feature-eigenvalue" combination. After the above-mentioned unique hash code, the 64-bit binary code is converted into a decimal index number, and then a large-scale embedded matrix is ​​used to look up the table to map the features and their corresponding values ​​to a low-dimensional dense vector, which effectively reduces the time complexity and space complexity of the model. The vector dimension in this embodiment is 16. The user's behavior data refers to the behavior (click, interact, browse, etc.) that the user completes on the microblog after the recommendation system exposes a microblog to the user.

[0179] Multi-objective first model construction: The model designed in the embodiment of the present invention is a deep neural network model, which has three shared networks at the bottom and six independent tower networks at the top. The input vector Vec of the model features is the vector Vec corresponding to the user attributes of the sample user Vector Vec corresponding to material attributes item And the user and material matching attribute feature vector Vec match A concatenated vector, Vec features =concat(Vec user ,Vec item ,Vec match ), the input vector Vec featuresInput three shared networks respectively to get three output vectors, where the internal structure of each shared network is a three-layer fully connected neural network; the independent tower network is divided into front and back networks, and the internal structure of the network is a three-layer fully connected neural network. The outputs of the three shared networks are spliced ​​to get a vector V1, and V1 is input into the first three independent tower front networks respectively. V1 is spliced ​​with the feature vectors of the material's location information and refresh information to get V2, and V2 is input into the last three independent tower front networks respectively. The outputs of the independent tower front networks are used as the inputs of the independent tower back networks respectively. Finally, six scores are output. The first three independent tower networks of the model train the model outside the first material, and the last three independent towers train the first material. The model structure diagram is as follows: Figure 7 As shown;

[0180] Model loss design: In the main network, the outputs a1 and a2 of the independent tower post-network 1 and 4 are the predicted values ​​of the click rate of non-first items and first items, respectively; the outputs b1 and b2 of the independent tower post-network 2 and 5 are the predicted values ​​of the interaction rate of non-first items and first items, respectively; the outputs c1 and c2 of the independent tower post-network 3 and 6 are the predicted values ​​of the consumption time of non-first items and first items, respectively. Among them, a1, a2, b1, and b2 are trained by cross entropy loss (loss1, loss4, loss2, and loss5), and the cross entropy loss function is as shown in formula (1). c1 and c2 are trained by MSE loss (loss3 and loss6), respectively. The MSE loss function is expressed as formula (2).

[0181] Model online learning: The embodiment of the present invention provides a method for online training of network models; when users browse blog posts in Weibo APP, a large number of real-time samples will be generated. The timer will start after the user finishes browsing the blog post, and the samples accumulated in a certain time window will be processed to form a sample stream; the sample stream will be sent to the neural network model mentioned above through the Flink stream task, and the user behavior prediction value will be obtained at the output end. According to the user's real behavior supervised learning, the weighted loss function result is calculated, and then the parameters of the neural network are updated using the stochastic gradient descent algorithm. After multiple rounds of iterations, a model with high user target prediction accuracy is obtained.

[0182] Model service: On the online service side of the recommendation system, the model will calculate six prediction values ​​online: a1, a2, b1, b2, c1, and c2, which represent the predicted values ​​of click-through rate, interaction rate, and viewing time for non-first items and first items, respectively. During the sorting process, different prediction values ​​are used for the first item and the non-first item. We combine the predicted values ​​of click-through rate, interaction rate, and viewing time, and recommend blog posts based on comprehensive sorting.

[0183] The embodiments of the present invention have the following beneficial effects:

[0184] The prior art only designs position features during training, and the degree of distinction between the first item and non-first item is not high. The embodiment of the present invention redefines the distinction between the first item and non-first item by combining the material refresh method and the position of the material in each refresh. In the model structure, the embodiment of the present invention additionally sets up a network structure to perform separate training for the first item and fine-tune the modeling of the first item. The embodiment of the present invention redefines the distinction between the first item and non-first item by combining the material refresh method and the position of the material in each refresh. In the multi-objective model structure, an additional network structure is set up to perform separate training for the first item and fine-tune the modeling of the first item.

[0185] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0186] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0187] The disclosed embodiments are described above to enable any person skilled in the art to implement or use the present invention. Various modifications of these embodiments are obvious to those skilled in the art, and the general principles defined herein may also be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0188] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is covered in a manner similar to the term "including". In addition, any term "or" used in the specification of the claims is intended to mean "non-exclusive or".

[0189] Those skilled in the art may also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention may be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.

[0190] The steps of the method or algorithm described in the embodiments of the present invention can be directly embedded in hardware, a software module executed by a processor, or a combination of the two. The software module can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or other storage media of any form in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be arranged in an ASIC, and the ASIC can be arranged in a user terminal. Optionally, the processor and the storage medium can also be arranged in different components in the user terminal.

[0191] In one or more exemplary designs, the above functions described in the embodiments of the present invention can be implemented in hardware, software, firmware or any combination of the three. If implemented in software, these functions can be stored on a computer-readable medium, or transmitted in the form of one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. The storage medium can be any available medium that can be accessed by any general or special computer. For example, such computer-readable media can include but are not limited to RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program codes in the form of instructions or data structures and other forms that can be read by general or special computers, or general or special processors. In addition, any connection can be appropriately defined as a computer-readable medium, for example, if the software is transmitted from a website site, server or other remote resource through a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wirelessly, such as infrared, wireless and microwave, it is also included in the defined computer-readable medium. The disk and disc include compact disk, laser disk, optical disk, DVD, floppy disk and blue-ray disk. Disks usually copy data magnetically, while discs usually copy data optically with lasers. The above combination can also be included in computer readable media.

[0192] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A model training method, characterized in that: include: Collecting a training data set online, each piece of training data in the training data set includes attribute information, display location, refresh method, and at least one user behavior record of the user on the material corresponding to the material pushed to the user; Determining a real frequency score of at least one user behavior of the user with respect to the material according to at least one user behavior record corresponding to the material in the training data; The attribute information corresponding to the material in the training data is converted into a first feature embedding vector through a first feature extraction model; Inputting the first feature embedding vector into a first non-first item rating model for prediction, and obtaining a non-first prediction frequency score of at least one user behavior of the user on the item; Obtaining a first feature concatenation embedding vector according to the display position and refresh mode corresponding to the material in the training data and the first feature embedding vector, inputting the first feature concatenation embedding vector into a first material rating model for prediction, and obtaining a first predicted frequency score of at least one user behavior of the user on the material; Determine whether the material is the first material according to the display position and refresh mode corresponding to the material in the training data and the preset first material determination rule; Determine a loss function value according to a result of determining whether the material is the first material, a real frequency score of at least one user behavior of the material, a predicted frequency score of a non-first material, and a predicted frequency score of a first material, and adjust model parameters of the first non-first material scoring model, the first first material scoring model, and the first feature extraction model according to the loss function value; Until the preset convergence condition is met, the first feature extraction model, the first non-first material scoring model and the first first material scoring model after training are obtained to construct a user behavior scoring prediction model for material sorting.

2. The training method of the model as claimed in claim 1, characterized in that: The method further comprises: determining a loss function value according to a result of determining whether the material is the first material, a real frequency score of at least one user behavior of the material, a predicted frequency score of a non-first material, and a predicted frequency score of a first material, and adjusting model parameters of the first non-first material scoring model, the first first material scoring model, and the first feature extraction model according to the loss function value, including: In the case where it is determined that the material is a non-first material, a loss value of a loss function corresponding to the first non-first material scoring model is determined according to a real frequency score of at least one user behavior of the material and a predicted frequency score of the non-first material, and model parameters of the first non-first material scoring model and the first feature extraction model are adjusted according to the loss value of the loss function corresponding to the first non-first material scoring model; the loss function corresponding to the first non-first material scoring model represents the gap between the predicted frequency score of the non-first material of at least one user behavior of the material and the real frequency score; When it is determined that the material is the first material, a loss value of a loss function corresponding to the first material scoring model is determined according to a real frequency score of at least one user behavior of the material and a predicted frequency score of the first material, and model parameters of the first first material scoring model and the first feature extraction model are adjusted according to the loss value of the loss function corresponding to the first first material scoring model; the loss function corresponding to the first first material scoring model represents the gap between the predicted frequency score of the first material of at least one user behavior of the material and the real frequency score.

3. The training method of the model as claimed in claim 1, characterized in that: The attribute information corresponding to the material includes multiple sets of key-value pairs consisting of attribute types and attribute values; The step of converting the attribute information corresponding to the material in the training data into a first feature embedding vector by using a first feature extraction model includes: For each group of key-value pairs in the attribute information corresponding to the material, determine the attribute type serial number corresponding to the attribute type in the key-value pair according to a preset attribute type and code mapping relationship table, perform hash calculation on the attribute value in the key-value pair to obtain the hash value corresponding to the attribute value, and concatenate the attribute type serial number corresponding to the attribute type with the hash value corresponding to the attribute value to obtain the concatenated code corresponding to the key-value pair; Using the concatenated code corresponding to the key-value pair as an index number, looking up a table in a preset attribute information vector matrix to obtain an embedded vector corresponding to the key-value pair; Concatenate the embedded vectors corresponding to all key-value pairs in the attribute information corresponding to the material to obtain a first attribute embedded vector corresponding to the material; The first attribute embedding vector is input into a first feature extraction model for feature extraction to obtain a first feature embedding vector.

4. The training method of the model as claimed in claim 1, characterized in that: The first material judgment rule includes: The material whose refresh mode is to display materials from the first position and whose display position is the first position is determined to be the first material; otherwise, it is determined to be a non-first material.

5. A user behavior rating prediction method, characterized in that: include: Obtain the refresh mode corresponding to the target user and the attribute information corresponding to the material to be pushed under the refresh mode; The attribute information corresponding to the material to be pushed is converted into a second feature embedding vector through a second feature extraction model; Inputting the second feature embedding vector into a second non-first item rating model for prediction, to obtain a non-first predicted frequency score of at least one user behavior of the target user on the item to be pushed; According to the refresh mode and the second feature embedding vector, a second feature splicing embedding vector is obtained, and the second feature splicing embedding vector is input into a second first item rating model for prediction to obtain a first predicted frequency score of at least one user behavior of the target user on the item to be pushed; Among them, the non-first predicted frequency score of at least one user behavior of the material to be pushed is used to determine the non-first interest score of the target user in the material to be pushed when the material to be pushed is a non-first material; the first predicted frequency score of at least one user behavior of the material to be pushed is used to determine the first interest score of the target user in the material to be pushed when the material to be pushed is a first material, and the non-first interest score and the first interest score of the material to be pushed are used for material sorting; Among them, the model parameters of the second feature extraction model, the second non-first material scoring model and the second first material scoring model are updated by the model parameters of the first feature extraction model, the first non-first material scoring model and the first first material scoring model having the same model structure; the first feature extraction model, the first non-first material scoring model and the first first material scoring model are trained using the model training method as described in any one of claims 1 to 4.

6. A user content shuffling method, characterized in that: include: In response to a recommendation request of a target user, obtaining a refresh mode corresponding to the target user and attribute information corresponding to each of a plurality of materials to be pushed under the refresh mode; For each material to be pushed, the refresh mode corresponding to the target user and the attribute information corresponding to the material to be pushed are input into the user behavior score prediction model to obtain the output non-first prediction frequency score and the first prediction frequency score of at least one user behavior corresponding to the material to be pushed; wherein the user behavior score prediction model includes a second feature extraction model, a second non-first material scoring model and a second first material scoring model, the model parameters of the second feature extraction model, the second non-first material scoring model and the second first material scoring model are updated by the model parameters of the first feature extraction model, the first non-first material scoring model and the first first material scoring model with the same model structure, and the first feature extraction model, the first non-first material scoring model and the first first material scoring model are trained using the model training method described in any one of claims 1 to 4; For each item to be pushed, determine the non-first interest score and the first interest score of the target user for the item to be pushed according to the non-first predicted frequency score and the first predicted frequency score of at least one user behavior corresponding to the item to be pushed; Selecting, from the multiple materials to be pushed, a first material to be pushed with the highest interest score as the first material to be pushed to the target user; From other materials to be pushed except the first material, a preset number of materials to be pushed are selected as non-first materials according to the non-first interest points; The first material and the non-first materials sorted from high to low according to the non-first interest scores constitute mixed content and are pushed to the target user.

7. A user behavior score prediction device, characterized in that: include: The second data collection unit is used to obtain the refresh mode corresponding to the target user and the attribute information corresponding to the material to be pushed under the refresh mode; A second feature embedding vector determining unit, configured to convert the attribute information corresponding to the material to be pushed into a second feature embedding vector through a second feature extraction model; A second non-first prediction frequency score determination unit, configured to input the second feature embedding vector into a second non-first material scoring model for prediction, to obtain a non-first prediction frequency score of at least one user behavior of the target user on the material to be pushed; A second first-item prediction frequency score determination unit is configured to obtain a second feature splicing embedding vector according to the refresh mode and the second feature embedding vector, input the second feature splicing embedding vector into a second first-item material scoring model for prediction, and obtain a first-item prediction frequency score of at least one user behavior of the target user on the material to be pushed; Among them, the non-first predicted frequency score of at least one user behavior of the material to be pushed is used to determine the non-first interest score of the target user in the material to be pushed when the material to be pushed is a non-first material; the first predicted frequency score of at least one user behavior of the material to be pushed is used to determine the first interest score of the target user in the material to be pushed when the material to be pushed is a first material, and the non-first interest score and the first interest score of the material to be pushed are used for material sorting; Among them, the model parameters of the second feature extraction model, the second non-first material scoring model and the second first material scoring model are updated by the model parameters of the first feature extraction model, the first non-first material scoring model and the first first material scoring model having the same model structure; the first feature extraction model, the first non-first material scoring model and the first first material scoring model are trained using the model training method described in any one of claims 1 to 4.

8. A user content shuffling device, characterized in that: include: The third data collection unit is used to respond to the recommendation request of the target user and obtain the refresh mode corresponding to the target user and the attribute information corresponding to each of the multiple materials to be pushed under the refresh mode; A frequency score determination unit is used to input the refresh mode corresponding to the target user and the attribute information corresponding to the material to be pushed into a user behavior score prediction model for each material to be pushed, and obtain the output non-first predicted frequency score and first predicted frequency score of at least one user behavior corresponding to the material to be pushed; wherein the user behavior score prediction model includes a second feature extraction model, a second non-first material scoring model and a second first material scoring model, and the model parameters of the second feature extraction model, the second non-first material scoring model and the second first material scoring model are updated by the model parameters of the first feature extraction model, the first non-first material scoring model and the first first material scoring model with the same model structure, and the first feature extraction model, the first non-first material scoring model and the first first material scoring model are trained using the model training method according to any one of claims 1 to 4; an interest score determination unit, for determining, for each item to be pushed, a non-first item interest score and a first item interest score of the target user for the item to be pushed according to a non-first item predicted frequency score and a first item predicted frequency score of at least one user behavior corresponding to the item to be pushed; a first item determination unit, configured to select a first item to be pushed with the highest interest score from the plurality of items to be pushed as the first item to be pushed to the target user; A non-first item sorting unit is used to select a preset number of items to be pushed as non-first items according to the non-first item interest from other items to be pushed except the first item. The content mixing unit is used to push the mixed content consisting of the first material and the non-first materials sorted from high to low according to the non-first interest scores to the target user.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. An electronic device, characterized in that: It includes: a processor and a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 6.

Images