Model training and material mixed arrangement method, device, equipment and medium

Abstract

The embodiment of the invention provides a model training and material mixed arrangement method and device, equipment and a medium. The method comprises the steps of obtaining user preferences of a target user, a user operation sequence and a material set corresponding to a material type pushed to the target user in real time; determining a material pre-estimated score of the material by using a pre-estimated score model corresponding to the material type of the material set; according to the material pre-estimation score of each material, a preset number of materials with the highest material pre-estimation score are selected as target materials, and a target material sequence corresponding to the material set is obtained; converting the user preference of the target user, the user operation sequence, and each target material and the material pre-estimated score thereof in the target material sequence into a feature coding sequence; and inputting the feature coding sequence into a first online calibration model to obtain a calibration score of the user operation corresponding to each target material in the target material sequence, and adjusting model parameters of the first online calibration model based on the calibration score of the user operation corresponding to each target material and the statistical score.

Description

Technical Field

[0001] The present invention relates to the field of online model calibration, and in particular to a model training and material mixing method, device, equipment and medium. Background Art

[0002] Current machine learning technologies, especially deep learning, have been widely used in recommendation systems to identify and predict user preferences through complex data patterns. These technologies continue to evolve, including real-time recommendations, context-aware recommendations, and personalized recommendations, aiming to provide a more accurate and personalized user experience. Social recommendation systems improve the accuracy and quality of recommendations by leveraging users' interactions and relationships in social networks. At the same time, they introduce non-interest materials by exploring areas that users may be interested in. Therefore, there will be many types of materials in such recommendation systems, such as attention materials, interest materials, and non-interest materials.

[0003] Model calibration technology is an important branch in the field of machine learning, which is mainly used to improve the accuracy and reliability of model predictions. For example, in the click-through rate of recommendation problems, calibration technology can ensure that the probability given by the recommendation system is close to the actual click probability distribution. Currently, commonly used calibration methods include platform scaling (Platt Scaling), isotonic regression, and Bayesian calibration. These methods adjust the output of the model offline to make the predicted output of the model closer to the actual probability distribution.

[0004] There are currently two common solutions for model calibration technology: the first is to directly add a linear transformation to the prediction result to change its distribution when the model predicts the output; the second is to bucket the model output results through offline statistical historical data after the model predicts the output.

[0005] In the process of implementing the present invention, the applicant discovered that there are at least the following problems in the prior art:

[0006] Existing model calibration technologies are not very real-time and cannot perform personalized calibration for specific users. Summary of the invention

[0007] The embodiments of the present invention provide a model training and material mixing method, device, equipment and medium, which solve the problems that the existing model calibration technology has low real-time performance and cannot perform personalized calibration for specific users.

[0008] To achieve the above objectives, in a first aspect, an embodiment of the present invention provides a model training method, comprising:

[0009] Real-time acquisition of user preferences and user operation sequences of target users, as well as material sets corresponding to at least one material type pushed to the target user; each material in the material set corresponds to a statistical score of at least one user operation, the statistical score representing the actual frequency of the target user performing the corresponding user operation on the material obtained based on log file statistics;

[0010] For each material set, using an estimated score model corresponding to the material type of the material set, determine the material estimated score of each material in the material set; the material estimated score represents the interest level of the target user in the material based on the corresponding estimated score model;

[0011] For each material set, according to the material estimated score of each material in the material set, a preset number of materials with the highest material estimated score are selected as target materials, and a target material sequence corresponding to the material set is obtained;

[0012] For the target material sequence corresponding to each material set, the user preference of the target user, the user operation sequence, and each target material in the target material sequence and its estimated material score are converted into a feature coding sequence of the target material sequence; the feature coding sequence meets the input data requirements of the first online calibration model;

[0013] The feature coding sequence of the target material sequence corresponding to each material set is input into the first online calibration model to obtain a calibration score of at least one user operation corresponding to each target material in the target material sequence; based on the calibration score of at least one user operation corresponding to each target material and the statistical score, the model parameters of the first online calibration model are adjusted; the calibration score represents the correction frequency of the target user performing the corresponding user operation on the material after unified calibration based on the first online calibration model.

[0014] In a second aspect, an embodiment of the present invention provides a material mixing method, comprising:

[0015] In response to a user's recommendation request, obtaining the user's user preference and user operation sequence, and at least two materials to be processed, classifying all materials to be processed according to their material types, and obtaining material sets corresponding to at least two material types;

[0016] For each material set, determine the material estimated score of each material in the material set by using the estimated score model corresponding to the material type of the material set; the material estimated score represents the user's interest in the material based on the corresponding estimated score model;

[0017] For each material set, the user preference and user operation sequence of the user, as well as each material in the material set and its estimated material score, are converted into a feature coding sequence corresponding to the material set; the feature coding sequence meets the input data requirements of a second online calibration model used for online business processing; the model parameters used by the second online calibration model are obtained from a first online calibration model used for online real-time training, the second online calibration model has the same model structure as the first online calibration model, and the first online calibration model uses the model training method described in any of the preceding items for online real-time training;

[0018] For each material set, input the feature code sequence corresponding to the material set into the second online calibration model to obtain a calibration score corresponding to at least one user operation corresponding to each material in the material set; the calibration score represents the frequency of correction of the corresponding user operation performed by the user on the material after unified calibration based on the second online calibration model;

[0019] Determine a total score corresponding to each material according to the calibration score of at least one user operation corresponding to each material in all material sets; the total score represents the user's interest in the material after unified calibration based on the second online calibration model;

[0020] According to the total score corresponding to each material in all material sets, all materials to be processed are mixed and sorted, and materials are recommended to the user based on the mixed sorting result.

[0021] In a third aspect, an embodiment of the present invention provides a model training device, comprising:

[0022] A data collection unit is used to obtain in real time the user preferences and user operation sequences of the target user, and a material set corresponding to each of the at least one material type pushed to the target user; each material in the material set corresponds to a statistical score of at least one user operation, and the statistical score represents the actual frequency of the target user performing the corresponding user operation on the material obtained based on the log file statistics;

[0023] A material estimated score determination unit, for determining, for each material set, a material estimated score of each material in the material set using an estimated score model corresponding to a material type in the material set; the material estimated score represents a target user's interest in the material based on the corresponding estimated score model;

[0024] a target material sequence determination unit, for each material set, selecting a preset number of materials with the highest material estimated scores as target materials according to the material estimated scores of each material in the material set, and obtaining a target material sequence corresponding to the material set;

[0025] an encoding unit, for converting, for a target material sequence corresponding to each material set, a user preference of the target user, a user operation sequence, and each target material in the target material sequence and its estimated material score, into a feature coding sequence of the target material sequence; the feature coding sequence meets the input data requirements of the first online calibration model;

[0026] A model parameter determination unit is used to input the feature coding sequence of the target material sequence corresponding to each material set into the first online calibration model, obtain the calibration score of at least one user operation corresponding to each target material in the target material sequence, and adjust the model parameters of the first online calibration model based on the calibration score and statistical score of at least one user operation corresponding to each target material; the calibration score represents the correction frequency of the target user performing the corresponding user operation on the material obtained after the unified calibration based on the first online calibration model.

[0027] In a fourth aspect, an embodiment of the present invention provides a material mixing device, comprising:

[0028] A material acquisition unit, configured to respond to a user's recommendation request, acquire the user's user preference and user operation sequence, and at least two materials to be processed, and classify all materials to be processed according to their material types to obtain a material set corresponding to at least two material types;

[0029] An online estimated score determination unit, for each material set, determines a material estimated score of each material in the material set by using an estimated score model corresponding to a material type of the material set; the material estimated score represents a user's interest in the material based on the corresponding estimated score model;

[0030] An online encoding unit, for converting, for each material set, the user preference and user operation sequence of the user, and each material in the material set and its estimated material score, into a feature coding sequence corresponding to the material set; the feature coding sequence meets the input data requirements of a second online calibration model for online business processing; the model parameters used by the second online calibration model are obtained from a first online calibration model for online real-time training, the second online calibration model has the same model structure as the first online calibration model, and the first online calibration model adopts the model training method as described in any of the preceding items for online real-time training;

[0031] an operation score determination unit, for each material set, inputting a feature code sequence corresponding to the material set into the second online calibration model to obtain a calibration score corresponding to at least one user operation corresponding to each material in the material set; the calibration score represents a correction frequency of the user performing the corresponding user operation on the material obtained after unified calibration based on the second online calibration model;

[0032] A sorting unit is used to determine a total score corresponding to each material according to the calibration score of at least one user operation corresponding to each material in all material sets, perform mixed sorting on all materials to be processed according to the total score corresponding to each material in all material sets, and recommend materials to the user based on the mixed sorting result; the total score represents the user's interest in the material after unified calibration based on the second online calibration model.

[0033] In a fifth aspect, an embodiment of the present invention provides a computer device, comprising at least one host, wherein the at least one host comprises a processor adapted to execute various instructions; and

[0034] A storage device, suitable for storing multiple instructions, wherein the instructions are suitable for being loaded by a processor and executed by any of the model training methods described above or the material mixing method described above.

[0035] In a sixth aspect, an embodiment of the present invention provides a storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the model training method as described in any one of the foregoing or the material mixing method as described in the foregoing.

[0036] The above technical solution has the following beneficial effects: in an online manner, the first online calibration model is trained using real-time interaction data between users and materials to determine the model parameters of the first online calibration model. The model parameters can be used to periodically update the model parameters of the calibration model with the same model structure as the first online calibration model in the online business system that provides services to users, thereby realizing online real-time training of the calibration model, based on which the periodic automatic update of the model parameters of the calibration model at the business end can be further realized, providing a technical basis for realizing automatic update and improvement of the performance of the business end system. In view of the problem that the material estimated scores obtained by using different estimated score models for different types of materials are not comparable, a real-time unified model calibration system is constructed. On the basis of comprehensively considering the material estimated scores of each estimated score model, the user's relevant information and materials are recombined to train the first online calibration model, so that the calibration scores of materials of different material types obtained based on the first online calibration model are comparable, so that based on the comparable calibration scores, the materials can be further sorted, and further business processing can be performed according to the sorting. The online model calibration phase uses the frequency of real user operations as the training target value, and updates the network (model) parameters of the first calibration model in real time through deep learning online training, which can capture online changes in real time and provide more accurate model calibration. Introduce user preference features and user operation sequences to achieve personalized model calibration. Input user preferences as features into the online model calibration network, and integrate user preferences into the network parameter update process to achieve personalized model calibration, so as to improve the accuracy of the overall shuffling effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 is a flow chart of a model training method according to one embodiment of the present invention;

[0039] Figure 2 It is a flow chart of a material mixing method according to one embodiment of the present invention;

[0040] Figure 3 It is an architecture diagram of a model training device according to one embodiment of the present invention;

[0041] Figure 4 This is a structural diagram of a material mixing device according to one embodiment of the present invention;

[0042] Figure 5 is another flow chart of a model training method according to one embodiment of the present invention;

[0043] Figure 6 is a schematic diagram of an online calibration model according to one embodiment of the present invention;

[0044] Figure 7 It is a schematic diagram of a model of a gating network according to one embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] First, as Figure 1 As shown, an embodiment of the present invention provides a model training method, comprising:

[0047] Step S10, obtaining in real time the user preference and user operation sequence of the target user, and a material set corresponding to at least one material type pushed to the target user; each material in the material set corresponds to a statistical score of at least one user operation, and the statistical score represents the actual frequency of the target user performing the corresponding user operation on the material obtained based on the log file statistics;

[0048] Step S11, for each material set, using an estimated score model corresponding to the material type of the material set, determining a material estimated score for each material in the material set; the material estimated score represents the interest level of the target user in the material based on the corresponding estimated score model;

[0049] Step S12, for each material set, according to the material estimated score of each material in the material set, select a preset number of materials with the highest material estimated score as target materials, and obtain a target material sequence corresponding to the material set;

[0050] Step S13, for the target material sequence corresponding to each material set, converting the user preference of the target user, the user operation sequence, and each target material in the target material sequence and its estimated material score into a feature coding sequence of the target material sequence; the feature coding sequence meets the input data requirements of the first online calibration model;

[0051] Step S14, inputting the characteristic coding sequence of the target material sequence corresponding to each material set into the first online calibration model to obtain the calibration score of at least one user operation corresponding to each target material in the target material sequence, and adjusting the model parameters of the first online calibration model based on the calibration score of at least one user operation corresponding to each target material and the statistical score; the calibration score represents the correction frequency of the target user performing the corresponding user operation on the material obtained after the unified calibration based on the first online calibration model.

[0052] In some embodiments, in a multi-type content mixed recommendation system or a multi-type material mixed system, since there are multiple material types, materials from multiple material types need to be mixed, so each material type will have a corresponding independent estimation score model. For example, material types include but are not limited to community materials, attention materials, non-attention materials, etc., which are classified according to the attributes of the materials or the business to which the materials belong. The estimation score model includes but is not limited to estimation models for community materials, estimation models for attention materials, estimation models for non-attention materials, etc. Each estimation score model may specifically include but is not limited to DIN deep neural network models, DeepFM deep neural network models and other neural network models or traditional expert systems. For each material, the original estimation score (material estimation score) is estimated through the estimation score model of the material type of the material, and the real-time obtained user preferences, user operation sequences, and each target material in the target material sequence and its material estimation score are used to train the first online calibration model in real time online, and update the model parameters of the first online calibration model online. Figure 5As shown, the input of the overall model is different types of materials. The original estimated scores (material estimated scores) are obtained by independent online reasoning through independent estimated score models. The materials ranked in the TopN of the original estimated scores are entered into the first online calibration model for training. Based on the training results of the first online calibration model, the calibration scores corresponding to each user operation on the target material are determined. The calibrated scores obtained after calibration are compared with the statistical scores, and the model parameters of the first online calibration model are adjusted to achieve real-time online calibration of the first online calibration model. User operations include but are not limited to clicks, views, interactions and / or forwarding. The actual frequency of user operations can be obtained based on the data such as the click rate, viewing time, and interaction rate of the user on the material recorded in the log. The embodiment of the present invention can be deployed as online real-time training. The materials pushed to the user can be classified into multiple material types, such as concerned materials, community materials, and non-concerned materials. For different material types, estimated score models corresponding to the material types are provided respectively. Each estimated score model can be a model of a different type, or a model of the same type but with different model parameters; the estimated score model can be used to determine the material estimated score of the material under the corresponding material type. Each material type uses an independent estimated score model. When training the estimated score model, the estimated score model corresponding to each material type will only be trained by the materials under this material type, and will not be input into the materials under other material types during the training. Therefore, during the training process of the estimated score model corresponding to this material type, the model parameters of the materials of other material types will not be generalized. The obtained estimated score model can more accurately predict the material estimated score of the materials under the corresponding material type, and the estimated score model for each material type can be optimized independently without affecting each other, which is conducive to the maintenance and upgrade of the system. The estimated score model can be a pre-built and trained neural network, expert system and other models. According to the material estimated score sorting of the materials, the preset number of materials with the highest material estimated score is taken as the target material sequence, and the materials with high user interest can be selected from a large number of materials, which is equivalent to roughly selecting a preset number of materials for each material type from a large number of materials through the estimated score model. In the subsequent prediction process, the preset number of materials of all material types can be put together and sorted uniformly according to the calibration score after calibration of the online calibration model. The training process can be processed based on the same method as the prediction process, and the preset number (TopN) of materials with the highest estimated scores for each material type are selected to train the online calibration model. The training data input to the online calibration model includes information such as materials, material estimated scores, user preferences corresponding to materials, and user operation sequences, so as to take into account the impact of user personal preferences and operations on material score calibration, so that the calibration results have user personalized characteristics.

[0053] The embodiment of the present invention has the following technical effects: in an online manner, the first online calibration model is trained using real-time interaction data between users and materials to determine the model parameters of the first online calibration model. The model parameters can be used to periodically update the model parameters of the calibration model with the same model structure as the first online calibration model in the online business system that provides services to users, thereby realizing online real-time training of the calibration model. Based on this, the periodic automatic update of the model parameters of the calibration model at the business end can be further realized, providing a technical basis for realizing automatic update and improvement of the system performance at the business end. In view of the problem that the material estimated scores obtained using different estimated score models for different types of materials are not comparable, a real-time unified model calibration system is constructed. On the basis of comprehensively considering the material estimated scores of each estimated score model, the user's relevant information and materials are recombined to train the first online calibration model, so that the calibration scores of each operation of materials of different material types obtained based on the first online calibration model are obtained. All materials of any material type are calibrated based on the same first online calibration model to make the calibration scores comparable, so that the total score of each material can be further determined based on the comparable calibration scores, and the total scores of different materials are also comparable, so that the materials can be sorted according to the total scores, and further business processing can be performed according to the sorting. The online model calibration link uses the frequency of real user operations as the training target value, and updates the network (model) parameters of the first calibration model in real time through deep learning online training, which can capture online changes in real time and provide more accurate model calibration. Introduce user preference features and user operation sequences to achieve personalized model calibration. Input user preferences as features into the online model calibration network, and integrate user preferences into the network parameter update process to achieve personalized model calibration, so as to improve the accuracy of the overall mixing effect. .

[0054] Further, for each material set, using an estimated score model corresponding to a material type of the material set to determine a material estimated score of each material in the material set includes:

[0055] For each material set, an estimated score model corresponding to the material type is used to obtain an estimated operation score for each user operation corresponding to each material in the material set, and a material estimated score for each material is determined based on the estimated operation score for each user operation corresponding to each material; the estimated operation score represents the predicted frequency of the target user performing the corresponding user operation based on the corresponding estimated score model.

[0056] In some embodiments, a corresponding estimated score model is provided for each material type. For the material corresponding to each material type, the material is predicted and analyzed through the estimated score model to obtain an estimated operation score for each user operation related to the material. User operations include but are not limited to interaction, clicks, views, etc. The size of the operation estimated score corresponds to the size of the interaction rate, click rate and viewing time. The material estimated score is determined based on the estimated operation scores of all user operations on the material. Specifically, each operation estimated score can be used as an independent variable and the material estimated score can be used as a dependent variable to form a certain functional relationship from the operation estimated score to the material estimated score, or each operation estimated score can be multiplied by a preset weight and then summed up as the material estimated score. The functional relationship can be determined by fitting analysis and other methods.

[0057] The embodiments of the present invention have the following technical effects: each estimated sub-model may be a model that has been used in other business modules, each estimated sub-model may be independently developed, maintained and updated by different business departments, there may be no coupling between them, and the estimated scores output by each estimated sub-model may not be comparable. The data of existing business systems can be fully utilized to avoid coupling or redevelopment of prior or upstream business systems in order to cooperate with mixed business.

[0058] Furthermore, for the target material sequence corresponding to each material set, the user preference of the target user, the user operation sequence, and each target material in the target material sequence and its estimated material score are converted into a feature coding sequence of the target material sequence, including:

[0059] For each target material in the target material sequence corresponding to each material set, the target material and the estimated material score of the target material, as well as the user preference and user operation sequence of the target user are mapped into a multi-dimensional embedded code corresponding to the target material;

[0060] The multi-dimensional embedded codes corresponding to all target materials in the target material sequence are combined into a characteristic coding sequence of the target material sequence.

[0061] In some embodiments, the multidimensional embedded code may be a 256-dimensional embedded code; the materials, material estimated scores, user preferences, and user operation sequences corresponding to the same material are vectorized respectively to obtain vector representations corresponding to the materials, material estimated scores, user preferences, and user operation sequences, and the obtained vector representations corresponding to each are spliced ​​to obtain the multidimensional embedded code corresponding to the material. The multidimensional embedded codes corresponding to all materials are spliced ​​into a feature code sequence corresponding to the target material sequence, and the feature code sequence may be a matrix composed of multiple multidimensional embedded codes.

[0062] Furthermore, the first online calibration model is specifically: a POSO model; the POSO model includes the same number of independent tower models as the at least one user operation, and each independent tower model outputs a calibration score corresponding to a user operation;

[0063] The step of inputting the characteristic coding sequence of the target material sequence corresponding to each material set into the first online calibration model to obtain a calibration score of at least one user operation corresponding to each target material in the target material sequence includes:

[0064] For the target material sequence corresponding to each material set, the feature coding sequence of the target material sequence is input into the POSO model, and the calibration score of the corresponding user operation of each target material in the target material sequence is outputted respectively through each independent tower model in the POSO model.

[0065] In some embodiments, Figure 6 As shown in the figure, the first online calibration model is specifically: POSO (Personalized COld Start MOdules) model. POSO learns the corresponding model for each user, which is not affected by other user samples and is completely based on the user's own personalization. POSO includes a fully connected structure (MLP, Multilayer Perceptron), a multi-head attention (MHA, multi-head attention mechanism) structure, and a PLE structure (Progressive Layered Extraction, hierarchical extraction structure). The output of each layer of the fully connected network is multiplied bit by bit with the output of the Gating Network (gated network). The multi-head attention structure learns autonomously according to the features and learns different weights for different users to achieve user-based personalization; PLE (Progressive Layered Extraction, hierarchical extraction structure) is for multiple tasks (including interaction estimation, click estimation, duration estimation, etc.), and different expert networks are constructed through the Gating Network module to realize a model for multi-task prediction.

[0066] Second, as Figure 2 As shown, an embodiment of the present invention provides a material mixing method, comprising:

[0067] Step S20, in response to the user's recommendation request, obtaining the user's user preference and user operation sequence, and at least two materials to be processed, classifying all materials to be processed according to their material types, and obtaining material sets corresponding to at least two material types;

[0068] Step S21, for each material set, determining a material estimated score of each material in the material set by using an estimated score model corresponding to the material type of the material set; the material estimated score represents the user's interest in the material based on the corresponding estimated score model;

[0069] Step S22: for each material set, convert the user preference and user operation sequence of the user, and each material in the material set and its estimated material score, into a feature coding sequence corresponding to the material set; the feature coding sequence meets the input data requirements of the second online calibration model used for online business processing; the model parameters used by the second online calibration model are obtained from the first online calibration model used for online real-time training, the second online calibration model has the same model structure as the first online calibration model, and the first online calibration model adopts the model training method described in any of the preceding items for online real-time training;

[0070] Step S23, for each material set, inputting the feature code sequence corresponding to the material set into the second online calibration model to obtain a calibration score corresponding to at least one user operation corresponding to each material in the material set; the calibration score represents the frequency of correction of the corresponding user operation performed by the user on the material after unified calibration based on the second online calibration model;

[0071] Step S24, determining a total score corresponding to each material according to the calibration score of at least one user operation corresponding to each material in all material sets; the total score represents the user's interest in the material after unified calibration based on the second online calibration model;

[0072] Step S25, performing mixed sorting on all materials to be processed according to the total score corresponding to each material in all material sets, and recommending materials to the user based on the mixed sorting result.

[0073] In some embodiments, the material mixing request may come from the user or from other business modules in the backend business system of the server, for example, it may be a user's recommendation request, or a material mixing request issued when other business modules need to sort materials. Through the embodiment of the present invention, sorted materials can be obtained, and materials of all material types determine their sorting positions according to their total scores. On this basis, materials of a preset number of materials with high total scores in the sorted materials can be selected to generate mixed content, and the mixed content can be sent to the user for display; the second online calibration model in the embodiment of the present invention periodically obtains model parameters from the first online calibration model trained online in real time, and uses the obtained model parameters to update the model parameters of the second online calibration model, so as to update the model parameters according to the latest data, adapt to the dynamic changes of material data, and ensure that the second online calibration model used for online business processing responds to the latest changes in user preferences, user operation sequences, materials, etc. in a timely manner, and gives a more accurate calibration score prediction, so that the material sorting result is more reasonable. The obtained material sorting includes materials of one or more material types, and the sorting result can be used in a multi-material type content mixing system.

[0074] Thirdly, Figure 3 As shown, an embodiment of the present invention provides a model training device, comprising:

[0075] The data collection unit 300 is used to obtain in real time the user preferences and user operation sequences of the target user, and a material set corresponding to each of the at least one material type pushed to the target user; each material in the material set corresponds to a statistical score of at least one user operation, and the statistical score represents the actual frequency of the target user performing the corresponding user operation on the material obtained based on the log file statistics;

[0076] The material estimated score determination unit 301 is used to determine the material estimated score of each material in the material set for each material set using the estimated score model corresponding to the material type of the material set; the material estimated score represents the interest level of the target user in the material based on the corresponding estimated score model;

[0077] A target material sequence determination unit 302 is used to select, for each material set, a preset number of materials with the highest material estimated scores as target materials according to the material estimated scores of each material in the material set, and obtain a target material sequence corresponding to the material set;

[0078] The encoding unit 303 is used to convert the user preference of the target user, the user operation sequence, and each target material and its estimated material score in the target material sequence into a feature coding sequence of the target material sequence for the target material sequence corresponding to each material set; the feature coding sequence meets the input data requirements of the first online calibration model;

[0079] The model parameter determination unit 304 is used to input the feature coding sequence of the target material sequence corresponding to each material set into the first online calibration model to obtain the calibration score of at least one user operation corresponding to each target material in the target material sequence, and adjust the model parameters of the first online calibration model based on the calibration score and statistical score of at least one user operation corresponding to each target material; the calibration score represents the correction frequency of the target user performing the corresponding user operation on the material obtained after the unified calibration based on the first online calibration model.

[0080] Furthermore, the material estimated score determination unit 301 is configured as follows:

[0081] For each material set, an estimated score model corresponding to the material type is used to obtain an estimated operation score for each user operation corresponding to each material in the material set, and the material estimated score for each material is determined based on the estimated operation score for each user operation corresponding to each material; the estimated operation score represents the predicted frequency of the target user performing the corresponding user operation based on the corresponding estimated score model.

[0082] Furthermore, the encoding unit 303 is configured as follows:

[0083] For each target material in the target material sequence corresponding to each material set, the target material and the estimated material score of the target material, as well as the user preference and user operation sequence of the target user, are mapped into the multidimensional embedded code corresponding to the target material; the multidimensional embedded codes corresponding to all target materials in the target material sequence are combined into a feature coding sequence of the target material sequence.

[0084] Furthermore, the first online calibration model is specifically: a POSO model; the POSO model includes the same number of independent tower models as the at least one user operation, and each independent tower model outputs a calibration score corresponding to a user operation;

[0085] The model parameter determination unit 304 is used to input the feature coding sequence of the target material sequence corresponding to each material set into the POSO model, and output the calibration score of the corresponding user operation of each target material in the target material sequence through each independent tower model in the POSO model.

[0086] The embodiment of the present invention is a device type embodiment that corresponds one-to-one to the aforementioned embodiment of a model training method. The embodiment of the present invention can be understood based on the aforementioned embodiment of a model training method, and will not be repeated here.

[0087] Fourthly, Figure 4 As shown, an embodiment of the present invention provides a material mixing device, comprising:

[0088] The material acquisition unit 400 is used to respond to the user's recommendation request, acquire the user's user preference and user operation sequence, and at least two materials to be processed, classify all the materials to be processed according to their material types, and obtain a material set corresponding to at least two material types;

[0089] An online estimated score determination unit 401 is used to determine, for each material set, a material estimated score of each material in the material set by using an estimated score model corresponding to the material type of the material set; the material estimated score represents the user's interest in the material based on the corresponding estimated score model;

[0090] The online coding unit 402 is used to convert the user preference and user operation sequence of the user, and each material in the material set and its estimated material score into a feature coding sequence corresponding to the material set for each material set; the feature coding sequence meets the input data requirements of the second online calibration model used for online business processing; the model parameters used by the second online calibration model are obtained from the first online calibration model used for online real-time training, the second online calibration model has the same model structure as the first online calibration model, and the first online calibration model adopts the model training method described in any of the preceding items for online real-time training;

[0091] The operation score determination unit 403 is used to input the feature code sequence corresponding to each material set into the second online calibration model to obtain a calibration score corresponding to at least one user operation corresponding to each material in the material set; the calibration score represents the correction frequency of the user performing the corresponding user operation on the material obtained after unified calibration based on the second online calibration model;

[0092] The sorting unit 404 is used to determine the total score corresponding to each material according to the calibration score of at least one user operation corresponding to each material in all material sets, perform mixed sorting on all materials to be processed according to the total score corresponding to each material in all material sets, and recommend materials to the user based on the mixed sorting result; the total score represents the user's interest in the material after unified calibration based on the second online calibration model.

[0093] The embodiment of the present invention is a device-type embodiment that corresponds one-to-one to the aforementioned embodiment of a material mixing method. The embodiment of the present invention can be understood based on the aforementioned embodiment of a material mixing method, and will not be repeated here.

[0094] In a fifth aspect, an embodiment of the present invention provides a computer device, comprising at least one host, wherein the at least one host comprises a processor adapted to execute various instructions; and

[0095] A storage device, suitable for storing multiple instructions, wherein the instructions are suitable for being loaded by a processor and executed by any of the model training methods described above or the material mixing method described above.

[0096] An embodiment of the present invention is an embodiment of a device that stores and executes any of the above-described model training methods or the above-described material mixing methods. The embodiment of the present invention can be understood based on the embodiments of the above-described model training methods or the above-described material mixing methods.

[0097] In a sixth aspect, an embodiment of the present invention provides a storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the model training method as described in any one of the foregoing or the material mixing method as described in the foregoing.

[0098] An embodiment of the present invention is an embodiment of a storage medium that stores instructions for a model training method as described in any one of the foregoing or a material mixing method as described in the foregoing. The embodiment of the present invention can be understood based on the embodiments of the foregoing model training method or the foregoing material mixing method.

[0099] The above technical solution of the embodiment of the present invention is described in detail below in conjunction with specific application examples. For technical details not introduced during the implementation process, please refer to the relevant description in the previous text.

[0100] In a mixed recommendation system for multiple types of content or a mixed material system, since there are multiple types of materials, each material will have a corresponding independent estimation model to obtain the original estimation score through estimation. Through online model calibration, the model estimation score is personalized calibrated based on a unified scenario, and finally the overall ranking is performed. Figure 5 The input of the overall model is materials of different material types. Figure 5It includes: concerned materials, non-concerned materials and community materials. Each material type is independently inferred online through an independent estimated score model corresponding to the material type to obtain the original estimated score (material estimated score). The materials ranked in the TopN (topN) of the original estimated score enter the online calibration link of the online calibration model (the first online calibration model during training or the second online calibration model when providing business services). After calibration, the estimated probability (calibration score) (such as the actual click-through rate) is obtained. The calibrated estimated probability determines the estimated user interest level (total score), and the materials are sorted by insertion according to the total score. Among them, the calibration score and statistical score can represent but are not limited to: click-through rate, viewing time, interaction rate, etc. Among them, the calibration score is obtained after calibration by the online calibration model, and the statistical score can be obtained from the log file through real statistics, which is the real value. The specific steps of the main solution for online model calibration are as follows:

[0101] There are multiple materials, each material requires multiple operations, each operation has its own estimated operation score, and the estimated operation scores of all operations of each material are combined into the total estimated score (material estimated score). By sorting the materials according to the total estimated score (material estimated score), multiple sequences of multiple materials can be obtained.

[0102] Take the topN of each sequence, encode the material sequence and the total estimated score (material estimated score) sequence (the two may be separated or one-to-one corresponding), and all material pairs correspond to a user's user characteristics and behavior sequence (user operation sequence). The user characteristics and behavior sequence are also encoded and converted to 256 dimensions. It may be that one material, one estimated score, one user preference, and one user behavior sequence correspond to one 256-dimensional sequence. Multiple materials correspond to multiple dimensions.

[0103] Input to the model, several operations, several models, one-to-one correspondence, corresponding real scores. Prediction and training are the same. During training, real values ​​of user operations, materials and statistics are obtained in real time, and supervised training is performed according to the input model above to update model parameters. When in use, new model parameters are obtained every hour, and the new model parameters are used as online model parameters to update the online business model, receive materials, and output the real value (total score), and select push materials based on the real value (total score).

[0104] First, the TopN materials obtained by sorting according to the original estimated scores enter the online model calibration stage as materials to be calibrated. Together with the materials to be calibrated, the original estimated score of each material (material estimated score) also enters the online calibration processing stage. This score will enter the calibration stage as an important feature.

[0105] The first step of online model calibration is feature embedding, which maps materials, original estimated scores, user preference features, and user behavior sequence features into 256-dimensional feature vectors through embedding. The vectors are then concat-processed and finally reshaped into the model input format size.

[0106] In the online calibration phase, each task (operation) will have a corresponding expert network, such as clicks, interactions, and viewing time. These expert networks train their own network-specific parameters based on actual efficiency values ​​in a unified scenario (such as interaction rate, click rate, and viewing time obtained from log statistics) to achieve the purpose of model calibration.

[0107] For user preference features, online model calibration uses the GatingNetwork approach to learn and capture user personalized characteristics. The specific approach is as follows: Figure 7 x1=Relu(x input *W 0 +b 0 ), where W 0 , b 0 is the parameter of the first layer of neural network, x out =Sigmoid(x1*W 1 +b 1 ), where W 1 , b 1 is the second layer neural network parameter, and finally the output result x of the Sigmoid activation function is out By multiplying it with the feature embedding, the user preference feature can be applied to the model calibration, that is, if the user likes a certain type of material, its corresponding x out The product with the feature embedding will be larger.

[0108] The embodiments of the present invention have the following technical effects: Compared with the prior art, the method in the present invention can be used to achieve online model calibration and update model calibration parameters in real time. By adopting the method of deep learning online training, the neural network parameters are updated in batches, which can achieve hourly parameter updates. For online businesses that change in real time, online changes can be captured more timely; the online calibration model uses the efficiency (such as click-through rate) in a unified scenario as a training label, and inputs the original model prediction score into the network as a feature. At the same time, multiple expert networks are introduced into the network, and different original prediction scores of different materials can be calibrated to achieve more accurate and reliable content recommendation distribution. Compared with the prior art, the method in the present invention can achieve personalized model calibration for different users, achieving a thousand-faced effect in the model calibration stage. The method in the present application uses the user's preference characteristics, such as material preference, interest preference and other characteristics, as an input for online model calibration, and implements personalized calibration based on user preferences through GatingNetwork technology. For materials that users like, the score after model calibration will be high, and for materials that users do not like, the score after model calibration will be low, so as to achieve more accurate distribution and improve overall mixing efficiency. For a multi-type content mixing or material mixing system based on online model calibration proposed in the present invention, a real-time unified model calibration system can be built for different models of different types of materials. After independent model estimation, different types of materials select TopN to enter the online model calibration link. The online model calibration link uses real efficiency as a label, and updates network parameters in real time through deep learning online training, which can capture online changes in real time and provide more accurate model calibration. A multi-type content mixing or material mixing system based on online model calibration proposed in the present invention can introduce user preference characteristics and implement personalized model calibration. User preferences are input as features into the online model calibration network, and user preferences are integrated into the network parameter update process through a gating network to achieve personalized model calibration and improve the accuracy of the overall mixing effect.

[0109] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0110] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0111] The disclosed embodiments are described above to enable any person skilled in the art to implement or use the present invention. Various modifications of these embodiments are obvious to those skilled in the art, and the general principles defined herein may also be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0112] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is covered in a manner similar to the term "including". In addition, any term "or" used in the specification of the claims is intended to mean "non-exclusive or".

[0113] Those skilled in the art may also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention may be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.

[0114] The various illustrative logic blocks or units described in the embodiments of the present invention can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of the above. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any conventional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0115] The steps of the method or algorithm described in the embodiments of the present invention can be directly embedded in hardware, a software module executed by a processor, or a combination of the two. The software module can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or other storage media of any form in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be arranged in an ASIC, and the ASIC can be arranged in a user terminal. Optionally, the processor and the storage medium can also be arranged in different components in the user terminal.

[0116] In one or more exemplary designs, the above functions described in the embodiments of the present invention can be implemented in hardware, software, firmware or any combination of the three. If implemented in software, these functions can be stored on a computer-readable medium, or transmitted in the form of one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. The storage medium can be any available medium that can be accessed by any general or special computer. For example, such computer-readable media can include but are not limited to RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program codes in the form of instructions or data structures and other forms that can be read by general or special computers, or general or special processors. In addition, any connection can be appropriately defined as a computer-readable medium, for example, if the software is transmitted from a website site, server or other remote resource through a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wirelessly, such as infrared, wireless and microwave, it is also included in the defined computer-readable medium. The disk and disc include compact disk, laser disk, optical disk, DVD, floppy disk and blue-ray disk. Disks usually copy data magnetically, while discs usually copy data optically with lasers. The above combination can also be included in computer readable media.

[0117] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A model training method, characterized in that: include: Real-time acquisition of user preferences and user operation sequences of target users, as well as material sets corresponding to at least one material type pushed to target users; Each material in the material set corresponds to a statistical score of at least one user operation, and the statistical score represents the actual frequency of the target user performing the corresponding user operation on the material obtained based on the log file statistics; For each material set, using an estimated score model corresponding to the material type of the material set, determine the material estimated score of each material in the material set; the material estimated score represents the interest level of the target user in the material based on the corresponding estimated score model; For each material set, according to the material estimated score of each material in the material set, a preset number of materials with the highest material estimated score are selected as target materials, and a target material sequence corresponding to the material set is obtained; For the target material sequence corresponding to each material set, convert the user preference of the target user, the user operation sequence, and each target material in the target material sequence and its estimated material score into a feature coding sequence of the target material sequence; The feature coding sequence meets the input data requirements of the first online calibration model; The feature coding sequence of the target material sequence corresponding to each material set is input into the first online calibration model to obtain a calibration score of at least one user operation corresponding to each target material in the target material sequence; based on the calibration score of at least one user operation corresponding to each target material and the statistical score, the model parameters of the first online calibration model are adjusted; the calibration score represents the correction frequency of the target user performing the corresponding user operation on the material after unified calibration based on the first online calibration model.

2. The model training method according to claim 1, characterized in that: For each material set, using an estimated score model corresponding to the material type of the material set to determine the material estimated score of each material in the material set includes: For each material set, an estimated score model corresponding to the material type is used to obtain an estimated operation score for each user operation corresponding to each material in the material set, and the material estimated score for each material is determined based on the estimated operation score for each user operation corresponding to each material; the estimated operation score represents the predicted frequency of the target user performing the corresponding user operation based on the corresponding estimated score model.

3. The model training method according to claim 1, characterized in that: The target material sequence corresponding to each material set, converting the user preference of the target user, the user operation sequence, and each target material and its estimated material score in the target material sequence into a feature coding sequence of the target material sequence, includes: For each target material in the target material sequence corresponding to each material set, the target material and the estimated material score of the target material, as well as the user preference and user operation sequence of the target user are mapped into a multi-dimensional embedded code corresponding to the target material; The multi-dimensional embedded codes corresponding to all target materials in the target material sequence are combined into a characteristic coding sequence of the target material sequence.

4. The model training method according to claim 1, characterized in that: The first online calibration model is specifically: a POSO model; the POSO model includes the same number of independent tower models as the at least one user operation, and each independent tower model outputs a calibration score corresponding to a user operation; Inputting the feature code sequence of the target material sequence corresponding to each material set into the first online calibration model to obtain a calibration score of at least one user operation corresponding to each target material in the target material sequence, including: For the target material sequence corresponding to each material set, the feature coding sequence of the target material sequence is input into the POSO model, and the calibration score of the corresponding user operation of each target material in the target material sequence is outputted respectively through each independent tower model in the POSO model.

5. A material mixing method, characterized in that: include: In response to a user's recommendation request, obtaining the user's user preference and user operation sequence, and at least two materials to be processed, classifying all materials to be processed according to their material types, and obtaining material sets corresponding to at least two material types; For each material set, determine the material estimated score of each material in the material set by using the estimated score model corresponding to the material type of the material set; the material estimated score represents the user's interest in the material based on the corresponding estimated score model; For each material set, convert the user preference and user operation sequence of the user, as well as each material in the material set and its estimated material score, into a feature coding sequence corresponding to the material set; The feature code sequence meets the input data requirements of the second online calibration model for online business processing; The model parameters used by the second online calibration model are obtained from the first online calibration model used for online real-time training, the second online calibration model has the same model structure as the first online calibration model, and the first online calibration model is trained online in real time using the model training method according to any one of claims 1 to 4; For each material set, inputting the feature coding sequence corresponding to the material set into the second online calibration model to obtain a calibration score corresponding to at least one user operation corresponding to each material in the material set; The calibration score represents the frequency of correction of the corresponding user operation performed by the user on the material obtained after unified calibration based on the second online calibration model; Determine a total score corresponding to each material according to the calibration score of at least one user operation corresponding to each material in all material sets; The total score represents the user's interest in the material after unified calibration based on the second online calibration model; According to the total score corresponding to each material in all material sets, all materials to be processed are mixed and sorted, and materials are recommended to the user based on the mixed sorting result.

6. A model training device, characterized in that: include: A data collection unit is used to obtain in real time the user preferences and user operation sequences of the target user, and a material set corresponding to each of the at least one material type pushed to the target user; each material in the material set corresponds to a statistical score of at least one user operation, and the statistical score represents the actual frequency of the target user performing the corresponding user operation on the material obtained based on the log file statistics; A material estimated score determination unit, for determining, for each material set, a material estimated score of each material in the material set using an estimated score model corresponding to a material type in the material set; the material estimated score represents a target user's interest in the material based on the corresponding estimated score model; a target material sequence determination unit, for each material set, selecting a preset number of materials with the highest material estimated scores as target materials according to the material estimated scores of each material in the material set, and obtaining a target material sequence corresponding to the material set; an encoding unit, for converting, for each target material sequence corresponding to each material set, a user preference of the target user, a user operation sequence, and each target material in the target material sequence and its estimated material score, into a feature encoding sequence of the target material sequence; The feature coding sequence meets the input data requirements of the first online calibration model; A model parameter determination unit is used to input the feature coding sequence of the target material sequence corresponding to each material set into the first online calibration model, obtain the calibration score of at least one user operation corresponding to each target material in the target material sequence, and adjust the model parameters of the first online calibration model based on the calibration score and statistical score of at least one user operation corresponding to each target material; the calibration score represents the correction frequency of the target user performing the corresponding user operation on the material obtained after the unified calibration based on the first online calibration model.

7. The model training device according to claim 6, characterized in that: The material estimated score determination unit is configured as follows: For each material set, using the estimated score model corresponding to the material type, obtain the estimated operation score of each user operation corresponding to each material in the material set, and determine the material estimated score of each material according to the estimated operation score of each user operation corresponding to each material; The operation prediction score represents the predicted frequency of the target user performing the corresponding user operation based on the corresponding prediction score model.

8. A material mixing device, characterized in that: include: A material acquisition unit, configured to respond to a user's recommendation request, acquire the user's user preference and user operation sequence, and at least two materials to be processed, and classify all materials to be processed according to their material types to obtain a material set corresponding to at least two material types; An online estimated score determination unit, for each material set, determines a material estimated score of each material in the material set by using an estimated score model corresponding to a material type of the material set; the material estimated score represents a user's interest in the material based on the corresponding estimated score model; An online encoding unit, for converting, for each material set, the user preference and user operation sequence of the user, and each material in the material set and its estimated material score, into a feature coding sequence corresponding to the material set; The feature code sequence meets the input data requirements of the second online calibration model for online business processing; The model parameters used by the second online calibration model are obtained from the first online calibration model used for online real-time training, the second online calibration model has the same model structure as the first online calibration model, and the first online calibration model is trained online in real time using the model training method according to any one of claims 1 to 4; an operation score determination unit, for each material set, inputting a feature code sequence corresponding to the material set into the second online calibration model to obtain a calibration score corresponding to at least one user operation corresponding to each material in the material set; The calibration score represents the frequency of correction of the corresponding user operation performed by the user on the material obtained after unified calibration based on the second online calibration model; A sorting unit is used to determine a total score corresponding to each material according to the calibration score of at least one user operation corresponding to each material in all material sets, perform mixed sorting on all materials to be processed according to the total score corresponding to each material in all material sets, and recommend materials to the user based on the mixed sorting result; the total score represents the user's interest in the material after unified calibration based on the second online calibration model.

9. A computer device, characterized in that: The device comprises at least one host, wherein the at least one host comprises a processor adapted to execute various instructions; and A storage device, adapted to store a plurality of instructions, wherein the instructions are adapted to be loaded by a processor and execute the method according to any one of claims 1 to 5.

10. A storage medium, characterized in that: A plurality of instructions are stored therein, and the instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 5.

Images