A model training method and device, equipment and storage medium

Abstract

The specification discloses a model training method and device, equipment and a storage medium. Based on the adaptation degree of a first training sample for training an information recommendation model corresponding to a target business type, a target training sample is selected from the first training sample, and the information recommendation model corresponding to the target business type to be trained is trained according to the target training sample and an actual recommendation effect representation value corresponding to the target training sample. It can be seen that the first training sample obtained from the historical page of a specified business type is used as a training sample for training the information recommendation model corresponding to the target business type based on the adaptation degree, sample migration from the specified business type to the target business type is realized, even if there are few user historical behaviors obtained from the historical page of the target business type, the information recommendation model corresponding to the target business type can also be trained based on the user historical behaviors obtained from the historical page of the specified business type, and the recommendation effect of the page of the specified business type is further improved.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a model training method, apparatus, device, and storage medium. Background Technology

[0002] When users browse information on the business platform through their terminals, the platform displays a recommendation list containing advertisements and various recommended information; that is, advertisements and recommended information are presented to users in a mixed layout. Improving the rationality of the sorting of advertisements and recommended information helps to improve the recommendation effect of the recommendation system.

[0003] In existing technologies, recommendation models used to rank advertisements and other information typically analyze user preferences based on their historical behavior, prioritizing advertisements and recommendations that match those preferences to improve recommendation effectiveness.

[0004] However, in order to meet the different needs of users, existing business platforms often provide users with display pages for various business types. This results in fewer visits to certain business type display pages and less historical user behavior data, which is insufficient to support the training of information recommendation models that match user preferences for that business type, leading to a decrease in recommendation effectiveness. Summary of the Invention

[0005] This specification provides a model training method and apparatus to partially solve the aforementioned problems existing in the prior art.

[0006] The following technical solution is adopted in this specification:

[0007] This manual provides a model training method, including:

[0008] Obtain historical recommendation information contained in historical pages of a specified business type as the first training sample;

[0009] Based on the first training sample, determine the suitability of the first training sample for training the information recommendation model corresponding to the target business type;

[0010] Based on the fitness level and the first training sample, the target training sample is determined;

[0011] Based on the user's operation results on the target training samples contained in the historical pages of the specified business type, the actual recommendation effect representation value of the target training samples is determined;

[0012] The target training sample is input into the information recommendation model corresponding to the target business type to be trained, and the first predicted recommendation effect representation value of the target training sample is obtained by the information recommendation model corresponding to the target business type to be trained. The information recommendation model corresponding to the target business type to be trained is trained with the goal of minimizing the difference between the first predicted recommendation effect representation value and the actual recommendation effect representation value of the target training sample.

[0013] Optionally, based on the first training sample, the suitability of the first training sample for training the information recommendation model corresponding to the target business type is determined, specifically including:

[0014] For each first training sample, a specified reward distribution and a target reward distribution for that first training sample are determined respectively; the specified reward distribution is used to characterize the user's operation results for the first training samples contained in several historical pages of the specified business type; the target reward distribution is used to characterize the user's operation results for the first training samples contained in several historical pages of the target business type.

[0015] Based on the target reward distribution and the specified reward distribution of the first training sample, the suitability of the first training sample for training the information recommendation model corresponding to the target business type is determined.

[0016] Optionally, for each first training sample, a specified reward distribution for that first training sample is determined, specifically including:

[0017] Using the first training sample as input, input several pre-trained reward determination models corresponding to the specified business type to obtain the specified reward of the first training sample output by each pre-trained reward determination model corresponding to the specified business type.

[0018] The specified reward of the first training sample output by the model is determined based on the pre-trained rewards corresponding to the specified business type, and the specified reward distribution of the first training sample is determined.

[0019] Optionally, for each first training sample, the target reward distribution of that first training sample is determined, specifically including:

[0020] Using the first training sample as input, input several pre-trained reward determination models corresponding to the target business type to obtain the target reward of the first training sample output by each pre-trained reward determination model corresponding to the target business type.

[0021] The target reward of the first training sample output by the model is determined based on the pre-trained rewards corresponding to the target business type, and the target reward distribution of the first training sample is determined.

[0022] Optionally, the target training sample is determined based on the fitness level and the first training sample, specifically including:

[0023] Obtain historical recommendation information contained in the historical pages of the target business type as a second training sample;

[0024] The second training sample and each of the first training samples whose fitness level is higher than the preset fitness level threshold are used as target training samples.

[0025] Optionally, before training the information recommendation model corresponding to the target business type with the training objective of minimizing the difference between the first predicted recommendation performance representation value and the actual recommendation performance representation value of the target training sample, the method further includes:

[0026] The target training sample is input into a pre-trained information recommendation model corresponding to a specified business type to obtain the second predicted recommendation effect characterization value corresponding to the target training sample output by the information recommendation model corresponding to the specified business type.

[0027] The training objective is to minimize the difference between the first predicted recommendation performance representation value and the actual recommendation performance representation value of the target training sample. The training of the information recommendation model corresponding to the target business type specifically includes:

[0028] The total loss is determined based on the first predicted recommendation performance characterization value, the second predicted recommendation performance characterization value, and the actual recommendation performance characterization value.

[0029] With minimizing the total loss as the training objective, an information recommendation model corresponding to the target business type to be trained is trained.

[0030] Optionally, the total loss is determined based on the first predicted recommendation performance characterization value, the second predicted recommendation performance characterization value, and the actual recommendation performance characterization value, specifically including:

[0031] The first loss is determined based on the difference between the first predicted recommendation effect characterization value and the second predicted recommendation effect characterization value;

[0032] The second loss is determined based on the difference between the actual predicted recommendation effect characterization value and the first predicted recommendation effect characterization value;

[0033] Based on the fitness level, determine the weight corresponding to the first loss;

[0034] The total loss is determined based on the first loss, the weight corresponding to the first loss, and the second loss.

[0035] This specification provides a model training apparatus, including:

[0036] The first training sample acquisition module is used to acquire historical recommendation information contained in the historical pages of a specified business type as the first training sample.

[0037] The fit determination module is used to determine the fit of the first training sample for training the information recommendation model corresponding to the target business type based on the first training sample.

[0038] A target training sample determination module is used to determine a target training sample based on the fitness and the first training sample.

[0039] The actual recommendation performance characterization value determination module is used to determine the actual recommendation performance characterization value of the target training sample based on the user's operation results on the target training sample contained in the historical page of the specified business type.

[0040] The training module is used to input the target training sample into the information recommendation model corresponding to the target business type to be trained, and obtain the first predicted recommendation effect representation value of the target training sample output by the information recommendation model corresponding to the target business type to be trained. The training objective is to minimize the difference between the first predicted recommendation effect representation value and the actual recommendation effect representation value of the target training sample.

[0041] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described model training method.

[0042] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the model training method described above.

[0043] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0044] This method uses a first training sample to determine the fit of the information recommendation model for the target business type. It then selects a target training sample from the first training sample and trains the information recommendation model for the target business type based on the target training sample and its corresponding actual recommendation performance representation value. Therefore, this method, based on the determined fit, uses the first training sample obtained from the historical pages of the specified business type as the training sample for the information recommendation model corresponding to the target business type. This achieves sample transfer from the specified business type to the target business type. Even if the historical user behavior obtained from the display pages of the target business type is limited, the information recommendation model for the target business type can still be trained based on the historical user behavior obtained from the display pages of the specified business type, thereby improving the recommendation performance of the display pages of the specified business type. Attached Figure Description

[0045] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0046] Figure 1A This is a schematic diagram of one type of display page in this specification;

[0047] Figure 1B This is a schematic diagram of one type of display page in this specification;

[0048] Figure 2 This is a flowchart illustrating one model training method described in this specification.

[0049] Figure 3 This is a flowchart illustrating one model training method described in this specification.

[0050] Figure 4 This is a schematic diagram of a model training device provided in this specification;

[0051] Figure 5 The corresponding information provided in this specification Figure 2 A schematic diagram of an electronic device. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0053] Additionally, it should be noted that all actions involving the acquisition of signals, information, or data in this invention are carried out in compliance with the relevant data protection laws and regulations of the country where the invention is located, and with authorization from the owner of the corresponding device.

[0054] With the booming development of the new retail industry, more and more merchants are placing advertisements on online business platforms to promote their products. To meet diverse user needs, existing business platforms often provide users with display pages for various business types. This means that when users browse these platforms on their devices, the platforms not only automatically recommend organic results based on the different business types offered, but also provide advertisements belonging to the same business type. By analyzing and mining user preferences and using this to rank advertisements and organic results, the recommendation effect of the recommendation system can be improved. The ranking method for advertisements and organic results is as follows: The input to the information recommendation model is two pre-sorted queues: an advertisement queue and a organic results queue. The output of the information recommendation model mixes these two queues into a single sequence, but does not change the internal order of the two queues.

[0055] Typically, a business platform can display multiple different business types on the initial page, such as Figure 1A As shown, each business type corresponds to the service provided by the business platform for that type. Users can click on a business type of interest to enter its display page. This page showcases recommended information that provides services of that type. For example, if a user clicks on business type A on the initial page, the page will redirect to the display page corresponding to business type A, which offers services of business type A. Figure 1B As shown, recommendations 1-5 can provide users with services of business type A.

[0056] It's clear that the purpose of setting up different display pages for different business types on a business platform is to provide users with more targeted services they desire. However, in real-world applications, the difference in visitor volume between these display pages can be significant. This results in some business type display pages having lower visitor volumes and less historical user behavior data, such as clicks, orders, and favorites. Consequently, it becomes difficult to effectively analyze and uncover user preferences, thus reducing the effectiveness of the recommendation system.

[0057] To address the aforementioned issues, embodiments of this application provide a training method for an information recommendation model based on transfer learning. This method uses historical user behavior data obtained from display pages corresponding to high-traffic business types as historical behavior data to analyze user preferences for services corresponding to low-traffic business types. By solving the problem of insufficient historical behavior data obtained from historical pages corresponding to low-traffic business types, this method supports the training of an information recommendation model for that business type that aligns with user preferences.

[0058] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0059] Figure 2 This is a flowchart illustrating one model training method described in this specification, which specifically includes the following steps:

[0060] S100: Obtain historical recommendation information contained in the historical pages of the specified business type as the first training sample.

[0061] Generally, a business platform can determine the recommendation order for each piece of information to be recommended using an information recommendation model, and then display the information to be recommended to the user in the sorted order. This specification provides a method for training an information recommendation model corresponding to a target business type, which can be executed by a server used for training the model.

[0062] The page for a specified business type is used to display recommended information for services of that specified business type to users, while the page for a target business type is used to display recommended information for services of that target business type to users.

[0063] In the method for training an information recommendation model based on transfer learning used in the embodiments of this application, since the amount of historical behavior data of users regarding each historical recommendation information obtained from the historical pages of a specified business type is large, while the amount of historical behavior data of users regarding each historical recommendation information obtained from the historical pages of a target business type is small, the similarity between the services corresponding to the specified business type provided on the display page of the specified business type and the services of the target business type provided on the display page of the target business type is utilized. The first training sample obtained from the historical pages of the specified business type is then applied to the process of training an information recommendation model for ranking the various recommendation information providing services of the target business type. The first training sample may include each historical recommendation information contained in the historical pages of the specified business type. Each historical recommendation information may be unsorted or sorted according to a certain sorting order, wherein the sorting order of each historical recommendation information may be determined by the pre-trained information recommendation model for the specified business type. Optionally, the first training sample may also include user characteristics, such as user age, gender, geographical location, and other features that reflect user attributes, and may also include contextual features, such as the time and duration of the user's browsing of the historical pages of the specified business type.

[0064] The relationship between the specified business type and the target business type can be hierarchical, such as a food type and a dessert type; it can also be business types at the same level under the same major business type category, such as a dessert type and a fast food type under the food type; or it can be business types belonging to different major business type categories, such as a guesthouse type under the hotel type and an exhibition type under the tourism type. The relationship between the specified business type and the target business type, as well as their specific types, can be determined based on the specific application scenario, and this specification does not limit this in the embodiments.

[0065] In addition, in the transfer learning method of using the first training sample obtained from the historical page of a specified business type to train the information recommendation model of the target business type, multiple specified business types can be selected, and the first training sample can be obtained from the historical page of each specified business type. In the embodiments of this specification, only one specified business type is used as an example to illustrate the specific scheme.

[0066] S102: Based on the first training sample, determine the suitability of the first training sample for training the information recommendation model corresponding to the target business type.

[0067] In practical applications, directly training a recommendation model for a target business type using first training samples obtained from historical pages of a specified business type may result in negative transfer. Negative transfer refers to the negative impact of some training samples in the first training sample on the training of the recommendation model for the target business type. This negative impact causes the recommendation performance of the trained model for the target business type to deviate from the user's preference for that service. Therefore, it is crucial to determine the suitability of the first training samples for training the recommendation model for the target business type, and based on the determined suitability, remove training samples that may lead to negative transfer, ensuring that the trained recommendation model for the target business type better aligns with user preferences.

[0068] S104: Determine the target training sample based on the fitness and the first training sample.

[0069] Normally, business platforms recommend information to users based on their historical behavior data and by mining user preferences. However, in certain cold start scenarios, it's necessary to analyze user interests and provide personalized recommendations when there's no historical user behavior data related to the recommended information in that scenario. Cold start scenarios here generally refer to the cold start of a business type. For example, a business platform adds a service of business type B. Obviously, the platform has never exposed the display page for business type B to users, so there's no historical user interaction with the recommended information contained in the display page for business type B. This makes it impossible to specifically analyze user preferences for the recommended information contained in the display page for business type B.

[0070] For cold start scenarios, in this step, a first training sample with a fit higher than the preset fit threshold can be selected as the target training sample. This allows the information recommendation model corresponding to the target business type to be trained using the first training sample that meets the fit condition obtained from the historical pages of the specified business type.

[0071] For scenarios where the amount of historical behavior data obtained from the historical pages of the target business type in step S100 is small, the historical recommendation information contained in the historical pages of the target business type can be used as a second training sample, together with the first training sample selected based on fit, as the target training sample. This is because although the amount of historical behavior data obtained from the historical pages corresponding to the target business type is small and insufficient to support training the information recommendation model corresponding to the target business type, the second training sample is obtained from the historical pages of the target business type. Obviously, the second training sample has a higher fit for training the information recommendation model corresponding to the target business type. Mixing the second training sample with the selected first training sample as the target training sample for training the information recommendation model corresponding to the target business type not only increases the sample size but also more closely reflects the user's actual preferences, thus improving the recommendation effect.

[0072] S106: Determine the actual recommendation effect representation value of the target training sample based on the user's operation results on the historical page of the specified business type containing the target training sample.

[0073] In this step, the actual recommendation effect representation value of the determined target training sample can also be regarded as the label of the target training sample. The purpose of the business platform in providing users with information recommendations for various business types is to place the recommended information that users prefer at the top of the recommendation sequence, in order to encourage users to take actions such as clicking on the recommended information. In different application scenarios, the actual recommendation effect representation value of the target training sample can be in various forms such as the click-through rate and conversion rate of the target training sample contained in historical pages of a specified business type within a user's historical time period. For example, if the number of times a historical page containing historical recommendation information 1 to 5 is exposed to a user is 5, and the user clicks on historical recommendation information 1 4 times in the 5 exposures, that is, the click-through rate is 0.8, the determined click-through rate of 0.8 can be used as the actual recommendation effect representation value of historical recommendation information 1.

[0074] Optionally, when the target training sample includes a second training sample obtained from the historical page of the target business type, the actual recommendation effect representation value corresponding to the target training sample can also be determined based on the user's operation result on the target training sample included in the historical page of the target business type.

[0075] S108: Input the target training sample into the information recommendation model corresponding to the target business type to be trained, and obtain the first predicted recommendation effect representation value of the target training sample output by the information recommendation model corresponding to the target business type to be trained. Train the information recommendation model corresponding to the target business type to be trained with the goal of minimizing the difference between the first predicted recommendation effect representation value and the actual recommendation effect representation value of the target training sample.

[0076] In practical applications, after determining the target training samples and their actual recommendation performance representation values, the server used to train the information recommendation model will train the information recommendation model corresponding to the target business type based on the target training samples and their actual recommendation performance representation values, with the goal of minimizing the difference between the first predicted recommendation performance representation value output by the model and the actual recommendation performance representation value of the target training samples. That is, when the specified business type is similar to the target business type, the recommendation performance of the information recommendation model corresponding to the target business type will conform to the user's preferences in the specified business type.

[0077] The above Figure 2 In the model training method shown, the first training sample is used to train the information recommendation model corresponding to the target business type to determine its fit. A target training sample is then selected from the first training sample, and the information recommendation model corresponding to the target business type is trained based on the target training sample and its corresponding actual recommendation performance representation value. It can be seen that by using the first training sample obtained from the historical pages of the specified business type as the training sample for the information recommendation model corresponding to the target business type based on the fit, sample transfer from the specified business type to the target business type is achieved. Even if the user historical behavior obtained from the historical pages of the target business type is limited, the information recommendation model corresponding to the target business type can still be trained based on the user historical behavior obtained from the historical pages of the specified business type, thereby improving the recommendation performance of the specified business type's pages.

[0078] In the embodiments described in this specification, such as Figure 2In step S102, when determining the suitability of the first training sample for training the information recommendation model corresponding to the target business type, the information recommendation model for the target business type to be trained can be modeled as a Markov Decision Process (MDP). Typically, based on the current state of each piece of information to be recommended, a recommendation sequence (action) is determined, and the display page containing the sorted information to be recommended is exposed to the user. The sorting method (strategy) of each piece of information to be recommended is adjusted according to the user's operation result (reward) on the display page containing the sorted information to be recommended, in order to ultimately obtain the maximum reward. The rewards in the above process can be determined based on the different results of the user's operation, combined with a reward function. For example, the reward function combines the platform commission and advertising billing in the business platform revenue. The platform commission is related to the conversion rate, that is, when the user places an order based on the recommended information on the display page. Advertising billing is related to the number of impressions, click-through rate, conversion rate, etc., such as cost per mille (CPM), cost per click (CPC), cost per sales (CPS), etc.

[0079] Optionally, rewards can also be presented in the form of returns, which typically refer to cumulative rewards. In this case, it could refer to the sum of the results of a user's actions on various historical recommendation information contained in multiple historical pages for a specified business type, and the same applies to the target business type. Additionally, there are cases where returns are discounted. Discounted returns apply a discount to potential future rewards; the further into the future the reward, the smaller the discount rate. Since the return is based on a discount rate of 1, in this embodiment, "return" refers to discounted returns.

[0080] The conventional method for determining rewards involves modeling the reward to obtain a value function and determining the expected value of this value function as the upper limit of future rewards, meaning that future rewards will not exceed the expected value of the value function. Specifically, in this case, each exposure of a historical page containing historical recommendation information is considered a decision step. The expected value of the N-step Return (NSR) function, determined by the user's historical actions over N decision steps, is used to judge the similarity between a service of a specified business type and a service of a target business type. However, in this process, the randomness in determining the sorting order of each piece of information to be recommended leads to randomness in the N-step Return function itself. Using only the expected NSR value to evaluate whether the first training sample can be used to train the information recommendation model corresponding to the target business type loses the complete random distribution information of NSR, potentially leading to an unreasonable determination of the target training sample due to an unreasonable fit.

[0081] Based on this, the embodiments of this application provide a method for determining fitness based on NSR distribution. To determine whether the obtained first training sample is suitable for training an information recommendation model corresponding to a target business type, for each first training sample, a specified reward distribution and a target reward distribution are determined respectively. Then, fitness is determined based on the specified reward distribution and the target reward distribution of the first training sample. The specified reward distribution is used to characterize the user's operation results on the first training samples contained in several historical pages of the specified business type, and the target reward distribution is used to characterize the user's operation results on the first training samples contained in several historical pages of the target business type. Figure 3 As shown, the specific steps include:

[0082] S200: For each first training sample, the first training sample is used as input to several pre-trained reward determination models corresponding to the specified business type, and the specified reward of the first training sample is obtained from the output of each pre-trained reward determination model corresponding to the specified business type.

[0083] In the embodiments of this specification, the idea of ​​ensemble learning is adopted to determine the specified reward of the first training sample output by the model using several rewards corresponding to a specified business type, thereby determining the specified reward distribution of the first training sample. The pre-trained reward determination model corresponding to the specified business type is the model for determining the NSR of the specified business type. It is trained based on historical recommendation information obtained from the historical pages of the specified business type, and the historical operation results of users on the historical recommendation information in the historical pages of the specified business type.

[0084] S202: Determine the specified reward of the first training sample output by the model based on the pre-trained rewards corresponding to the specified business type, and determine the specified reward distribution of the first training sample.

[0085] Generally, the designated reward of the first training sample output by the model is determined based on the pre-trained rewards corresponding to the designated business type. The mean and variance of each designated reward corresponding to the first training sample are calculated, and then the designated reward distribution of the first training sample is obtained.

[0086] S204: Using the first training sample as input, input several pre-trained reward determination models corresponding to the target business type to obtain the target reward of the first training sample output by each pre-trained reward determination model corresponding to the target business type.

[0087] In this step, the pre-trained reward determination model corresponding to the target business type is the model for determining the NSR of the target business type. It is trained based on the historical recommendation information obtained from the historical pages of the target business type, and the user's historical operation results on the historical recommendation information in the historical pages of the target business type. The method for determining the target reward of the first training sample is similar to step S200, and will not be repeated here.

[0088] S206: Determine the target reward of the first training sample output by the model based on the pre-trained rewards corresponding to the target business type, and determine the target reward distribution of the first training sample.

[0089] In this step, the method for determining the target reward distribution of the first training sample is similar to that in step S202, and will not be repeated here.

[0090] S208: Based on the target reward distribution of the first training sample and the specified reward distribution of the first training sample, determine the suitability of the first training sample for training the information recommendation model corresponding to the target business type.

[0091] In the embodiments of this specification, the fit of the first training sample for training the information recommendation model corresponding to the target business type can be defined as the similarity between the target reward distribution of the first training sample and the specified reward distribution of the first training sample. That is, if the operation result of the first training sample contained in the historical page of the specified business type is similar to the operation result of the first training sample contained in the historical page of the target business type, then the first training sample can be considered suitable for training the information recommendation model corresponding to the target business type.

[0092] The similarity between the target reward distribution and the specified reward distribution of the first training sample can be calculated using existing methods for calculating the similarity of distribution functions, such as Euclidean distance, Hellinger distance, and relative entropy. Taking relative entropy as an example, the formula for calculating the fit of the first training sample for training the information recommendation model corresponding to the target business type is as follows:

[0093] ω=1-KL(p S ||p T )

[0094] Where ω is the fit of the first training sample for training the information recommendation model corresponding to the target business type, and p T p is the target reward distribution of the first training sample. S The specified reward distribution for the first training sample.

[0095] In the embodiments described in this specification, such as Figure 2In step S108, during the training process of the information recommendation model corresponding to the target business type, which minimizes the difference between the first predicted recommendation effect representation value output by the model and the actual recommendation effect representation value of the target training sample, in addition to transferring the target training sample and its actual recommendation effect representation value obtained from the historical page of the specified business type to the training process of the information recommendation model of the target business type, a pre-trained information recommendation model of the specified business type can also be used to train the information recommendation model of the target business type. Specifically, this includes the following steps:

[0096] First, the target training sample is input into a pre-trained information recommendation model corresponding to a specified business type to obtain the second predicted recommendation effect representation value of the target training sample output by the information recommendation model corresponding to the specified business type.

[0097] In this step, since the information recommendation model corresponding to the specified business type is pre-trained using the first training sample and the historical operation results of the user on the historical page corresponding to the specified business type, the training of the information recommendation model corresponding to the target business type can be guided by the second predicted recommendation effect representation value corresponding to the output target training sample.

[0098] Optionally, in order to make the second predicted recommendation performance representation value better reflect user preferences, for each target training sample, the target training sample is input into the information recommendation model corresponding to the specified business type to obtain the second predicted recommendation performance representation value corresponding to the target training sample. Several samples with larger second predicted recommendation performance representation values ​​are selected from each target training sample as training samples for the information recommendation model corresponding to the actual training target business type to avoid overestimation.

[0099] Next, a first loss is determined based on the difference between the first predicted recommendation effect characterization value and the second predicted recommendation effect characterization value; a second loss is determined based on the difference between the actual predicted recommendation effect characterization value and the first predicted recommendation effect characterization value; the weight corresponding to the first loss is determined based on the fitness; and the total loss is determined based on the first loss, the weight corresponding to the first loss, and the second loss.

[0100] The first loss characterizes the correlation between the information recommendation model corresponding to the current target business type and the pre-trained information recommendation model corresponding to the specified business type. Taking the mean squared error calculation method as an example, the formula for calculating the first loss is as follows:

[0101] Loss1=(Q S -Q T ) 2

[0102] Where loss1 represents the first loss, Q T Q represents the performance characteristic of the first prediction recommendation. S This represents the performance value of the second prediction recommendation.

[0103] The weights corresponding to the first loss are determined based on the fit, where the fit represents the similarity between a service of a specified business type and a service of a target business type. A higher fit indicates greater guidance from the pre-trained information recommendation model for the specified business type for training the information recommendation model for the target business type. The second loss represents the degree to which the recommendation performance of the current information recommendation model for the target business type matches user preferences. It should be noted that the loss functions used to calculate the first and second losses can be any existing loss function, determined according to the specific application scenario; this application specification does not impose any limitations on this.

[0104] Based on the first loss, the weight corresponding to the first loss, and the second loss, the formula for determining the total loss is as follows:

[0105] Loss = ωLoss1 + Loss2

[0106] Where Loss represents the total loss, loss1 represents the first loss, loss2 represents the second loss, and ω represents the weight corresponding to the first loss.

[0107] Optionally, the weights corresponding to the first loss can also be determined by inputting the target training samples into several pre-trained reward determination models corresponding to the specified business type and several pre-trained reward determination models corresponding to the target business type, respectively, to obtain the specified reward distribution and the target reward distribution corresponding to the target training samples, and then determining the weights corresponding to the first loss based on the specified reward distribution and the target reward distribution corresponding to the target samples. At this time, the weights corresponding to the first loss can still represent the similarity between the service of the specified business type and the service of the target business type.

[0108] Then, with minimizing the total loss as the training objective, the information recommendation model corresponding to the target business type to be trained is trained.

[0109] By taking the minimization of the total loss as the training objective, it is possible to achieve the goal of using a pre-trained information recommendation model corresponding to a specified business type to guide the training process of an information recommendation model corresponding to a target business type, so that the recommendation effect of the information recommendation model corresponding to the target business type is similar to that of the information recommendation model corresponding to the specified business type. It is also possible to make the information recommendation model corresponding to the target business type more closely aligned with user preferences.

[0110] Alternatively, in an optional embodiment of this specification, the method utilizes a method based on... Figure 2The model training method trains a target business type information recommendation model that can sort the information to be recommended that provides services of the target business type to users, put the sorted recommendation sequence into the display page of the target business type, and expose it to users. This ensures that the information to be recommended that matches the user's preferences is ranked higher in the recommendation sequence, thereby improving the recommendation effect of the display page of the target business type. The specific steps are as follows:

[0111] First, the features of each piece of information to be recommended are obtained as input features.

[0112] In this step, the information to be recommended is typically information that can provide services of the target business type to the user. For example, if the target business type is supermarket delivery, the information to be recommended could be various supermarket merchants that can provide supermarket delivery services. In addition, user characteristics and contextual characteristics can be obtained and used together with the characteristics of the information to be recommended as input features. User characteristics may include features that reflect user attributes, such as user age, gender, and geographical location. Contextual characteristics may include statistical features such as the time and duration of the user's browsing history of pages related to the target business type.

[0113] Optionally, the information to be recommended can also be a pre-sorted queue of advertisements and a queue of natural results. The advertisement queue contains advertisements that advertisers place on the display page of the target business type through the business platform. The natural results queue can contain natural results that are automatically recommended to users based on their historical search records and user preferences.

[0114] Next, the input features are input into the trained information recommendation model corresponding to the target business type to obtain the recommendation sequence containing each recommendation information output by the trained information recommendation model corresponding to the target business type.

[0115] Then, the display page containing the recommended sequence of the target business type is returned to the user, so that the user can operate on the recommended information in the order of the recommended sequence contained in the display page of the target business type.

[0116] As can be seen, by using the first training samples obtained from the historical pages of the specified business type as training samples for the information recommendation model corresponding to the target business type based on adaptability, sample transfer from the specified business type to the target business type is achieved. Combined with the second predicted recommendation effect representation value output by the pre-trained information recommendation model corresponding to the specified business type, the information recommendation model corresponding to the target business type is trained. Even if there are few user historical behaviors obtained from the historical pages of the target business type, the information recommendation model corresponding to the target business type can still be trained based on the user historical behaviors obtained from the historical pages of the specified business type, thereby improving the recommendation effect of the specified business type pages.

[0117] The above describes one or more embodiments of the model training method provided in this specification. Based on the same idea, this specification also provides corresponding model training devices, such as... Figure 4 As shown.

[0118] Figure 4 A schematic diagram of a model training device provided in this specification specifically includes:

[0119] The first training sample acquisition module 300 is used to acquire historical recommendation information contained in the historical pages of a specified business type as the first training sample.

[0120] The adaptation determination module 302 is used to determine the adaptation of the first training sample to train the information recommendation model corresponding to the target business type based on the first training sample.

[0121] The target training sample determination module 304 is used to determine the target training sample based on the fitness and the first training sample.

[0122] The actual recommendation effect representation value determination module 306 is used to determine the actual recommendation effect representation value of the target training sample based on the user's operation results on the target training sample contained in the historical page of the specified business type.

[0123] The training module 308 is used to input the target training sample into the information recommendation model corresponding to the target business type to be trained, to obtain the first predicted recommendation effect representation value of the target training sample output by the information recommendation model corresponding to the target business type to be trained, and to train the information recommendation model corresponding to the target business type to be trained with the goal of minimizing the difference between the first predicted recommendation effect representation value and the actual recommendation effect representation value of the target training sample.

[0124] Optionally, the adaptation determination module 302 is specifically used to determine, for each first training sample, a specified reward distribution and a target reward distribution for that first training sample; the specified reward distribution is used to characterize the user's operation results for the first training samples contained in several historical pages of the specified business type; the target reward distribution is used to characterize the user's operation results for the first training samples contained in several historical pages of the target business type.

[0125] Based on the target reward distribution and the specified reward distribution of the first training sample, the suitability of the first training sample for training the information recommendation model corresponding to the target business type is determined.

[0126] Optionally, the fitness determination module 302 is specifically used to take the first training sample as input, input several pre-trained reward determination models corresponding to the specified business type, and obtain the specified reward of the first training sample output by each pre-trained reward determination model corresponding to the specified business type.

[0127] The specified reward of the first training sample output by the model is determined based on the pre-trained rewards corresponding to the specified business type, and the specified reward distribution of the first training sample is determined.

[0128] Optionally, the fit determination module 302 is specifically used to take the first training sample as input, input several pre-trained reward determination models corresponding to the target business type, and obtain the target reward of the first training sample output by each pre-trained reward determination model corresponding to the target business type.

[0129] The target reward of the first training sample output by the model is determined based on the pre-trained rewards corresponding to the target business type, and the target reward distribution of the first training sample is determined.

[0130] Optionally, the target training sample determination module 304 is specifically used to: obtain historical recommendation information contained in the historical pages of the target business type as the second training sample; and use the second training sample and each of the first training samples whose fit is higher than the preset fit threshold as the target training sample.

[0131] Optionally, before the training module 308 trains the information recommendation model corresponding to the target business type with the goal of minimizing the difference between the first predicted recommendation effect representation value and the actual recommendation effect representation value of the target training sample, the training module 308 is further configured to input the target training sample into the pre-trained information recommendation model corresponding to the specified business type to obtain the second predicted recommendation effect representation value of the target training sample output by the information recommendation model corresponding to the specified business type.

[0132] Optionally, the training module 308 is specifically used to determine the total loss based on the first predicted recommendation effect representation value, the second predicted recommendation effect representation value, and the actual recommendation effect representation value; and to train the information recommendation model corresponding to the target business type to be trained with the goal of minimizing the total loss.

[0133] Optionally, the training module 308 is specifically configured to: determine a first loss based on the difference between the first predicted recommendation performance characterization value and the second predicted recommendation performance characterization value; determine a second loss based on the difference between the actual predicted recommendation performance characterization value and the first predicted recommendation performance characterization value; determine the weight corresponding to the first loss based on the fitness; and determine the total loss based on the first loss, the weight corresponding to the first loss, and the second loss.

[0134] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 2 The provided model training method.

[0135] This instruction manual also provides Figure 5 The diagram shows a schematic structural representation of the electronic device. Figure 5 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 2 The model training method described herein. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0136] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0137] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0138] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0139] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0140] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0142] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0143] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0144] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0145] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0146] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0147] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0148] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0149] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0150] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0151] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A model training method, characterized in that, include: Obtain historical recommendation information contained in historical pages of a specified business type as the first training sample; Based on the first training sample, determine the suitability of the first training sample for training the information recommendation model corresponding to the target business type; Each first training sample whose fitness score is higher than the preset fitness score threshold is determined as the target training sample. Based on the user's operation results on the target training samples contained in the historical pages of the specified business type, the actual recommendation effect representation value of the target training samples is determined. The actual recommendation effect representation value is the click-through rate or conversion rate of the target training samples contained in the historical pages of the specified business type within the user's historical time period. The target training sample is input into the information recommendation model corresponding to the target business type to be trained, and the first predicted recommendation effect representation value of the target training sample is obtained by the information recommendation model corresponding to the target business type to be trained. The first predicted recommendation effect representation value is the predicted recommendation effect value output by the information recommendation model corresponding to the target business type to be trained. The training objective is to minimize the difference between the first predicted recommendation effect representation value of the target training sample and the actual recommendation effect representation value. The information recommendation model corresponding to the target business type to be trained is then trained.

2. The method of claim 1, wherein, Based on the first training sample, determine the suitability of the first training sample for training the information recommendation model corresponding to the target business type, specifically including: For each first training sample, determine the specified reward distribution and the target reward distribution for that first training sample. The specified reward distribution is used to characterize the user's operation results for the first training samples contained in several historical pages of the specified business type; The target reward distribution is used to characterize the user's operation results for the first training samples contained in several historical pages of the target business type; Based on the target reward distribution and the specified reward distribution of the first training sample, the suitability of the first training sample for training the information recommendation model corresponding to the target business type is determined.

3. The method of claim 2, wherein, For each first training sample, determine the specified reward distribution for that first training sample, specifically including: Using the first training sample as input, input several pre-trained reward determination models corresponding to the specified business type to obtain the specified reward of the first training sample output by each pre-trained reward determination model corresponding to the specified business type. The specified reward of the first training sample output by the model is determined based on the pre-trained rewards corresponding to the specified business type, and the specified reward distribution of the first training sample is determined.

4. The method of claim 2, wherein, For each first training sample, determine the target reward distribution for that first training sample, specifically including: Using the first training sample as input, input several pre-trained reward determination models corresponding to the target business type to obtain the target reward of the first training sample output by each pre-trained reward determination model corresponding to the target business type; and determine the target reward distribution of the first training sample based on the target rewards output by each pre-trained reward determination model corresponding to the target business type.

5. The method of claim 1, wherein, Based on the fitness level and the first training sample, the target training sample is determined, specifically including: Obtain historical recommendation information contained in the historical pages of the target business type as a second training sample; The second training sample and each of the first training samples whose fitness level is higher than the preset fitness level threshold are used as target training samples.

6. The method of claim 1, wherein, Before training the information recommendation model corresponding to the target business type with the training objective of minimizing the difference between the first predicted recommendation performance representation value and the actual recommendation performance representation value of the target training sample, the method further includes: The target training sample is input into a pre-trained information recommendation model corresponding to a specified business type to obtain the second predicted recommendation effect characterization value corresponding to the target training sample output by the information recommendation model corresponding to the specified business type. The information recommendation model corresponding to the target business type is trained with the goal of minimizing the difference between the first predicted recommendation effect representation value and the actual recommendation effect representation value of the target training sample. Specifically, this includes: determining the total loss based on the first predicted recommendation effect representation value, the second predicted recommendation effect representation value and the actual recommendation effect representation value. With minimizing the total loss as the training objective, an information recommendation model corresponding to the target business type to be trained is trained.

7. The method of claim 6, wherein, The total loss is determined based on the first predicted recommendation effect characterization value, the second predicted recommendation effect characterization value, and the actual recommendation effect characterization value, specifically including: determining the first loss based on the difference between the first predicted recommendation effect characterization value and the second predicted recommendation effect characterization value; The second loss is determined based on the difference between the actual predicted recommendation performance value and the first predicted recommendation performance value; the weight corresponding to the first loss is determined based on the fitness. The total loss is determined based on the first loss, the weight corresponding to the first loss, and the second loss.

8. A device for training a model according to any one of claims 1-7, characterized in that, include: The first training sample acquisition module is used to acquire historical recommendation information contained in the historical pages of a specified business type as the first training sample. The fit determination module is used to determine the fit of the first training sample for training the information recommendation model corresponding to the target business type based on the first training sample. The target training sample determination module is used to determine each first training sample with a fitness score higher than a preset fitness score threshold as a target training sample. The actual recommendation effect representation value determination module is used to determine the actual recommendation effect representation value of the target training sample based on the user's operation results on the target training sample contained in the historical page of the specified business type. The actual recommendation effect representation value is the click-through rate or conversion rate of the target training sample contained in the historical page of the specified business type within the user's historical time period. The training module is used to input the target training sample into the information recommendation model corresponding to the target business type to be trained, and obtain the first predicted recommendation effect representation value of the target training sample output by the information recommendation model corresponding to the target business type to be trained. The first predicted recommendation effect representation value is the predicted recommendation effect value output by the information recommendation model corresponding to the target business type to be trained. The training objective is to minimize the difference between the first predicted recommendation effect representation value and the actual recommendation effect representation value of the target training sample.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 7.

Images