Hyperparameter learning, intelligent recommendation, keyword and multimedia recommendation method and device

Abstract

The application discloses a kind of super parameter learning, intelligent recommendation, keyword recommendation method and device.The machine learning model super parameter learning method includes: obtaining the user feedback behavior data of machine learning model;According to the user behavior feedback data, from the preset super parameter continuous interval, determine the optimal super parameter subinterval where the super parameter combination of the machine learning model is located;In the optimal super parameter subinterval, the optimal super parameter combination is selected as the super parameter combination of the machine learning model.The application realizes the autonomous learning of the super parameter combination of the recommendation system, so that the recommendation system can track the needs and interests of users in real time, thereby ensuring that the recommendation system has good recommendation effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, in particular to a super parameter learning, intelligent recommendation, keyword and multimedia recommendation method and device. BACKGROUND

[0002] With the rapid development of the Internet, especially the development of mobile Internet, recommendation systems based on various scenarios play an increasingly important role.

[0003] In recent years, the core problem of the recommendation system is how to accurately capture the user's personalized interest in real time to achieve accurate recommendation services, such as recommending goods, multimedia, advertisements and product functions that the user may be interested in based on the user's preferences.

[0004] In order to solve the problem of personalized recommendation, the recommendation system often uses machine learning algorithms to describe the user's interest, such as various linear regression models, neural network models, etc. The super parameters that describe the properties of these models are often specified by the system builder before the learning process, which increases the workload of the model builder. More importantly, relying solely on manually setting model super parameters, if the super parameter setting is unreasonable, it will lead to the unreasonable of the recommendation model itself, and the recommendation system naturally cannot achieve better recommendation results. SUMMARY

[0005] In view of the above problems, the present application is proposed to provide a super parameter learning, intelligent recommendation, keyword recommendation method and device to overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect, the embodiments of the present application provide a machine learning model super parameter learning method, comprising:

[0007] Obtaining user feedback behavior data of a machine learning model;

[0008] According to the user behavior feedback data, determining an optimal super parameter sub-interval in which the super parameter combination of the machine learning model is located from a preset super parameter continuous interval;

[0009] Selecting an optimal super parameter combination in the optimal super parameter sub-interval as the super parameter combination of the machine learning model.

[0010] In an embodiment, according to the user feedback behavior data, determining the optimal super parameter sub-interval in which the super parameter combination is located, comprises:

[0011] Inputting the user feedback behavior data into the machine learning model;

[0012] The user feedback behavior data is used to traverse preset sub-intervals in the continuous interval, and the optimal hyperparameter sub-interval in which the hyperparameter combination of the machine learning model is located is determined.

[0013] In an embodiment, the preset continuous hyperparameter interval adopts a tree structure; a root node of the tree structure is the continuous interval, and child nodes are sub-intervals of the interval corresponding to parent nodes.

[0014] The user feedback behavior data is used to traverse preset sub-intervals in the continuous interval, and the optimal hyperparameter sub-interval in which the hyperparameter combination of the machine learning model is located is determined, specifically including:

[0015] For each node in the tree structure, starting from the root node, the user feedback behavior data is used to evaluate the revenue estimation value of each node at the same level, and according to the evaluation result, the next node to be traversed is selected, and the above steps are repeated until the leaf node is reached.

[0016] The leaf node is taken as the optimal hyperparameter sub-interval in which the hyperparameter combination of the machine learning model is located.

[0017] In an embodiment, after the optimal hyperparameter combination is selected in the optimal hyperparameter sub-interval, the method further includes:

[0018] In the tree structure, according to the start end and the end end of the sub-interval corresponding to the leaf node and the optimal hyperparameter combination, the leaf node is split into at least two child nodes.

[0019] Starting from the child nodes obtained after the splitting, the entire tree structure is traversed back, and the revenue estimation value of each node corresponding to the sub-interval is updated.

[0020] In an embodiment, after the optimal hyperparameter combination is selected in the optimal hyperparameter sub-interval, the method specifically includes:

[0021] One or more of a black box optimization algorithm, a grid search optimization, a Bayesian optimization, a random search optimization, and a gradient-based optimization are used to select the optimal hyperparameter combination contained in the optimal hyperparameter sub-interval.

[0022] In an embodiment, the preset continuous hyperparameter interval is obtained by the following method:

[0023] A user-input initial range of hyperparameters is received, and the initial range of hyperparameters is a continuous interval of hyperparameters.

[0024] After the optimal hyperparameter combination is selected, the method further includes:

[0025] outputting the optimal hyperparameter combination, and determining whether an adjustment instruction for the optimal hyperparameter combination is received;

[0026] if the adjustment instruction is received, adjusting the optimal hyperparameter combination according to the adjustment instruction and then outputting again.

[0027] In a second aspect, an embodiment of the present application provides a method for intelligent recommendation, comprising:

[0028] determining at least two candidate objects to be recommended;

[0029] obtaining a current optimal hyperparameter combination of a recommendation model;

[0030] scoring the at least two candidate objects by using the recommendation model and the current optimal hyperparameter combination;

[0031] recommending the candidate objects to a user based on the score values of the at least two candidate objects;

[0032] The optimal hyperparameter combination is obtained by using the machine learning model hyperparameter learning method as described above.

[0033] In an embodiment, the method for intelligent recommendation further comprises:

[0034] periodically collecting user feedback behavior data corresponding to the recommendation model;

[0035] updating features of the candidate objects by using the feedback behavior data, and inputting the feedback behavior data as training sample data into the recommendation model to retrain the recommendation model.

[0036] In an embodiment, if the optimal hyperparameter combination contains a hyperparameter affecting the structure of the recommendation model, the retraining of the recommendation model specifically comprises:

[0037] updating the structure of the recommendation model according to the hyperparameter affecting the structure of the recommendation model;

[0038] inputting the feedback behavior data as training sample data into the updated recommendation model to retrain the updated recommendation model.

[0039] In an embodiment, the determination of the at least two candidate objects to be recommended comprises one or a combination of the following:

[0040] determining at least two candidate objects matching the geographic location information from a candidate database according to the geographic location information of the user:

[0041] determine at least two candidate objects matching the attribute information from a candidate database according to the attribute information of the user;

[0042] determine at least two candidate objects matching the preference information from a candidate database according to the preference information of the user;

[0043] determine at least two candidate objects matching the preference information of the associated user from a candidate database according to the preference information of the user.

[0044] In a third aspect, an embodiment of the present application provides a search keyword recommendation method, comprising:

[0045] determining at least two candidate search keywords to be recommended;

[0046] obtaining a current optimal hyperparameter combination of a keyword recommendation model;

[0047] scoring at least two candidate search keywords by using the keyword recommendation model and the optimal hyperparameter combination;

[0048] selecting a preset number of candidate search keywords from the highest score value as recommended search keywords according to the order of the score values, and returning the recommended search keywords;

[0049] The optimal hyperparameter combination is obtained by using the machine learning model hyperparameter learning method.

[0050] In an embodiment, the optimal hyperparameter combination includes at least two of the following parameters: a Jaccard distance and an edit distance for limiting the number of recalled commodities under the same category, a similarity distance for filtering similar commodities, and a number of categories for displaying different categories of commodities.

[0051] In a fourth aspect, an embodiment of the present application provides a multimedia recommendation method, comprising:

[0052] determining at least two candidate multimedia types to be recommended;

[0053] obtaining a current optimal hyperparameter combination of a multimedia recommendation model;

[0054] scoring a user preference index of the candidate multimedia types by using the keyword recommendation model and the optimal hyperparameter combination;

[0055] selecting a preset number of multimedia types from the highest score value as recommended multimedia types according to the order of the score values of the preference index;

[0056] selecting at least one multimedia from a multimedia library corresponding to the recommended multimedia types and pushing the multimedia to a user.

[0057] The optimal hyperparameter combination is obtained by using the machine learning model hyperparameter learning method as described above.

[0058] In a fifth aspect, an embodiment of the present application provides a machine learning model hyperparameter learning device, comprising:

[0059] An acquisition module is configured to acquire user feedback behavior data of a recommendation model.

[0060] An interval determination module is configured to determine, from a preset hyperparameter continuous interval, an optimal hyperparameter subinterval in which a hyperparameter combination of the machine learning model is located, according to the user behavior feedback data.

[0061] A hyperparameter selection module is configured to select, from the optimal hyperparameter subinterval, an optimal hyperparameter combination as the hyperparameter of the machine learning model.

[0062] In a sixth aspect, an embodiment of the present application provides an intelligent recommendation device, comprising:

[0063] A determination module is configured to determine at least two candidate objects to be recommended.

[0064] A hyperparameter combination acquisition module is configured to acquire a current optimal hyperparameter combination of a recommendation model.

[0065] A scoring module is configured to score the at least two candidate objects by using the recommendation model and the current optimal hyperparameter combination.

[0066] A recommendation module is configured to recommend the candidate objects to a user based on the score values of the at least two candidate objects.

[0067] The optimal hyperparameter combination is obtained by using the machine learning model hyperparameter learning method as described above.

[0068] In a seventh aspect, an embodiment of the present application provides a search keyword recommendation device, comprising:

[0069] A candidate keyword determination module is configured to determine at least two candidate search keywords to be recommended.

[0070] A first acquisition module is configured to acquire a current optimal hyperparameter combination of a keyword recommendation model.

[0071] A first scoring module is configured to score the at least two candidate search keywords by using the keyword recommendation model and the optimal hyperparameter combination.

[0072] The search keyword recommendation module is configured to select a preset number of candidate search keywords with the highest score values in a descending order of the score values as recommended search keywords, and return the recommended search keywords.

[0073] The optimal hyperparameter combination is obtained by using the machine learning model hyperparameter learning method.

[0074] In an eighth aspect, an embodiment of the present application provides a multimedia recommendation device, comprising:

[0075] The determining module is configured to determine at least two candidate multimedia theme types to be recommended.

[0076] The second obtaining module is configured to obtain a current optimal hyperparameter combination of the multimedia recommendation model.

[0077] The second scoring module is configured to score user preference indexes of the candidate multimedia theme types by using the keyword recommendation model and the optimal hyperparameter combination.

[0078] The selecting module is configured to select a preset number of multimedia theme types with the highest score values in a descending order of the score values as recommended multimedia theme types.

[0079] The recommendation module is configured to select at least one multimedia from a multimedia library corresponding to the recommended multimedia theme types and push the multimedia to a user.

[0080] The optimal hyperparameter combination is obtained by using the machine learning model hyperparameter learning method.

[0081] In a ninth aspect, an embodiment of the present application provides a recommendation system, comprising:

[0082] The feedback collection module is configured to collect user feedback behavior data of the recommendation model.

[0083] The recommendation model module is configured to train the recommendation model by using the user feedback behavior data as a sample, select at least two candidate objects from a candidate pool, score the candidate objects by using the recommendation model and a hyperparameter combination of the recommendation model, and return a recommendation result according to a score result.

[0084] The candidate pool is configured to store data of candidate objects to be recommended.

[0085] The online learning module is configured to determine an optimal hyperparameter subinterval of the hyperparameter combination of the machine learning model from a preset hyperparameter continuous interval according to the user feedback behavior data collected by the feedback collection module, and output the hyperparameter combination of the recommendation model to the recommendation model module.

[0086] An hyperparameter optimization module is configured to select an optimal hyperparameter combination as the hyperparameter combination of the recommendation model in the optimal hyperparameter subinterval and return to the online learning module.

[0087] In a tenth aspect, the information processing device provided by the embodiments of the present application includes a memory and a processor, wherein the memory stores a computer program, and the program, when executed by the processor, can implement the machine learning model hyperparameter learning method, the intelligent recommendation method, the search keyword recommendation method, or the multimedia recommendation method.

[0088] In a ninth aspect, the computer readable storage medium provided by the embodiments of the present application stores computer instructions, and the instructions, when executed by a processor, can implement the machine learning model hyperparameter learning method, the intelligent recommendation method, the search keyword recommendation method, or the multimedia recommendation method.

[0089] The above technical solutions provided by the embodiments of the present application have at least the following beneficial effects:

[0090] The hyperparameter learning, intelligent recommendation, keyword and multimedia recommendation method and device provided by the embodiments of the present application can obtain an optimal hyperparameter subinterval through exploration, and then determine an optimal hyperparameter combination of a recommendation model in the optimal hyperparameter subinterval, thereby realizing autonomous learning of the hyperparameter combination of the recommendation system, enabling the recommendation system to track the needs and interests of users in real time, thereby ensuring that the recommendation system has a good recommendation effect, and the use of hyperparameter self-learning can also greatly reduce the workload of model builders.

[0091] Further, in the embodiments of the present application, the bandit algorithm is combined with the hyperparameter optimization method, the bandit algorithm is used to find an optimal hyperparameter subinterval of hyperparameters, and then the hyperparameter optimization method is used to select an extreme point in the subinterval to determine the optimal hyperparameter combination, which not only reduces the size of the search interval of the hyperparameter optimization algorithm and the system overhead, but also improves the accuracy of the optimal hyperparameter combination, and the overall efficiency is high. The bandit algorithm can well balance exploration and utilization, has good exploration ability for unknown situations, and can well utilize the results of previous exploration to select a candidate object with the highest cumulative income as the final recommended object, thereby achieving a good recommendation effect.

[0092] Other features and advantages of the present application will be set forth in the following specification, and in part will be apparent from the description, or can be learned by practice of the application. The objects and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

[0093] The technical solutions of the present application are described in detail below with the help of the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0094] The accompanying drawings are included to provide a further understanding of the present application and are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and explain the principles of the present application, and are not intended to limit the present application. In the drawings:

[0095] Figure 1 Flow chart of the machine learning model hyperparameter learning method provided for the embodiment of the present application;

[0096] Figure 2 Schematic diagram of the tree structure of the continuous interval provided for the embodiment of the present application;

[0097] Figure 3 Another schematic diagram of the tree structure provided for the embodiment of the present application;

[0098] Figure 4 Flow chart of the intelligent recommendation method provided for the embodiment of the present application;

[0099] Figure 5A Flow chart of the search keyword recommendation method provided for the embodiment of the present application;

[0100] Figure 5B Flow chart of the multimedia recommendation method provided for the embodiment of the present application;

[0101] Figure 6 Architectural schematic diagram of the recommendation system provided for the embodiment one of the present application;

[0102] Figure 7 Workflow diagram of the recommendation model module provided for the embodiment one of the present application;

[0103] Figure 8 Workflow diagram of the online learning module provided for the embodiment one of the present application;

[0104] Figure 9 Schematic diagram of the traversal path provided for the embodiment one of the present application;

[0105] Figure 10 Workflow diagram of the hyperparameter optimization module provided for the embodiment one of the present application;

[0106] Figure 11A workflow diagram of the feedback collection module provided for the first embodiment of the present application;

[0107] Figure 12 A workflow diagram of the candidate pool module provided for the first embodiment of the present application;

[0108] Figure 13 A schematic diagram of the search interface provided for the second embodiment of the present application;

[0109] Figure 14 A schematic diagram of the architecture of the recommendation system provided for the second embodiment of the present application;

[0110] Figure 15 A structural schematic diagram of the learning device of the machine learning model hyperparameters provided for the embodiments of the present application;

[0111] Figure 16 A structural schematic diagram of the intelligent recommendation device provided for the embodiments of the present application;

[0112] Figure 17 A structural schematic diagram of the recommendation device of the search keywords provided for the embodiments of the present application;

[0113] Figure 18 A structural schematic diagram of the recommendation device of the multimedia provided for the embodiments of the present application;

[0114] Figure 19 A structural schematic diagram of the recommendation system provided for the embodiments of the present application. DETAILED DESCRIPTION

[0115] Exemplary embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure can be embodied in various forms without being limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0116] In order to solve the problem of personalized recommendation, the recommendation system often uses machine learning algorithms to characterize the interests and hobbies of users, such as linear regression models, deep learning models, etc. No matter what type of machine learning algorithm is used, there will be two types of parameters, namely model parameters and hyperparameters. Model parameters are used to characterize the relationship between training data and labels, while hyperparameters are used to characterize the properties of the model. Since hyperparameters are used to characterize the model, they are often pre-set by model designers in the prior art.

[0117] The machine learning model is essentially a calculation process in which a machine learns data through an algorithm. The model obtained through learning is essentially a series of numbers, such as the numbers on each node of a tree model that determine whether to belong to the left or right child, or a one-dimensional array in a logistic regression model. These parameters are referred to as model parameters, and hyperparameters are parameters that define model properties or define the training process.

[0118] Taking a deep learning model as an example, the model parameters include parameters corresponding to each neuron. These neuron corresponding values depict the relationship between the input sample and the output label. The hyperparameters include, for example, the number of layers of the neural network and the number of neurons in each layer of the neural network. Obviously, the selection of hyperparameters has a great influence on the final effect of the model. For a complete set of recommendation systems, many machine learning models are often involved, and each model includes multiple hyperparameters. If all these hyperparameters need to be specified by the system builder, the workload of the system builder is greatly increased. Moreover, the setting of the hyperparameters has a great influence on the final effect of the model. If the system builder only presets the hyperparameters of the model, the recommendation system is difficult to achieve an optimal effect.

[0119] To solve the problem that the recommendation system is difficult to achieve an optimal effect due to the setting mode of the model hyperparameters of the recommendation system in the prior art, the embodiment of the present application provides a machine learning model hyperparameter learning method, an intelligent recommendation method, a search keyword recommendation method and device, and a recommendation system. The machine learning model hyperparameter learning method can explore a set of optimal hyperparameter combinations through self-learning, thereby greatly improving the effect of the recommendation system.

[0120] The specific implementation of the machine learning model hyperparameter learning method, the intelligent recommendation method, the search keyword recommendation method and device, and the recommendation system provided by the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0121] The machine learning model hyperparameter learning method provided by the embodiment of the present application, with reference to Figure 1 includes the following steps:

[0122] S11, obtaining user feedback behavior data of a machine learning model;

[0123] S12, determining an optimal hyperparameter sub-interval in which a hyperparameter combination of the machine learning model is located, from a preset hyperparameter continuous interval, according to the user behavior feedback data;

[0124] S13, selecting an optimal hyperparameter combination in the optimal hyperparameter sub-interval as the hyperparameter combination of the machine learning model.

[0125] The machine learning model provided by the embodiment of the present application can be applied to machine learning models in various scenarios, for example: linear regression model, deep learning model, various neural network models, graph learning model, etc. The embodiment of the present application is not limited to the machine learning model used by the recommendation system.

[0126] After the machine learning model outputs the result, for example, the recommendation model, after outputting the recommendation result, feedback data of the user on the recommendation result can be received. These data can reflect the implicit relationship between the recommended object and the user preference or hobby, and are used to further correct the accuracy of the recommendation model. These user feedback behavior data are related to specific application scenarios.

[0127] For example, the recommendation system, these machine learning models can be applied to the following recommendation scenarios, but are not limited to the following scenarios:

[0128] 1) recommendation of goods:

[0129] In the field of e-commerce, merchants will publish many goods on websites or APPs, and each user has his own unique preferences. Therefore, the recommendation system needs to consider the user's interest, select a suitable good from the large number of goods published by the merchant, and recommend it to the user, so as to meet the user's interest and hobby, and thus improve the transaction conversion rate of the goods.

[0130] In the recommendation scenario of goods, user feedback behavior data may be, for example, behavior log data, such as log data related to behaviors such as browsing a certain good, clicking on a certain good, purchasing a certain good, and collecting a certain good.

[0131] For example, the user clicks on a good, which generates behavior log data in the following format:

[0132] 2019-08-16 15:03:10\t u222\t 7b93ba75-4fdc-4116-8e3e-5a0e81ce42e6\ti1111\t click

[0133] The log data contains multiple parts, which are separated by \t. The first part indicates the time of the behavior, the second part is the ID of the user, the third part is the unique identifier of the log, the fourth part is the ID of the good clicked by the user, and the fifth part is the type of behavior ( "click" indicates that the user clicks on the good).

[0134] 2) recommendation of video material pictures:

[0135] With the development of mobile internet, more and more users use fragmented time to watch videos through mobile terminals. For example, for a video APP, there are many popular TV series or movies, and these video contents often have many recommended pictures, which may be designed by designers or may come from screenshots of the video contents. For the same video, which picture is selected as the cover picture may have different answers for different users, so the recommendation system needs to recommend the most favorite picture as the cover picture for different users.

[0136] In the recommendation scenario of the video material library, the user feedback data can also come from the user behavior log data, such as the time length, frequency, and viewing times of the user watching a certain video or a certain type of video.

[0137] 3) Function and service recommendation:

[0138] Taking the data service access of the mobile terminal on the Internet as an example, there are usually many data access methods, such as 3G, 4G, Wifi, etc. At the same time, there are usually many ways to access the Internet for data transmission. Different users may have different preferences and different use environments, so personalized recommendation is needed.

[0139] Similarly, in the function and service recommendation scenario, the user feedback data can be, for example, the time length, frequency, and times of the user using a certain function and / or service, or the time, times, and frequency of the function and / or service switching, and the like.

[0140] In the above step S12, the optimal hyperparameter subinterval in which the hyperparameter combination is located is determined according to the user feedback behavior data. In specific implementation, for example, the following methods can be used:

[0141] The user feedback behavior data is input into the machine learning model; the user feedback behavior data is used to traverse the subintervals with gradually reduced ranges in the preset continuous hyperparameter interval, until the optimal hyperparameter subinterval in which the hyperparameter combination of the machine learning model is located is determined.

[0142] Specifically, the above preset continuous hyperparameter interval is stored in a tree structure, and the root node of the tree structure is the continuous interval itself. The child nodes between different levels are the subintervals of the corresponding interval of the parent node;

[0143] For example, the continuous hyperparameter interval is assumed to be (s, e), and s and e are the upper and lower limits of the continuous interval, respectively. The interval corresponding to the root node is (s, e), and the subintervals corresponding to the next level child nodes of the root node are (s, y) and (y, e), respectively. The intervals of the nodes at the lower level are similar.

[0144] inputting the user feedback behavior data into the machine learning model, for each node in the tree structure, starting from the root node, using the user feedback behavior data to evaluate the revenue estimation value of each node in the same layer (for example, using the upper bound method of the confidence interval, etc.), and according to the evaluation result, selecting the next node to be traversed, repeating the above steps until reaching the leaf node;

[0145] the leaf node is the optimal hyperparameter subinterval in which the hyperparameter combination of the machine learning model is located.

[0146] As can be seen from the above process, the process of determining the optimal hyperparameter subinterval can be regarded as a process of continuously narrowing the range in the entire continuous interval. In this process, each step of narrowing the range is a process of selecting one of the two subintervals with a better revenue estimation value as the next reached subinterval according to the principle of optimal revenue estimation value. Therefore, only the next level of subnodes to be traversed need to be compared in terms of revenue to select the optimal hyperparameter subinterval. The stopping condition of the repeated process can be various, such as reaching a preset time or reaching a stable confidence interval (convergence, such as reaching the leaf node). The embodiments of the present application do not limit this.

[0147] In the embodiments of the present application, the revenue estimation value refers to the expected revenue brought by the selected subinterval, that is, the quantification of the recommendation effect. For different application scenarios, the meaning of revenue is related to the recommendation purpose of the recommendation model itself. For example, for a search engine, the measurement standard of whether the recommendation is successful or not is whether the user gets the content he wants, so the meaning of "revenue" is the click revenue (number of clicks, etc.) brought by the keyword. For example, in the scenario of commodity recommendation, the specific meaning of "revenue" may include, for example, the number of commodity views, the number of visitor purchases, the amount, etc. Here, they are not listed one by one.

[0148] In the embodiments of the present application, the real-time feedback user feedback behavior data is input into the machine learning model, and the Multi-armed bandit (MAB) algorithm is used to realize the maximization of cumulative revenue by balancing exploration and utilization.

[0149] The hyperparameter continuous interval is an initial search interval of a preset hyperparameter. It can be preset by the constructor of the machine learning model or obtained by preliminary processing through other optimization methods after being preset by the constructor. The hyperparameter continuous interval is an interval of the hyperparameter preset by the model constructor for selecting the optimal hyperparameter combination. The interval is a continuous numerical interval. The optimal hyperparameter subinterval is a subinterval in the hyperparameter continuous interval, which is obtained by exploring the above-mentioned hyperparameter continuous interval through, for example, the value evaluation method.

[0150] The continuous interval of hyperparameters is stored in a tree structure, and a tree structure of a continuous interval is shown in FIG. 1. Figure 2 As shown in FIG. 1, each parent node contains two child nodes, and the sum of the two intervals corresponding to the two child nodes (hereinafter referred to as child nodes) is the child interval of the interval corresponding to the parent node (hereinafter referred to as parent node).

[0151] When traversing the entire tree structure, starting from the root node, compare the estimated return values of the left child node and the right child node. If the estimated return value of one child node is larger, continue to traverse the child node with the larger estimated return value. If the estimated return values of the two child nodes are equal, randomly select one child node according to equal probability, and repeat the above process until the leaf node of the entire tree structure is reached. The child interval corresponding to this leaf node is the current optimal hyperparameter child interval.

[0152] In one embodiment, after the optimal hyperparameter combination is selected in the optimal hyperparameter child interval in step S13, the method can further perform the following steps:

[0153] In the tree structure of the continuous interval, according to the start and end of the child interval corresponding to the leaf node and the optimal hyperparameter combination, the leaf node is split into at least two child nodes.

[0154] Starting from the child nodes obtained after splitting, traverse the entire tree structure back, and update the estimated return value of the child interval corresponding to each node.

[0155] In the original tree structure, the leaf node is the deepest level node (the farthest from the root node). After the optimal hyperparameter combination is selected, a new layer is added to the original structure, and the tree structure dynamically changes. As the machine learning system runs, the optimal hyperparameter combination is constantly selected in real time, and the depth of the entire tree structure becomes deeper and deeper. The child interval corresponding to the current leaf node becomes smaller and smaller, and the optimal hyperparameter combination is approached or converged. The process can be referred to in FIG. 2. Figure 3

[0156] Figure 3 In the example shown in FIG. 2, the nodes of the hyperparameter continuous interval stored in the tree structure are split and extended from bottom to top. The root node of this tree structure (i.e. Figure 3 ​The bottommost node represents the continuous interval of the hyperparameter itself. In the second layer, the root node splits into two nodes, which are the child nodes of the root node. The corresponding intervals are the sub-intervals of the root node. In the third layer, each node in the second layer continues to split into two nodes, and so on. After the optimal hyperparameter is selected each time, the tree structure will go deeper into the tree, generating new leaf nodes for use in the next search process for the optimal hyperparameter sub-interval, so that the interval corresponding to the leaf node is continuously reduced.

[0157] This method, which uses a tree structure to gradually narrow down the continuous interval until the optimal hyperparameter sub-interval is obtained, not only reduces the range of the search interval for hyperparameter combinations, but also improves the accuracy of finding the optimal hyperparameter combination, thereby improving the overall performance of the machine learning model.

[0158] In the above embodiments of the invention, a tree structure is used to divide continuous intervals, where child nodes are sub-intervals of their parent nodes. Furthermore, during the statistical data update process, the statistical data of leaf nodes is used to update all their ancestor nodes. The advantage of this tree structure is that parent nodes share the statistical data of all their child nodes, which can significantly accelerate the convergence speed of node statistical data.

[0159] In the above embodiments of the invention, for example, the Upper Confidence Bound (UCB) method can be used to evaluate the estimated value of the revenue corresponding to the sub-interval of each node in the tree structure. This method is a common value evaluation method in the bandit algorithm, which uses the upper limit of the confidence interval of the mean bonus to represent the estimated value of the revenue corresponding to the node. The formula is as follows:

[0160] Where μ i It represents the expected return, where n is the total number of attempts (rounds of trials). i The number of attempts is the maximum. As can be seen from the formula, the more attempts are made on i, the smaller the difference between its estimated value and the upper confidence bound, meaning the higher the confidence level. When the UCB method traverses the continuous interval of hyperparameters stored in the tree structure, it needs to calculate the upper confidence bounds of the two nodes at each level as the estimated revenue, and then select the node with the larger estimated revenue.

[0161] The method has the advantages of good balance between exploration and utilization, ability to maintain exploration for unknown conditions, and good utilization of previous exploration results, so that the cumulative income of the interval obtained by exploration is the highest. The bandit algorithm is adopted in the embodiment of the application, which can realize self-learning of the hyperparameter combination. For the recommendation model, the learning process can automatically and intelligently explore the unknown state, so that the hyperparameter combination learned can ensure that the recommendation model can more diversifiedly explore user interests and capture real-time changes in user interests, thereby achieving better recommendation effect.

[0162] Other existing algorithms such as Epsilon-Greedy algorithm and Thompson Sampling algorithm can also be used to evaluate the estimated income value, and the specific implementation can refer to the prior art, which will not be described in detail here.

[0163] In the above step S13, the optimal hyperparameter combination is selected in the optimal hyperparameter sub-interval. In specific implementation, one or more of black box optimization algorithm, grid search optimization, Bayesian optimization, random search optimization and gradient-based optimization can be used to select the optimal hyperparameter combination contained in the optimal hyperparameter sub-interval.

[0164] Taking the black box optimization algorithm as an example, the optimal hyperparameter sub-interval [X s ,X e ] obtained by the foregoing steps and part of the data samples {(x0,y0),...,(x n ,y n} are used as algorithm inputs, wherein the data samples satisfy f(x)=y, but the function f is unknown. The black box optimization algorithm is used to fit the unknown function f by using the data samples, so as to find the extreme point of the function f in the interval [X s ,X e ]. The extreme point is the optimal hyperparameter combination.

[0165] The black box optimization algorithm has wide adaptability and high universality. Moreover, the extreme point is searched in the foregoing determined optimal hyperparameter sub-interval, which greatly reduces the performance overhead compared with optimization on the whole interval. Moreover, for the recommendation model, the hyperparameter combination in the optimal hyperparameter sub-interval has better recommendation effect compared with other hyperparameter combinations, so that the recommendation effect is better when the extreme point is searched in this interval.

[0166] It should be noted that the hyperparameter combination composed of multiple hyperparameters is a point in a high-dimensional space, and the foregoing intervals and sub-intervals of the embodiment of the application are also high-dimensional spaces, so that the extreme point is determined, that is, the corresponding hyperparameter combination is found.

[0167] The partial data samples are generated according to the user feedback data.

[0168] For example, Bayesian optimization can generate a prior probability model from a set of initial observations (optimal hyperparameter subinterval) by a Gaussian process, search the next evaluation point by a chaotic particle swarm method, evaluate the new evaluation point to obtain the observation value corresponding to the new evaluation point, update the Gaussian process probability model by updating the observation set, and obtain the optimal hyperparameter combination by multiple iterations.

[0169] The above are only a few examples of implementing hyperparameter combination optimization, and other methods can refer to prior art, which will not be listed one by one.

[0170] On the basis of the machine learning model hyperparameter learning method, the embodiment of the application further provides an intelligent recommendation method, referring to Figure 4 as shown, comprising:

[0171] S41, determining at least two candidate objects to be recommended;

[0172] S42, obtaining a current optimal hyperparameter combination of the recommendation model;

[0173] S43, scoring the at least two candidate objects by the recommendation model and the current optimal hyperparameter combination;

[0174] S44, recommending the candidate objects to the user based on the score values of the at least two candidate objects;

[0175] The optimal hyperparameter combination can be obtained by the machine learning model hyperparameter learning method in the foregoing embodiments.

[0176] In the embodiment of the application, the candidate objects can be different according to different scenes suitable for the recommendation system, such as goods to be recommended, search keywords to be recommended, services to be recommended, friends to be recommended (in a social network), etc. The scenes suitable for the recommendation model are as described above, such as goods, services, multimedia, friends, etc. Various possible scenes will not be listed one by one.

[0177] For example, in a friend recommendation model in a social network, a plurality of types of candidate recommended friends to be recommended can be determined according to an actual application scenario, for example, the candidate recommended friends can be obtained from a user's address book, an association list, and the like. Then, the optimal hyperparameter combination obtained according to the foregoing method is input into the friend recommendation model, and the similarity between the candidate recommended friends and the user is scored by using user feedback data, for example, geographic location information of the user, behavior information (i.e., behavior feedback data) of the user on a product or a service, such as purchase, use, and the like, and a few candidate recommended friends with higher scores are selected as final recommended friends, and related friend information is pushed.

[0178] The recommendation model can also be one or more of a plurality of machine learning models, for example, a linear regression model, a deep learning model, various neural network models, a graph learning model, and the like. Embodiments of the present application do not limit this.

[0179] In the step S42, the optimal hyperparameter combination in the current optimal hyperparameter combination of the recommendation model is obtained. The hyperparameters can be obtained by referring to the hyperparameter learning method of the machine learning model, that is, according to the user feedback data of the recommendation model (i.e., the machine learning model), the optimal hyperparameter subinterval in which the recommendation model is located is first determined, and then the optimal hyperparameter combination of the recommendation model is selected in the optimal hyperparameter subinterval. The specific implementation manner can refer to the description of the foregoing embodiments.

[0180] In an embodiment, in the step S41, the at least two candidate objects to be recommended can be determined in a plurality of ways, for example, in any one or a combination of the following ways:

[0181] 1. According to the geographic location information of the user, at least two candidate objects matching the geographic location information are determined from the candidate database:

[0182] For example, according to the geographic region of the user, it is determined that the current user is located in Beijing, and objects in the candidate data that belong to the geographic region of Beijing are selected as candidate objects.

[0183] 2. According to the attribute information of the user, at least two candidate objects matching the attribute information are determined from the candidate database;

[0184] For example, according to the gender, age, and the like in the attribute information of the user, objects matching the gender and age, and the like are determined from the candidate database.

[0185] 3. According to the preference information of the user, at least two candidate objects matching the preference information are determined from the candidate database;

[0186] For example, the object matching the preference of the user is selected from the database by using the preference information of the user set in advance or the preference of the user obtained by learning in advance.

[0187] 4. The at least two candidate objects matching the preference information are determined from the candidate database according to the preference information of the associated user of the user.

[0188] The associated user of the user, for example, other users followed by the user in a social network, friends of the user, and the like.

[0189] The method for selecting the candidate object is only an example, and the embodiments of the present application are not limited to the specific implementation of how to select the candidate object.

[0190] In the embodiments of the present application, the user feedback behavior data corresponding to the recommendation model can also be collected periodically; the features of the candidate object are updated by using the user feedback behavior data, and the user feedback behavior data is input as the training sample data into the recommendation model to retrain the recommendation model.

[0191] The preference and interest of the user can change, so it is necessary to update the weight of the recommendation model in real time, and therefore, the user feedback behavior data is collected periodically, the features of the candidate object are updated by using the user feedback behavior data, and the user feedback behavior data is also used to retrain the recommendation model.

[0192] The retraining of the recommendation model is essentially the updating of the weight of each feature in the model. Because the interest of the user changes, it is necessary to update the weight in the model in real time. For example, if the user likes the red type related goods before, then the weight of the feature related to the red color in the recommendation model is high, and if the interest of the user changes and the user likes the blue type related goods, then the weight of the feature related to the blue color should be increased, and the embodiments of the present application can track the change of the interest of the user in real time by retraining the model to maintain the accuracy of the recommendation.

[0193] The updating of the features of the candidate object usually refers to the updating due to the change of some attributes of the candidate object, for example, the publishing duration of the goods, the price of the goods, the category to which the goods belong, and the like, and the updating of the features of the candidate object is not for tracking the change of the preference or interest of the user, but for real-time description of the attributes of the candidate object.

[0194] In an embodiment, if the optimal hyperparameter combination contains a hyperparameter that affects the structure of the recommendation model, the recommendation model can also be retrained according to the hyperparameter. Specifically, the structure of the recommendation model is updated according to the hyperparameter that affects the structure of the recommendation model; and the feedback behavior data is input as training sample data into the updated recommendation model to retrain the updated recommendation model.

[0195] For example, the number of layers of a deep neural network model is 9527 layers, which is a hyperparameter that affects the structure of the recommendation model. If the optimal hyperparameter combination contains such a parameter, the recommendation model can be updated and retrained according to the hyperparameter.

[0196] The embodiment of the present application also provides a recommendation method of a search keyword. Referring to the flowchart shown in FIG. 1, the method comprises the following steps: Figure 5A

[0197] S51, determining at least two candidate search keywords to be recommended;

[0198] For example, when a user clicks on a search box, the recommendation system will recommend some search keywords (query words) to the user. When the recommendation system is processed, a series of candidate search keywords to be recommended need to be determined in advance. The selection of these keywords can be based on one or more of the following: geographic information of the user, historical behavior data of the user, preference information set by the user, and the like.

[0199] S52, obtaining a current optimal hyperparameter combination of a keyword recommendation model;

[0200] The method for obtaining the current optimal hyperparameter combination of the keyword recommendation model is described in the foregoing embodiments.

[0201] In an embodiment, in the scenario of an e-commerce transaction system, the recommendation system can use the foregoing method to obtain the optimal hyperparameter combination currently required by the candidate search keyword model of a commodity. In an example of the hyperparameter combination, at least two of the following parameters are contained: a Jaccard distance and an edit distance for limiting the number of recalled commodities under the same category, a similarity distance for filtering similar commodities, and a number of categories for displaying different categories of commodities.

[0202] Of course, the combination of the above-mentioned hyperparameters is only an example, and the embodiment of the present application does not limit what specific hyperparameters are contained in the optimal hyperparameter combination. The system can be pre-set according to the needs.

[0203] S53, scoring at least two candidate search keywords by using the keyword recommendation model and the optimal hyperparameter combination; ​

[0204] S54, selecting preset number of candidate search terms starting from the highest score as recommended search keywords according to the order of the scores, and returning the recommended search keywords;

[0205] The optimal hyperparameter combination in step S52 is obtained by using the learning method of the hyperparameters of the machine learning model.

[0206] In another embodiment, the embodiment of the present application also provides a multimedia recommendation method, which refers to the flowchart shown in Figure 5B The method comprises the following steps:

[0207] S51', determining at least two candidate multimedia types to be recommended;

[0208] S52', obtaining a current optimal hyperparameter combination of a multimedia recommendation model;

[0209] S53', scoring the user preference indexes of the candidate multimedia types by using the multimedia keyword recommendation model and the optimal hyperparameter combination;

[0210] S54', selecting preset number of multimedia types starting from the highest score as recommended multimedia types according to the order of the scores of the preference indexes;

[0211] S55', selecting at least one multimedia from the multimedia library corresponding to the recommended multimedia types and pushing the multimedia to the user;

[0212] Similarly, the optimal hyperparameter combination can also be obtained by using the learning method of the hyperparameters of the machine learning model.

[0213] The multimedia types in the method can be subject types of multimedia, or format types of multimedia, or style types of multimedia, etc.

[0214] Taking the recommendation scenario of the video material picture as an example, the system can score the user preference indexes of the material pictures in the video material library according to the material picture recommendation model and the optimal hyperparameter combination, and select the material picture with the highest score as the material picture of the video to be recommended and push the material picture to the user.

[0215] In order to better illustrate the learning method of the hyperparameters, the intelligent recommendation method and the recommendation method of the search keywords provided by the embodiments of the present application, two specific embodiments of the recommendation system are described in detail below.

[0216] Embodiment one:

[0217] In the embodiment one, the architecture of the recommendation system refers to Figure 6As shown, in the system, there are a recommendation model module, a candidate pool module, a feedback collection module, an online learning module, and a hyperparameter optimization module.

[0218] In the above architecture, the candidate pool module and / or the feedback collection module can be integrated into the same module, and the hyperparameter optimization module can also be integrated into the online learning module. The above architecture diagram is only one of the many possible implementation manners, and each method of the embodiments of the present application can be implemented, and is not limited to the form in the above architecture diagram.

[0219] In the architecture of the recommendation system, the functions of each module are briefly described as follows:

[0220] The recommendation model module is used to train the recommendation model by taking the user feedback behavior data as samples, select at least two candidate objects from the candidate pool module, score the candidate objects by the recommendation model and the hyperparameter combination of the recommendation model, and return the recommendation result according to the scoring result.

[0221] The online learning module is used to determine the optimal hyperparameter subinterval in which the hyperparameter combination of the machine learning model is located from the preset continuous interval of hyperparameters according to the user feedback behavior data collected by the feedback collection module, and output the hyperparameter combination of the recommendation model to the recommendation model module.

[0222] For example, the module obtains the real-time feedback behavior data of the user from the feedback collection module, and explores and utilizes the subinterval in which the hyperparameter combination is located by using the Bandit algorithm, while exploring the optimal hyperparameter combination contained therein by using the hyperparameter optimization module, so that the entire recommendation system can find the optimal hyperparameter combination more quickly.

[0223] The hyperparameter optimization module is used to select the optimal hyperparameter combination as the hyperparameter combination of the recommendation model in the optimal hyperparameter subinterval, and return to the online learning module.

[0224] The hyperparameter optimization module has a hyperparameter optimization capability, which can select an optimal point (i.e., a specific hyperparameter combination) from the optimal hyperparameter subinterval, and then return it to the recommendation model for use by the online learning module.

[0225] The feedback collection module is used to collect the user feedback behavior data of the recommendation model.

[0226] After the feedback collection module collects the user's behavior feedback data, the behavior feedback data can be processed as necessary, parsed into a data format that the recommendation model and online learning module can support, etc., so as to be used by the recommendation model training and the online learning module. The processing process includes: data cleaning (removing incorrect or duplicate data), data normalization (for example, user behavior log data generated by different platforms can be different and needs to be normalized), etc.

[0227] The candidate pool module is used for storing data of candidate objects to be recommended;

[0228] In a specific implementation, the candidate pool module can be a database for storing recommended candidate objects. Some simple conditional query functions can be supported, so that the recommendation model can recall a specific candidate object set instead of the entire data set, thereby reducing the pressure of scoring and sorting the recommended objects and improving the overall performance of the recommendation system.

[0229] The working processes of the above-mentioned several modules will be described below.

[0230] The functions of the recommendation model module include: training the model according to the user behavior data, and scoring each candidate object by using the hyperparameter combination output by the online learning module as the hyperparameters of the recommendation model. Meanwhile, the module will update the features of the candidate objects and retrain the recommendation model according to the latest user feedback behavior data, so that the recommendation system can capture the changes in user interest in real time. The working process diagram of the recommendation model module is shown in Figure 7 .

[0231] Figure 7 The steps shown in

[0232] 1) Training the model; first, a recommendation model M is pre-trained by using historical data. In the embodiment of the present application, the selection of the model type is not limited and can be any machine learning model, such as: decision tree, logistic regression, deep neural network, etc.

[0233] 2) Scoring the candidate objects: at the current time t, the recommendation model M will predict a score S a .

[0234] 3) Judging whether there is user feedback behavior data: if there is feedback behavior data, the process continues, otherwise, the process is directly ended.

[0235] 4) Collecting user feedback behavior data; the module will use the output of the feedback collection module as the input, and after a period of time, the features of the candidate objects of the recommendation model will be updated and the recommendation model will be retrained.

[0236] The recommendation model module can fully utilize historical data to train the recommendation model, and there is no limitation on the type of model, which can support any machine learning or deep learning model, and has high universality. At the same time, the use of the recommendation model can greatly improve the recommendation accuracy of the system.

[0237] In addition, the features are updated by using the latest user feedback data, and the recommendation model is retrained, so that the recommendation model can capture the changes of user interest in real time.

[0238] The functions of the online learning module include: taking the real-time user feedback data output by the feedback collection module as input, and using the continuous interval bandit algorithm to explore the optimal hyperparameter sub-interval under the current state, and then using the hyperparameter optimization module to calculate an optimal value from the optimal hyperparameter sub-interval, and returning the value as the optimal hyperparameter to the recommendation model for model prediction scoring. The detailed process is shown in FIG. 8, including the following specific steps:

[0239] 1) Obtain user feedback behavior data. Obtain the current real-time user feedback behavior data from the feedback collection module, and process the data into a numerical type, denoted as Y.

[0240] 2) Update sub-interval data. The hyperparameter continuous interval and the sub-interval are stored in a tree structure, and the interval corresponding to the child node is the sub-interval of the interval corresponding to the parent node. For all nodes n on the path from the sub-interval (s n ,e n ) to the root node of the tree structure, update according to the following formula:

[0241] T n ←T n +1

[0242]

[0243] Update all nodes n in the tree structure according to the upper bound of the confidence interval:

[0244]

[0245] The above represents the average return of the sub-interval;

[0246] The above U n is the upper bound of the confidence interval of the return of the sub-interval, that is, the highest return value that the sub-interval can reach;

[0247] The above B n is the estimated return value of the sub-interval, which is calculated by the formula in step 6, and U nwhich takes more global information into account;

[0248] The above t is the current total number of experimental rounds, h is the depth of node n in the tree structure, T n represents the number of times the sub-interval corresponding to node n is selected.

[0249] 3) Evaluate the prediction. That is, traverse the entire tree structure according to the following rules to find the current optimal minimum sub-interval (corresponding to the leaf node of the tree structure): start from the root node of the tree (the current node n points to the root node), if B n|left >B n|right , then the node n points to the node n|left (where n|left and n|right represent the left child and right child of node n respectively), if B n|left <B n|right , then the node n points to the node n|right, if B n|left =B n|right , then randomly select a child node according to equal probability. Repeat the above process until the current node n points to a leaf node n t , then this leaf node n t corresponding sub-interval is the current optimal hyperparameter sub-interval (s n , e n ).

[0250] The specific traversal path is shown in Figure 9 , starting from the root node at the top and traversing layer by layer downwards, in the second layer, according to the size of the revenue estimate value of the sub-interval corresponding to the two nodes, the right node (the node marked B h,i in Figure 9 ) with the larger revenue estimate value is selected, and the traversal continues from this node to the third layer, and the revenue estimate values of the sub-intervals corresponding to the left node and the right node, i.e. B h+1，2i-1 and B h+1，2i , are compared again, and the left node (the node marked B h+1，2i-1 in Figure 9 ) with the larger revenue estimate value is selected, and so on, until the leaf node is reached, the sub-interval corresponding to this node is (H n , I n ), and finally the optimal point X n , i.e. the optimal hyperparameter combination, is selected from this (H n , I n ).

[0251] X nis a point in high dimensional space, representing a combination of hyperparameters, in the implementation, this point in high dimensional space can be represented by a matrix, for example, assuming that an optimal hyperparameter combination contains three hyperparameters, represented by matrix {[0.0, 0.5, 1.5]} and matrix {[1.0, 2.0, 3.0]}, 0 represents the start point of the interval corresponding to the first hyperparameter in the optimal hyperparameter combination, 1.0 represents the end point of the interval corresponding to the first hyperparameter, 0.5 represents the start point of the interval corresponding to the second hyperparameter in the optimal hyperparameter combination, 2.0 represents the end point of the interval corresponding to the second hyperparameter, and so on, 1.5 represents the start point of the interval corresponding to the third hyperparameter, and 3.0 represents the end point of the interval corresponding to the third hyperparameter.

[0252] 4) Output the optimal hyperparameter sub-interval to the hyperparameter optimization module. That is, output the optimal hyperparameter sub-interval selected in the evaluation and prediction step and the previously accumulated user behavior data to the hyperparameter optimization module.

[0253] 5) Obtain the optimal value from the hyperparameter optimization module. The hyperparameter optimization module uses a hyperparameter optimization method to select the optimal data value X from the optimal hyperparameter sub-interval and returns it to the online learning module.

[0254] 6) Update the internal state of the recommendation model: split the currently selected leaf node n t , add two child nodes n t |leftand n t |rightto the next layer, corresponding to intervals (s n ,X) and (X,e n ), respectively. And traverse the entire tree structure from the leaf node and update B n according to the following formula:

[0255] B n ←min{U n ,max{B n|left ,B n|right}}

[0256] In the online learning module, a tree structure is used to divide the continuous interval, and the child nodes are the sub-intervals of the parent nodes. At the same time, in the statistical data updating process, the statistical data of the leaf nodes is used to update all their ancestors (parent nodes, upper nodes of the parent nodes, etc.). The advantage of this tree structure is that the parent nodes share the statistical data of all child nodes, which greatly accelerates the convergence speed of the node statistical data.

[0257] Furthermore, the tree structure is constructed dynamically. As the system runs online, previous leaf nodes will continuously split into smaller sub-intervals, thus increasing the depth of the tree structure and making the sub-interval corresponding to the current leaf node smaller. The advantage of this dynamic splitting is that as the system continues to run online, the exploration interval for the optimal hyperparameters will gradually shrink, eventually converging to the optimal hyperparameter values.

[0258] This online learning module uses the upper bound of confidence intervals to evaluate each node in the tree structure. This method is a common value evaluation method in the Bandit algorithm. Its advantage is that it effectively balances exploration and utilization, allowing the system to maintain its ability to explore unknown situations while also making good use of previous exploration results, thus maximizing the system's cumulative reward. This embodiment uses the Bandit algorithm, enabling the recommendation system to have online learning capabilities and explore unknown states. This allows the recommendation system to explore user interests in a more diversified way and capture changes in user interests in real time.

[0259] The online learning module can directly output hyperparameter combinations for use by the recommendation model. For example, these hyperparameter combinations include: para_jaccard_distance (Jaccard distance, used to limit the number of recalled products within the same category), para_common_distance (edit distance, similar to Jaccard distance, also used to limit the number of recalled products within the same category), cate_distinct_cnt (similarity distance, used to filter similar products and increase the diversity of displayed products), and para_edit_distance (number of categories, the number of different categories of displayed products, used to control the diversity of displayed products).

[0260] The hyperparameter optimization module's functions include: the hyperparameter optimization module uses data range [X] s ,X e ] and partial data samples {(x0,y0),...,(x n ,y n The data samples are taken as input, where the data samples satisfy f(x) = y, and the function f is an unknown function. The hyperparameter optimization module uses a preset hyperparameter optimization algorithm to fit the unknown function f using the data samples, thereby achieving a fit within the interval [X]. s ,X e Find the extreme points of the function f in the [] Detailed process as follows Figure 10 As shown, it includes the following steps:

[0261] 1) Obtain the data interval: Obtain the current optimal hyperparameter sub-interval from the online learning module.

[0262] 2) Obtain data samples: Obtain user feedback data from the online learning module, and construct partial data samples.

[0263] 3) Calculate the optimal value: Calculate the extreme point in the current interval according to the black box optimization algorithm.

[0264] The above process can support any black box optimization algorithm, has high universality, and can be combined with specific problems to use the corresponding optimization algorithm, so that the system performance is optimal. On the other hand, only in the optimal hyperparameter sub-interval given by the online learning module, compared with optimizing in the whole interval, the performance overhead is also greatly reduced.

[0265] The functions of the feedback collection module include collecting user feedback behavior data, and performing corresponding format analysis on the data for the online learning module and the recommendation model. The specific process is shown in Figure 11 , which includes the following steps:

[0266] 1) Collect user feedback: Obtain user feedback behavior data from the front-end interaction page;

[0267] 2) Determine whether the user has feedback: If the user has feedback behavior, continue to execute, otherwise, end directly;

[0268] 3) Process feedback behavior data: Process the user feedback behavior data and parse it into a data format that can be supported by downstream modules (recommendation model and online learning module).

[0269] The above process can collect user feedback behavior in real time, and preprocess the data. The advantage of real-time processing data is that it can capture the changes of user interest points in time, thereby improving the overall performance of the recommendation system.

[0270] The candidate pool module contains a candidate pool, which is a database that supports conditional queries. For example, in the context of an e-commerce transaction system, if the user's geographic location is Beijing, the candidate pool can be used to obtain goods published in the geographic area of Beijing. The specific process is shown in Figure 12 , which includes the following steps:

[0271] 1) Query condition analysis;

[0272] 2) Obtain data that meets the conditions from the database.

[0273] The candidate pool module can support conditional queries, and the recommendation model can obtain specific candidate data sets through some combined conditional queries, which greatly reduces the pressure of the recommendation model scoring and sorting, and improves the overall performance of the system.

[0274] Embodiment two:

[0275] In embodiment two, a recommendation system for providing search keywords for a shopping website, when a user clicks on the webpage or APP of the shopping website, referring to the interface schematic diagram shown in the figure, the recommendation system will recommend some search keywords for the user, the purpose of recommending search keywords for the user can excavate the potential purchase demand of the user, increase the use stickiness of the user and improve the overall commodity transaction quantity, at the same time, save the browsing time of the user, and facilitate the user to quickly find the goods that the user is interested in and really needs. Figure 13

[0276] The schematic diagram of the architecture of the recommendation system is shown in the figure, the system comprises a feedback collection module, a model training module, a recommendation model, a candidate pool and an optimal hyperparameter combination learning module (equivalent to integrating an online learning module and an online learning module). Wherein: Figure 14 The model training module is used for training the recommendation model.

[0277] The recommendation model is used for real-time prediction of the demand of the user according to the trained model and recommendation of the recommended object with better effect, i.e. the Query word, to the user.

[0278] The optimal hyperparameter combination learning module is used for exploring the optimal hyperparameter subinterval by using the continuous interval Bandit algorithm, and then finding the optimal hyperparameter combination in the optimal hyperparameter subinterval by using the Bayesian optimization method, and outputting to the recommendation model.

[0279] The feedback collection module, for example, can be used as a real-time data processing platform based on visualization and componentization, is used for real-time processing of user feedback behavior data, and is used for the model training module and the optimal hyperparameter combination learning module.

[0280] The candidate pool provides data storage and data query functions, and can support multiple storage modes, for example, a graph storage mode can be used.

[0281] The hyperparameter learning, intelligent recommendation and keyword recommendation method provided by the embodiment of the application can improve the recommendation effect of the system, for example, in the scene of search keywords, compared with the prior art, the visitor number (UV) parameter can be improved by 5.68%, the commodity browsing quantity (PV) parameter can be improved by 5.53%, and the UV usage rate, guide pen number, guide amount and other parameters will be improved by two digits.

[0282]

[0283] ​​Based on the same inventive concept, the embodiments of the present application further provide a machine learning model hyperparameter learning device, an intelligent recommendation device, a search keyword recommendation device, a multimedia recommendation device, a recommendation system and an information processing equipment. Since the principles of the problems solved by the devices and the client are similar to the aforementioned hyperparameter learning method, intelligent recommendation method, search keyword recommendation method and multimedia recommendation method, the implementation of the device, system and equipment can be referred to the implementation of the aforementioned method, and the repeated parts will not be described here.

[0284] The machine learning model hyperparameter learning device provided by the embodiments of the present application, as shown in Figure 15 comprises:

[0285] The acquisition module 151 is configured to acquire user feedback behavior data of a recommendation model.

[0286] The interval determination module 152 is configured to determine, according to the user behavior feedback data, an optimal hyperparameter subinterval in which a hyperparameter combination of the machine learning model is located, from a preset hyperparameter continuous interval.

[0287] The hyperparameter selection module 153 is configured to select an optimal hyperparameter combination in the optimal hyperparameter subinterval as the hyperparameter of the machine learning model.

[0288] The intelligent recommendation device provided by the embodiments of the present application, as shown in Figure 16 comprises:

[0289] The determination module 161 is configured to determine at least two candidate objects to be recommended.

[0290] The hyperparameter combination acquisition module 162 is configured to acquire a current optimal hyperparameter combination of a recommendation model.

[0291] The scoring module 163 is configured to score the at least two candidate objects by using the recommendation model and the current optimal hyperparameter combination.

[0292] The recommendation module 164 is configured to recommend the candidate objects to a user based on the score values of the at least two candidate objects.

[0293] The optimal hyperparameter combination is obtained by using the aforementioned machine learning model hyperparameter learning method.

[0294] The search keyword recommendation device provided by the embodiments of the present application, as shown in Figure 17 comprises:

[0295] The candidate keyword determination module 171 is configured to determine at least two candidate search keywords to be recommended.

[0296] The first obtaining module 172 is configured to obtain a current optimal hyperparameter combination of a keyword recommendation model.

[0297] The first scoring module 173 is configured to score at least two candidate search keywords by using the keyword recommendation model and the optimal hyperparameter combination.

[0298] The search keyword recommendation module 174 is configured to select, in a high-to-low order of scores, a preset number of candidate search keywords with the highest scores as recommended search keywords, and return the recommended search keywords.

[0299] The optimal hyperparameter combination is obtained by using the learning method of the machine learning model hyperparameters.

[0300] The multimedia recommendation device provided by the embodiment of the application, as shown in Figure 18 , comprises:

[0301] The determining module 181 is configured to determine at least two candidate multimedia theme types to be recommended.

[0302] The second obtaining module 182 is configured to obtain a current optimal hyperparameter combination of a multimedia recommendation model.

[0303] The second scoring module 183 is configured to score user preference indexes of the candidate multimedia theme types by using the keyword recommendation model and the optimal hyperparameter combination.

[0304] The selecting module 184 is configured to select, in a high-to-low order of scores of the preference indexes, a preset number of multimedia theme types with the highest scores as recommended multimedia theme types.

[0305] The recommendation module 185 is configured to select at least one multimedia from a multimedia library corresponding to the recommended multimedia theme types and push the multimedia to a user.

[0306] The optimal hyperparameter combination can also be obtained by using the learning method of the machine learning model hyperparameters.

[0307] The recommendation system provided by the embodiment of the application, as shown in Figure 19 , comprises:

[0308] The feedback collecting module 191 is configured to collect user feedback behavior data of a recommendation model.

[0309] The recommendation model module 192 is configured to train the recommendation model by using the user feedback behavior data as samples, select at least two candidate objects from a candidate pool, score the candidate objects by using the recommendation model and a hyperparameter combination of the recommendation model, and return a recommendation result according to a score result.

[0310] a candidate pool 193, configured to store data of candidate objects to be recommended;

[0311] an online learning module 194, configured to determine, according to user feedback behavior data collected by the feedback collection module 191, an optimal hyperparameter subinterval in which a hyperparameter combination of the machine learning model is located, from preset hyperparameter continuous intervals; and output the hyperparameter combination of the recommendation model to the recommendation model module 192;

[0312] a hyperparameter optimization module 195, configured to select an optimal hyperparameter combination as the hyperparameter combination of the recommendation model in the optimal hyperparameter subinterval, and return to the online learning module 194.

[0313] The information processing device provided by the embodiment of the present application comprises a memory and a processor, wherein the memory stores a computer program, and the program can realize the machine learning model hyperparameter learning method, the intelligent recommendation method, or the search keyword recommendation method when executed by the processor.

[0314] The computer readable storage medium provided by the embodiment of the present application stores computer instructions, and the instructions can realize the machine learning model hyperparameter learning method, the intelligent recommendation method, or the search keyword recommendation method when executed by the processor.

[0315] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can be in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer usable program code.

[0316] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one flow or multiple flows and / or blocks Figure 1means for performing the function specified by the block or blocks.

[0317] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a Figure 1 one or more flowcharts and / or blocks Figure 1 one or more blocks.

[0318] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that execute on the computer or other programmable apparatus provide steps for implementing the flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks Figure 1 one or more blocks. Obviously, numerous modifications and variations are possible in accordance with the principles of the present application without departing from the scope of the present application. Accordingly, it is intended that the present application embrace all such modifications and variations as fall within the scope of the appended claims and their equivalents.

Claims

1. A method for learning hyperparameters of a machine learning model, characterized in that, include: Obtain user feedback behavior data for machine learning models; Based on the user feedback behavior data, the optimal hyperparameter sub-interval where the hyperparameter combination of the machine learning model is located is determined from the preset continuous hyperparameter interval; The preset hyperparameter continuous interval adopts a tree structure; the root node of the tree structure is the continuous interval, and the child nodes are the sub-intervals of the interval corresponding to the parent node; For each node in the tree structure, starting from the root node, the estimated revenue of each node at the same level is evaluated using the user feedback behavior data. Based on the evaluation results, the next node to be traversed is selected, and the above steps are repeated until the leaf node is reached. The leaf node is taken as the optimal hyperparameter subinterval where the hyperparameter combination of the machine learning model is located; Within the optimal hyperparameter sub-interval, the optimal hyperparameter combination is selected as the hyperparameter combination of the machine learning model; After selecting the optimal hyperparameter combination from the optimal hyperparameter sub-interval, in the tree structure, according to the start and end of the sub-interval corresponding to the leaf node and the optimal hyperparameter combination, the leaf node is split into at least two child nodes to increase the depth of the tree structure. Starting from the child nodes obtained after splitting, backtrack through the entire tree structure and update the estimated revenue of the sub-interval corresponding to each node.

2. The method as described in claim 1, characterized in that, Based on the user feedback behavior data, the optimal hyperparameter sub-interval where the hyperparameter combination lies is determined, including: Input user feedback behavior data into the machine learning model; Using the user feedback behavior data, the system iterates through the sub-intervals that are gradually narrowed within the preset continuous interval until the optimal hyperparameter sub-interval containing the hyperparameter combination of the machine learning model is determined.

3. The method as described in claim 1 or 2, characterized in that, Within the optimal hyperparameter sub-interval, the optimal hyperparameter combination is selected, specifically including: Using one or more of the following optimization algorithms—black-box optimization, grid search optimization, Bayesian optimization, stochastic search optimization, and gradient-based optimization—select the optimal combination of hyperparameters contained within the optimal hyperparameter sub-interval.

4. The method as described in claim 1 or 2, characterized in that, The preset hyperparameter continuous interval is obtained in the following way: Receive the initial range of hyperparameters input by the user, wherein the initial range of hyperparameters is a continuous interval of hyperparameters; After selecting the optimal combination of hyperparameters, the method further includes: Output the optimal hyperparameter combination and determine whether an adjustment instruction for the optimal hyperparameter combination has been received; If the adjustment instruction is received, the optimal hyperparameter combination is adjusted according to the adjustment instruction and then output again.

5. A method for intelligent recommendation, characterized in that, include: Identify at least two candidate entities to be recommended; Obtain the current optimal combination of hyperparameters for the recommendation model; The recommendation model and the current optimal hyperparameter combination are used to score the at least two candidate objects; Based on the rating values ​​of the at least two candidate objects, the candidate objects are recommended to the user; The optimal hyperparameter combination is obtained using the hyperparameter learning method of the machine learning model as described in any one of claims 1-4.

6. The method as described in claim 5, characterized in that, The determination of at least two candidate objects to be recommended includes one or more combinations of the following: Based on the user's geographic location information, at least two candidate objects matching the geographic location information are identified from the candidate database: Based on the user's attribute information, at least two candidate objects matching the attribute information are determined from the candidate database; Based on the user's preference information, at least two candidate objects that match the preference information are identified from the candidate database; Based on the user's associated user preference information, at least two candidate objects matching the preference information are identified from the candidate database.

7. A method for recommending search keywords, characterized in that, include: Identify at least two candidate search keywords to be recommended; Obtain the current optimal combination of hyperparameters for the keyword recommendation model; The keyword recommendation model and the optimal hyperparameter combination are used to score at least two candidate search keywords; Based on the ranking of the scores, a predetermined number of candidate search terms, starting from the highest score, are selected as recommended search keywords, and the recommended search keywords are returned. The optimal hyperparameter combination is obtained using the hyperparameter learning method of the machine learning model as described in any one of claims 1-4.

8. The method as described in claim 7, characterized in that, The optimal hyperparameter combination includes at least two of the following parameters: Jaccard distance and edit distance for limiting the number of recalled products under the same category, similarity distance for filtering similar products, and the number of different categories of products displayed.

9. A multimedia recommendation method, characterized in that, include: Identify at least two candidate multimedia types to be recommended; Obtain the current optimal hyperparameter combination for the multimedia recommendation model; The user preference index of the candidate multimedia types is scored using the multimedia recommendation model and the optimal hyperparameter combination. Based on the rating of the preference index, a preset number of multimedia types, starting from the highest rating, are selected as recommended multimedia types. Select at least one multimedia from the multimedia library corresponding to the recommended multimedia type and push it to the user; The optimal hyperparameter combination is obtained using the hyperparameter learning method of the machine learning model as described in any one of claims 1-4.

10. A learning device for hyperparameters of a machine learning model, characterized in that, include: The acquisition module is used to acquire user feedback behavior data for the recommendation model; The interval determination module is used to determine the optimal hyperparameter sub-interval containing the hyperparameter combination of the machine learning model from a preset continuous hyperparameter interval based on the user feedback behavior data. The preset continuous hyperparameter interval adopts a tree structure. The root node of the tree structure is the continuous interval, and the child nodes are the sub-intervals of the interval corresponding to the parent node. For each node in the tree structure, starting from the root node, the estimated revenue of each node at the same level is evaluated using the user feedback behavior data. Based on the evaluation result, the next node to be traversed is selected, and the above steps are repeated until a leaf node is reached. The leaf node is taken as the optimal hyperparameter sub-interval containing the hyperparameter combination of the machine learning model. After selecting the optimal hyperparameter combination in the optimal hyperparameter sub-interval, the leaf node is split into at least two child nodes in the tree structure according to the start and end of the sub-interval corresponding to the leaf node and the optimal hyperparameter combination. Starting from the child nodes obtained after splitting, the entire tree structure is traversed backtracking, and the estimated revenue of the sub-interval corresponding to each node is updated. The hyperparameter selection module is used to select the optimal combination of hyperparameters from the optimal hyperparameter sub-interval as the hyperparameters of the machine learning model.

11. An intelligent recommendation device, characterized in that, include: The determination module is used to identify at least two candidate objects to be recommended; The hyperparameter combination acquisition module is used to obtain the current optimal hyperparameter combination of the recommendation model; The scoring module is used to score the at least two candidate objects using the recommendation model and the current optimal hyperparameter combination. The recommendation module is used to recommend candidate objects to the user based on the rating values ​​of the at least two candidate objects; The optimal hyperparameter combination is obtained using the hyperparameter learning method of the machine learning model as described in any one of claims 1-4.

12. A recommendation device for search keywords, characterized in that, include: The candidate keyword determination module is used to determine at least two candidate search keywords to be recommended; The first acquisition module is used to acquire the current optimal hyperparameter combination of the keyword recommendation model; The first scoring module is used to score at least two candidate search keywords using the keyword recommendation model and the optimal hyperparameter combination. The search keyword recommendation module is used to select a preset number of candidate search terms, starting from the highest score, as recommended search keywords according to their scores, and return the recommended search keywords. The optimal hyperparameter combination is obtained using the hyperparameter learning method of the machine learning model as described in any one of claims 1-4.

13. A multimedia recommendation device, characterized in that, include: The determination module is used to identify at least two candidate multimedia topic types to be recommended; The second acquisition module is used to acquire the current optimal hyperparameter combination of the multimedia recommendation model; The second scoring module is used to score the user preference index of the candidate multimedia topic types using the multimedia recommendation model and the optimal hyperparameter combination. The selection module is used to select a preset number of multimedia theme types as recommended multimedia theme types, starting from the highest score, according to the score of the preference index. The recommendation module is used to select at least one multimedia from the multimedia library corresponding to the recommended multimedia topic type and push it to the user. The optimal hyperparameter combination is obtained using the hyperparameter learning method of the machine learning model as described in any one of claims 1-4.

14. A recommendation system, characterized in that, include: The feedback collection module is used to collect user feedback behavior data for the recommendation model; The recommendation model module is used to train the recommendation model using user feedback behavior data as samples, select at least two candidate objects from the candidate pool, score the candidate objects using the recommendation model and its hyperparameter combination, and return the recommendation result based on the score result. The candidate pool module is used to store data on candidate objects to be recommended; The online learning module is used to determine the optimal hyperparameter sub-interval of the hyperparameter combination of the recommendation model from the preset hyperparameter continuous interval based on the user feedback behavior data collected by the feedback collection module. And output the hyperparameter combination of the recommendation model to the recommendation model module; the preset continuous interval of hyperparameters adopts a tree structure; the root node of the tree structure is the continuous interval, and the child nodes are the sub-intervals of the interval corresponding to the parent node; The step of determining the optimal hyperparameter sub-interval where the hyperparameter combination of the recommendation model is located from the preset continuous hyperparameter interval includes: for each node in the tree structure, starting from the root node, using the user feedback behavior data, evaluating the revenue prediction value of each node in the same layer, and selecting the next node to be traversed based on the evaluation result, repeating the above steps until a leaf node is reached; taking the leaf node as the optimal hyperparameter sub-interval where the hyperparameter combination of the recommendation model is located; and, after selecting the optimal hyperparameter combination in the optimal hyperparameter sub-interval, splitting the leaf node into at least two child nodes in the tree structure according to the start and end of the sub-interval corresponding to the leaf node and the optimal hyperparameter combination, so as to increase the depth of the tree structure; starting from the child node obtained after splitting, backtracking through the entire tree structure, and updating the revenue prediction value of the sub-interval corresponding to each node; The hyperparameter optimization module is used to select the optimal hyperparameter combination as the hyperparameter combination of the recommendation model from the optimal hyperparameter sub-interval and return it to the online learning module.

15. An information processing device, characterized in that, include: A memory and a processor; wherein the memory stores a computer program, which, when executed by the processor, is capable of implementing the hyperparameter learning method of the machine learning model as described in any one of claims 1-4, or the intelligent recommendation method as described in claim 5 or 6, or the search keyword recommendation method as described in claim 7 or 8, or the multimedia recommendation method as described in claim 9.

16. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, enable the implementation of a machine learning model hyperparameter learning method as described in any one of claims 1-4, or an intelligent recommendation method as described in claim 5 or 6, or a search keyword recommendation method as described in claim 7 or 8, or a multimedia recommendation method as described in claim 9.

Images