Recommended methods and devices

Abstract

The embodiments of this specification provide a recommendation method and device, wherein the method includes first receiving the tth access request; obtaining the attribute characteristics of the user, the object characteristics of n candidate objects for the user, and the cross-combination characteristics of the user and the candidate objects based on the user identifier of the user in the access request to obtain a high-dimensional initial feature; then automatically encoding and reducing the dimension of the high-dimensional initial feature according to a first machine learning model to obtain a low-dimensional target feature with a much lower dimension than the high-dimensional initial feature; and then inputting the low-dimensional target feature into a second machine learning model to quickly and accurately obtain the prediction result of each candidate object.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and in particular to recommendation methods. One or more embodiments of this specification also relate to recommendation devices, video cover recommendation methods and devices, application recommendation methods and devices, product recommendation methods and devices, computing devices, and computer-readable storage media. Background Art

[0002] With the continuous development of Internet technology, how to accurately capture users' personalized interests in real time and provide users with accurate recommendation services (such as product recommendations and personalized material recommendations) has become a hot research project; although the existing offline recommendation model can adopt a complex model structure and learn users' long-term interests through rich historical data, it is difficult for the offline recommendation model to capture rapid changes in user interests in big promotion scenarios. Although the existing online recommendation model can continuously learn with the real-time feedback of the online system, it is difficult to select and determine features. Unreasonable feature selection will directly lead to poor prediction accuracy of the model. In addition, due to the requirement of real-time computing, the supported feature dimensions are usually small, resulting in poor accuracy.

[0003] Based on this, a more effective and accurate recommendation method needs to be provided. Summary of the invention

[0004] In view of this, the present specification provides a recommendation method. One or more embodiments of the present specification also relate to a recommendation device, a video cover recommendation method and device, an application recommendation method and device, a product recommendation method and device, four computing devices, and four computer-readable storage media to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a recommendation method is provided, including:

[0006] Receive a t-th access request, wherein the access request carries a user identifier of a user, and t is a positive integer;

[0007] Acquire initial features including m dimensions based on the user identifier;

[0008] According to a first machine learning model, which is an offline model, the initial features are reduced in dimension to obtain target features including d dimensions, wherein m and d are both positive integers, and m>>d, and the first machine learning model is trained based on the initialization parameters of the second machine learning model;

[0009] Inputting the target feature into the second machine learning model, which is an online learning model, to obtain a prediction result of the candidate object;

[0010] The candidate objects whose prediction results meet the preset conditions are recommended to the user.

[0011] According to a second aspect of an embodiment of this specification, a video cover recommendation method is provided, including:

[0012] receiving a recommendation request for a video cover, wherein the recommendation request carries a user identifier of a user;

[0013] Based on the user identifier, acquiring attribute features of the user, image features of n video images of the user, and cross-combination features of the user and the video images to obtain initial features including m dimensions;

[0014] Performing dimensionality reduction on the initial features according to the depth encoder to obtain target features including d dimensions, wherein m and d are both positive integers, and m>d;

[0015] Inputting the target feature into a contextual bandit model to obtain a prediction result of the video image;

[0016] The video image whose prediction result meets the preset condition is recommended to the user as a video cover.

[0017] According to a third aspect of an embodiment of this specification, a method for recommending an application is provided, including:

[0018] receiving a recommendation request for an application, wherein the recommendation request carries a user identifier of a user;

[0019] Based on the user identifier, acquiring attribute features of the user, attribute features of n applications for the user, and cross-combination features of the user and the applications to obtain initial features including m dimensions;

[0020] Performing dimensionality reduction on the initial features according to the depth encoder to obtain target features including d dimensions, wherein m and d are both positive integers, and m>d;

[0021] Inputting the target feature into a contextual bandit model to obtain a prediction result of the application;

[0022] An application program whose prediction result meets a preset condition is recommended to the user as a target application program.

[0023] According to a fourth aspect of the embodiments of this specification, a product recommendation method is provided, including:

[0024] receiving a recommendation request for a product, wherein the recommendation request carries a user identifier of a user;

[0025] Acquire attribute characteristics of the user, attribute characteristics of n commodities for the user, and cross-combination characteristics of the user and the commodities based on the user identifier to obtain initial characteristics including m dimensions;

[0026] Performing dimensionality reduction on the initial features according to the depth encoder to obtain target features including d dimensions, wherein m and d are both positive integers, and m>d;

[0027] Inputting the target feature into a contextual bandit model to obtain a prediction result of the product;

[0028] The commodities whose prediction results meet the preset conditions are recommended to the user.

[0029] According to a fifth aspect of the embodiments of this specification, a recommendation method is provided, including:

[0030] Displaying a target input interface for the caller based on a call request of the caller;

[0031] receiving a t-th access request generated by the caller through the target input interface, wherein the access request carries a user identifier of a user and n candidate objects for the user, and t is a positive integer;

[0032] Acquire initial features including m dimensions based on the user identifier and n candidate objects for the user;

[0033] Performing dimensionality reduction on the initial features according to the first machine learning model to obtain target features including d dimensions, wherein m and d are both positive integers, and m>d, and the first machine learning model is trained based on the initialization parameters of the second machine learning model;

[0034] Inputting the target feature into the second machine learning model to obtain a prediction result of the candidate object;

[0035] The candidate objects whose prediction results meet the preset conditions are recommended to the user through the caller.

[0036] According to a sixth aspect of the embodiments of this specification, a recommendation method is provided, including:

[0037] Receive the t-th access request sent by the caller, wherein the access request carries a user identifier of the user and n candidate objects for the user, and t is a positive integer;

[0038] Acquire initial features including m dimensions based on the user identifier and n candidate objects for the user;

[0039] Performing dimensionality reduction on the initial features according to the first machine learning model to obtain target features including d dimensions, wherein m and d are both positive integers, and m>d, and the first machine learning model is trained based on the initialization parameters of the second machine learning model;

[0040] Inputting the target feature into the second machine learning model to obtain a prediction result of the candidate object;

[0041] The candidate objects whose prediction results meet the preset conditions are recommended to the user through the caller.

[0042] According to a seventh aspect of the embodiments of this specification, a recommendation device is provided, including:

[0043] A first request receiving module is configured to receive a t-th access request, wherein the access request carries a user identifier of a user, and t is a positive integer;

[0044] A first high-dimensional feature acquisition module is configured to acquire initial features including m dimensions based on the user identifier;

[0045] A first low-dimensional feature acquisition module is configured to reduce the dimension of the initial feature according to a first machine learning model to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d, and the first machine learning model is trained based on the initialization parameters of the second machine learning model;

[0046] A first prediction module is configured to input the target feature into a second machine learning model to obtain a prediction result of the candidate object;

[0047] The first recommendation module is configured to recommend candidate objects whose prediction results meet preset conditions to the user.

[0048] According to an eighth aspect of the embodiments of this specification, a video cover recommendation device is provided, including:

[0049] A second request receiving module is configured to receive a recommendation request for a video cover, wherein the recommendation request carries a user identifier of a user;

[0050] A second high-dimensional feature acquisition module is configured to acquire attribute features of the user, image features of n video images of the user, and cross-combination features of the user and the video images based on the user identifier to obtain initial features including m dimensions;

[0051] A second low-dimensional feature acquisition module is configured to reduce the dimension of the initial feature according to the depth encoder to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d;

[0052] A second prediction module is configured to input the target feature into a contextual bandit model to obtain a prediction result of the video image;

[0053] The second recommendation module is configured to recommend the video image whose prediction result meets the preset condition as the video cover to the user.

[0054] According to a ninth aspect of the embodiments of this specification, there is provided an application recommendation device, including:

[0055] A third request receiving module is configured to receive a recommendation request for an application, wherein the recommendation request carries a user identifier of a user;

[0056] A third high-dimensional feature acquisition module is configured to acquire attribute features of the user, attribute features of n applications for the user, and cross-combination features of the user and the applications based on the user identifier to obtain initial features including m dimensions;

[0057] A third low-dimensional feature acquisition module is configured to reduce the dimension of the initial feature according to the depth encoder to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d;

[0058] A third prediction module is configured to input the target feature into a contextual bandit model to obtain a prediction result of the application;

[0059] The third recommendation module is configured to recommend an application whose prediction result meets a preset condition as a target application to the user.

[0060] According to a tenth aspect of the embodiments of this specification, a commodity recommendation device is provided, including:

[0061] a fourth request receiving module, configured to receive a recommendation request for a product, wherein the recommendation request carries a user identifier of a user;

[0062] a fourth high-dimensional feature acquisition module, configured to acquire attribute features of the user, attribute features of n commodities for the user, and cross-combination features of the user and the commodities based on the user identifier, so as to obtain initial features including m dimensions;

[0063] A fourth low-dimensional feature acquisition module is configured to reduce the dimension of the initial feature according to the depth encoder to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d;

[0064] a fourth prediction module, configured to input the target feature into a contextual bandit model to obtain a prediction result of the commodity;

[0065] The fourth recommendation module is configured to recommend commodities whose prediction results meet preset conditions to the user.

[0066] According to an eleventh aspect of the embodiments of this specification, there is provided a recommendation device, including:

[0067] An interface determination module, configured to display a target input interface to the caller based on a call request of the caller;

[0068] a fifth request receiving module, configured to receive a t-th access request generated by the caller through the target input interface, wherein the access request carries a user identifier of a user and n candidate objects for the user, and t is a positive integer;

[0069] a fifth high-dimensional feature acquisition module, configured to generate initial features of m dimensions based on the user identifier and n candidate objects for the user;

[0070] a fifth low-dimensional feature acquisition module, configured to reduce the dimension of the initial feature according to the first machine learning model to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d, and the first machine learning model is trained based on the initialization parameters of the second machine learning model;

[0071] A fifth prediction module is configured to input the target feature into the second machine learning model to obtain a prediction result of the candidate object;

[0072] The fifth recommendation module is configured to recommend the candidate objects whose prediction results meet the preset conditions to the user through the caller.

[0073] According to a twelfth aspect of the embodiments of this specification, a recommendation device is provided, including:

[0074] a sixth request receiving module, configured to receive a t-th access request sent by a caller, wherein the access request carries a user identifier of a user and n candidate objects for the user, and t is a positive integer;

[0075] a sixth high-dimensional feature acquisition module, configured to generate initial features of m dimensions based on the user identifier and n candidate objects for the user;

[0076] a sixth low-dimensional feature acquisition module, configured to reduce the dimension of the initial feature according to the first machine learning model to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d, and the first machine learning model is trained based on the initialization parameters of the second machine learning model;

[0077] a sixth prediction module, configured to input the target feature into the second machine learning model to obtain a prediction result of the candidate object;

[0078] The sixth recommendation module is configured to recommend the candidate objects whose prediction results meet the preset conditions to the user through the caller.

[0079] According to a thirteenth aspect of the embodiments of this specification, a computing device is provided, including:

[0080] Memory and processor;

[0081] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, wherein when the processor executes the computer-executable instructions, it implements the steps of any one of the recommendation methods, or implements the steps of the video cover recommendation method, or implements the steps of the application recommendation method, or implements the steps of the product recommendation method.

[0082] According to the fourteenth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, which, when executed by a processor, implement the steps of any one of the recommendation methods, or the steps of the video cover recommendation method, or the steps of the application recommendation method, or the steps of the product recommendation method.

[0083] An embodiment of the present specification implements a recommendation method and device, wherein the recommendation method includes first receiving the tth access request; obtaining the attribute characteristics of the user, the object characteristics of n candidate objects for the user, and the cross-combination characteristics of the user and the candidate objects based on the user identifier of the user in the access request to obtain a high-dimensional initial feature; then automatically encoding and reducing the dimension of the high-dimensional initial feature through a first machine learning model obtained by training based on the initialization parameters of the second machine learning to obtain a low-dimensional target feature with a much lower dimension than the high-dimensional initial feature; then inputting the low-dimensional target feature into the second machine learning model to quickly and accurately obtain the prediction result of each candidate object; finally, recommending the candidate object whose prediction result meets the preset conditions to the user; through the joint learning mechanism of the first machine learning and the second machine learning, the online and offline learning processes can be integrated, and a more accurate feature vector with a dimension sufficient to support the online effect can be automatically learned, and then the high-dimensional features are automatically optimized and integrated through the first machine learning model without reducing the real-time prediction efficiency of the second machine learning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1This is an example diagram of a specific application scenario of a recommended method provided by an embodiment of this specification;

[0085] Figure 2 is a flow chart of a first recommended method provided by an embodiment of this specification;

[0086] Figure 3 It is a system deployment diagram of a recommended method provided by an embodiment of this specification applied to a specific system;

[0087] Figure 4 It is a schematic diagram of specific online and offline training of a contextual bandit and a deep encoder in a recommended method provided by an embodiment of this specification;

[0088] Figure 5 is a flow chart of a video cover recommendation method provided by an embodiment of this specification;

[0089] Figure 6 is a flow chart of an application recommendation method provided by an embodiment of this specification;

[0090] Figure 7 is a flow chart of a commodity recommendation method provided by an embodiment of this specification;

[0091] Figure 8 is a flow chart of a second recommended method provided by an embodiment of this specification;

[0092] Fig. 9 is a flow chart of a third recommended method provided by an embodiment of this specification;

[0093] Fig.10 It is a structural schematic diagram of a first recommended device provided by an embodiment of this specification;

[0094] Fig.11 It is a structural schematic diagram of a video cover recommendation device provided by an embodiment of this specification;

[0095] Fig.12 is a structural diagram of an application recommendation device provided by an embodiment of this specification;

[0096] Fig.13 It is a structural schematic diagram of a commodity recommendation device provided by an embodiment of this specification;

[0097] Fig.14 is a schematic diagram of the structure of a second type of recommended device provided by an embodiment of this specification;

[0098] Fig.15 is a schematic diagram of the structure of a third recommended device provided by an embodiment of this specification;

[0099] Fig.16 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0100] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0101] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0102] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0103] First, the terms involved in one or more embodiments of this specification are explained.

[0104] Bandit: The name of an algorithm used to balance exploration and exploitation in solution selection problems.

[0105] Contextual Bandit: English full name: Contextual Bandit, a deep learning-based prediction model used in recommendation systems.

[0106] Online decision-making: The system makes decision recommendations based on user requests in real time.

[0107] MLP: English full name: Multilayer Perceptron, Chinese full name: Multilayer Perceptron.

[0108] CNN: English full name: Convolutional Neural Networks, Chinese full name: Convolutional Neural Network.

[0109] DNN: English full name: Deep Neural Network, Chinese full name: Deep Neural Network.

[0110] GCN: English full name: Graph Convolutional Network, Chinese full name: Graph Convolutional Neural Network.

[0111] In this specification, a recommendation method is provided. One or more embodiments of this specification also involve a recommendation device, a video cover recommendation method and device, an application recommendation method and device, a product recommendation method and device, four computing devices, and four computer-readable storage media, which are described in detail one by one in the following embodiments.

[0112] See also Figure 1 , Figure 1 An example diagram showing a specific application scenario of a recommendation method provided according to an embodiment of the present specification is shown.

[0113] Figure 1 The application scenario includes a terminal and a server. Specifically, a user accesses a shopping website through a terminal. The terminal generates an access request and sends it to the server. After receiving the access request, the server queries the database to obtain the attribute information of the user corresponding to the user identifier, the attribute information of the goods on the shopping website, and the cross-combination information of the user and the goods based on the user identifier carried in the access request. Then, the above information is spliced ​​together to obtain the original high-dimensional feature vector X reflecting the context information of this access request. (a,t) ;

[0114] Then X (a,t) The first machine learning model has multiple layers of perception parameters, which will transform the original high-dimensional feature vector X through nonlinear transformation. (a,t) The dimension of the target low-dimensional feature vector V is reduced to a fixed d-dimensional target (a,t) In the example above, the dimension of the target low-dimensional feature vector is much smaller than that of the original high-dimensional feature vector;

[0115] Then the d-dimensional target low-dimensional feature vector V obtained after dimensionality reduction (a,t) The data is passed to the second machine learning model, which scores each product in the shopping website online through the online decision evaluation function of the second machine learning model, for example, by using a confidence interval upper bound method based on ridge regression to perform decision evaluation;

[0116] Finally, among all the scored products, 1 or k products are selected and returned to the user as the recommendation result for the user. The specific number of products selected and returned to the user is determined based on the application scenario of the specific application party.

[0117] See also Figure 2 , Figure 2 A flowchart of a first recommendation method provided according to an embodiment of the present specification is shown, including steps 202 to 210.

[0118] Step 202: Receive the tth access request, wherein the access request carries a user identifier of the user, and t is a positive integer.

[0119] Wherein, t is a positive integer greater than 0. If t is 1, receiving the tth access request is equal to receiving the first access request.

[0120] In actual applications, the received access requests may be multiple access requests generated when the same user accesses the terminal multiple times, or may be multiple access requests generated when each of different users accesses the terminal; and each access request carries the user identifier of the user initiating the access request and / or some other context environment parameters, wherein the user identifier includes an identifier named by a variable, function and / or array, etc., which can uniquely define itself; the context environment parameters include but are not limited to the time when the access request is received and the model of the terminal device used by the user initiating the access request, etc.

[0121] Step 204: Acquire initial features including m dimensions based on the user identifier.

[0122] The initial features of the m dimensions include attribute features of the user, object features of n candidate objects for the user, and cross-combination features of the user and the candidate objects.

[0123] Specifically, after receiving the tth access request carrying a user identifier, based on the user identifier, the database is queried to obtain the attribute characteristics of the user corresponding to the user identifier, the object characteristics of n candidate objects for the user, and the cross-combination characteristics of the user and the candidate objects, and then the above characteristics are spliced ​​to obtain the initial characteristics including m dimensions that reflect the context information of this access request; wherein the attribute characteristics of the user include but are not limited to the user's name, age, mobile phone number, historical shopping behavior, historical click behavior and / or historical browsing behavior, etc.; the object characteristics of the candidate object include but are not limited to the name, type and / or attribute characteristics of the associated user of the candidate object; and the cross-combination characteristics of the user and the candidate object are the characteristics obtained by crossing the attribute characteristics of the user with the object characteristics of the candidate object.

[0124] In specific implementation, the candidate objects include but are not limited to video covers, mobile phone applications, network connection methods and / or commodities; when the candidate object is a video cover, the object characteristics of the candidate object are image characteristics of the video image, including but not limited to the type of the video cover and the image information in the video cover; when the candidate object is a mobile phone application, the object characteristics of the candidate object are attribute characteristics of the mobile phone application, including but not limited to the type, function, memory usage and security performance of the mobile phone application; when the candidate object is a commodity, the object characteristics of the candidate object are attribute characteristics of the commodity, including but not limited to the type, name, price, origin and seller information of the commodity.

[0125] The following is a detailed description of the cross-combination features of the user and the candidate object. Taking the candidate object as a product as an example, users of different ages have different interests in the types of products. However, from the perspective of the user's age and the attribute characteristics of the product alone, the differences between the types of products that users of different age groups are interested in cannot be portrayed. In order to solve this problem, the attribute characteristics of the user are cross-combined with the attribute characteristics of the product to construct a cross-combination feature of the user and the candidate object. For example, the attribute characteristic of the user's age of 18 is cross-combined with the attribute characteristic of the product type of "clothing" to form a cross-combination feature of the user and the candidate object; the attribute characteristic of the user's age of 28 is cross-combined with the attribute characteristic of the product type of "kitchen utensils" to form a cross-combination feature of the user and the candidate object. By using this cross-combination feature of the user and the candidate object, a high-dimensional initial feature that reflects the context information of the access request can be obtained in a more fine-grained manner, so that the subsequent second machine learning model can make a more accurate prediction of the score of the candidate object based on the acquisition of finer-grained features.

[0126] Step 206: Reduce the dimension of the initial features according to the first machine learning model to obtain target features including d dimensions.

[0127] Wherein, m and d are both positive integers, and m>d, and the first machine learning model is trained based on the initialization parameters of the second machine learning model.

[0128] Specifically, after obtaining the attribute characteristics of the user, the object characteristics of n candidate objects for the user, and the cross-combination characteristics of the user and the candidate objects based on the user identifier, and obtaining the high-dimensional initial characteristics that reflect the context information of this access request, in order to avoid the high-dimensional initial characteristics from affecting the prediction efficiency of the second machine learning model when predicting the scores of candidate objects and to improve the prediction efficiency of the second machine learning model in predicting the scores of candidate objects, the initial characteristics can be reduced in dimension through the first machine learning model to obtain a target feature with a much lower dimension than the initial characteristics.

[0129] After the initial features are reduced in dimension according to the first machine learning model, the number of dimensions of the d-dimensional target features obtained can be determined according to the processable number of dimensions of the second machine learning model. For example, the second machine learning model can only process 500 dimensions. Then, after the initial features are reduced in dimension according to the first machine learning model, the number of dimensions of the d-dimensional target features obtained cannot exceed 500 dimensions. In practical applications, a suitable first machine learning model for dimensionality reduction can also be selected according to the processable number of dimensions of the second machine learning model. The two complement each other.

[0130] Regardless of whether the initial features are reduced in dimension through the first machine learning model, or the scores of the candidate objects are subsequently predicted through the second machine learning model, the first machine learning model and the second machine learning model need to be determined and trained in advance.

[0131] In specific implementation, if there is historical sample data when training the first machine learning model and the second machine learning model, the first machine learning model and the second machine learning model are trained based on the historical sample data. The specific implementation method is as follows:

[0132] The second machine learning model includes a prediction model;

[0133] Accordingly, the training method of the prediction model includes:

[0134] Acquire historical sample data of the first dimension, wherein the historical sample data includes attribute features of a sample user, object features of a sample candidate object corresponding to the sample user, cross-combination features of the sample user and the sample candidate object, and corresponding labels;

[0135] Performing dimensionality reduction processing on the historical sample data of the first dimension to obtain historical sample data of a second dimension, wherein the first dimension is greater than the second dimension;

[0136] The prediction model is trained according to the historical sample data of the second dimension to obtain the trained prediction model and initialization parameters of the prediction model, and the prediction model outputs the scores of the sample candidate objects.

[0137] The labels include the true scores of the candidate objects.

[0138] Specifically, when the system is cold started, a first machine learning model is first determined, and the parameters in the first machine learning model are randomly initialized so that the first machine learning model can realize the mapping of high-dimensional features to low-dimensional features. After obtaining the historical sample data of the first dimension (i.e., high dimension), the historical sample data of the first dimension is reduced in dimension through the randomly initialized first machine learning to obtain the historical sample data of the second dimension (i.e., low dimension). The parameters in the prediction model are trained offline using the historical sample data of the second dimension to obtain the trained prediction model and the initialization parameters W of the prediction model.

[0139] The offline training method and low-dimensional historical sample data are used to obtain the offline trained prediction model, so that the prediction model can quickly and accurately predict the score of each candidate object after receiving the low-dimensional target features.

[0140] After obtaining the initialization parameter W of the second machine learning model, the first machine learning model is trained based on the initialization parameter W. The specific implementation method is as follows:

[0141] The first machine learning model includes a dimensionality reduction model;

[0142] Accordingly, the training method of the dimensionality reduction model includes:

[0143] Obtaining historical sample data of the first dimension and initialization parameters of the prediction model;

[0144] The dimension reduction model is trained according to the historical sample data of the first dimension and the initialization parameters of the prediction model to obtain the trained dimension reduction model,

[0145] The dimensionality reduction model outputs historical sample data of the second dimension corresponding to the historical sample data of the first dimension.

[0146] Specifically, after training the second machine learning model and the initialization parameter W of the second machine learning model, the first machine learning can be trained by using the historical sample data of the first dimension and the initialization parameter W of the second machine learning model in combination with the stochastic gradient descent algorithm, and then the first machine learning model is updated to obtain the trained first machine learning model.

[0147] Among them, the first machine learning model includes a supervision layer and a loss function layer, wherein the supervision layer uses the initialization parameters in the first machine learning model to make forward predictions, and the loss function layer calculates the difference between the predicted results of the candidate objects predicted by the supervision layer and the actual scores of the candidate objects.

[0148] Through the above-mentioned training method and using the initialization parameter W in the second machine learning model for training, the first machine learning model can automatically optimize and fuse high-dimensional features so that the feature space dimension supported by the second machine learning model can be expanded to infinite dimensions, so that its effect can approach the theoretical upper limit.

[0149] In the embodiments of this specification, the first machine learning model for feature dimensionality reduction and the second machine learning model for prediction are both the first machine learning model and the second machine learning model trained by the above method.

[0150] In another embodiment of the present specification, if there is no historical sample data when training the first machine learning model and the second machine learning model, the first machine learning model and the second machine learning model are trained, and the specific implementation method is as follows:

[0151] Determine the second machine learning model and set initialization parameters for the second machine learning model;

[0152] Determine the first machine learning model, train the first machine learning model based on the initialization parameters, and reduce the dimension of the input of the second machine learning by the first machine learning model.

[0153] Specifically, a second machine learning model is first determined, and then the parameters in the second machine learning model are randomly initialized according to the existing technology, and then the first machine learning model is trained based on the initialization parameters to obtain a trained first machine learning model for online service.

[0154] In the embodiments of this specification, the first machine learning model obtained in the above manner can also achieve dimensionality reduction of high-dimensional features, so that the second machine learning model can quickly predict the score of each candidate element based on the target features after dimensionality reduction.

[0155] In another possible implementation, in the absence of historical sample data, the first machine learning model and the second machine learning model can also be initially trained based on similar historical sample data. For example, historical data samples in similar fields can be obtained to perform initial training on the first machine learning model and the second machine learning model. The embodiments of this specification do not impose any limitations on this.

[0156] In a specific implementation, the initial features are reduced in dimension according to the first machine learning model to obtain target features including d dimensions, that is, it can be understood that the initial features are reduced in dimension according to the dimensionality reduction model obtained in any of the above methods to obtain target features including d dimensions, wherein the number of dimensions m in the initial features is greater than the number of dimensions d in the target features.

[0157] Step 208: Input the target feature into a second machine learning model to obtain a prediction result for the candidate object.

[0158] Specifically, by using the second machine learning model trained in the above manner, and inputting the target features after dimensionality reduction into the second machine learning model, the prediction results of each candidate object can be obtained, wherein the prediction results can represent the user's preference for the candidate object.

[0159] In another embodiment of the present specification, the first machine learning model includes a multi-layer perceptron model, a convolutional neural network model, a deep neural network model or a graph convolutional neural network model;

[0160] The second machine learning model includes a contextual bandit model.

[0161] Step 210: Recommend candidate objects whose prediction results meet preset conditions to the user.

[0162] Among them, the preset conditions can be set according to the needs of the application party in the actual application scenario. For example, the recommendation method is applied in a video cover recommendation scenario. Since only one video cover is needed, the preset condition can be the video cover with the highest prediction result.

[0163] Then recommending the candidate objects whose prediction results meet the preset conditions to the user means recommending the video cover with the highest prediction result to the user.

[0164] When the recommendation method is applied in a mobile application recommendation scenario, the preset condition may be to take the top 10 mobile applications after sorting in descending order; then the candidate objects whose prediction results meet the preset conditions are recommended to the user, that is, the top 10 mobile applications with the highest prediction results are recommended to the user.

[0165] The recommendation method provided in the embodiments of this specification automatically optimizes and fuses high-dimensional features through the first machine learning model without reducing the real-time efficiency of the second machine learning model of online learning, so that the dimension of the feature space supported by the second machine learning model is expanded to infinite dimensions. When high-dimensional features are obtained, the prediction results of candidate objects are quickly and accurately obtained through the second machine learning model.

[0166] In another embodiment of the present specification, after the candidate object whose prediction result meets the preset conditions is recommended to the user, the dimension reduction model and the prediction model are cyclically trained based on the user's behavioral feedback information on the recommended candidate object, so that the prediction model can quickly learn the user's interests based on the learned mapping function of high-dimensional features to low-dimensional features and rely on each online user's behavioral feedback information; and the dimension reduction model learns a more appropriate mapping function of high-dimensional features to low-dimensional features based on the prediction function of mapping the learned low-dimensional features to the user's interests, ensuring that the parameters of the online prediction model and the multi-layer perception parameters in the dimension reduction model are trained in a complementary manner to obtain a better prediction model and dimension reduction model, and the specific implementation method is as follows:

[0167] After recommending the candidate object whose prediction result meets the preset condition to the user, the method further includes:

[0168] Receive behavior feedback information from the user regarding the candidate object whose prediction result meets a preset condition.

[0169] Among them, the behavioral feedback information is the user's click, purchase and / or abandonment behavior for the recommended candidate objects whose prediction results meet the preset conditions; for example, after seeing the recommended candidate objects, the user clicks on one or several candidate objects (such as commodities) in the recommended candidate object list according to his or her own interests, or abandons all candidate elements recommended this time, and initiates an access request again.

[0170] In an embodiment of the present specification, the user's behavioral feedback information on the candidate object whose prediction result meets the preset conditions is received, and the behavioral feedback information will be processed by the feedback collector. The behavioral feedback information will then be combined with the original high-dimensional features to re-iterate the training of the first machine learning model and the second machine learning model of the embodiment of the present specification to obtain more accurate first machine learning model and second machine learning model.

[0171] In specific implementation, after receiving the user's behavioral feedback information on the recommended candidate object, it is associated with the original high-dimensional features at the time of the system decision, that is, the retraining process of the prediction model is started to enhance the prediction effect of the prediction model. The specific implementation method is as follows:

[0172] After receiving the user's behavior feedback information on the candidate object whose prediction result meets the preset condition, the method further includes:

[0173] Updating the initial feature based on the behavior feedback information to obtain an updated initial feature;

[0174] According to the dimensionality reduction model, the updated initial features are reduced in dimension to obtain updated target features;

[0175] The prediction model is retrained based on the updated target feature to obtain a retrained prediction model and updated initialization parameters of the retrained prediction model.

[0176] Specifically, the initial features are updated based on the user's actual behavioral feedback to obtain high-dimensional updated initial features, and then the dimensionality reduction model is used to reduce the dimensionality of the high-dimensional updated initial features to obtain low-dimensional updated target features. Finally, based on the updated target features and the labels corresponding to each updated target feature, the prediction model is retrained to obtain the retrained prediction model and the updated initialization parameters of the retrained prediction model, wherein the label corresponding to each updated target feature is the true score of the candidate element.

[0177] In practical applications, the updated initial features include the original high-dimensional initial features and the new high-dimensional initial features added based on the user's behavioral feedback information. This feature accumulation method is used to obtain more fine-grained high-dimensional training data samples, so that the training effect of the prediction model is better. Specifically, the specific implementation method of updating the initial features is as follows:

[0178] The updating of the initial feature based on the behavior feedback information to obtain the updated initial feature includes:

[0179] Determine, based on the behavior feedback information, newly added attribute information of the user, object features of n candidate objects for the user, and cross-combination features of the user and the candidate objects to obtain newly added initial features;

[0180] The newly added initial feature is superimposed on the initial feature to obtain the updated initial feature.

[0181] Specifically, after receiving the user's behavioral feedback information on the recommended candidate object, it is associated with the original high-dimensional initial features of the decision system, and then superimposed in the above manner to form an updated initial feature, and then the low-dimensional updated target feature is obtained by dimensionality reduction of the dimensionality reduction model. In actual applications, the low-dimensional updated target feature can be obtained without dimensionality reduction of the dimensionality reduction model. For example, during online decision-making, the updated target feature corresponding to the updated initial feature has been dynamically written into the database. In actual application, the updated target feature can be directly selected to retrain the prediction model.

[0182] In another embodiment of the present specification, after retraining the prediction model based on the updated target feature to obtain the retrained prediction model and the updated initialization parameters of the prediction model, the method further includes:

[0183] Determine whether t is greater than d 2 And whether t is greater than n,

[0184] If so, the dimensionality reduction model is trained according to the updated initial features and the updated initialization parameters to obtain a retrained dimensionality reduction model.

[0185] Among them, t is the number of access requests, d is the number of dimensions of the target feature, and n is the number of candidate elements.

[0186] Specifically, after receiving the user's behavioral feedback information on the recommended candidate object, the prediction model is retrained using the above method to obtain the updated initialization parameters of the retrained prediction model, and then it is determined whether t is greater than d 2 And whether t is greater than n, if so, the retraining process of the dimensionality reduction model is started, the dimensionality reduction model is trained according to the updated initial features and the updated initialization parameters to obtain the retrained dimensionality reduction model, and then the dimensionality reduction model actually used in the recommendation method is updated; that is, the end condition of the retraining of the prediction model and the start condition of the retraining of the dimensionality reduction model are: whether t is greater than d 2 And whether t is greater than n; in practical applications, based on experience, in order to obtain a higher quality dimensionality reduction coding effect, the dimensionality reduction model requires training sample data that is 10 times the number of parameters used. Since only one access request can collect one training sample data, the minimum number of convergence rounds required can be calculated. Therefore, the prediction model needs to be predicted only when the above conditions are met, which will trigger the offline training process of the dimensionality reduction model for batch data, so that the dimensionality reduction model can obtain a higher quality dimensionality reduction coding effect.

[0187] And judge whether t is greater than d 2 And whether t is greater than n, if not, t is incremented by 1 and the step of receiving the tth access request is continued.

[0188] Specifically, when t is less than or equal to d 2 And when t is less than or equal to n, the stop training condition of the prediction model is not met. At this time, the next access request will be obtained, that is, t will be incremented by 1, and the tth access request will be received. For example, if the original t is 1, then t after incrementing by 1 is 2, that is, the second access request will be received. The second access request still carries the user ID of the user. The high-dimensional initial features are obtained through the user ID, and the high-dimensional initial features are reduced in dimension through the updated dimensionality reduction model to obtain low-dimensional target features. The low-dimensional target features are then input into the updated prediction model to obtain the prediction results of each candidate element, and then the candidate objects whose prediction results meet the prediction conditions are recommended to the user, and the updated prediction model is retrained based on the user's behavior feedback information until it is satisfied whether t is greater than d 2And only after the condition that t is greater than n is met, the above processing steps will be ended and the offline training process of the dimensionality reduction model based on batch data will be started;

[0189] Finally, the dimension reduction model is trained based on all high-dimensional training sample data of the prediction model trained iteratively and the final fixed update initialization parameters of the prediction model to obtain a trained dimension reduction model.

[0190] In the embodiment of the present specification, the online learning process of the prediction model based on each access request is mainly aimed at fixing the parameters of the dimensionality reduction model and the mapping function of the low-dimensional target features learned online to the user's interest in the candidate objects, so that the user's interest can be accurately predicted and the candidate objects can be accurately distinguished; and based on batch data (i.e., whether t is greater than d 2 The main purpose of the offline learning process of the dimensionality reduction model is to fix the initialization parameters of the online learning prediction model and learn a better mapping function from high-dimensional initial features to low-dimensional target features.

[0191] Based on the user's behavioral feedback information, the prediction model is trained online with streaming data (once for each access request), and the dimensionality reduction model is trained offline with batch data to ensure that the parameters of the prediction model and the multi-layer perception parameters of the dimensionality reduction model are trained in a complementary manner. Without reducing the real-time efficiency of the prediction model, the high-dimensional initial features are automatically optimized and fused through the dimensionality reduction model, so that the effect can approach the theoretical upper limit. The dimensionality reduction model can efficiently model the nonlinear relationship between features and feedback, so that the feature space dimension supported by the prediction model is expanded to infinite dimensions.

[0192] See also Figure 3 , Figure 3 A schematic diagram of system deployment of a recommended method provided by an embodiment of this specification is shown in a specific system.

[0193] like Figure 3 As shown, in addition to the user entity, the specific system applied by the recommendation method mainly includes two parts. One part is online decision-making, which involves components including a high-dimensional feature acquirer, a deep encoder (that is, the above-mentioned first machine learning model or dimensionality reduction model), a contextual gambling machine (that is, the above-mentioned second machine learning model or prediction model) and a feedback collector; the other part is a joint learning mechanism, which involves components including a deep encoder offline trainer and an online learning component.

[0194] The following describes in detail the recommended method in the embodiment of this specification applied to the specific system.

[0195] Step 1: The user accesses the system in the form of a request string. For example, when the user opens a shopping website, the shopping website will generate a request string and send it to the system server. The request string carries a user identifier (ID) and some other context parameters.

[0196] Step 2: The high-dimensional feature acquirer of the system queries the database based on the user identifier in the request string to obtain information related to the user, basic information of the candidate elements, and cross-combination information of the user and each candidate element. The user-related information can be specifically referred to as the attribute characteristics of the user, the basic information of the candidate elements can be specifically referred to as the object characteristics of the candidate objects, and the cross-combination information of the user and each candidate element can be specifically referred to as the cross-combination characteristics of the user and the candidate objects. The candidate elements include but are not limited to heterogeneous information such as natural language text, image pixel, and geographic location information. The above information is spliced ​​together to obtain the original high-dimensional feature vector (X) reflecting the context information of this access request. (a,t) ), and if the original high-dimensional features of all candidate elements are concatenated, an n*m dimensional matrix can be obtained, where n is the number of candidate elements and m is the dimension of X(a,t).

[0197] Step 3: Convert the original high-dimensional feature vector (X (a,t) ) is passed to the deep encoder, which has multiple layers of perception parameters inside. The original high-dimensional feature vector (X (a,t) ) is reduced to a fixed d-dimensional low-dimensional feature vector (V (a,t) ), and d is much smaller than m.

[0198] Step 4: The low-dimensional feature vector (V (a,t) ) is passed to the contextual gambling machine, and the online decision evaluation function of the contextual gambling machine is based on the low-dimensional feature vector (V (a,t) ) performs online scoring for each candidate element, for example, performs decision evaluation for each candidate element by using a confidence interval upper bound method based on ridge regression, and then selects 1 or k from all the scored candidate elements and returns them to the user as a recommendation result, and the actual number of selected candidate elements is set according to the needs of the user in actual applications, and no requirements are made here, wherein the online decision evaluation function of the contextual gambling machine may be different according to different contextual gambling machine algorithms, and there is no limitation in the embodiments of this specification, and it can be applicable to any contextual gambling machine algorithm based on model features.

[0199] Step 5: After receiving the recommendation results, the user will give feedback on the results according to his or her own interests, such as clicking on a product in the recommendation result list or giving up the current result and initiating another access request; the user's feedback behavior will be processed by the feedback collector, and the streaming samples will be passed into the online learning component of the joint learning mechanism in a streaming manner, and the batch samples will be passed into the deep encoder offline trainer of the joint learning mechanism in a batch processing manner. The joint learning mechanism ensures that the online model parameters (model parameters involved in the contextual gambling machine) and the multi-layer perception parameters in the deep encoder are trained in a complementary manner. The contextual gambling machine is responsible for the mapping function from the learned high-dimensional features to the low-dimensional online feature vectors, and relies on each online user feedback to quickly learn the user's interests, while the deep encoder learns a more appropriate mapping function from the high-dimensional feature vector to the low-dimensional feature vector based on the prediction function of the learned low-dimensional features to the user's interests.

[0200] The system adopts the recommendation method provided in the embodiment of this specification to automatically optimize and fuse high-dimensional features through a deep encoder without reducing the real-time efficiency of the online contextual bandit algorithm, so that the effect can approach the theoretical upper limit, and the deep encoder can efficiently model the nonlinear relationship between features and feedback. This method can theoretically expand the feature space dimension supported by the online contextual bandit algorithm to infinite dimensions.

[0201] For the specific process of online and offline training of contextual bandits and deep encoders based on the above joint learning mechanism, see Figure 4 , Figure 4 A schematic diagram showing specific online and offline training of a context bandit and a deep encoder in a recommendation method provided by an embodiment of the present specification is shown.

[0202] Depend on Figure 4 It can be seen that the entire joint learning mechanism consists of two parts. One part is the offline learning process of the deep encoder based on batch data. Its main purpose is to fix the initialization parameter W of the online learning context bandit and learn a better mapping function from high-dimensional features to low-dimensional features, that is, the deep encoder. The second part is the online learning process of the context bandit based on streaming data. Its main purpose is to fix the parameters of the deep encoder and learn the mapping function from low-dimensional features to user interests in candidate elements online, so that user interests can be accurately predicted and candidate elements can be accurately distinguished.

[0203] Firstly, the online learning process of contextual bandit based on streaming data is introduced in detail.

[0204] Step 1: In the specific implementation, after the feedback collector collects a piece of user feedback, it is associated with the original high-dimensional feature vector X of the online decision of the above system(a,t) After that, the online learning process of the contextual gambling machine based on the streaming data is started.

[0205] Step 2: The original high-dimensional feature X obtained in step 1 (a,t) After being processed by the deep encoder, a low-dimensional feature vector V is obtained (a,t) , and then the low-dimensional feature vector V (a,t) The process involved in the online learner of the input contextual gambling machine is learned and updated to obtain the updated initialization parameter W. The online learner of the contextual gambling machine used here supports different online learners according to the selected algorithms. In addition, the Woodbury formula can be used to reduce the computational load of matrix inversion to the square level instead of the cubic level. In practical applications, V(a, t) for online learning of the contextual gambling machine can also be obtained without deep encoder operation. For example, the original high-dimensional feature vector X is dynamically converted into (a,t) The corresponding low-dimensional feature vector V (a,t) Write to the database, then you can directly select and use it in this step.

[0206] Step 3: Update using the updated initialization parameter W Figure 3 The context bandit component in , and judge whether t is greater than d 2 And whether t is greater than n, if so, start the offline learning process of the deep encoder based on batch data; if not, repeat steps 1 and 2 until t>d 2 And t>n, (where t is the number of access requests of the user, n is the number of candidate elements, and d is the dimension of the low-dimensional feature vector after dimensionality reduction).

[0207] Then at t>d 2 And under the condition of t>n, start the offline learning process of the deep encoder based on batch data. The specific implementation method is as follows:

[0208] Step 1: Obtain the model initialization parameter W in the context bandit from the trained online learning process (the W can be randomly generated during initialization). Specifically, W usually involves two types of parameters, one is the model parameter for predicting the user preference feedback (reward) of the candidate element, and the other is the parameter for calculating the upper bound of the confidence interval, which may be different for different context bandit algorithms.

[0209] Step 2: Use the previous T batches of data (i.e., the historical training sample data set consisting of the original high-dimensional features involved in the request log and the actual user feedback data) to iteratively train the parameters involved in the deep encoder using the stochastic gradient algorithm (SGD) or other optimization algorithms until convergence.

[0210] Step 3: Get the trained new deep encoder model and update Figure 3 The deep encoder in .

[0211] Specifically, Figure 4 The deep encoder in includes a supervision layer and a loss function layer, wherein the supervision layer is a prediction layer constructed using the parameter W in the context bandit, and the supervision layer only performs a forward prediction process and does not need to perform a reverse gradient update process, that is, in the entire offline learning process of the deep encoder based on batch data, W remains unchanged; and the loss function selected by the loss function layer is the root mean square error (RMSE) / mean square error (MSE), which calculates the difference between the expected value of the user preference feedback of the input candidate element predicted by the supervision layer and the true value of the feedback generated by the actual user.

[0212] In actual use, the deep encoder can be constructed using any deep network without any restrictions. For example, common structures that can be supported include DNN, CNN, GCN, MLP, etc. In addition, the dimension of the output V(a, t) of the deep encoder during the training process is a constant d-dimensional vector, and the dimension is much smaller than X. (a,t) Dimension.

[0213] In the embodiments of the present specification, the recommendation method automatically encodes the original high-dimensional features by integrating a deep encoder to obtain a low-dimensional feature with a much lower dimension than the original high-dimensional feature, and constructs an online contextual bandit model based on the low-dimensional feature; through the joint learning mechanism of the above-mentioned deep encoder and contextual bandit, the online and offline learning processes can be integrated, and a more accurate feature vector with a dimension sufficient to support the online effect can be automatically learned, and then, without reducing the real-time efficiency of the contextual bandit algorithm, the high-dimensional features are automatically optimized and integrated through the deep encoder, and the linear relationship assumption between the features and the feedback is broken, and complex nonlinear relationships are modeled, so that the effect can approach the theoretical upper limit; the recommendation method can theoretically expand the feature space dimension supported by the contextual bandit algorithm to infinite dimensions.

[0214] In practical applications, the recommendation method can be regarded as an online decision-making selection problem, which can be described as given a candidate set containing n candidate elements, k candidate elements need to be selected from the n candidate elements for sorting, and finally the selected k elements are displayed on the page according to the sorting results. In specific implementation, an instance of a selection problem can be called a machine, and the candidate elements therein can be called arms. Users will give feedback to the displayed arms according to their own interests and other factors. For example, in a specific application, if a user clicks on a displayed arm, it is called positive feedback, and if a user complains about a displayed arm, it is called negative feedback. A set of the above display and feedback processes is called an experiment. The solution to this problem is usually an online optimization algorithm (i.e., online learning). The goal of the algorithm is to minimize the cumulative regret (Regret) that can be achieved after performing the above display and feedback process in an iterative order for n rounds of experiments, that is, to minimize the gap between it and the cumulative maximum value (maximum positive feedback), so as to maximize the positive feedback. Usually after each round of experiments, the algorithm will refresh (retrain) the decision model (i.e. the above-mentioned dimensionality reduction model and prediction model) according to the feedback from users in that round to achieve the algorithm goal. Therefore, it is called an online algorithm, and the recommendation method is based on this online algorithm to achieve reasonable and accurate recommendation of candidate elements.

[0215] Specifically, the above recommendation method is applied to a video first picture material recommendation system of a certain video app software and described in detail.

[0216] There are multiple videos in the video app. These videos generally have many poster images and video plot screenshots. These images can be presented to users as video cover images. Since different users have different interests, they have different preferences for cover images of different categories. Displaying cover images that users like and are interested in can increase the click-through rate of the video and the viewing time of the video. Therefore, the video cover recommendation method provided in the embodiment of this specification can be used to select the most appropriate cover image from n different cover images and present it to the user.

[0217] In actual applications, each image is considered an arm, and each video contains a group of candidate images, which is a machine. The system is responsible for selecting a candidate image from each group of candidate images to display to the user. After the user sees the candidate image, if the user clicks to watch it, it can be regarded as a positive feedback. The duration is used as a quantitative indicator of the feedback. If the user does not click, the feedback is 0.

[0218] Specifically, for each image, the high-dimensional features include the following three parts: (1) context parameters of the request, such as the video categories that the user has watched and the types of cover images that have been clicked in the past; (2) static features of the image and the user, such as the image elements in the image, the creator, the user's name, and the user's address; (3) cross-combination features of the image and the user. The features of the above three parts are then spliced ​​to form a high-dimensional feature X. (a,t) , for example, the high-dimensional feature is 5000 dimensions.

[0219] Then the system uses a wide-deep network as the deep encoder. The wide-deep network is built by combining the 16-ResNet and converts the high-dimensional features X (a,t) After being encoded by the deep encoder, a low-dimensional feature V of dimension d is obtained. (a,t) , where d = 100 dimensions, where the supervision layer of the deep encoder uses W and X(a, t) of ridge regression for forward prediction, and the loss function layer uses the root mean square error RMSE.

[0220] Finally, the low-dimensional feature V (a,t) Input the context bandit based on ridge regression to predict the scores of n different cover images respectively, and then recommend the cover image with the highest score to the user.

[0221] In specific implementation, the parameters in the deep encoder are randomly initialized when the system is cold started. If there are 1,000 historical data sets, during initialization, the parameters in the context bandit are first trained offline using the historical data sets to obtain the initial parameters W of the context bandit; then the deep encoder is offline batch trained in combination with the stochastic gradient descent algorithm. During the training process, the parameters W of the context bandit are fixed to obtain a pre-trained deep encoder.

[0222] The behavior of the system after it goes online is as follows: assuming that there are 10,000 access requests, during the process of access requests from 0 to 10,000 (100*100), the online learning process of the contextual bandit based on streaming data is used to continuously tune the parameters of the contextual bandit on the basis of a fixed deep encoder until the 10,000th request is processed, and then the offline learning process of the deep encoder based on batch data is started. When the parameter W involved in the contextual bandit remains unchanged, the deep encoder is retrained using 1,000 historical data sets and 10,000 online incremental data sets. After the deep encoder is updated, 10,000 data are processed again, and then the above method is continuously iterated.

[0223] In the actual effect of the embodiments of this specification, the delay of the online request and learning system (i.e., the system that applies the recommendation method combined with online and offline) is compared with that of a simple 100-dimensional context bandit. It has no additional delay, but can continuously optimize the feature vector on which it is based. Therefore, its effect is much lower than the cumulative regret of a simple context bandit.

[0224] See also Figure 5 , Figure 5 A video cover recommendation method provided by an embodiment of the present specification is shown, including steps 502 to 510.

[0225] Step 502: Receive a recommendation request for a video cover, wherein the recommendation request carries a user identifier of a user.

[0226] Step 504: Based on the user identifier, the attribute features of the user, the image features of the n video images of the user, and the cross-combination features of the user and the video images are obtained to obtain initial features including m dimensions.

[0227] Step 506: Reduce the dimension of the initial features according to the depth encoder to obtain target features including d dimensions, where m and d are both positive integers and m>d.

[0228] Step 508: Input the target feature into the contextual bandit model to obtain the prediction result of the video image.

[0229] Step 510: recommending the video image whose prediction result meets the preset conditions to the user as a video cover.

[0230] Among them, the attribute characteristics of the user, the image characteristics of the video image, and the cross-combination characteristics of the user and the video image in the embodiments of this specification can all be referred to the specific introduction of the above embodiments. The deep encoder and contextual bandit model used are the better deep encoder and contextual bandit model obtained by iterative training in the above manner, and will not be repeated here.

[0231] Specifically, the recommendation method of the above embodiment is applied to video cover recommendation. After receiving a recommendation request for a video cover, the high-dimensional initial features obtained are first reduced to low-dimensional target features through a deep encoder obtained through offline training. Finally, the target features are input into a context encoder obtained through online training to obtain a prediction result for each video image. The video image with the highest prediction result is then recommended to the user as the video cover. This active recommendation method can avoid unnecessary search requests generated by users frequently searching for video images, reduce the burden on video websites or video app servers, and improve user experience.

[0232] See also Figure 6 , Figure 6 An application recommendation method provided by an embodiment of the present specification is shown, including steps 602 to 610.

[0233] Step 602: Receive a recommendation request for an application, wherein the recommendation request carries a user identifier of a user.

[0234] Step 604: Based on the user identifier, the attribute characteristics of the user, the attribute characteristics of the n applications for the user, and the cross-combination characteristics of the user and the applications are obtained to obtain initial characteristics including m dimensions.

[0235] Step 606: Reduce the dimension of the initial features according to the depth encoder to obtain target features including d dimensions, where m and d are both positive integers and m>d.

[0236] Step 608: Input the target feature into the contextual bandit model to obtain the prediction result of the application.

[0237] Step 610: recommending the application whose prediction result meets the preset condition to the user as the target application.

[0238] Among them, the attribute characteristics of the user, the attribute characteristics of the application, and the cross-combination characteristics of the user and the application in the embodiments of this specification can all be referred to the specific introduction of the above embodiments. The deep encoder and contextual bandit model used are the better deep encoder and contextual bandit model obtained by iterative training in the above manner, and will not be repeated here.

[0239] Specifically, the recommendation method of the above embodiment is applied to application recommendation. After receiving a recommendation request for an application, the high-dimensional initial features obtained are first reduced to low-dimensional target features through a deep encoder obtained through offline training. Finally, the target features are input into a context encoder obtained through online training to obtain a prediction result for each application. Then, the applications whose prediction results meet preset conditions (for example, the top 10 scores) are recommended to the user as target applications. This active recommendation method can increase the user download and usage rate of the application and improve the exposure of the recommended application.

[0240] See also Figure 7 , Figure 7 A commodity recommendation method provided by an embodiment of this specification is shown, including steps 702 to 710.

[0241] Step 702: Receive a recommendation request for a product, wherein the recommendation request carries a user ID of a user.

[0242] Step 704: Based on the user identifier, the attribute characteristics of the user, the attribute characteristics of the n commodities for the user, and the cross-combination characteristics of the user and the commodities are obtained to obtain initial characteristics including m dimensions.

[0243] Step 706: Reduce the dimension of the initial features according to the depth encoder to obtain target features including d dimensions, where m and d are both positive integers and m>d.

[0244] Step 708: Input the target feature into the contextual bandit model to obtain the prediction result of the product.

[0245] Step 710: Recommending products whose prediction results meet preset conditions to the user.

[0246] Among them, the attribute characteristics of the user, the attribute characteristics of the product, and the cross-combination characteristics of the user and the product in the embodiments of this specification can all be referred to the specific introduction of the above embodiments. The deep encoder and contextual bandit model used are the better deep encoder and contextual bandit model obtained by iterative training in the above manner, and will not be repeated here.

[0247] Specifically, the recommendation method of the above embodiment is applied to product recommendation. After receiving a recommendation request for a product, the high-dimensional initial features obtained are first reduced to low-dimensional target features through a deep encoder obtained through offline training. Finally, the target features are input into a contextual encoder obtained through online training to obtain a prediction result for each product. Then, the products whose prediction results meet preset conditions (for example, the top 20 scores) are recommended to the user as target products. This active recommendation method can reduce users' frequent and untargeted searches, reduce the burden on the server, increase work efficiency, and improve user experience.

[0248] In addition, the recommended method provided in the embodiments of this specification can also be applied to the selection of mobile phone Internet access methods. For example, the mobile phone Internet access methods include 3G, 4G, Wifi, etc. At the same time, the above-mentioned recommended method can be used to select a suitable Internet access method to realize data transmission.

[0249] See also Figure 8 , Figure 8 A second recommended method provided by an embodiment of this specification is shown, comprising the following steps:

[0250] Step 802: Displaying a target input interface to the caller based on the call request of the caller.

[0251] The caller includes but is not limited to merchants or individual business operators.

[0252] In actual applications, the recommendation method is applied to the server. After receiving the call request from the caller, the server first displays the target input interface to the caller to prompt the caller to enter reasonable required parameters, such as the user identifier and candidate objects mentioned below.

[0253] Step 804: Receive the tth access request generated by the caller through the target input interface, wherein the access request carries a user identifier of the user and n candidate objects for the user, and t is a positive integer.

[0254] Step 806: Acquire initial features including m dimensions based on the user identifier and the n candidate objects for the user.

[0255] Step 808: Reduce the dimension of the initial features according to the first machine learning model to obtain target features including d dimensions, where m and d are both positive integers and m>d, and the first machine learning model is trained based on the initialization parameters of the second machine learning model.

[0256] Step 810: Input the target feature into the second machine learning model to obtain a prediction result for the candidate object.

[0257] Step 812: Recommend candidate objects whose prediction results meet preset conditions to the user through the caller.

[0258] The recommendation method provided in the embodiment of this specification, after receiving the call request from the caller, first displays the target input interface to the caller, so that the caller can generate an access request based on the target input interface, and then processes the caller's access request to achieve fast and accurate recommendation of candidate objects for the caller's user.

[0259] See also Fig. 9 , Fig. 9 A third recommended method provided by an embodiment of this specification is shown, comprising the following steps:

[0260] Step 902: Receive the t-th access request sent by the caller, wherein the access request carries a user identifier of the user and n candidate objects for the user, and t is a positive integer.

[0261] The caller includes but is not limited to merchants or individual business operators.

[0262] In actual applications, the recommendation method is applied to the server, and an API interface is provided for the caller, and the caller can directly use the API interface to call the recommendation method.

[0263] Step 904: Acquire initial features including m dimensions based on the user identifier and n candidate objects for the user.

[0264] Step 906: Reduce the dimension of the initial features according to the first machine learning model to obtain target features including d dimensions, where m and d are both positive integers and m>d, and the first machine learning model is trained based on the initialization parameters of the second machine learning model.

[0265] Step 908: Input the target feature into the second machine learning model to obtain the prediction result of the candidate object.

[0266] Step 910: Recommend candidate objects whose prediction results meet preset conditions to the user through the caller.

[0267] The recommendation method provided in the embodiment of this specification is applied to the local server and provides an API interface for the caller. After receiving the API call request sent by the caller, the data processing method receives the access request sent by the caller, and then quickly and accurately recommends candidate objects to the user of the caller based on the user identifier and candidate objects carried in the access request through the recommendation method, and returns the candidate objects to the caller to improve the user experience.

[0268] Corresponding to the above method embodiment, this specification also provides a first recommended device embodiment, Fig.10 FIG. 1 shows a schematic diagram of the structure of a first recommended device provided by an embodiment of this specification. Fig.10 As shown, the device comprises:

[0269] The first request receiving module 1002 is configured to receive a t-th access request, wherein the access request carries a user identifier of a user, and t is a positive integer;

[0270] A first high-dimensional feature acquisition module 1004 is configured to acquire initial features including m dimensions based on the user identifier;

[0271] A first low-dimensional feature acquisition module 1006 is configured to reduce the dimension of the initial feature according to a first machine learning model to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d, and the first machine learning model is trained based on the initialization parameters of the second machine learning model;

[0272] A first prediction module 1008 is configured to input the target feature into a second machine learning model to obtain a prediction result of the candidate object;

[0273] The first recommendation module 1010 is configured to recommend candidate objects whose prediction results meet preset conditions to the user.

[0274] Optionally, the second machine learning model includes a prediction model;

[0275] Accordingly, the training method of the prediction model includes:

[0276] Acquire historical sample data of the first dimension, wherein the historical sample data includes attribute features of a sample user, object features of a sample candidate object corresponding to the sample user, cross-combination features of the sample user and the sample candidate object, and corresponding labels;

[0277] Performing dimensionality reduction processing on the historical sample data of the first dimension to obtain historical sample data of a second dimension, wherein the first dimension is greater than the second dimension;

[0278] The prediction model is trained according to the historical sample data of the second dimension to obtain the trained prediction model and initialization parameters of the prediction model, and the prediction model outputs the scores of the sample candidate objects.

[0279] Optionally, the first machine learning model includes a dimensionality reduction model;

[0280] Accordingly, the training method of the dimensionality reduction model includes:

[0281] Obtaining historical sample data of the first dimension and initialization parameters of the prediction model;

[0282] The dimension reduction model is trained according to the historical sample data of the first dimension and the initialization parameters of the prediction model to obtain the trained dimension reduction model,

[0283] The dimensionality reduction model outputs historical sample data of the second dimension corresponding to the historical sample data of the first dimension.

[0284] Optionally, the device further includes:

[0285] The feedback information receiving module is configured to receive the user's behavior feedback information on the candidate object whose prediction result meets the preset condition.

[0286] Optionally, the device further includes:

[0287] an initial feature acquisition module, configured to update the initial feature based on the behavior feedback information to obtain an updated initial feature;

[0288] A target feature acquisition module is configured to reduce the dimension of the updated initial feature according to the dimension reduction model to obtain an updated target feature;

[0289] The first retraining module is configured to retrain the prediction model based on the updated target feature to obtain a retrained prediction model and updated initialization parameters of the retrained prediction model.

[0290] Optionally, the initial feature acquisition module is further configured to:

[0291] Determine, based on the behavior feedback information, newly added attribute information of the user, object features of n candidate objects for the user, and cross-combination features of the user and the candidate objects to obtain newly added initial features;

[0292] The newly added initial feature is superimposed on the initial feature to obtain the updated initial feature.

[0293] Optionally, the device further includes:

[0294] A judgment module is configured to judge whether t is greater than d 2 And whether t is greater than n,

[0295] The second retraining module is configured to, if yes, train the dimensionality reduction model according to the updated initial features and the updated initialization parameters to obtain a retrained dimensionality reduction model.

[0296] Optionally, the device further includes:

[0297] The execution module is configured to determine whether t is greater than d 2 And if the result of whether t is greater than n is no, t is incremented by 1 and the step of receiving the tth access request is continued.

[0298] Optionally, the device further includes:

[0299] A first initialization module is configured to determine the second machine learning model and set initialization parameters for the second machine learning model;

[0300] The second initialization module is configured to determine the first machine learning model, train the first machine learning model based on the initialization parameters, and the first machine learning model reduces the dimension of the input of the second machine learning.

[0301] Optionally, the first machine learning model includes a multi-layer perceptron model, a convolutional neural network model, a deep neural network model or a graph convolutional neural network model;

[0302] The second machine learning model includes a contextual bandit model.

[0303] Optionally, the first machine learning model includes a supervision layer and a loss function layer, wherein the supervision layer uses the initialization parameters in the first machine learning model to make forward predictions, and the loss function layer calculates the difference between the predicted result of the candidate object predicted by the supervision layer and the actual score of the candidate object.

[0304] Optionally, the candidate objects include video covers, mobile phone applications, network connection methods and / or commodities.

[0305] The above is a schematic scheme of the first recommendation device of this embodiment. It should be noted that the technical scheme of the target data determination device and the technical scheme of the first recommendation method mentioned above are of the same concept, and the details of the technical scheme of the recommendation device that are not described in detail can all be referred to the description of the technical scheme of the first recommendation method mentioned above.

[0306] Corresponding to the above method embodiment, this specification also provides a video cover recommendation device embodiment. Fig.11 A schematic structural diagram of a video cover recommendation device provided by an embodiment of the present specification is shown.

[0307] like Fig.11 As shown, the device comprises:

[0308] The second request receiving module 1102 is configured to receive a recommendation request for a video cover, wherein the recommendation request carries a user identifier of a user;

[0309] The second high-dimensional feature acquisition module 1104 is configured to acquire attribute features of the user, image features of n video covers of the user, and cross-combination features of the user and the video covers based on the user identifier, so as to obtain initial features including m dimensions;

[0310] A second low-dimensional feature acquisition module 1106 is configured to reduce the dimension of the initial feature according to the depth encoder to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d;

[0311] The second prediction module 1108 is configured to input the target feature into a contextual bandit model to obtain a prediction result of the video cover;

[0312] The second recommendation module 1110 is configured to recommend the video cover whose prediction result meets the preset condition to the user.

[0313] The above is a schematic scheme of a video cover recommendation device of this embodiment. It should be noted that the technical scheme of the video cover recommendation device and the technical scheme of the above-mentioned video cover recommendation method belong to the same concept, and the details not described in detail in the technical scheme of the video cover recommendation device can be referred to the description of the technical scheme of the above-mentioned video cover recommendation method.

[0314] Corresponding to the above method embodiment, this specification also provides an application recommendation device embodiment. Fig.12 A schematic diagram of the structure of an application recommendation device provided by an embodiment of the present specification is shown.

[0315] like Fig.12 As shown, the device comprises:

[0316] The third request receiving module 1202 is configured to receive a recommendation request for an application, wherein the recommendation request carries a user identifier of a user;

[0317] The third high-dimensional feature acquisition module 1204 is configured to acquire attribute features of the user, attribute features of n applications for the user, and cross-combination features of the user and the applications based on the user identifier to obtain initial features including m dimensions;

[0318] A third low-dimensional feature acquisition module 1206 is configured to reduce the dimension of the initial feature according to the depth encoder to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d;

[0319] The third prediction module 1208 is configured to input the target feature into the contextual bandit model to obtain a prediction result of the application;

[0320] The third recommendation module 1210 is configured to recommend the application program whose prediction result meets the preset condition to the user.

[0321] The above is a schematic scheme of an application recommendation device of this embodiment. It should be noted that the technical scheme of the application recommendation device and the technical scheme of the above application recommendation method belong to the same concept, and the details not described in detail in the technical scheme of the application recommendation device can be referred to the description of the technical scheme of the above application recommendation method.

[0322] Corresponding to the above method embodiment, this specification also provides a product recommendation device embodiment. Fig.13 FIG. 1 is a schematic diagram showing the structure of a commodity recommendation device provided by an embodiment of the present specification. Fig.13 As shown, the device comprises:

[0323] The fourth request receiving module 1302 is configured to receive a recommendation request for a product, wherein the recommendation request carries a user identifier of a user;

[0324] The fourth high-dimensional feature acquisition module 1304 is configured to acquire attribute features of the user, attribute features of n commodities for the user, and cross-combination features of the user and the commodities based on the user identifier, so as to obtain initial features including m dimensions;

[0325] A fourth low-dimensional feature acquisition module 1306 is configured to reduce the dimension of the initial feature according to the depth encoder to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d;

[0326] The fourth prediction module 1308 is configured to input the target feature into the contextual bandit model to obtain a prediction result of the product;

[0327] The fourth recommendation module 1310 is configured to recommend commodities whose prediction results meet preset conditions to the user.

[0328] The above is a schematic scheme of a product recommendation device of this embodiment. It should be noted that the technical scheme of the product recommendation device and the technical scheme of the above product recommendation method belong to the same concept, and the details not described in detail in the technical scheme of the product recommendation device can be referred to the description of the technical scheme of the above product recommendation method.

[0329] Corresponding to the above method embodiment, this specification also provides a second recommended device embodiment, Fig.14 FIG. 2 shows a schematic diagram of the structure of a second recommended device provided by an embodiment of this specification. Fig.14 As shown, the device comprises:

[0330] The interface determination module 1402 is configured to display a target input interface for the caller based on the call request of the caller;

[0331] The fifth request receiving module 1404 is configured to receive the tth access request generated by the caller through the target input interface, wherein the access request carries a user identifier of the user and n candidate objects for the user, and t is a positive integer;

[0332] A fifth high-dimensional feature acquisition module 1406 is configured to acquire initial features including m dimensions based on the user identifier and n candidate objects for the user;

[0333] a fifth low-dimensional feature acquisition module 1408, configured to reduce the dimension of the initial feature according to the first machine learning model to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d, and the first machine learning model is trained based on the initialization parameters of the second machine learning model;

[0334] A fifth prediction module 1410 is configured to input the target feature into the second machine learning model to obtain a prediction result of the candidate object;

[0335] The fifth recommendation module 1412 is configured to recommend the candidate objects whose prediction results meet the preset conditions to the user through the caller.

[0336] The above is a schematic scheme of the second recommendation device of this embodiment. It should be noted that the technical scheme of the product recommendation device and the technical scheme of the second product recommendation method mentioned above are of the same concept, and the details not described in detail in the technical scheme of the product recommendation device can be found in the description of the technical scheme of the second product recommendation method mentioned above.

[0337] Corresponding to the above method embodiment, this specification also provides a third recommended device embodiment, Fig.15 FIG. 2 shows a schematic diagram of the structure of a third recommended device provided by an embodiment of this specification. Fig.15 As shown, the device comprises:

[0338] The sixth request receiving module 1502 is configured to receive a t-th access request sent by the caller, wherein the access request carries a user identifier of the user and n candidate objects for the user, and t is a positive integer;

[0339] A sixth high-dimensional feature acquisition module 1504 is configured to acquire initial features including m dimensions based on the user identifier and n candidate objects for the user;

[0340] a sixth low-dimensional feature acquisition module 1506, configured to reduce the dimension of the initial feature according to the first machine learning model to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d, and the first machine learning model is trained based on the initialization parameters of the second machine learning model;

[0341] A sixth prediction module 1508 is configured to input the target feature into the second machine learning model to obtain a prediction result of the candidate object;

[0342] The sixth recommendation module 1510 is configured to recommend the candidate objects whose prediction results meet the preset conditions to the user through the caller.

[0343] The above is a schematic scheme of the third recommendation device of this embodiment. It should be noted that the technical scheme of the product recommendation device and the technical scheme of the third product recommendation method mentioned above are of the same concept, and the details not described in detail in the technical scheme of the product recommendation device can be found in the description of the technical scheme of the third product recommendation method mentioned above.

[0344] Fig.16 The block diagram of a computing device 1600 according to one embodiment of the present specification is shown. The components of the computing device 1600 include but are not limited to a memory 1610 and a processor 1620. The processor 1620 is connected to the memory 1610 via a bus 1630, and the database 1650 is used to store data.

[0345] The computing device 1600 also includes an access device 1640 that enables the computing device 1600 to communicate via one or more networks 1660. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1640 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0346] In one embodiment of the present specification, the above components of the computing device 1600 and Fig.16 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.16 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0347] Computing device 1600 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. Computing device 1600 may also be a mobile or stationary server.

[0348] Among them, processor 1620 is used to execute the following computer-executable instructions, and when the processor executes the computer-executable instructions, it implements the steps of any one of the recommendation methods, or implements the steps of the video cover recommendation method, or implements the steps of the application recommendation method, or implements the steps of the product recommendation method.

[0349] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the recommendation method, or video cover recommendation method, or application recommendation method, or commodity recommendation method mentioned above are of the same concept. For details not described in detail in the technical scheme of the computing device, please refer to the description of the technical scheme of the recommendation method, or video cover recommendation method, or application recommendation method, or commodity recommendation method mentioned above.

[0350] An embodiment of the present specification also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of any one of the recommendation methods, or the steps of the video cover recommendation method, or the steps of the application recommendation method, or the steps of the product recommendation method.

[0351] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the recommendation method, or video cover recommendation method, or application recommendation method, or product recommendation method described above are of the same concept. For details not described in detail in the technical scheme of the storage medium, please refer to the description of the technical scheme of the recommendation method, or video cover recommendation method, or application recommendation method, or product recommendation method described above.

[0352] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0353] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0354] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0355] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0356] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A recommendation method, comprising: Receive a t-th access request, wherein the access request carries a user identifier of a user, and t is a positive integer; Acquire initial features including m dimensions based on the user identifier; According to the first machine learning model, the initial feature is reduced in dimension to obtain a target feature including d dimensions, wherein m and d are both positive integers and m>d, the first machine learning model is trained based on the initialization parameters of the second machine learning model, the number of dimensions of the d dimensions of the target feature is determined according to the processable number of dimensions of the second machine learning model, the first machine learning model is selected according to the processable number of dimensions of the second machine learning model, and the first machine learning model includes a dimensionality reduction model; Inputting the target feature into the second machine learning model to obtain a prediction result of the candidate object, wherein the second machine learning model includes a prediction model; The candidate objects whose prediction results meet the preset conditions are recommended to the user, wherein after the candidate objects whose prediction results meet the preset conditions are recommended to the user, the method further includes: receiving behavioral feedback information from the user regarding the candidate objects whose prediction results meet the preset conditions; updating the initial features based on the behavioral feedback information to obtain updated initial features; reducing the dimensions of the updated initial features according to the dimensionality reduction model to obtain updated target features; retraining the prediction model based on the updated target features to obtain a retrained prediction model and updated initialization parameters of the retrained prediction model; and determining whether t is greater than d 2 And whether t is greater than n, t is the number of access requests, d is the number of dimensions of the target feature, and n is the number of candidate elements; if so, the dimensionality reduction model is trained according to the updated initial features and the updated initialization parameters to obtain a retrained dimensionality reduction model.

2. According to the recommendation method of claim 1, the training method of the prediction model comprises: Acquire historical sample data of the first dimension, wherein the historical sample data includes attribute features of a sample user, object features of a sample candidate object corresponding to the sample user, cross-combination features of the sample user and the sample candidate object, and corresponding labels; Performing dimensionality reduction processing on the historical sample data of the first dimension to obtain historical sample data of a second dimension, wherein the first dimension is greater than the second dimension; The prediction model is trained according to the historical sample data of the second dimension to obtain the trained prediction model and initialization parameters of the prediction model, and the prediction model outputs the scores of the sample candidate objects.

3. According to the recommendation method of claim 2, the training method of the dimensionality reduction model comprises: Obtaining historical sample data of the first dimension and initialization parameters of the prediction model; The dimension reduction model is trained according to the historical sample data of the first dimension and the initialization parameters of the prediction model to obtain the trained dimension reduction model, The dimensionality reduction model outputs historical sample data of the second dimension corresponding to the historical sample data of the first dimension.

4. The recommendation method according to claim 1, wherein the updating of the initial features based on the behavior feedback information to obtain the updated initial features comprises: Determine, based on the behavior feedback information, newly added attribute information of the user, object features of n candidate objects for the user, and cross-combination features of the user and the candidate objects to obtain newly added initial features; The newly added initial feature is superimposed on the initial feature to obtain the updated initial feature.

5. The recommendation method according to claim 1, wherein the step of determining whether t is greater than d 2 And whether t is greater than n, it also includes: If not, t is incremented by 1 and the step of receiving the tth access request is continued.

6. The recommendation method according to claim 1, before receiving the tth access request from the user, further comprising: Determine the second machine learning model and set initialization parameters for the second machine learning model; Determine the first machine learning model, train the first machine learning model based on the initialization parameters, and reduce the dimension of the input of the second machine learning by the first machine learning model.

7. The recommendation method according to claim 6, wherein the first machine learning model comprises a multi-layer perceptron model, a convolutional neural network model, a deep neural network model or a graph convolutional neural network model; The second machine learning model includes a contextual bandit model.

8. According to any one of claims 1 to 7, the first machine learning model comprises a supervision layer and a loss function layer, wherein: The supervision layer uses the initialization parameters in the first machine learning model to make forward predictions, and the loss function layer calculates the difference between the predicted result of the candidate object predicted by the supervision layer and the actual score of the candidate object.

9. According to the recommendation method of claim 1, the initial features of the m dimensions include attribute features of the user, object features of n candidate objects for the user, and cross-combination features of the user and the candidate objects.

10. The recommendation method according to claim 1, wherein the candidate objects include video covers, mobile phone applications, network connection methods and / or commodities.

11. A video cover recommendation method, comprising: receiving a recommendation request for a video cover, wherein the recommendation request carries a user identifier of a user; Based on the user identifier, acquiring attribute features of the user, image features of n video images of the user, and cross-combination features of the user and the video images to obtain initial features including m dimensions; Performing dimensionality reduction on the initial feature according to the deep encoder to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d, the number of dimensions of the d dimensions of the target feature is determined according to the number of processable dimensions of the second machine learning model, and the deep encoder includes a dimensionality reduction model; Inputting the target feature into a contextual gambling machine model to obtain a prediction result of the video image, wherein the contextual gambling machine model includes a prediction model; The video image whose prediction result meets the preset conditions is recommended to the user as a video cover, wherein after the video image whose prediction result meets the preset conditions is recommended to the user as a video cover, it also includes: receiving the user's behavioral feedback information on the video image whose prediction result meets the preset conditions as a video cover; updating the initial features based on the behavioral feedback information to obtain updated initial features; reducing the dimension of the updated initial features according to the dimensionality reduction model to obtain updated target features; retraining the prediction model based on the updated target features to obtain a retrained prediction model and updated initialization parameters of the retrained prediction model; judging whether t is greater than d 2 And whether t is greater than n, t is the number of access requests, d is the number of dimensions of the target feature, and n is the number of candidate elements; if so, the dimensionality reduction model is trained according to the updated initial features and the updated initialization parameters to obtain a retrained dimensionality reduction model.

12. A method for recommending an application, comprising: receiving a recommendation request for an application, wherein the recommendation request carries a user identifier of a user; Based on the user identifier, acquiring attribute features of the user, attribute features of n applications for the user, and cross-combination features of the user and the applications to obtain initial features including m dimensions; Performing dimensionality reduction on the initial feature according to the deep encoder to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d, the number of dimensions of the d dimensions of the target feature is determined according to the number of processable dimensions of the second machine learning model, and the deep encoder includes a dimensionality reduction model; Inputting the target feature into a contextual bandit model to obtain a prediction result of the application, wherein the contextual bandit model includes a prediction model; An application whose prediction result meets a preset condition is recommended to the user as a target application, wherein after the application whose prediction result meets the preset condition is recommended to the user as a target application, the method further includes: receiving behavioral feedback information from the user regarding the application whose prediction result meets the preset condition as the target application; updating the initial features based on the behavioral feedback information to obtain updated initial features; reducing the dimension of the updated initial features according to the dimensionality reduction model to obtain updated target features; retraining the prediction model based on the updated target features to obtain a retrained prediction model and updated initialization parameters of the retrained prediction model; judging whether t is greater than d 2 And whether t is greater than n, t is the number of access requests, d is the number of dimensions of the target feature, and n is the number of candidate elements; if so, the dimensionality reduction model is trained according to the updated initial features and the updated initialization parameters to obtain a retrained dimensionality reduction model.

13. A product recommendation method, comprising: receiving a recommendation request for a product, wherein the recommendation request carries a user identifier of a user; Based on the user identifier, the attribute characteristics of the user, the attribute characteristics of n commodities for the user, and the cross-combination characteristics of the user and the commodities are obtained to obtain initial characteristics including m dimensions; Performing dimensionality reduction on the initial feature according to the deep encoder to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d, the number of dimensions of the d dimensions of the target feature is determined according to the number of processable dimensions of the second machine learning model, and the deep encoder includes a dimensionality reduction model; Inputting the target feature into a contextual bandit model to obtain a prediction result of the commodity, wherein the contextual bandit model includes a prediction model; The product whose prediction result meets the preset condition is recommended to the user, wherein after the product whose prediction result meets the preset condition is recommended to the user, the method further includes: receiving the user's behavioral feedback information on the product whose prediction result meets the preset condition; updating the initial feature based on the behavioral feedback information to obtain an updated initial feature; reducing the dimension of the updated initial feature according to the dimensionality reduction model to obtain an updated target feature; retraining the prediction model based on the updated target feature to obtain a retrained prediction model and updated initialization parameters of the retrained prediction model; judging whether t is greater than d 2 And whether t is greater than n, t is the number of access requests, d is the number of dimensions of the target feature, and n is the number of candidate elements; if so, the dimensionality reduction model is trained according to the updated initial features and the updated initialization parameters to obtain a retrained dimensionality reduction model.

14. A recommendation method, comprising: Displaying a target input interface for the caller based on a call request of the caller; receiving a t-th access request generated by the caller through the target input interface, wherein the access request carries a user identifier of a user and n candidate objects for the user, and t is a positive integer; Acquire initial features including m dimensions based on the user identifier and n candidate objects for the user; According to the first machine learning model, the initial feature is reduced in dimension to obtain a target feature including d dimensions, wherein m and d are both positive integers and m>d, the first machine learning model is trained based on the initialization parameters of the second machine learning model, the number of dimensions of the d dimensions of the target feature is determined according to the processable number of dimensions of the second machine learning model, the first machine learning model is selected according to the processable number of dimensions of the second machine learning model, and the first machine learning model includes a dimensionality reduction model; Inputting the target feature into the second machine learning model to obtain a prediction result of the candidate object, wherein the second machine learning model includes a prediction model; The candidate objects whose prediction results meet the preset conditions are recommended to the user through the caller, wherein after the candidate objects whose prediction results meet the preset conditions are recommended to the user through the caller, it also includes: receiving the user's behavioral feedback information about the candidate objects whose prediction results meet the preset conditions through the caller; updating the initial features based on the behavioral feedback information to obtain updated initial features; reducing the dimensions of the updated initial features according to the dimensionality reduction model to obtain updated target features; retraining the prediction model based on the updated target features to obtain a retrained prediction model and updated initialization parameters of the retrained prediction model; judging whether t is greater than d 2 And whether t is greater than n, t is the number of access requests, d is the number of dimensions of the target feature, and n is the number of candidate elements; if so, the dimensionality reduction model is trained according to the updated initial features and the updated initialization parameters to obtain a retrained dimensionality reduction model.

15. A recommendation method, comprising: Receive the t-th access request sent by the caller, wherein the access request carries a user identifier of the user and n candidate objects for the user, and t is a positive integer; Acquire initial features including m dimensions based on the user identifier and n candidate objects for the user; According to the first machine learning model, the initial feature is reduced in dimension to obtain a target feature including d dimensions, wherein m and d are both positive integers and m>d, the first machine learning model is trained based on the initialization parameters of the second machine learning model, the number of dimensions of the d dimensions of the target feature is determined according to the processable number of dimensions of the second machine learning model, the first machine learning model is selected according to the processable number of dimensions of the second machine learning model, and the first machine learning model includes a dimensionality reduction model; Inputting the target feature into the second machine learning model to obtain a prediction result of the candidate object, wherein the second machine learning model includes a prediction model; The candidate objects whose prediction results meet the preset conditions are recommended to the user through the caller, wherein after the candidate objects whose prediction results meet the preset conditions are recommended to the user through the caller, it also includes: receiving the user's behavioral feedback information about the candidate objects whose prediction results meet the preset conditions through the caller; updating the initial features based on the behavioral feedback information to obtain updated initial features; reducing the dimensions of the updated initial features according to the dimensionality reduction model to obtain updated target features; retraining the prediction model based on the updated target features to obtain a retrained prediction model and updated initialization parameters of the retrained prediction model; judging whether t is greater than d 2 And whether t is greater than n, t is the number of access requests, d is the number of dimensions of the target feature, and n is the number of candidate elements; if so, the dimensionality reduction model is trained according to the updated initial features and the updated initialization parameters to obtain a retrained dimensionality reduction model.

16. A recommendation device, comprising: A first request receiving module is configured to receive a t-th access request, wherein the access request carries a user identifier of a user, and t is a positive integer; A first high-dimensional feature acquisition module is configured to acquire initial features including m dimensions based on the user identifier; A first low-dimensional feature acquisition module is configured to reduce the dimension of the initial feature according to a first machine learning model to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d, the first machine learning model is trained based on the initialization parameters of the second machine learning model, the number of dimensions of the d dimensions of the target feature is determined according to the processable number of dimensions of the second machine learning model, the first machine learning model is selected according to the processable number of dimensions of the second machine learning model, and the first machine learning model includes a dimensionality reduction model; A first prediction module is configured to input the target feature into a second machine learning model to obtain a prediction result of the candidate object, wherein the second machine learning model includes a prediction model; The first recommendation module is configured to recommend the candidate object whose prediction result meets the preset condition to the user, wherein the first recommendation module is further configured to: receive the user's behavioral feedback information on the candidate object whose prediction result meets the preset condition; update the initial feature based on the behavioral feedback information to obtain an updated initial feature; reduce the dimension of the updated initial feature according to the dimensionality reduction model to obtain an updated target feature; retrain the prediction model based on the updated target feature to obtain a retrained prediction model and an updated initialization parameter of the retrained prediction model; determine whether t is greater than d 2 And whether t is greater than n, t is the number of access requests, d is the number of dimensions of the target feature, and n is the number of candidate elements; if so, the dimensionality reduction model is trained according to the updated initial features and the updated initialization parameters to obtain a retrained dimensionality reduction model.

17. A video cover recommendation device, comprising: A second request receiving module is configured to receive a recommendation request for a video cover, wherein the recommendation request carries a user identifier of a user; A second high-dimensional feature acquisition module is configured to acquire attribute features of the user, image features of n video images of the user, and cross-combination features of the user and the video images based on the user identifier to obtain initial features including m dimensions; A second low-dimensional feature acquisition module is configured to reduce the dimension of the initial feature according to the deep encoder to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d, and the number of dimensions of the d dimensions of the target feature is determined according to the processable number of dimensions of the second machine learning model, and the deep encoder includes a dimensionality reduction model; A second prediction module is configured to input the target feature into a contextual gambling machine model to obtain a prediction result of the video image, wherein the contextual gambling machine model includes a prediction model; The second recommendation module is configured to recommend the video image whose prediction result meets the preset conditions as the video cover to the user, wherein the second recommendation module is further configured to: receive the user's behavioral feedback information regarding the video image whose prediction result meets the preset conditions as the video cover; update the initial features based on the behavioral feedback information to obtain updated initial features; reduce the dimension of the updated initial features according to the dimensionality reduction model to obtain updated target features; retrain the prediction model based on the updated target features to obtain a retrained prediction model and updated initialization parameters of the retrained prediction model; determine whether t is greater than d 2 And whether t is greater than n, t is the number of access requests, d is the number of dimensions of the target feature, and n is the number of candidate elements; if so, the dimensionality reduction model is trained according to the updated initial features and the updated initialization parameters to obtain a retrained dimensionality reduction model.

18. An application recommendation device, comprising: A third request receiving module is configured to receive a recommendation request for an application, wherein the recommendation request carries a user identifier of a user; A third high-dimensional feature acquisition module is configured to acquire attribute features of the user, attribute features of n applications for the user, and cross-combination features of the user and the applications based on the user identifier to obtain initial features including m dimensions; A third low-dimensional feature acquisition module is configured to reduce the dimension of the initial feature according to the deep encoder to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d, the number of dimensions of the d dimensions of the target feature is determined according to the processable number of dimensions of the second machine learning model, and the deep encoder includes a dimensionality reduction model; A third prediction module is configured to input the target feature into a contextual bandit model to obtain a prediction result of the application, wherein the contextual bandit model includes a prediction model; The third recommendation module is configured to recommend the application whose prediction result meets the preset conditions as the target application to the user, wherein the third recommendation module is further configured to: receive the user's behavioral feedback information regarding the application whose prediction result meets the preset conditions as the target application; update the initial features based on the behavioral feedback information to obtain updated initial features; reduce the dimension of the updated initial features according to the dimensionality reduction model to obtain updated target features; retrain the prediction model based on the updated target features to obtain a retrained prediction model and updated initialization parameters of the retrained prediction model; determine whether t is greater than d 2 And whether t is greater than n, t is the number of access requests, d is the number of dimensions of the target feature, and n is the number of candidate elements; if so, the dimensionality reduction model is trained according to the updated initial features and the updated initialization parameters to obtain a retrained dimensionality reduction model.

19. A commodity recommendation device, comprising: a fourth request receiving module, configured to receive a recommendation request for a product, wherein the recommendation request carries a user identifier of a user; a fourth high-dimensional feature acquisition module, configured to acquire attribute features of the user, attribute features of n commodities for the user, and cross-combination features of the user and the commodities based on the user identifier, so as to obtain initial features including m dimensions; a fourth low-dimensional feature acquisition module, configured to reduce the dimension of the initial feature according to the deep encoder to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d, the number of dimensions of the d dimensions of the target feature is determined according to the processable number of dimensions of the second machine learning model, and the deep encoder includes a dimensionality reduction model; a fourth prediction module, configured to input the target feature into a contextual bandit model to obtain a prediction result of the commodity, wherein the contextual bandit model includes a prediction model; The fourth recommendation module is configured to recommend the product whose prediction result meets the preset condition to the user, wherein the fourth recommendation module is further configured to: receive the user's behavioral feedback information on the product whose prediction result meets the preset condition; update the initial feature based on the behavioral feedback information to obtain an updated initial feature; reduce the dimension of the updated initial feature according to the dimensionality reduction model to obtain an updated target feature; retrain the prediction model based on the updated target feature to obtain a retrained prediction model and an updated initialization parameter of the retrained prediction model; determine whether t is greater than d 2 And whether t is greater than n, t is the number of access requests, d is the number of dimensions of the target feature, and n is the number of candidate elements; if so, the dimensionality reduction model is trained according to the updated initial features and the updated initialization parameters to obtain a retrained dimensionality reduction model.

20. A recommendation device, comprising: An interface determination module, configured to display a target input interface to the caller based on a call request of the caller; a fifth request receiving module, configured to receive a t-th access request generated by the caller through the target input interface, wherein the access request carries a user identifier of a user and n candidate objects for the user, and t is a positive integer; a fifth high-dimensional feature acquisition module, configured to acquire initial features including m dimensions based on the user identifier and n candidate objects for the user; a fifth low-dimensional feature acquisition module, configured to reduce the dimension of the initial feature according to the first machine learning model to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d, the first machine learning model is trained based on the initialization parameters of the second machine learning model, the number of dimensions of the d dimensions of the target feature is determined according to the processable number of dimensions of the second machine learning model, the first machine learning model is selected according to the processable number of dimensions of the second machine learning model, and the first machine learning model includes a dimensionality reduction model; A fifth prediction module is configured to input the target feature into the second machine learning model to obtain a prediction result of the candidate object, wherein the second machine learning model includes a prediction model; The fifth recommendation module is configured to recommend the candidate objects whose prediction results meet the preset conditions to the user through the caller, wherein the fifth recommendation module is further configured to: receive the user's behavioral feedback information on the candidate objects whose prediction results meet the preset conditions through the caller; update the initial features based on the behavioral feedback information to obtain updated initial features; reduce the dimension of the updated initial features according to the dimensionality reduction model to obtain updated target features; retrain the prediction model based on the updated target features to obtain a retrained prediction model and updated initialization parameters of the retrained prediction model; determine whether t is greater than d 2 And whether t is greater than n, t is the number of access requests, d is the number of dimensions of the target feature, and n is the number of candidate elements; if so, the dimensionality reduction model is trained according to the updated initial features and the updated initialization parameters to obtain a retrained dimensionality reduction model.

21. A recommendation device, comprising: a sixth request receiving module, configured to receive a t-th access request sent by a caller, wherein the access request carries a user identifier of a user and n candidate objects for the user, and t is a positive integer; a sixth high-dimensional feature acquisition module, configured to acquire initial features including m dimensions based on the user identifier and n candidate objects for the user; a sixth low-dimensional feature acquisition module, configured to reduce the dimension of the initial feature according to the first machine learning model to obtain a target feature including d dimensions, wherein m and d are both positive integers, and m>d, the first machine learning model is trained based on the initialization parameters of the second machine learning model, the number of dimensions of the d dimensions of the target feature is determined according to the processable number of dimensions of the second machine learning model, the first machine learning model is selected according to the processable number of dimensions of the second machine learning model, and the first machine learning model includes a dimensionality reduction model; a sixth prediction module, configured to input the target feature into the second machine learning model to obtain a prediction result of the candidate object, wherein the second machine learning model includes a prediction model; The sixth recommendation module is configured to recommend the candidate objects whose prediction results meet the preset conditions to the user through the caller, wherein the sixth recommendation module is further configured to: receive the user's behavioral feedback information on the candidate objects whose prediction results meet the preset conditions through the caller; update the initial features based on the behavioral feedback information to obtain updated initial features; reduce the dimension of the updated initial features according to the dimensionality reduction model to obtain updated target features; retrain the prediction model based on the updated target features to obtain a retrained prediction model and updated initialization parameters of the retrained prediction model; determine whether t is greater than d 2 And whether t is greater than n, t is the number of access requests, d is the number of dimensions of the target feature, and n is the number of candidate elements; if so, the dimensionality reduction model is trained according to the updated initial features and the updated initialization parameters to obtain a retrained dimensionality reduction model.

22. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, wherein when the processor executes the computer-executable instructions, it implements the steps of the recommendation method described in any one of claims 1 to 10, 14, and 15, or implements the steps of the video cover recommendation method described in claim 11, or implements the steps of the application recommendation method described in claim 12, or implements the steps of the product recommendation method described in claim 13.

23. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the recommendation method described in any one of claims 1 to 10, 14, and 15, or the steps of the video cover recommendation method described in claim 11, or the steps of the application recommendation method described in claim 12, or the steps of the product recommendation method described in claim 13.

Images