Adaptive recommendation method and system based on context dobby machine model

Through the adaptive recommendation method based on the context multi-arm slot machine model, using deep neural network and noise adjustment technology, the recommendation problem of the recommendation system when new users and new items is solved, real-time update and efficient calculation of the recommendation system are realized. Improved recommendation accuracy and personalization.

CN118982407APending Publication Date: 2024-11-19CHANGZHOU UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410997681.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

The recommendation system is difficult to find similar users or make effective recommendations when new users and new items are cold-started, and the lack of scoring information leads to a decrease in recommendation accuracy.

Method used

Adaptive recommendation method based on the context multi-arm slot machine model is adopted, and the product context features are processed through deep neural networks, the variance of noise input is dynamically adjusted, and the model parameters are updated in combination with user feedback to achieve adaptive recommendation.

Benefits of technology

It effectively solves the cold start problem, explores and utilizes dynamically, ensures real-time updates and efficient calculations of the recommendation system, and improves the accuracy and personalization of recommendations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982407A_ABST
    Figure CN118982407A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of information technology, in particular to a self-adaptive recommendation method and system based on a context multi-arm machine model, and the method comprises the steps: initializing a deep neural network model which processes the context features of a commodity and calculates a prediction reward; calculating a predicted reward of each commodity according to the context features of all the to-be-recommended commodities; selecting and recommending the commodity with the highest predicted reward to the user to obtain recommended commodity information; checking the selection times of the recommended commodity, and dynamically adjusting the variance of noise input according to the selection times to obtain an adjusted noise variance; and recording feedback of the user to recommended commodities, adjusting noise, updating parameters of the deep neural network by minimizing a loss function, and performing adaptive recommendation based on the updated deep neural network model. According to the method, the cold start problem is effectively solved, exploration and utilization are dynamically balanced, and real-time updating and efficient calculation of a recommendation system are ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of information technology technology, and in particular to an adaptive recommendation method and system based on a contextual multi-armed bandit model. Background Art

[0002] With the rise of artificial intelligence and the rapid development of the Internet economy, the accuracy of recommendation systems has been continuously improved. However, in practical applications, recommendation systems often face the cold start problem. When a new user enters the system, due to the lack of rating information, it is difficult for the system to find similar users for effective recommendations. Similarly, new items also face similar challenges because no users have rated them.

[0003] The information disclosed in this background technology section is only intended to deepen the understanding of the overall background technology of the present disclosure, and should not be regarded as acknowledging or suggesting in any form that the information constitutes the prior art known to those skilled in the art. Summary of the invention

[0004] The present invention provides an adaptive recommendation method and system based on a contextual multi-armed bandit model, thereby effectively solving the problems pointed out in the background technology.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] An adaptive recommendation method based on a contextual multi-armed bandit model, comprising:

[0007] Initializing a deep neural network model based on a contextual multi-armed bandit, the deep neural network model processes the contextual features of the product and calculates the predicted reward, and obtains initial model parameters;

[0008] Based on the deep neural network model, the predicted reward of each product is calculated according to the contextual features of all the products to be recommended, so as to obtain the predicted reward value of each product;

[0009] Select the product with the highest predicted reward and recommend it to the user to obtain the recommended product information;

[0010] Check the number of times the recommended product is selected, and dynamically adjust the variance of the noise input according to the number of selections to obtain the adjusted noise variance;

[0011] Record user feedback on recommended products, adjust noise, and update the parameters of the deep neural network by minimizing the loss function, and make adaptive recommendations based on the updated deep neural network model.

[0012] Furthermore, a formula for dynamically adjusting noise intensity is designed, and the formula is:

[0013]

[0014] Among them, σ i is the noise intensity, t k is the number of times product k is selected, α and β are the upper and lower limits of the noise variance, and ζ is the adjustment coefficient.

[0015] Furthermore, the calculation formula of the predicted reward is:

[0016] a t = argmax i∈[k] f(x i ;θ)

[0017] Among them, a t is the recommended product in round t, argmax i∈[k] represents the selection set [k] that makes f(x i ;θ) the largest index i, f(x i ; θ) represents the predicted reward of commodity i calculated by the neural network model, x i is the context feature of product i, and θ is the parameter of the neural network model, which is used to weight the input features.

[0018] Furthermore, we explore adaptive noise addition methods in the decision-making process, including:

[0019] In each decision round, adding a noise randomly drawn from a multidimensional normal distribution with a mean of 0 and a variance of a dynamically adjusted value to the actual reward, wherein the dynamically adjusted value is calculated according to the dynamically adjusted noise intensity formula;

[0020] The noise is added to the actual reward to form a new noisy reward.

[0021] Furthermore, after the adaptive noise addition method in the exploration decision process, the optimization problem is defined as:

[0022]

[0023] in, is the loss function, t is the number of decision rounds, r i is the actual reward value of product i, γ i is the noise value, is the mean square error of the sum of the predicted reward and the noise reward, λ is the regularization parameter, is the regularization term, and θ0 is the initial parameter.

[0024] Furthermore, we explore adaptive noise addition methods in parameter space, including:

[0025] At the beginning of each iteration, adding noise to the parameters of the model, the noise being randomly drawn from a multidimensional normal distribution with a mean of 0 and a variance of a dynamically adjusted value, the dynamic adjustment value being calculated according to the dynamically adjusted noise intensity formula;

[0026] The noise is added to the parameters of the model to generate new perturbation parameters.

[0027] Furthermore, the deep neural network model includes:

[0028] Input layer, used to receive context features of products;

[0029] Hidden layers, used to process input features and extract useful information;

[0030] The output layer is used to calculate the predicted reward for each item;

[0031] Regularization layer to prevent overfitting;

[0032] Batch normalization layer, used to normalize each batch of data;

[0033] Activation function, used to introduce nonlinear transformation.

[0034] Furthermore, the context features include: user attributes, user historical behaviors, product attributes and environmental information.

[0035] Furthermore, the method further comprises:

[0036] Split the recommendation task into multiple subtasks corresponding to different user groups and product categories;

[0037] Sharing model parameters among the plurality of subtasks, optimizing the parameters through a meta-learning algorithm, updating the shared model parameters, and obtaining optimized shared model parameters;

[0038] When faced with new tasks, use a small amount of training data to quickly adjust the model parameters to obtain model parameters that are suitable for the new task.

[0039] An adaptive recommendation system based on a contextual multi-armed bandit model, the system comprising:

[0040] A model initialization module is used to initialize a deep neural network model based on a contextual multi-armed bandit, wherein the deep neural network model processes the contextual features of the product and calculates the predicted reward, and obtains initial model parameters;

[0041] A reward value acquisition module, based on the deep neural network model, calculates the predicted reward of each product according to the contextual features of all the products to be recommended, and obtains the predicted reward value of each product;

[0042] The product information acquisition module selects the product with the highest predicted reward and recommends it to the user to obtain the recommended product information;

[0043] The noise variance adjustment module checks the number of times the recommended product is selected and dynamically adjusts the variance of the noise input according to the number of times selected to obtain the adjusted noise variance;

[0044] The model parameter updating module records the user's feedback on the recommended products, adjusts the noise and updates the parameters of the deep neural network by minimizing the loss function, and performs adaptive recommendations based on the updated deep neural network model.

[0045] The technical solution of the present invention can achieve the following technical effects:

[0046] The present invention effectively solves the cold start problem, dynamically balances exploration and utilization, and ensures real-time updating and efficient calculation of the recommendation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A flowchart of an adaptive recommendation method based on a contextual multi-armed bandit model is shown;

[0049] Figure 2 Schematic diagram of the process of exploring adaptive noise addition methods in decision-making processes;

[0050] Figure 3 Schematic diagram of the process for exploring adaptive noise addition methods in parameter space;

[0051] Figure 4 Schematic diagram of the subsequent optimization steps of an adaptive recommendation method based on a contextual multi-armed bandit model. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0054] Embodiment 1

[0055] like Figure 1 As shown, the present invention provides an adaptive recommendation method based on a contextual multi-armed bandit model, the method comprising:

[0056] S1: Initialize the deep neural network model based on the contextual multi-armed bandit. The deep neural network model processes the contextual features of the product and calculates the predicted reward, and obtains the initial model parameters;

[0057] Specifically, the deep neural network model based on contextual multi-armed bandit is a recommendation system model that combines contextual information, multi-armed bandit algorithm and deep neural network. The model aims to make accurate recommendations by utilizing the contextual features of users and products, while balancing exploration and utilization. By introducing contextual information, the personalization and accuracy of recommendations are improved, and effective recommendations can be made based on current contextual information even in the absence of historical data. First, the contextual features required for the recommendation system are determined, and then the collected data is preprocessed, such as data cleaning, missing value filling and feature normalization; then, the architecture of the deep neural network is designed, and all model parameters are initialized, and the model is built and compiled using a deep learning framework (such as TensorFlow or PyTorch), and the loss function, optimization algorithm and evaluation index are configured; finally, the model is preliminarily verified to ensure that it can process contextual features normally and output predicted reward values, so as to obtain the initial model parameters and prepare for subsequent recommendation tasks.

[0058] S2: Based on the deep neural network model, the predicted reward of each product is calculated according to the contextual features of all the recommended products to obtain the predicted reward value of each product;

[0059] Specifically, first, the context features of all the items to be recommended are collected and converted into feature vectors; then, these feature vectors are standardized or normalized to ensure the consistency of the scale of the input features; then, the preprocessed feature vectors are input into the deep neural network model, and the features are extracted by processing and applying nonlinear activation functions, and finally the predicted reward value of each item is generated. In order to improve the computational efficiency, batch processing and parallel computing methods can be used to process the feature vectors of multiple items at the same time. Finally, the model outputs the predicted reward value of each item, which provides a basis for subsequent recommendation decisions.

[0060] S3: Select the product with the highest predicted reward and recommend it to the user to obtain the recommended product information;

[0061] Specifically, first, the predicted reward values ​​of all recommended products output from the deep neural network model are collected; then these reward values ​​are compared to determine which product has the highest predicted reward value; then, the product with the highest predicted reward value is selected as the recommended product, and the relevant information of the product is recorded, including product ID, name, and category; then, a recommendation result containing detailed information of the recommended product is generated, and a log of the recommendation decision is recorded for subsequent analysis and debugging; finally, the information of the recommended product is passed to the user interface or caller, displayed or returned to the user, so as to provide personalized and accurate recommendation services.

[0062] S4: Check the number of times the recommended product is selected, and dynamically adjust the variance of the noise input according to the number of times selected to obtain the adjusted noise variance;

[0063] Specifically, first, check the number of times each recommended item has been selected in historical recommendation rounds, and obtain the total number of times the current recommended item has been selected; then, dynamically adjust the variance of the noise input according to the number of selections, and smooth the growth of the selection frequency through a logarithmic function; then, increase the variance of the noise input for items with fewer selections to promote exploration, and reduce the variance of the noise input for items with more selections to improve stability; finally, record the adjusted noise variance for use in subsequent recommendations.

[0064] S5: Record user feedback on recommended products, adjust noise, and update the parameters of the deep neural network by minimizing the loss function. Perform adaptive recommendations based on the updated deep neural network model.

[0065] Specifically, first, record user feedback on recommended items, such as clicks, ratings, or purchase behaviors, and associate these feedbacks with the contextual features of the recommended items, predicted reward values, and other information; then, generate a noise value for each recommended item based on the previously dynamically adjusted noise variance; then, use user feedback and noise values ​​to calculate actual feedback, and calculate the loss function based on actual feedback and model prediction values. Through gradient descent or other optimization algorithms, update model parameters according to the gradient of the loss function, so that the model can reflect the latest user feedback and noise adjustments. The update formula is: Among them, η is the learning rate, which controls the step size of each parameter update; finally, the updated parameters are applied to the deep neural network model so that the model reflects the latest user feedback and noise adjustment, and the next round of recommendations is performed based on the updated model, so that the model can better adapt to user needs and feedback, and improve the accuracy and personalization of recommendations.

[0066] The present invention effectively solves the cold start problem, dynamically balances exploration and utilization, and ensures real-time updating and efficient calculation of the recommendation system.

[0067] As a preferred embodiment of the above, a formula for dynamically adjusting noise intensity is designed, and the formula is:

[0068]

[0069] Among them, σ i is the noise intensity, t k is the number of times product k is selected, α and β are the upper and lower limits of the noise variance, and ζ is the adjustment coefficient.

[0070] Specifically, σ i represents the adjusted noise intensity, which is based on the number of selections t of product k k Dynamic changes. Select times t k reflects the frequency of the product being selected in the historical recommendation rounds. k As the frequency of product recommendations and user feedback increases, the adjustment coefficient ζ controls the rate at which the noise intensity changes with the number of selections. A larger ζ value will result in a larger initial noise intensity. The logarithmic function ln(t k +1) smoothes the growth of the number of selections. Since the logarithmic function grows slowly, it can effectively slow down the impact of the number of selections on the noise intensity, making the noise intensity gradually decrease. At the same time, adding 1 avoids taking the logarithm of 0. The minimum function Ensure that the noise intensity does not exceed the upper limit β, even if the number of selections is small, σ i It will not exceed β, thus avoiding excessive noise affecting the stability of the recommendation. The maximum function max(α, ...) ensures that the noise intensity is not lower than the lower limit α. Even if the number of selections is large, σ i It will not be lower than α, thus ensuring a certain degree of exploration. Through this method of dynamically adjusting the noise intensity, the recommendation system can find a balance between exploration and utilization, improving the overall effect and efficiency of the recommendation.

[0071] As a preferred embodiment of the above embodiment, the calculation formula for predicting the reward is:

[0072] a t = argmax i∈[k] f(x i ;θ)

[0073] Among them, a t is the recommended product in round t, argmax i∈[k] represents the selection set [k] that makes f(x i ;θ) the largest index i, f(x i; θ) represents the predicted reward of item i calculated by the neural network model, x i is the context feature of product i, and θ is the parameter of the neural network model, which is used to weight the input features.

[0074] Specifically, the formula means that in each recommendation round, the predicted reward value of each item is calculated through the neural network model based on the contextual features of all items to be recommended, and the item with the highest predicted reward is selected for recommendation. This process ensures that the recommendation system selects the best item for recommendation in each decision-making round, thereby improving the accuracy of recommendations and user satisfaction.

[0075] As a preferred embodiment of the above, Figure 2 As shown, the adaptive noise addition method in the decision-making process is explored, including:

[0076] A10: In each decision round, a noise randomly drawn from a multidimensional normal distribution with a mean of 0 and a variance of a dynamic adjustment value is added to the actual reward. The dynamic adjustment value is calculated according to the dynamic adjustment noise intensity formula.

[0077] A20: Add noise to the actual reward to form a new noisy reward.

[0078] Specifically, in each decision round, the noise variance is first adjusted dynamically according to the current environment and historical data. The adjustment of the noise variance is based on a predefined formula, which takes into account factors such as the number of times the product is selected, so that the noise variance can change dynamically; then, the calculated noise variance is used to define a multidimensional normal distribution with a mean of 0, and a noise value is randomly drawn from the distribution; the drawn noise value is added to the actual reward to form a new noise reward. The purpose of this step is to introduce a certain degree of randomness on the basis of the actual reward to encourage the system to explore between different products and choices. In this way, the recommendation system can avoid falling into a local optimal solution too early and improve the overall performance of the recommendation system.

[0079] As a preferred embodiment of the above, after exploring the adaptive noise adding method in the decision-making process, the optimization problem is defined as:

[0080]

[0081] in, is the loss function, t is the number of decision rounds, r i is the actual reward value of product i, γ i is the noise value, is the mean square error of the sum of the predicted reward and the noise reward, λ is the regularization parameter, is the regularization term, and θ0 is the initial parameter.

[0082] Specifically, is the loss function that needs to be minimized during the optimization process, which is used to measure the difference between the model prediction value and the actual feedback; t represents the total number of decision-making rounds. In each round, the model makes a recommendation decision based on the current context features and parameters; r i That is, the actual feedback of users on the product, such as clicks, ratings, or purchases; i Add to the actual reward r i The noise values ​​in are randomly drawn from a normal distribution with mean 0 and a fixed variance. It introduces randomness to facilitate exploration; is the prediction reward f(x i ;θ) and the noise reward r i +γ i The mean square error of the sum is used to measure the difference between the model prediction value and the noise reward. By minimizing this error, the model parameter θ can better fit the actual feedback; λ is the regularization parameter, which is used to control the complexity of the model and prevent overfitting. A larger λ value can increase the weight of the regularization term and force the model parameters to tend to the initial value θ0; It is a regularization term. The regularization term ensures that the model maintains a certain stability and generalization ability during the training process by punishing excessive deviations of model parameters.

[0083] As a preferred embodiment of the above, Figure 3 As shown, adaptive noise addition methods in parameter space are explored, including:

[0084] B10: At the beginning of each iteration, noise is added to the parameters of the model. The noise is randomly drawn from a multidimensional normal distribution with a mean of 0 and a variance of a dynamic adjustment value. The dynamic adjustment value is calculated according to the dynamic adjustment noise intensity formula;

[0085] B20: Add noise to the parameters of the model to generate new perturbation parameters.

[0086] Specifically, at the beginning of each iteration, the noise variance is first dynamically adjusted according to the number of times the product is selected. The more times the product is selected, the smaller the noise variance; the fewer times the product is selected, the larger the noise variance. The purpose of this design is to add more noise in the early stage of exploration to promote exploration, and to reduce noise in the later stage to enhance stability; next, the calculated noise variance is used to define a multidimensional normal distribution with a mean of 0. A noise vector is randomly extracted from this multidimensional normal distribution, and this noise vector has the same dimension as the model parameters; finally, the extracted noise vector is added element by element to the parameters of the model to generate new perturbation parameters. In this way, at the beginning of each iteration, the parameters of the model will be perturbed by random noise to form a new parameter combination. This noise perturbation helps the model to explore in the parameter space, thereby improving the model's adaptability and generalization ability to different scenarios.

[0087] The main differences between the adaptive noise addition method in the exploration decision process and the adaptive noise addition method in the exploration parameter space are the application level, the target and the location of noise addition. The adaptive noise addition method in the exploration decision process is to add noise randomly drawn from a normal distribution with a mean of 0 and a fixed variance to the actual reward in each decision round to form a new noise reward, so as to introduce randomness in the recommendation decision, promote the exploration of new products, and increase the diversity of recommendations; while the adaptive noise addition method in the exploration parameter space is to add noise randomly drawn from a multidimensional normal distribution with a mean of 0 and a fixed variance to the parameters of the model at the beginning of each model iteration, generate new perturbation parameters, so as to introduce randomness in the parameter update process, prevent the model from falling into the local optimal solution, and enhance the generalization ability and robustness of the model.

[0088] As a preferred embodiment of the above, the deep neural network model includes:

[0089] Input layer, used to receive context features of products;

[0090] Hidden layers, used to process input features and extract useful information;

[0091] The output layer is used to calculate the predicted reward for each item;

[0092] Regularization layer to prevent overfitting;

[0093] Batch normalization layer, used to normalize each batch of data;

[0094] Activation function, used to introduce nonlinear transformation.

[0095] Specifically, the model receives the contextual features of the product and converts these features into feature vectors that the model can process through the input layer. The hidden layer consists of multiple neurons, which transform the input features through weighted summation and activation functions (such as ReLU) to extract high-level useful information. The output layer is a fully connected layer that calculates the predicted reward value of each product and outputs the expected return of the product in the current context. The regularization layer prevents the model from overfitting and improves the generalization ability by adding L2 or L1 regularization terms. The batch normalization layer normalizes the input data in each batch training so that the data has zero mean and unit variance to ensure the stability of training and accelerate model convergence. The activation function is applied between the hidden layer and the output layer, introducing nonlinear transformations to enhance the expressiveness of the model and enable it to learn complex function mapping relationships. Through these layers and processing steps, the deep neural network model can effectively perform recommendation tasks and improve the accuracy of recommendations and user satisfaction.

[0096] As a preferred embodiment of the above embodiment, the context features include: user attributes, user historical behaviors, product attributes and environmental information.

[0097] Specifically, the reason for selecting user attributes, user historical behaviors, product attributes, and environmental information as contextual features is that these features can comprehensively and accurately capture the personalized needs and preferences of users, thereby ensuring that the recommendation system can provide accurate and diverse recommendations. User attributes (such as age, gender, location, etc.) provide important information about the personalized needs and preferences of users. The attributes of different users may affect their interests and needs for products; user historical behaviors (such as browsing history, purchase records, etc.) can reflect the interests and consumption habits of users. Past behaviors are strong indicators for predicting future behaviors; product attributes (such as categories, prices, brands, etc.) help the recommendation system identify and classify products to ensure the relevance and diversity of recommendations; in addition, environmental information (such as time, location, weather, etc.) can affect user needs and behaviors. In different environments, user needs may change. For example, weather information can help the recommendation system recommend warm clothes in cold weather, time information can help the recommendation system recommend breakfast products in the morning and leisure and entertainment products in the evening, and the user's location can help the recommendation system provide localized services or product recommendations. By integrating user attributes, user historical behavior, product attributes and environmental information, the recommendation system can better understand user needs in different situations and provide more personalized recommendation services, thereby improving the relevance of recommendations and user satisfaction.

[0098] As a preferred embodiment of the above, Figure 4 As shown, the method also includes:

[0099] C10: Split the recommendation task into multiple subtasks corresponding to different user groups and product categories;

[0100] C20: Share model parameters among multiple subtasks, optimize the parameters through a meta-learning algorithm, update the shared model parameters, and obtain the optimized shared model parameters;

[0101] C30: When faced with a new task, use a small amount of training data to quickly adjust the model parameters to obtain model parameters that are suitable for the new task.

[0102] Specifically, the recommendation task is first subdivided into multiple subtasks so that the model can be optimized for different user groups and product categories, improving the accuracy and diversity of recommendations. This is achieved by classifying user groups and product categories according to user attributes, historical behaviors, and product attributes; then, the model parameters are shared among multiple subtasks, and the shared model parameters are updated through meta-learning algorithms (such as MAML) on multiple subtasks to improve the generalization ability of the model; finally, when encountering a new recommendation task, the shared model parameters optimized from the meta-learning algorithm are used as the initial value, and fine-tuned through a small amount of new task training data to quickly obtain the model parameters that adapt to the new task. This not only reduces training time and data requirements, but also improves the recommendation effect of new tasks. In this way, the recommendation system can improve the generalization ability and adaptability of the model when dealing with recommendation tasks for different user groups and product categories, thereby providing more accurate and efficient recommendation services.

[0103] Embodiment 2

[0104] Based on the same inventive concept as the adaptive recommendation method based on a contextual multi-armed bandit model in the aforementioned embodiment, the present invention further provides an adaptive recommendation system based on a contextual multi-armed bandit model, the system comprising:

[0105] Model initialization module, which initializes the deep neural network model based on the contextual multi-armed bandit. The deep neural network model processes the contextual features of the product and calculates the predicted reward, and obtains the initial model parameters;

[0106] The reward value acquisition module calculates the predicted reward of each product based on the contextual features of all recommended products based on the deep neural network model to obtain the predicted reward value of each product;

[0107] The product information acquisition module selects the product with the highest predicted reward and recommends it to the user to obtain the recommended product information;

[0108] The noise variance adjustment module checks the number of times the recommended product is selected and dynamically adjusts the variance of the noise input according to the number of times selected to obtain the adjusted noise variance;

[0109] The model parameter update module records user feedback on recommended products, adjusts noise, and updates the parameters of the deep neural network by minimizing the loss function. Based on the updated deep neural network model, adaptive recommendations are performed.

[0110] The above-mentioned recommendation system in the present invention can effectively implement an adaptive recommendation method based on a contextual multi-armed bandit model, and the technical effects that can be achieved are as described in the above-mentioned embodiments, which will not be repeated here.

[0111] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the accompanying drawings are merely exemplary illustrations of the present application as defined therein, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application is intended to include these modifications and variations.

Claims

1. An adaptive recommendation method based on a contextual multi-armed bandit model, characterized in that: include: Initializing a deep neural network model based on a contextual multi-armed bandit, the deep neural network model processes the contextual features of the product and calculates the predicted reward, and obtains initial model parameters; Based on the deep neural network model, the predicted reward of each product is calculated according to the contextual features of all the products to be recommended, so as to obtain the predicted reward value of each product; Select the product with the highest predicted reward and recommend it to the user to obtain the recommended product information; Check the number of times the recommended product is selected, and dynamically adjust the variance of the noise input according to the number of selections to obtain the adjusted noise variance; Record user feedback on recommended products, adjust noise, and update the parameters of the deep neural network by minimizing the loss function, and make adaptive recommendations based on the updated deep neural network model.

2. The adaptive recommendation method based on the contextual multi-armed bandit model according to claim 1, characterized in that: A formula for dynamically adjusting noise intensity is designed, which is: Among them, σ i is the noise intensity, t k is the number of times product k is selected, α and β are the upper and lower limits of the noise variance, and ζ is the adjustment coefficient.

3. The adaptive recommendation method based on the contextual multi-armed bandit model according to claim 1, characterized in that: The calculation formula of the prediction reward is: a t =argmax i∈[k] f(x i ; θ) Among them, a t is the recommended product in round t, argmax i∈[k] represents the selection set [k] that makes f(x i ;θ) the largest index i, f(x i ; θ) represents the predicted reward of commodity i calculated by the neural network model, x i is the context feature of product i, and θ is the parameter of the neural network model, which is used to weight the input features.

4. The adaptive recommendation method based on the contextual multi-armed bandit model according to claim 1, characterized in that: Explore methods for adaptive noise addition in the decision-making process, including: In each decision round, adding a noise randomly drawn from a multidimensional normal distribution with a mean of 0 and a variance of a dynamically adjusted value to the actual reward, wherein the dynamically adjusted value is calculated according to the dynamically adjusted noise intensity formula; The noise is added to the actual reward to form a new noisy reward.

5. The adaptive recommendation method based on the contextual multi-armed bandit model according to claim 4, characterized in that: After the adaptive noise addition method in the exploration decision process, the optimization problem is defined as: in, is the loss function, t is the number of decision rounds, r i is the actual reward value of product i, γ i is the noise value, is the mean square error of the sum of the predicted reward and the noise reward, λ is the regularization parameter, is the regularization term, and θ0 is the initial parameter.

6. The adaptive recommendation method based on the contextual multi-armed bandit model according to claim 1, characterized in that: Explore adaptive noise addition methods in parameter space, including: At the beginning of each iteration, adding noise to the parameters of the model, the noise being randomly drawn from a multidimensional normal distribution with a mean of 0 and a variance of a dynamically adjusted value, the dynamic adjustment value being calculated according to the dynamically adjusted noise intensity formula; The noise is added to the parameters of the model to generate new perturbation parameters.

7. The adaptive recommendation method based on the contextual multi-armed bandit model according to claim 1, characterized in that: The deep neural network model comprises: Input layer, used to receive context features of products; Hidden layers, used to process input features and extract useful information; The output layer is used to calculate the predicted reward for each item; Regularization layer to prevent overfitting; Batch normalization layer, used to normalize each batch of data; Activation function, used to introduce nonlinear transformation.

8. The adaptive recommendation method based on the contextual multi-armed bandit model according to claim 1, characterized in that: The context features include: user attributes, user historical behaviors, product attributes and environmental information.

9. The adaptive recommendation method based on the contextual multi-armed bandit model according to claim 1, characterized in that: The method further comprises: Split the recommendation task into multiple subtasks corresponding to different user groups and product categories; Sharing model parameters among the plurality of subtasks, optimizing the parameters through a meta-learning algorithm, updating the shared model parameters, and obtaining optimized shared model parameters; When faced with new tasks, use a small amount of training data to quickly adjust the model parameters to obtain model parameters that are suitable for the new task.

10. An adaptive recommendation system based on a contextual multi-armed bandit model, characterized in that: The system comprises: A model initialization module is used to initialize a deep neural network model based on a contextual multi-armed bandit, wherein the deep neural network model processes the contextual features of the product and calculates the predicted reward, and obtains initial model parameters; A reward value acquisition module, based on the deep neural network model, calculates the predicted reward of each product according to the contextual features of all the products to be recommended, and obtains the predicted reward value of each product; The product information acquisition module selects the product with the highest predicted reward and recommends it to the user to obtain the recommended product information; The noise variance adjustment module checks the number of times the recommended product is selected and dynamically adjusts the variance of the noise input according to the number of selections to obtain the adjusted noise variance; The model parameter updating module records the user's feedback on the recommended products, adjusts the noise and updates the parameters of the deep neural network by minimizing the loss function, and performs adaptive recommendations based on the updated deep neural network model.