Method and device for information recommendation

Abstract

The present specification discloses a method and device for information recommendation, which includes: obtaining the business data of a user and the scenario data of a recommendation scenario in which the user is currently located; determining the intention information corresponding to the user based on the business data, wherein the intention information is used to characterize the operation tendency of the user when facing different types of recommendation information in the recommendation scenario; inputting the business data, the scenario data and the intention information into a parameter reinforcement learning model selected to match the intention corresponding to the intention information, and determining the intention parameters corresponding to the user; for each candidate recommendation information, adjusting the initial recommendation degree corresponding to the candidate recommendation information based on the intention information and the intention parameters, and obtaining the adjusted recommendation degree corresponding to the candidate recommendation information; and recommending information to the user based on the adjusted recommendation degree corresponding to each candidate recommendation information.

Description

Technical Field

[0001] The present invention relates to the field of information recommendation, and in particular to a method and device for information recommendation. Background Art

[0002] With the development of information recommendation technology, recommending information to users based solely on the click-through rate of recommended information can no longer meet the current user and market needs. Frequently recommending to users the type of information they have purchased or clicked on will not only make users tired of this type of information, but also make the recommended content of the recommendation system increasingly limited, seriously affecting the user experience.

[0003] In the currently used technologies, in order to increase the diversity of information recommended to users, a fixed proportion is usually set for different types of recommended information, and information is recommended to users according to the proportion. For example, a certain proportion is set for information with a high user click rate and information that has not been recommended to users, and the number of recommendations for different types of information is determined based on the proportion, and then information is recommended to users. In addition, the information type is forced to be different as much as possible each time information is recommended to users to increase the diversity of recommended information. However, the above methods ignore the browsing habits of different users. Some users have clear browsing purposes when browsing information, while some users prefer to browse different types of information. If information is recommended to users blindly for the purpose of increasing the diversity of recommended information, it is bound to fail to meet the browsing habits of all users, thereby affecting the user experience of some users.

[0004] Therefore, how to enhance the diversity of recommended information while ensuring user experience based on the browsing habits of different users is an urgent problem to be solved. Summary of the invention

[0005] This specification provides a method and device for information recommendation to partially solve the above problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a method for information recommendation, including:

[0008] Obtaining the user's business data and the scenario data of the recommended scenario in which the user is currently located;

[0009] Determining, according to the business data, intention information corresponding to the user, the intention information being used to characterize the operation tendency of the user when facing different types of recommendation information in the recommendation scenario;

[0010] Inputting the business data, the scenario data and the intent information into a parameter reinforcement learning model selected to match the intent corresponding to the intent information, and determining the intent parameter corresponding to the user;

[0011] For each candidate recommendation information, adjusting the initial recommendation degree corresponding to the candidate recommendation information according to the intention information and the intention parameter to obtain an adjusted recommendation degree corresponding to the candidate recommendation information;

[0012] Recommend information to the user according to the adjusted recommendation degree corresponding to each candidate recommendation information.

[0013] Optionally, determining the intention information corresponding to the user according to the business data specifically includes:

[0014] For each type of recommendation information, judging, based on the business data, whether the user has clicked on the recommendation information of this type within a preset time range in the past, and obtaining a judgment result;

[0015] According to the judgment result, the intention information corresponding to the user is determined.

[0016] Optionally, determining the intention information corresponding to the user according to the business data specifically includes:

[0017] For each type of recommended information, determine the number of times the recommended information of this type has been recommended to the user within a preset time range in the past, and the number of clicks on the recommended information of this type by the user, based on the business data;

[0018] According to the number of recommendations and the click situation, the intention information corresponding to the user is determined.

[0019] Optionally, selecting a parameter model that matches the intent corresponding to the intent information specifically includes:

[0020] Determine, according to the business data, the user category to which the user belongs and the category characteristics corresponding to the user category;

[0021] The category features and the current time are input into a preset selection model so as to select a parameter reinforcement learning model that matches the intent corresponding to the intent information through the selection model.

[0022] Optionally, selecting a parameter reinforcement learning model that matches the intent corresponding to the intent information through the selection model specifically includes:

[0023] For each parameter reinforcement learning model, predict the probability of the parameter reinforcement learning model obtaining different reward values ​​through the selection model;

[0024] According to the predicted probability of each parameter reinforcement learning model obtaining different reward values, a parameter reinforcement learning model that matches the intention corresponding to the intention information is selected.

[0025] Optionally, for each candidate recommendation information, adjusting the initial recommendation degree corresponding to the candidate recommendation information according to the intention information and the intention parameter to obtain the adjusted recommendation degree corresponding to the candidate recommendation information specifically includes:

[0026] For each candidate recommendation information, the initial recommendation degree corresponding to the candidate recommendation information is weighted according to the determined different types of intent information and the intent parameters corresponding to each type of intent information to obtain the adjusted recommendation degree corresponding to the candidate recommendation information.

[0027] Optionally, training a parameter reinforcement learning model includes:

[0028] Obtaining business data of a specified user and historical scenario data of a historically recommended scenario in which the specified user is located;

[0029] Determining historical intention information corresponding to the designated user according to the business data of the designated user;

[0030] Inputting the business data of the designated user, the historical scenario data and the historical intent information into a parameter reinforcement learning model that matches the intent corresponding to the historical intent information, and determining the historical intent parameters corresponding to the designated user;

[0031] For each piece of historical candidate recommendation information, adjusting the initial recommendation degree corresponding to the historical candidate recommendation information according to the historical intention information and the historical intention parameter, to obtain an adjusted recommendation degree corresponding to the historical candidate recommendation information;

[0032] Recommend information to the designated user according to the adjusted recommendation degree corresponding to each historical candidate recommendation information to obtain a historical recommendation result;

[0033] Based on the historical intention information and the historical operation records of the designated user on each recommendation information in the historical recommendation results, the comprehensive reward value corresponding to the historical recommendation results is determined, and the parameter reinforcement learning model is trained with maximizing the comprehensive reward value as the optimization goal.

[0034] Optionally, determining the comprehensive reward value corresponding to the historical recommendation result according to the historical intention information and the operation record of the designated user on each recommendation information in the historical recommendation result specifically includes:

[0035] For each piece of historical candidate recommendation information, determine a reward value corresponding to the historical candidate recommendation information according to the historical intention information and the historical operation record of the designated user on the historical candidate recommendation information;

[0036] Determine the weight of the reward value corresponding to the historical candidate recommendation information according to the ranking position corresponding to the historical candidate recommendation information in the historical recommendation result, wherein the higher the ranking position corresponding to the historical candidate recommendation information, the greater the weight of the reward value corresponding to the historical candidate recommendation information;

[0037] According to the reward value corresponding to each historical candidate recommendation information and the weight of the reward value corresponding to each historical candidate recommendation information, the comprehensive reward value corresponding to the parameter reinforcement learning model is determined.

[0038] This specification provides an information recommendation device, including:

[0039] An acquisition module is used to acquire the user's business data and the scenario data of the recommended scenario in which the user is currently located;

[0040] A determination module, which determines the intention information corresponding to the user according to the business data, wherein the intention information is used to characterize the operation tendency of the user when facing different types of recommendation information in the recommendation scenario;

[0041] An input module, inputting the business data, the scenario data and the intent information into a parameter reinforcement learning model selected to match the intent corresponding to the intent information, to determine the intent parameter corresponding to the user;

[0042] an adjustment module, for each candidate recommendation information, adjusting the initial recommendation degree corresponding to the candidate recommendation information according to the intention information and the intention parameter, to obtain an adjusted recommendation degree corresponding to the candidate recommendation information;

[0043] The recommendation module recommends information to the user according to the adjusted recommendation degree corresponding to each candidate recommendation information.

[0044] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned information recommendation method is implemented.

[0045] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned information recommendation method when executing the program.

[0046] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0047] In the information recommendation method provided in this specification, the user's intention information can be determined based on the business data, and the determined intention information, scenario data and business data can be input into a parameter reinforcement learning model selected to match the intention corresponding to the user's intention information to obtain the user's corresponding intention parameters, and then the initial recommendation degree of the candidate recommendation information is adjusted, and information is recommended to the user based on the adjusted recommendation degree.

[0048] It can be seen from the above method that when recommending information to users, this manual does not use the same parameter reinforcement learning model for all users and in all scenarios. Instead, it will select an intent parameter model that matches the intent corresponding to the current user's intent information, and determine the intent parameter corresponding to the user through the intent parameter reinforcement learning model to adjust the recommendation degree of the candidate recommended information, so that the information finally recommended to the user can meet the browsing intention of each user, avoiding the impact on the user experience of different users when information is recommended to all users in the same recommendation form for the sake of user click-through rate or diversity of recommended content, thereby improving the accuracy of information recommendation to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0050] Figure 1 A flowchart of a method for recommending a method for providing information in this specification;

[0051] Figure 2 A schematic diagram of the training process of a parameter reinforcement learning model provided in this specification;

[0052] Figure 3 A flowchart of a recommended information provided in this manual;

[0053] Figure 4 A schematic diagram of a device recommended for providing information in this specification;

[0054] Figure 5 A method corresponding to the Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0056] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0057] Figure 1 A flowchart of a method for recommending information provided in this specification, comprising the following steps:

[0058] S101: Acquire the user's business data and the scenario data of the recommended scenario in which the user is currently located.

[0059] When users perform tasks such as online shopping and takeout reservations, recommendation systems often predict the click rates of users for different types of products, and then recommend information to users based on the prediction results. However, due to the different browsing habits of different users, the browsing intentions of users in different recommendation scenarios are also different. This requires that the recommended information finally recommended to users can fully meet the browsing intentions of different users in different recommendation scenarios, so as to maximize the user experience.

[0060] For example, in the online shopping scenario, some users usually have a clear purpose and hope to find the target product they want and place an order directly, and will not browse other types of products too much. Therefore, it is necessary to recommend products that meet their purchase goals to such users as much as possible. Some users do not have a clear purpose when shopping online. Such users are usually accustomed to browsing various types of products, and click or place an order when they find the products that they are more interested in. Therefore, when recommending information to such users, it is necessary to make the recommended information more abundant. For another example, in the takeaway reservation scenario, users usually pay more attention to breakfast information during the breakfast time period. At this time, it is obviously inappropriate to recommend dinner content to them. During other non-meal times, users may be more inclined to browse different types of food information.

[0061] If we blindly recommend information to users based on their historical preferences, the types of information that can be recommended to users will become fewer and fewer, which will prevent users from discovering other recommended information that may interest them, thus narrowing their interests and ultimately leading to only a few fixed types of recommended information being recommended to users each time. If we recommend information to users simply for the sake of pursuing diversity in information content, it will inevitably affect the user experience for users who want to find their purchase targets as quickly as possible. Therefore, it is necessary to make the recommended information to users fully meet the browsing intentions of different users, and to improve the diversity of recommended content while meeting the needs of different users.

[0062] Based on this, the present specification provides a method for information recommendation, in which it is necessary to obtain the user's business data and the scenario data of the recommended scenario in which the user is located. Among them, the business data may include data of the user's own information and the browsing data generated by the user during the actual browsing process. The scenario data may include the time, geographical location, and the user's current browsing depth in the recommended scenario in which the current user is located (for example, when the user browses the page through the client, the exclusive page for online shopping and the exclusive page for takeaway reservation can be regarded as different recommended scenarios). The browsing depth is used to characterize the number of pages visited by the user during this browsing process (for example, the more times the user swipes the recommended page during the browsing process and the deeper the browsing, the greater the browsing depth).

[0063] It should be noted that in this specification, before obtaining the user's business data, the user needs to be authorized first, and the user's business data can be obtained only after the user authorizes. If the user refuses to authorize, the user's business data will not be obtained. If the user first authorizes and then cancels the authorization, all the user's business data obtained after the user's authorization will be deleted.

[0064] In this specification, the execution entity used to implement the information recommendation method may refer to a designated device such as a server set up on the business platform. For the sake of ease of description, this specification only takes the server as the execution entity as an example to illustrate an information recommendation method provided in this specification.

[0065] In addition, in this specification, business scenarios for recommending information to users may include but are not limited to the above-mentioned business scenarios such as online shopping and takeout reservations, and may also include other business scenarios such as video browsing, information query, recommended attractions, recommended review information, etc. This specification does not make specific limitations on this.

[0066] S102: Determine intention information corresponding to the user according to the business data, where the intention information is used to characterize the operation tendency of the user when facing different types of recommendation information in the recommendation scenario.

[0067] After the server obtains the user's business data and the scenario data of the current recommendation scenario, it can determine the user's intention information based on the business data. Among them, the intention information can include multiple types, which are used to characterize the user's various operation tendencies when facing different types of recommended information in the recommendation scenario. For example, when the user faces information that is more in line with his purchase intention, there is often a greater probability that he will place an order for the information. When the user faces information that is relatively "novel" to him that he has not been exposed to before, he often clicks on such information out of curiosity, but he does not necessarily place an order for such information. When the user faces information that has been recommended to him many times and does not meet his purchase intention, he usually feels "tired" of such information and performs operations such as swiping away or closing such information.

[0068] Therefore, the above-mentioned user intention information is the intention information corresponding to when the user faces each type of recommended information. The type of the intention information can be specifically represented by at least one of: the user performs an order operation when facing this type of information, the user performs a click operation but does not perform an order operation when facing this type of information, and the user performs a swipe or close operation when facing this type of information.

[0069] From the above content, we can see that the intent information can actually further characterize whether the user, in the recommendation environment, is more inclined to browse various types of recommended information, or purposefully wants to place an order for a certain product or category of products or services.

[0070] Specifically, the server may input the above business data into a preset willingness value determination model, thereby determining the user's willingness values ​​for different types of information through the willingness value determination model, and then determining the user's intention information based on the willingness value. For example, when recommending products to users, the server may determine the probability of the user placing an order for each type of product through the above willingness value determination model, and use the probability as the user's willingness value for each type of product, and then determine the user's intention information for each type of product based on the willingness value.

[0071] Before using the above willingness value determination model, the server needs to first train the willingness value model, and only after the training is completed can it be deployed in the server to determine the willingness value corresponding to the user.

[0072] In this specification, the training subject used for training each model may refer to a server or a designated device such as a desktop computer or a laptop computer. For the sake of ease of expression, the following only takes the server as the training subject for model training as an example to illustrate the training of each model in this specification.

[0073] In the process of training the willingness value determination model, the server needs to obtain training samples containing historical interaction data between a specified user and different types of historical recommendation information. Then, the user characteristics of the user are input into the willingness value determination model to determine the probability of the user placing an order for information of different categories, and the willingness value determination model is trained with the optimization goal of minimizing the deviation between the determined probability of the user placing an order for information of different categories and the probability of the user actually placing an order for information of different categories.

[0074] In the process of recommending information to users, some information is not clicked or browsed by the user before. This information is relatively "novel" to the user, which can arouse the user's interest and make the user click and browse this information. Therefore, for each type of recommended information, the server can determine whether the user has clicked on this type of recommended information within the past preset time range based on the historical interaction data between the user and this type of recommended information contained in the business data, obtain the judgment result, and determine the user's corresponding intention information based on the judgment result. Among them, if the user has not clicked on this type of recommended information within the past preset time range, it means that this type of information is relatively "novel" to the user, and the user is more likely to perform a click operation on this type of information.

[0075] In addition, if the user has clicked or placed an order for a certain type of recommended information in the past, the server may mistakenly believe that the user prefers this type of recommended information, and thus recommend this type of recommended information to the user for many consecutive times, causing the user to feel "tired" of this type of information, and then perform operations such as swiping away or closing this type of recommended information. Therefore, for each type of recommended information, the server can also determine the number of times the server recommended this type of recommended information to the user within the past preset time range, and the user's clicks on this type of recommended information based on the historical interaction data between the user and the historical information contained in the business data, and determine the user's corresponding intention information based on the number of recommendations and clicks. Among them, if the user does not click on this type of recommended information when the preset number of times this type of recommended information is recommended to the user within the preset time range, it means that the user has felt "tired" of this type of recommended information, and the user is more likely to swipe away or close this type of information.

[0076] In this specification, the above-mentioned preset time range and preset number of times can be set according to actual conditions, and this specification does not make any specific limitations on this.

[0077] It should be noted that, in the actual application process, the information type corresponding to the above-mentioned recommendation information can be determined according to the specific business scenario of recommending information to users. For example, when recommending goods to users, the information type can be different types of goods such as clothing, toys, books, electrical appliances, etc., and when recommending food to users, the information type can be different types of food such as cooked food, desserts, vegetables, fruits, beverages, etc. This manual does not make specific limitations on the specific information type of the recommended information.

[0078] S103: Input the business data, the scenario data and the intent information into a parameter reinforcement learning model selected to match the intent corresponding to the intent information, and determine the intent parameters corresponding to the user.

[0079] In actual application, due to the differences in browsing intentions corresponding to different user categories (such as the user's occupational range, age group, etc.) in different time periods, different user categories can be understood as representing different types of users. In other words, a user category can represent a user group) there are certain differences. Therefore, when recommending information to users, the server can select a parameter reinforcement learning model that matches the intention corresponding to the user's intention information based on the current time and the user category to which the current user belongs, so as to meet the intentions of different user categories or users in each time period.

[0080] For example, during meal times such as breakfast, lunch, and dinner, users usually have a clear browsing purpose when browsing information, that is, to purchase breakfast, lunch, and dinner. In this case, you can choose a parameter reinforcement learning model that focuses more on user orders, so that the determined user's corresponding intention parameters are more appropriate. For another example, for young users with more free time on a daily basis, such as college students and white-collar workers, they tend to focus more on browsing different types of recommended information during non-meal times. In this case, you can choose a parameter reinforcement learning model that focuses more on the diversity of recommended content.

[0081] Specifically, the server can train multiple different parameter reinforcement learning models, and each parameter reinforcement learning model focuses on different intent information. Therefore, even for the same user in the same recommendation scenario, the intent parameters corresponding to different types of intent information determined by different parameter reinforcement learning models are also different.

[0082] When recommending information to a user, the server can first determine the user category to which the user belongs and the category features corresponding to the user category, and input the category features and the current time into a preset selection model. Through the selection model, the probability of each parameter reinforcement learning model obtaining different reward values ​​is predicted, and based on the predicted probability of each parameter reinforcement learning model obtaining different reward values, the parameter reinforcement learning model that matches the intention corresponding to the user's intention information is selected.

[0083] For example, when the user intention is clear, the selection model will determine the probability distribution of different reward values ​​obtained by each parameter reinforcement learning model for the same user intention. The server can select a parameter reinforcement learning model with a larger reward value and a higher probability of obtaining the reward value based on the above probability distribution as the parameter reinforcement learning model that matches the intention corresponding to the user intention information.

[0084] The server can then update the probability of each parameter reinforcement learning model obtaining different reward values ​​based on the actual reward value obtained when recommending information to the user this time, so that the next time information is recommended to the user, the server can select the parameter reinforcement learning model that matches the intent corresponding to the user's intent information based on the probability of each updated parameter reinforcement learning model obtaining different reward values.

[0085] After the server selects a parameter reinforcement learning model that matches the intent corresponding to the user's intent information, it can input the business data, scenario data and the determined intent information into the selected parameter reinforcement learning model that matches the intent corresponding to the intent information to determine the intent parameters corresponding to the user, where the intent parameters can include multiple types, so that each type of intent information corresponds to an intent parameter.

[0086] Before using the above parameter reinforcement learning model, the server needs to train each parameter reinforcement learning model first. After the training is completed, it can be deployed in the server to determine the user's corresponding intention parameters.

[0087] In the process of training the parameter reinforcement learning model, the server needs to obtain training samples, which include the business data of the specified user and the historical scene data of the historical recommendation scene in which the specified user is located. The server can determine the historical intention information corresponding to the specified user based on the business data of the specified user, and then input the business data, historical scene data and historical intention information of the specified user into the parameter reinforcement learning model that matches the intention corresponding to the historical intention information, determine the historical intention parameters corresponding to the specified user, and for each historical candidate recommendation information, adjust the initial recommendation degree corresponding to the candidate recommendation information according to the above historical intention information and historical intention parameters to obtain the adjusted recommendation degree corresponding to the historical candidate recommendation information, and recommend information to the specified user according to the adjusted recommendation degree corresponding to each historical candidate recommendation information to obtain the historical recommendation result.

[0088] In actual applications, the server can sort each historical candidate recommendation information from large to small according to the recommendation degree corresponding to each adjusted historical candidate recommendation information, thereby selecting the historical candidate recommendation information before the specified position, and recommending the information to the user in the order of the sorting position, and using it as the historical recommendation result.

[0089] For each historical candidate recommendation information, the server may determine the reward value corresponding to the historical candidate recommendation information according to the historical intent information and the historical operation record of the designated user on the historical candidate recommendation information. The reward value corresponding to the historical candidate recommendation information may be expressed by the following formula:

[0090] reward(i)=reward click (i)+reward pay (i)+reward tired (i)

[0091] Among them, reward(i) is the reward value corresponding to the historical candidate recommendation information i, reward click (i) is the reward value obtained when the specified user clicks on the historical candidate recommendation information i but does not place an order. pay (i) is the reward value obtained by the specified user when placing an order for the historical candidate recommendation information i, reward tired (i) is the reward value obtained by the specified user when he swipes away or closes the historical candidate recommendation information i.

[0092] Furthermore, the reward value obtained when the designated user performs a click operation on the historical candidate recommendation information i but does not perform an order operation can be determined based on the historical operation record of the designated user on the historical candidate recommendation information and the user's intention information. Then, the reward value obtained when the designated user performs a click operation on the historical candidate recommendation information i but does not perform an order operation can be expressed by the following formula:

[0093] reward click (i)=I click (i)*(1+α1*I novel (cate i ))

[0094] Among them, I click (i) indicates whether the user clicks on the historical candidate recommendation information i. If the user clicks on the historical candidate recommendation information i, then I click (i)=1, otherwise I click (i) = 0, I novel (cate i ) is the user's corresponding intention information. If the user has not clicked on the same type of recommended information as the historical candidate recommended information i within the past preset time range, then I novel (cate i )=1, otherwise I novel (cate i )=0, α1 is the model parameter corresponding to the parameter reinforcement learning model.

[0095] In addition, the reward value obtained when the designated user performs an order operation on the historical candidate recommendation information i can be determined based on the historical operation record of the designated user on the historical candidate recommendation information and the user's intention information. Then, the reward value obtained when the designated user performs an order operation on the historical candidate recommendation information i can be expressed by the following formula:

[0096] reward pay (i) = α2*I pay (i)*(1+α3*I novel (cate i ))*price i

[0097] Among them, α2 and α3 are the model parameters corresponding to the parameter reinforcement learning model, I pay (i) indicates whether the user has placed an order for the historical candidate recommendation information i. If the user has placed an order for the historical candidate recommendation information i, then I pay (i)=1, otherwise I pay (i)=0,price iis the price corresponding to the historical candidate recommendation information i.

[0098] The reward value obtained when a specified user performs a swipe away or close operation on historical candidate recommendation information i can be determined according to the user's intention information. The reward value obtained when a specified user performs a swipe away or close operation on historical candidate recommendation information i can be expressed by the following formula:

[0099] reward tired (i) = -α4*I tired (cate i )

[0100] Among them, α4 is the model parameter corresponding to the parameter reinforcement learning model, I tired (cate i ) is the user's corresponding intention information. If the user does not click on the same type of recommended information as the historical candidate recommended information i when the recommended information is recommended to the user for a preset number of times within the past preset time range, then I tired (cate i )=1, otherwise I tired (cate i )=0.

[0101] From the above formula, we can see that for the historical candidate recommendation information that the user clicks or places an order, the historical candidate recommendation information will receive a larger reward value, while for the historical candidate recommendation information that the user does not click or place an order, the reward is pay and reward click are all 0. At this time, the reward value corresponding to the candidate recommendation information is only composed of reward tired To make the determination, in this case the candidate recommendation information will receive a smaller reward value, thereby suppressing the information type that has been recommended to the user many times in the past and the user has not clicked on it, so that the number of times this type of recommended information is recommended to the user in the future will be reduced, or this type of recommended information will not be recommended to the user.

[0102] After the server determines the reward value corresponding to each historical candidate recommendation information, the server can determine the weight of the reward value corresponding to each historical candidate recommendation information based on the corresponding ranking position of each historical candidate recommendation information in the historical recommendation results, wherein, for each historical candidate recommendation information, the higher the ranking position corresponding to the historical candidate recommendation information, the greater the weight of the reward value corresponding to the historical candidate recommendation information.

[0103] Then, the server can determine the comprehensive reward value corresponding to the above historical recommendation result according to the reward value corresponding to each historical candidate recommendation information and the weight of the reward value corresponding to each historical candidate recommendation information. The comprehensive reward value corresponding to the historical recommendation result can be expressed by the following formula:

[0104]

[0105] Among them, Reward is the comprehensive reward value corresponding to the above historical recommendation results, W π(i) is the weight corresponding to the historical candidate recommendation information i, where the higher the ranking position of the historical candidate recommendation information i, the higher the weight W π(i) The larger the value, the later the ranking position of the historical candidate recommendation information i. π(i) The smaller.

[0106] The server can train the parameter reinforcement learning model with maximizing the above-mentioned comprehensive reward value as the optimization goal until the parameter reinforcement learning model meets the training goal and deploy it in the server, wherein the training goal can be: the reward value of the parameter reinforcement learning model reaches a preset threshold, or reaches a preset number of training times, wherein the above-mentioned preset threshold and preset number of training times can be set according to actual conditions, and this manual does not make specific limitations on this.

[0107] For ease of understanding, this manual also provides a training flow diagram of a parameter reinforcement learning model, such as Figure 2 shown.

[0108] Figure 2 A schematic diagram of the training process of a parameter reinforcement learning model provided in this manual.

[0109] Among them, the business data of the specified user, the historical intention information of the specified user and the historical scene data are used as the state input of the parameter reinforcement learning model. The agent of the parameter reinforcement learning model outputs the corresponding action (that is, the intention parameter corresponding to the user), and then the historical recommendation results of each historical candidate recommendation information are obtained through the intention parameter. The comprehensive reward value corresponding to the above-mentioned historical candidate recommendation information is determined according to the historical recommendation result, and then the model parameters of the parameter reinforcement learning model are updated according to the comprehensive reward value.

[0110] S104: For each candidate recommendation information, adjust the initial recommendation degree corresponding to the candidate recommendation information according to the intention information and the intention parameter to obtain an adjusted recommendation degree corresponding to the candidate recommendation information.

[0111] After the server obtains the intention parameters corresponding to the user, for each candidate recommendation information, the server can weight the initial recommendation degree corresponding to the candidate recommendation information according to the determined different types of intent information and the intention parameters corresponding to each type of intent information, to obtain the adjusted recommendation degree corresponding to the candidate recommendation information.

[0112] Specifically, the server may determine the recommendation degree determined by each type of intent information according to each type of intent information and the intent parameters corresponding to each type of intent information, and then, according to the recommendation degree determined by each type of intent information, weight the initial recommendation degree corresponding to the candidate recommendation information to obtain the adjusted recommendation degree corresponding to the candidate recommendation information, and the adjusted recommendation degree corresponding to the candidate recommendation information may be expressed by the following formula:

[0113] ReRankScore i

[0114] =RankScore i *IntentionScore i *NovelScore i *TiredScore i -1

[0115] Among them, ReRankScore i is the adjusted recommendation degree corresponding to the candidate recommendation information i, RankScore i is the initial recommendation degree corresponding to the candidate recommendation information i, IntentionScore i is the recommendation degree determined according to the user's willingness value. Accordingly, the recommendation degree can be expressed by the following formula:

[0116] IntentionScore i =1+w1*Intention(cate i )

[0117] IntentionScore i Intention(cate i ) is the intention value determined by the intention value determination model, and w1 is the intention parameter corresponding to this type of intention information.

[0118] NovelScore i The recommendation degree is determined based on the intention information obtained based on whether the user has clicked on the recommendation information of the same type as the candidate recommendation information i within the past preset time range. Accordingly, the recommendation degree can be expressed by the following formula:

[0119] NovelScore i =1+w2*I novel (cate i )

[0120] w2 is the intent parameter corresponding to the type of intent information. If the user has not clicked on the same type of recommendation information as the historical candidate recommendation information i within the past preset time range, then I novel (cate i )=1, otherwise I novel (cate i )=0.

[0121] TiredScore i -1 The recommendation degree is determined based on the number of times the same type of recommendation information as the historical candidate recommendation information i is recommended to the user within the past preset time range, and the intention information obtained from the user's clicks on the recommendation information of this type. Accordingly, the recommendation degree can be expressed by the formula:

[0122] TiredScore i =1+w2*I tired (cate i )

[0123] w3 is the intention parameter corresponding to the type of intention information. If the user has not clicked on any of the recommended information of the same type as the candidate recommendation information i recommended to the user for a preset number of times within the past preset time range, then I tired (cate i )=1, otherwise I tired (cate i )=0.

[0124] It can be seen from the above formula that if the user does not click on the same type of recommended information when it is recommended to the user a preset number of times within the preset time range in the past, the larger the intention parameter corresponding to the user's intention of this type, the smaller the adjusted recommendation degree corresponding to the candidate recommendation information; the smaller the intention parameter corresponding to the user's intention of this type, the larger the adjusted recommendation degree corresponding to the candidate recommendation information.

[0125] In addition, since IntentionScore i 、NovelScore i and TiredScore i , and the initial recommendation RankScore i The relationship between is a multiplication relationship, so it can be considered that through IntentionScore i 、NovelScorei 、TiredScore i The initial recommendation degree RankScore is weighted, that is, according to the determined different types of intent information and the intent parameters corresponding to each type of intent information, the initial recommendation degree corresponding to the candidate recommendation information is weighted to obtain the adjusted recommendation degree corresponding to the candidate recommendation information.

[0126] It should be noted that the above initial recommendation degree can be determined by the server through a click-throughrate (CTR) estimation model, and the click-through rates corresponding to different types of recommendation information are determined through the CTR model, and then the initial recommendation degree corresponding to each candidate recommendation information is determined according to the size of the click-through rate corresponding to each candidate recommendation information. Of course, it can also be determined according to the matching degree of each candidate recommendation information with the current user, and this specification does not make specific limitations on this.

[0127] It should be pointed out that there is an essential difference between the reward value mentioned in this specification and the adjusted recommendation degree. The reward value mentioned in this specification is determined based on the user's user intention, the recommendation result of information recommendation to the user and the user's actual operation record of the recommended information. Although the user will obtain different reward values ​​when performing different operations, the reward value and the final comprehensive reward value are mainly used in the training stage of the parameter reinforcement learning model, and the adjusted recommendation degree refers to the weighted recommendation degree of the user's initial recommendation degree according to the user's intention information and intention parameters. The adjusted recommendation degree is mainly used in the actual application stage of recommending information to users, so as to determine the recommendation result of information recommendation to users in the actual application process through the adjusted recommendation degree.

[0128] S105: Recommending information to the user according to the adjusted recommendation degree corresponding to each candidate recommendation information.

[0129] After the server determines the adjusted recommendation degree corresponding to each candidate recommendation information, it can sort each candidate recommendation information in descending order according to the adjusted recommendation degree corresponding to each candidate recommendation information, so that in the sorting result, each candidate recommendation information before the specified sorting position or a specified number of candidate recommendation information is selected as the information to be recommended, and then the information is recommended to the user in the order of the sorting position.

[0130] For ease of understanding, this manual also provides a flowchart of information recommendation, such as Figure 3 shown.

[0131] Figure 3 A flowchart of information recommendation provided in this manual.

[0132] Among them, before recommending information to the user, the category characteristics of the user category to which the user belongs and the time range corresponding to the current time are first determined, and the category characteristics and time range are input into the selection model, so that the parameter reinforcement learning model that matches the intention corresponding to the user intention information is selected from the pre-trained parameter reinforcement learning models through the selection model, and then the business data, scenario data and user intention information are used as the state input of the parameter reinforcement learning model, and the corresponding action (that is, the intention parameter corresponding to the user) is input through the intelligent agent of the reinforcement learning model, so as to determine the final recommendation result for each candidate recommendation information according to the intention parameter.

[0133] It should be noted that all actions of acquiring signals, information or data in this manual are performed in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0134] It can be seen from the above method that when recommending information to users, this manual will select an intent parameter model that matches the intent corresponding to the current user's intent information, and determine the intent parameters corresponding to different types of user intent information through the intent parameter reinforcement learning model, so as to adjust the initial recommendation degree of each candidate recommended information according to each intent information and intent parameter, so that the information finally recommended to the user can meet the browsing intention of each user, avoiding the forced recommendation of information to all users in the same recommendation form for the sake of user click-through rate or diversity of recommended content.

[0135] The above are one or more methods for implementing information recommendation in this specification. Based on the same idea, this specification also provides a corresponding information recommendation device, such as Figure 3 shown.

[0136] Figure 4 A schematic diagram of a device for providing information recommended for this specification, comprising:

[0137] The acquisition module 401 is used to acquire the user's business data and the scenario data of the recommended scenario in which the user is currently located;

[0138] A determination module 402 is used to determine the intention information corresponding to the user according to the business data, where the intention information is used to characterize the operation tendency of the user when facing different types of recommendation information in the recommendation scenario;

[0139] An input module 403, configured to input the business data, the scenario data and the intent information into a parameter reinforcement learning model selected to match the intent corresponding to the intent information, and determine the intent parameter corresponding to the user;

[0140] An adjustment module 404 is used to adjust, for each candidate recommendation information, an initial recommendation degree corresponding to the candidate recommendation information according to the intention information and the intention parameter, to obtain an adjusted recommendation degree corresponding to the candidate recommendation information;

[0141] The recommendation module 405 is used to recommend information to the user according to the adjusted recommendation degree corresponding to each candidate recommendation information.

[0142] Optionally, the determination module 402 is specifically used to, for each type of recommendation information, determine, based on the business data, whether the user has clicked on the recommendation information of this type within a preset time range in the past, and obtain a determination result; based on the determination result, determine the intention information corresponding to the user.

[0143] Optionally, the determination module 402 is specifically used to determine, for each type of recommendation information, based on the business data, the number of times that this type of recommendation information has been recommended to the user within a preset time range in the past, as well as the number of clicks on this type of recommendation information by the user; and determine the user's corresponding intention information based on the number of recommendations and the clicks.

[0144] Optionally, the input module 403 is specifically used to determine, based on the business data, the user category to which the user belongs and the category characteristics corresponding to the user category; input the category characteristics and the current time into a preset selection model, so as to select a parameter reinforcement learning model that matches the intention corresponding to the intention information through the selection model.

[0145] Optionally, the input module 403 is specifically used to predict, for each parameter reinforcement learning model, the probability of the parameter reinforcement learning model obtaining different reward values ​​through the selection model; and select a parameter reinforcement learning model that matches the intention corresponding to the intention information based on the predicted probability of each parameter reinforcement learning model obtaining different reward values.

[0146] Optionally, the adjustment module 404 is specifically used to weight the initial recommendation degree corresponding to each candidate recommendation information according to the determined different types of intent information and the intent parameters corresponding to each type of intent information, so as to obtain the adjusted recommendation degree corresponding to the candidate recommendation information.

[0147] Optionally, the device further comprises:

[0148] Training module 406 is used to obtain the business data of a designated user and the historical scenario data of the historical recommendation scenario in which the designated user is located; determine the historical intention information corresponding to the designated user based on the business data of the designated user; input the business data of the designated user, the historical scenario data and the historical intention information into a parameter reinforcement learning model that matches the intention corresponding to the historical intention information to determine the historical intention parameters corresponding to the designated user; for each historical candidate recommendation information, adjust the initial recommendation degree corresponding to the historical candidate recommendation information based on the historical intention information and the historical intention parameters to obtain the adjusted recommendation degree corresponding to the historical candidate recommendation information; recommend information to the designated user based on the adjusted recommendation degree corresponding to each historical candidate recommendation information to obtain a historical recommendation result; determine the comprehensive reward value corresponding to the historical recommendation result based on the historical intention information and the historical operation records of the designated user on each recommended information in the historical recommendation result, and train the parameter reinforcement learning model with maximizing the comprehensive reward value as the optimization goal.

[0149] Optionally, the training module 406 is specifically used to determine, for each historical candidate recommendation information, a reward value corresponding to the historical candidate recommendation information according to the historical intention information and the historical operation records of the designated user on the historical candidate recommendation information; determine the weight of the reward value corresponding to the historical candidate recommendation information according to the ranking position corresponding to the historical candidate recommendation information in the historical recommendation result, wherein the higher the ranking position corresponding to the historical candidate recommendation information, the greater the weight of the reward value corresponding to the historical candidate recommendation information; determine the comprehensive reward value corresponding to the historical recommendation result according to the reward value corresponding to each historical candidate recommendation information and the weight of the reward value corresponding to each historical candidate recommendation information.

[0150] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A method of providing information recommendations.

[0151] This manual also provides Figure 5 The one shown corresponds to Figure 1 A schematic diagram of the electronic device. Figure 5 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0152] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0153] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0154] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0155] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0156] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0157] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0158] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0159] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0160] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0161] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0162] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0163] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0164] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0165] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing scenarios where tasks are performed by remote processing devices connected through a communication network. In distributed computing scenarios, program modules may be located in local and remote computer storage media including storage devices.

[0166] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0167] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A method for information recommendation, characterized in that: include: Obtaining the user's business data and the scenario data of the recommended scenario in which the user is currently located; Determining, according to the business data, intention information corresponding to the user, the intention information being used to characterize the operation tendency of the user when facing different types of recommendation information in the recommendation scenario; Inputting the business data, the scenario data and the intent information into a parameter reinforcement learning model selected to match the intent corresponding to the intent information, and determining the intent parameter corresponding to the user; For each candidate recommendation information, adjusting the initial recommendation degree corresponding to the candidate recommendation information according to the intention information and the intention parameter to obtain an adjusted recommendation degree corresponding to the candidate recommendation information; Recommending information to the user according to the adjusted recommendation degree corresponding to each candidate recommendation information; Training parameter reinforcement learning model, including: Obtaining business data of a specified user and historical scenario data of a historically recommended scenario in which the specified user is located; Determining historical intention information corresponding to the designated user according to the business data of the designated user; Inputting the business data of the designated user, the historical scenario data and the historical intent information into a parameter reinforcement learning model that matches the intent corresponding to the historical intent information, and determining the historical intent parameters corresponding to the designated user; For each piece of historical candidate recommendation information, adjusting the initial recommendation degree corresponding to the historical candidate recommendation information according to the historical intention information and the historical intention parameter, to obtain an adjusted recommendation degree corresponding to the historical candidate recommendation information; Recommend information to the designated user according to the adjusted recommendation degree corresponding to each historical candidate recommendation information to obtain a historical recommendation result; Based on the historical intention information and the historical operation records of the designated user on each recommendation information in the historical recommendation results, the comprehensive reward value corresponding to the historical recommendation results is determined, and the parameter reinforcement learning model is trained with maximizing the comprehensive reward value as the optimization goal.

2. The method according to claim 1, characterized in that Determining the intention information corresponding to the user according to the business data specifically includes: For each type of recommendation information, judging, based on the business data, whether the user has clicked on the recommendation information of this type within a preset time range in the past, and obtaining a judgment result; According to the judgment result, the intention information corresponding to the user is determined.

3. The method according to claim 1, characterized in that Determining the intention information corresponding to the user according to the business data specifically includes: For each type of recommended information, determine the number of times the recommended information of this type has been recommended to the user within a preset time range in the past, and the number of clicks on the recommended information of this type by the user, based on the business data; According to the number of recommendations and the click situation, the intention information corresponding to the user is determined.

4. The method according to claim 1, characterized in that Selecting a parameter model that matches the intent corresponding to the intent information specifically includes: Based on the business data, determine the user category to which the user belongs and the category features corresponding to the user category; input the category features and the current time into a preset selection model to select a parameter reinforcement learning model that matches the intent corresponding to the intent information through the selection model.

5. The method according to claim 4, characterized in that By means of the selection model, a parameter reinforcement learning model matching the intent corresponding to the intent information is selected, specifically including: For each parameter reinforcement learning model, predict the probability of the parameter reinforcement learning model obtaining different reward values ​​through the selection model; According to the predicted probability of each parameter reinforcement learning model obtaining different reward values, a parameter reinforcement learning model that matches the intention corresponding to the intention information is selected.

6. The method according to claim 1, characterized in that For each candidate recommendation information, adjusting the initial recommendation degree corresponding to the candidate recommendation information according to the intention information and the intention parameter to obtain the adjusted recommendation degree corresponding to the candidate recommendation information specifically includes: For each candidate recommendation information, the initial recommendation degree corresponding to the candidate recommendation information is weighted according to the determined different types of intent information and the intent parameters corresponding to each type of intent information to obtain the adjusted recommendation degree corresponding to the candidate recommendation information.

7. The method according to claim 1, characterized in that Determining the comprehensive reward value corresponding to the historical recommendation result according to the historical intention information and the operation record of the designated user on each recommendation information in the historical recommendation result specifically includes: For each piece of historical candidate recommendation information, determine a reward value corresponding to the historical candidate recommendation information according to the historical intention information and the historical operation record of the designated user on the historical candidate recommendation information; Determine the weight of the reward value corresponding to the historical candidate recommendation information according to the ranking position corresponding to the historical candidate recommendation information in the historical recommendation result, wherein the higher the ranking position corresponding to the historical candidate recommendation information, the greater the weight of the reward value corresponding to the historical candidate recommendation information; The comprehensive reward value corresponding to the historical recommendation result is determined according to the reward value corresponding to each historical candidate recommendation information and the weight of the reward value corresponding to each historical candidate recommendation information.

8. An information recommendation device, characterized in that: include: An acquisition module is used to acquire the user's business data and the scenario data of the recommended scenario in which the user is currently located; A determination module, which determines the intention information corresponding to the user according to the business data, wherein the intention information is used to characterize the operation tendency of the user when facing different types of recommendation information in the recommendation scenario; An input module inputs the business data, the scenario data and the intent information into a selected parameter reinforcement learning model that matches the intent corresponding to the intent information, determines the intent parameter corresponding to the user, and trains the parameter reinforcement learning model, specifically including: Obtaining business data of a specified user and historical scenario data of a historically recommended scenario in which the specified user is located; Determining historical intention information corresponding to the designated user according to the business data of the designated user; Inputting the business data of the designated user, the historical scenario data and the historical intent information into a parameter reinforcement learning model that matches the intent corresponding to the historical intent information, and determining the historical intent parameters corresponding to the designated user; For each piece of historical candidate recommendation information, adjusting the initial recommendation degree corresponding to the historical candidate recommendation information according to the historical intention information and the historical intention parameter, to obtain an adjusted recommendation degree corresponding to the historical candidate recommendation information; Recommend information to the designated user according to the adjusted recommendation degree corresponding to each historical candidate recommendation information to obtain a historical recommendation result; Determine the comprehensive reward value corresponding to the historical recommendation result according to the historical intention information and the historical operation record of the designated user on each recommendation information in the historical recommendation result, and train the parameter reinforcement learning model with maximizing the comprehensive reward value as the optimization goal; an adjustment module, for each candidate recommendation information, adjusting the initial recommendation degree corresponding to the candidate recommendation information according to the intention information and the intention parameter, to obtain an adjusted recommendation degree corresponding to the candidate recommendation information; The recommendation module recommends information to the user according to the adjusted recommendation degree corresponding to each candidate recommendation information.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images