Recommendation method, terminal and storage medium

Abstract

The present invention discloses a recommendation method, a terminal and a storage medium. The method comprises the following steps: inputting search keywords into a recommendation model based on a reinforcement learning framework to obtain an initial product recommendation list and sending the list to a user terminal; taking the initial product recommendation list as the user's current state information, calculating the current state information through a state value function, and obtaining an estimated value of each product keyword in the initial product recommendation list; obtaining user behavior information of the user with respect to the initial product recommendation list through the user terminal, and taking the user behavior information and the initial product recommendation list as next state information; calculating the next state information through an action state value function, and obtaining a true value of each product keyword; determining the search keyword for the next recommendation process based on the estimated value and the true value to input the recommendation model, obtaining the next product recommendation list and sending the list to the user terminal, so as to effectively ensure the accuracy of product recommendations when making product recommendations using non-user privacy information.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a recommendation method, a terminal and a storage medium. Background Art

[0002] The emergence and popularization of the Internet has brought a large amount of information to users, meeting their demand for information in the information age. However, with the rapid development of the Internet, the amount of information online has increased significantly, making it impossible for users to obtain truly useful information when faced with a large amount of information. Instead, their efficiency in using information has decreased, which has led to the problem of information overload.

[0003] To address the problem of information overload, recommendation systems have emerged. They are widely used in e-commerce, film and television recommendations, music radio station recommendations, social networks, personalized reading, personalized advertising, and other fields.

[0004] Current recommendation systems often make recommendations to users based on their historical behavior records (such as historical purchase records, historical browsing records, etc.). For example, if user A purchases novel C by author B, the purchase record is analyzed, the analysis results are quantified and ranked, and the similarities between items are established as the basis for the recommendation system.

[0005] The user's historical behavior records mean that their private information will be collected, stored, and used for recommendations. However, users do not want their private data to be leaked, but protecting user privacy will inevitably lead to a loss in recommendation performance, resulting in a decrease in recommendation accuracy and affecting the user experience.

[0006] Based on this, how to avoid the loss of product recommendation accuracy when using non-user privacy information for product recommendations has become a technical problem that needs to be solved urgently. Summary of the Invention

[0007] The main purpose of the present invention is to provide a recommendation method, terminal and storage medium, aiming to solve the problem in the prior art of loss of accuracy in product recommendation when using non-user privacy information for product recommendation.

[0008] In order to achieve the above object, the present invention provides a recommendation method, which includes:

[0009] Based on the search keywords in the user's search request and the recommendation model based on the reinforcement learning framework, an initial product recommendation list corresponding to the search request is obtained and sent to the user terminal;

[0010] Using the initial product recommendation list as the current state information of the user, calculating the current state information using a state value function, and using the obtained first value as an estimated value;

[0011] Obtaining user behavior information of the user with respect to the initial product recommendation list, and using the user behavior information and the initial product recommendation list as next state information;

[0012] Calculating the next state information using an action-value function, and using the obtained second value as a true value;

[0013] Based on the estimated value and the true value, a search keyword for a next recommendation process is determined, so as to obtain a next product recommendation list through the recommendation model and send the list to the user terminal.

[0014] Optionally, obtaining user behavior information of the user with respect to the initial product recommendation list specifically includes:

[0015] Obtaining feedback from the user regarding the initial product recommendation list;

[0016] The feedback behavior includes at least one of: click, purchase, leave, browse, and favorite;

[0017] Determine user behavior information of the user for the initial product recommendation category based on the user's feedback behavior.

[0018] Optionally, the user behavior information includes at least one of viewing behavior status information, purchasing behavior status information, leaving behavior status information, browsing behavior status information, and collecting behavior status information.

[0019] Optionally, calculating the current state information using a state value function to obtain a corresponding estimated value specifically includes:

[0020] Obtain product keywords in the initial product recommendation list using a preset word segmentation algorithm;

[0021] Extracting features of the product keywords to obtain product features;

[0022] Each of the commodity features is calculated using the state value function, and the obtained first numerical value is used as the estimated value of the corresponding commodity keyword.

[0023] Optionally, calculating the next state information using an action-value function to obtain a corresponding true value specifically includes:

[0024] Obtaining products in the initial product recommendation list corresponding to the user behavior information as user feedback products;

[0025] Extracting features from the user behavior information to obtain product-user features;

[0026] The product-user feature is calculated using an action-value function, and the obtained second value is used as the true value of the corresponding product keyword of the user.

[0027] Optionally, determining a search keyword for a next recommendation process based on the estimated value and the true value specifically includes:

[0028] updating the weight of each of the product keywords according to the estimated value and the actual value of each of the product keywords;

[0029] The search keywords used in the next recommendation process are determined according to the updated weights of the commodity keywords.

[0030] Optionally, updating the weight of each product keyword according to the estimated value and the actual value of each product keyword specifically includes:

[0031] Performing word frequency statistics on the product keywords in the initial product recommendation list to determine the word frequency of each product keyword;

[0032] The word frequency of each product keyword is used as the initial weight of the corresponding product keyword;

[0033] Adjust the initial weight of each product keyword according to the estimated value and the actual value of each product keyword.

[0034] Optionally, determining the search keywords for the next recommendation process based on the updated weights of the product keywords specifically includes:

[0035] The search keywords used for the next recommendation process are determined according to a preset greedy algorithm and the updated weights of the product keywords.

[0036] In order to achieve the above objectives, the present invention also provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in any of the recommendation methods described above.

[0037] In order to achieve the above-mentioned object, the present invention also provides a terminal, characterized in that it includes: a processor and a memory; the memory stores a computer-readable program that can be executed by the processor; when the processor executes the computer-readable program, it implements the steps in the recommendation method described in any one of the above items.

[0038] The present invention inputs the search keywords in the user's search request into a recommendation model based on a reinforcement learning framework to obtain an initial product recommendation list corresponding to the search request and sends it to the user terminal; then the initial product recommendation list is used as the user's current state information, and the current state information is calculated through a state value function to obtain an estimated value of each product keyword in the initial product recommendation list; then the user's user behavior information for the above-mentioned initial product recommendation list is obtained through the user terminal, and the user behavior information and the initial product recommendation list are used as the next state information; the next state information is calculated through an action state value function to obtain the true value of each product keyword; then, based on the estimated value and the true value, the search keyword of the next recommendation process is determined to input the above-mentioned recommendation model, and the next product recommendation list is obtained and sent to the user terminal. In the product recommendation process of the above-mentioned recommendation method, when making product recommendations using non-user privacy information, the accuracy of product recommendations can be effectively guaranteed, the user experience can be improved, and the leakage of user privacy data can be prevented. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flowchart of a recommended method provided by an embodiment of the present invention;

[0040] Figure 2 Another schematic diagram of a recommended method provided by an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of the structure of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0043] Recommender systems are widely used in various fields, such as e-commerce, video and music recommendations, social networks, personalized reading, and personalized advertising. For example, in e-commerce, recommendation systems have significantly boosted sales across the industry. Research has found that when e-commerce platforms use recommendation systems to provide personalized services to users, they can identify potential sellers, stimulate potential customer purchases, and enhance user loyalty by improving the accuracy of product recommendations, thereby increasing sales. This phenomenon has intensified competition among major e-commerce companies. Based on extensive data on user personal information, shopping preferences, and product reviews, these companies are leveraging recommendation systems to offer more product recommendations, discounts, and other services, attracting more sellers and merchants to their platforms, thereby increasing sales and boosting market share.

[0044] However, existing recommendation systems often rely on users' private data, which means that their historical behavior records and private information will be collected and stored, and users do not want their private data to be leaked. Therefore, using users' non-private data for recommendations will inevitably lead to a loss of recommendation accuracy.

[0045] Based on this, the present invention provides a recommendation method, a terminal and a storage medium to solve the above problems.

[0046] First, the present invention provides a recommendation method, such as Figure 1 As shown, the recommended method may include at least the following steps:

[0047] S101 : Based on the search keywords in the user's search request and the recommendation model based on the reinforcement learning framework, an initial product recommendation list corresponding to the search request is obtained and sent to the user terminal.

[0048] In an embodiment of the present invention, a user enters a search keyword into a user terminal. The user terminal generates a corresponding search request based on the search keyword and sends it to a server. The server extracts the search keyword from the search request from the user terminal. The server invokes a recommendation model based on a reinforcement learning framework, inputs the search keyword from the search request into the recommendation model, and outputs an initial list of recommended products corresponding to the search request. The server then sends this initial list to the user terminal, which then displays the received initial list to the user.

[0049] For example, a user searches for the keyword "mobile phone case" on the user terminal. The user terminal generates a search request based on the search keyword and sends it to the server. After receiving the search request from the user terminal, the server extracts the search keyword "mobile phone case" in the search request, and then calls the corresponding recommendation model based on the reinforcement learning framework and inputs the search keyword "mobile phone case", and outputs the initial product recommendation list corresponding to "mobile phone case".

[0050] It can be understood that the above-mentioned server is only an example of an execution subject of the recommendation method provided in the embodiment of the present invention, and is not limited to the above-mentioned server.

[0051] The application scenarios of the aforementioned recommendation model based on the reinforcement learning framework are divided into agents and environments. The agent refers to the machine that learns and makes decisions. In the recommendation scenario, the recommendation algorithm serves as the agent in the reinforcement learning framework. The environment refers to the set of all objects that interact with the agent, other than the agent, that the agent cannot control. In the recommendation scenario, the environment refers to the set of all objects other than the recommendation algorithm. The recommendation task can be modeled as a Markov process, where the recommendation model recommends an item to the user, and the user provides feedback to the recommendation model at the next moment. The user's feedback is related to the recommended item in the previous state.

[0052] It should be noted that the recommended products in the above-mentioned initial product recommendation list may refer to physical products or virtual products (for example, videos, music, recharge cards, etc.), which is not specifically limited in the embodiment of the present invention.

[0053] Furthermore, the environment in the recommendation model based on the reinforcement learning framework records the initial product recommendation list corresponding to the search keyword as the current state. The initial product recommendation list is the current state information, and the environment feeds back the product name of each product in the initial product recommendation list to the intelligent agent as a reward.

[0054] S102: Using the initial product recommendation list as the user's current state information, calculating the current state information using a state value function, and using the obtained first value as an estimated value.

[0055] Specifically, the product keywords in the above-mentioned initial product recommendation list can be obtained through a preset word segmentation algorithm; then, the features of the above-mentioned product keywords are extracted respectively to obtain product features; each product feature is calculated through the above-mentioned state value function to obtain a first numerical value, and the first numerical value is used as the estimated value of the corresponding product keyword.

[0056] In an embodiment of the present invention, the agent uses a state value function to calculate the current state information to obtain an estimated value of each commodity keyword.

[0057] Furthermore, the product names of all recommended products in the initial product recommendation list can be obtained first, and then each product name can be segmented using a preset word segmentation algorithm to obtain multiple product keywords, and all product keywords can be used as product keywords in the above initial product recommendation list.

[0058] Furthermore, if Figure 2As shown, the product keywords in the initial product recommendation list can be input into the product feature extraction network to obtain the product features of each product keyword. The product features are then input into the state value network for calculation to obtain the estimated value of each product keyword. In this embodiment of the present invention, the state value network is constructed using the state value function.

[0059] S103: Obtain user behavior information of the user with respect to the initial product recommendation list, and use the user behavior information and the initial product recommendation list as next state information.

[0060] Specifically, the user's feedback behavior on the initial product recommendation list is obtained, and user behavior information of the user on the initial product recommendation category is determined based on the user's feedback behavior.

[0061] In the embodiment of the present invention, after the environment learns that the user has provided feedback on the initial product recommendation list, the current state is changed to the next state.

[0062] The feedback behavior includes at least one of click, purchase, exit, browse, and favorite. The user behavior status information includes at least one of click behavior status information, purchase behavior status information, exit behavior status information, browse behavior status information, and favorite behavior status information.

[0063] In the embodiment of the present invention, after the user terminal displays the received initial product recommendation list to the user, the user takes corresponding actions on the initial product recommendation list, and the above actions are used as feedback behaviors.

[0064] S104: Calculate the next state information using the action-value function, and use the obtained second value as the true value.

[0065] Specifically, first obtain the products in the initial product recommendation list corresponding to the user behavior as the user feedback product; then perform feature extraction on the above user behavior state information to obtain product-user features; calculate the product-user features through the action value function, and use the second numerical value obtained as the true value of the product keyword of the user feedback product.

[0066] In an embodiment of the present invention, the intelligent agent calculates the next state information using the action value function to obtain the true value of each product keyword.

[0067] The above-mentioned product-user feature is used to represent the interaction with a product in the initial product recommendation list. The product-user feature is represented as a two-dimensional matrix, which only stores product features and user features.

[0068] From the above, it can be seen that user behavior includes at least one of: clicking, purchasing, leaving, browsing, and collecting. It can be understood that clicking means that the user has viewed one or several products in the initial product recommendation list, purchasing means that the user has purchased one or several products in the initial product recommendation list, leaving means that the user has returned to the interface displaying the initial product recommendation list, browsing means that the user has browsed the initial product list for a period of time, and collecting means that the user has collected one or several products in the initial product recommendation list. It can be seen that user behavior corresponds to the products in the initial product recommendation list, and the above-mentioned leaving and browsing refer to all products in the initial product recommendation list.

[0069] Therefore, in the embodiment of the present invention, the products in the initial product recommendation list corresponding to the user behavior can be determined first as the user feedback products. Figure 2 As shown, user behavior state information is input into the product-user feature extraction network, which outputs the corresponding product-user features. These product-user features are then input into the action value network for calculation to obtain a second value. This second data serves as the true value of the product keywords corresponding to the user behavior state information. The action value network is constructed using the above-mentioned action value function.

[0070] It is understandable that there may be a situation where some product keywords in the initial product recommendation list do not have matching real values. For example, the user behavior status information is click behavior status information, that is, the user only clicks on some of the products in the initial product recommendation list, then only the product keywords of the clicked products have real values. In an embodiment of the present invention, the real values ​​of other product keywords that do not have real values ​​can be uniformly set to 0.

[0071] S105 : Based on the estimated value and the true value, determine the search keyword for the next recommendation process, so as to obtain the next product recommendation list through the recommendation model and send it to the user terminal.

[0072] Specifically, the weight of each commodity keyword is updated according to the estimated value and the actual value of each commodity keyword; and the search keyword for the next recommendation process is determined according to the updated weight of each commodity keyword.

[0073] like Figure 2 As shown, in an embodiment of the present invention, the estimated value output by the state value network and the real value input by the action value network are input into the feature fusion network, and the weight of each product keyword is updated based on the estimated value and real value of each product keyword through the feature fusion network.

[0074] Furthermore, the search keywords used for the next recommendation process may be determined based on a preset greedy algorithm and the updated weight values ​​of the product keywords.

[0075] In an embodiment of the present invention, a preset greedy algorithm is used to find product keywords whose weight values ​​rank in a preset number of places (for example, the top three weight values), and use them as search keywords for the next recommendation process.

[0076] Furthermore, the word frequency statistics of the product keywords in the initial product recommendation list can be performed to determine the word frequency of each product keyword; the word frequency of each product keyword can be used as the initial weight of the corresponding product keyword; then the adjustment value can be determined based on the estimated value and actual value of each product keyword, and the initial weight of each product keyword can be updated based on the adjustment value.

[0077] Specifically, the above adjustment value is calculated by the following formula:

[0078] V'(S t )=V(S t )+α(R t+1 +γV(S t+1 )-V(S t ));

[0079] Among them, V'(S t ) is the adjustment value; V(S t ) is the estimated value of the product keyword; V(S t+1 ) is the true value of the product keyword; α represents the learning rate; R t+1 It represents the immediate reward after taking an action in the current state; γ represents the discount factor.

[0080] The sum of the adjusted value and the initial weight is used as the updated weight of each product keyword.

[0081] In an embodiment of the present invention, the intelligent agent performs word frequency statistics on each commodity keyword according to the reward fed back by the environment, and stores the word frequency of each commodity keyword in a memory module.

[0082] After obtaining the updated weight of each product keyword, the product of the updated weight and the characteristic value of the product feature of the product keyword is used as the comprehensive score of the product keyword, and each product keyword is ranked according to the comprehensive score to determine the search keyword for the next recommendation process.

[0083] The recommendation method provided by the present invention inputs the search keywords in the user's search request into a recommendation model based on a reinforcement learning framework to obtain an initial product recommendation list corresponding to the search request and sends it to the user terminal; then the initial product recommendation list is used as the user's current state information, and the current state information is calculated through a state value function to obtain an estimated value of each product keyword in the initial product recommendation list; the user's user behavior information for the above-mentioned initial product recommendation list is obtained through the user terminal, and the user behavior information and the initial product recommendation list are used as the next state information; the next state information is calculated through an action state value function to obtain the true value of each product keyword; then, based on the estimated value and the true value, the search keyword of the next recommendation process is determined to input the above-mentioned recommendation model, obtain the next product recommendation list and send it to the user terminal. In the product recommendation process of the above recommendation method, the user's privacy information is not used, which can effectively prevent the leakage of the user's privacy information. On the other hand, without using the user's privacy information, further product recommendations are made to the user through the user's feedback on the initial product recommendation list, and the search keywords for the next recommendation process are determined based on the estimated value and the true value. When the user's privacy information is not used to recommend to the user, the loss of product recommendation accuracy can be effectively avoided, and the effect of effectively ensuring the accuracy of product recommendations when using non-user privacy information for product recommendations is achieved.

[0084] Based on the above recommendation method, the present invention also provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the recommendation method described in the above embodiment.

[0085] Based on the above-mentioned recommended method, the present invention also provides a terminal, such as Figure 3 As shown, it includes at least one processor 30; a display screen 31; and a memory 32. It may also include a communications interface 33 and a bus 34. The processor 30, display screen 31, memory 32, and communications interface 33 can communicate with each other via bus 34. The display screen 31 is configured to display a preset user guidance interface in the initial setup mode. The communications interface 33 can transmit information. The processor 30 can call the logic instructions in the memory 32 to execute the recommended method described in the above embodiment.

[0086] In addition, the logic instructions in the memory 32 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0087] The memory 32 is a computer-readable storage medium that can be configured to store software programs or computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 30 executes the software programs, instructions, or modules stored in the memory 32 to perform functional applications and data processing, thereby implementing the methods in the above embodiments.

[0088] The memory 32 may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data created based on the use of the terminal. Furthermore, the memory 32 may include high-speed random access memory and non-volatile memory. For example, various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, may also be transient storage media.

[0089] The various embodiments in this application are described in a progressive manner. Similar portions between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device and medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simple. For relevant portions, refer to the descriptions of the method embodiments.

[0090] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects to their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0091] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0092] Of course, those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium that can be read by a computer. When the program is executed, it can include the processes in the above-described method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0093] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A recommendation method, characterized in that: The recommended methods include: Based on the search keywords in the user's search request and the recommendation model based on the reinforcement learning framework, an initial product recommendation list corresponding to the search request is obtained and sent to the user terminal; Using the initial product recommendation list as the current state information of the user, calculating the current state information using a state value function, and using the obtained first value as an estimated value; Obtaining user behavior information of the user with respect to the initial product recommendation list, and using the user behavior information and the initial product recommendation list as next state information; Calculating the next state information using an action-value function, and using the obtained second value as a true value; Determining search keywords for a next recommendation process based on the estimated value and the true value, so as to obtain a next product recommendation list through the recommendation model and send the list to the user terminal; The calculation of the current state information by the state value function to obtain the corresponding estimated value specifically includes: Obtain product keywords in the initial product recommendation list using a preset word segmentation algorithm; Extracting features of the product keywords to obtain product features; Calculating each of the product features using the state-value function, and using the obtained first value as an estimated value of the corresponding product keyword; The calculation of the next state information by the action value function to obtain the corresponding real value specifically includes: Obtaining products in the initial product recommendation list corresponding to the user behavior information as user feedback products; Extracting features from the user behavior information to obtain product-user features; Calculating the product-user feature using an action-value function, and using the obtained second value as the true value of the corresponding product keyword of the user; The determining of the search keyword for the next recommendation process based on the estimated value and the true value specifically includes: updating the weight of each of the product keywords according to the estimated value and the actual value of each of the product keywords; Determining the search keywords to be used in the next recommendation process according to the updated weights of the product keywords; The weight of each product keyword is updated according to the estimated value and the actual value of each product keyword, specifically including: Performing word frequency statistics on the product keywords in the initial product recommendation list to determine the word frequency of each product keyword; The word frequency of each product keyword is used as the initial weight of the corresponding product keyword; The initial weight of each of the product keywords is adjusted according to the estimated value and the actual value of each of the product keywords.

2. The recommendation method according to claim 1, characterized in that Obtaining user behavior information of the user with respect to the initial product recommendation list, specifically including: Obtaining feedback from the user regarding the initial product recommendation list; The feedback behavior includes at least one of: click, purchase, leave, browse, and favorite; Determine user behavior information of the user for the initial product recommendation category based on the user's feedback behavior.

3. The recommendation method according to claim 1, characterized in that: The user behavior information includes at least one of viewing behavior status information, purchasing behavior status information, leaving behavior status information, browsing behavior status information, and collecting behavior status information.

4. The recommendation method according to claim 1, characterized in that: The step of determining the search keywords to be used in the next recommendation process based on the updated weights of the product keywords specifically includes: The search keywords used for the next recommendation process are determined according to a preset greedy algorithm and the updated weights of the product keywords.

5. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the recommendation method according to any one of claims 1 to 4.

6. A terminal, characterized in that: include: processor and memory; The memory stores a computer-readable program that can be executed by the processor; when the processor executes the computer-readable program, the steps of the recommendation method according to any one of claims 1 to 4 are implemented.

Images