Personalized Recommenders with Limited Data Availability

Abstract

In some examples, a system for generating personalized recommendations includes a processor that can perform initial training for a deep reinforcement learning (DRL) model using domain knowledge, available user data, and a list of items. The processor also inputs the user data and the list of items into the trained DRL model to generate an initial list of recommended items. The processor also inputs the initial list of recommended items and the user profile into a content-based filter to generate a final recommendation list for a target user.

Description

Background Art

[0001] The present disclosure relates to techniques for generating recommendations using deep reinforcement learning and content-based filtering. Summary of the invention

[0002] According to an embodiment described herein, a system for generating personalized recommendations includes a processor that can perform initial training for a deep reinforcement learning (DRL) model using domain knowledge and available user data and a list of items. The processor also inputs the user data and the list of items into the trained DRL model to generate an initial list of recommended items. The processor also inputs the initial list of recommended items and the user profile into a content-based filter to generate a final recommendation list for a target user.

[0003] In some embodiments, a method for generating personalized recommendations includes: performing initial training for a deep reinforcement learning (DRL) model using domain knowledge and available user data and a list of items, inputting the user data and the list of items into the trained DRL model to generate an initial list of recommended items, and inputting the initial list of recommended items and a user profile into a content-based filter to generate a final recommendation list for a target user.

[0004] In yet another embodiment, a computer program product for generating personalized recommendations may include a computer-readable storage medium having program instructions embodied therein, wherein the computer-readable storage medium is not the transient signal itself. The program instructions may be executed by a processor to cause the processor to perform initial training for a deep reinforcement learning (DRL) model using domain knowledge and available user data and item lists, input the user data and item lists into the trained DRL model to generate an initial list of recommended items, and input the initial list of recommended items and the user profile into a content-based filter to generate a final recommendation list for a target user. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 depicts a block diagram of an example computing system that can generate personalized recommendations according to embodiments described herein;

[0006] Figure 2 is a block diagram illustrating an example recommender system that generates recommendations using a deep reinforcement learning (DRL) model and content-based filters;

[0007] Figure 3 is a block diagram illustrating another example recommender system using a deep reinforcement learning (DRL) model and content-based filters to generate recommendations;

[0008] Figure 4is a tangible, non-transitory computer-readable medium that can generate personalized recommendations according to the embodiments described herein;

[0009] Figure 5 depicts an illustrative cloud computing environment according to embodiments described herein; and

[0010] Figure 6 Depicted is a set of functional abstraction layers provided by a cloud computing environment according to embodiments described herein. DETAILED DESCRIPTION

[0011] This disclosure describes techniques for generating recommendations using deep reinforcement learning and content-based filtering. As used herein, a recommendation is an identification of one or more items that may be of interest to a particular person. For example, a media platform may generate recommendations to present recommended content to its users. An online retailer may also generate recommendations to present products to its users that the user is more likely to purchase. Various websites may also generate recommendations to identify advertisements that may be useful in attracting user interest. The purpose of such recommendations is to present items that may be of interest to the user.

[0012] If there is enough information about the interests of a user or a large group of users, a machine learning algorithm can provide useful recommendations. However, when little data is available, typical machine learning techniques may not be suitable for generating useful recommendations. This disclosure describes techniques for generating useful information when large amounts of user interest data are not available.

[0013] Reference now Figure 1 , shows an example computing device that can generate personalized recommendations. Computing device 100 can be, for example, a server, a desktop computer, a laptop computer, a tablet computer, or a smart phone. In some examples, computing device 100 can be a cloud computing node. Computing device 100 can be described in the general context of computer system executable instructions (e.g., program modules) executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. Computing device 100 can be practiced in a distributed cloud computing environment in which tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located in both local and remote computer system storage media including memory storage devices.

[0014] The computing device 100 may include a processor 102 adapted to execute stored instructions, and a memory device 104 for providing temporary memory space for the operation of the instructions during operation. The processor may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory device 104 may include random access memory (RAM), read-only memory, flash memory, or any other suitable memory system.

[0015] Processor 102 may be connected via system interconnect 106 (e.g., The computing device 100 is connected to an input / output (I / O) device interface 108 suitable for connecting the computing device 100 to one or more I / O devices 110. The I / O devices 110 may include, for example, a keyboard and a pointing device, wherein the pointing device may include a touch pad or a touch screen, etc. The I / O devices 110 may be built-in components of the computing device 100, or may be devices connected to the computing device 100 from the outside.

[0016] The processor 102 may also be linked to a display interface 112 adapted to connect the computing device 100 to a display device 114 via the system interconnect 106. The display device 114 may include a display screen that is a built-in component of the computing device 100. The display device 114 may also include a computer monitor, a television, or a projector, etc., that is externally connected to the computing device 100. In addition, a network interface controller (NIC) 116 may be adapted to connect the computing device 100 to a network 118 via the system interconnect 106. In some embodiments, the NIC 116 may use any suitable interface or protocol (e.g., Internet Small Computer System Interface, etc.) to transmit data. The network 118 may be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN), or the Internet, etc. A remote device 120 may be connected to the computing device 100 via the network 118.

[0017] In some examples, processor 102 may be linked to storage device 122 via system interconnect 106, which may include: deep reinforcement learning (DRL) model 124, content-based filters 126, domain knowledge 128, user profile 130, and item list 132. DRL model 124 uses deep neural networks in conjunction with reinforcement learning techniques to generate recommendations. DRL model 124 may be trained using domain knowledge 128, which represents the interests of a large number of users. Content-based filters are used to personalize recommendations to specific target users based on the user's interests captured by user profile 130.

[0018] The item list is a list of all or a subset of potential items that can be recommended. The items in the item list will vary depending on the specific implementation. For example, in the case of a streaming platform, the items can be media files, including audio files, video files, audio / video files. For example, media files can include music, music videos, movies, video clips, television series, etc. In some embodiments, the items can be retail products or services, advertisements, etc. Each item can be associated with a set of features that describe the characteristics of the item.

[0019] There are several techniques by which the DRL model 124 can be used in conjunction with content-based filters to generate personalized recommendations. Figure 2 and 3 Some example embodiments of the technology are described.

[0020] It should be understood that Figure 1 The block diagram is not intended to indicate that computing device 100 will include Figure 1 Instead, computing device 100 may include fewer components or Figure 1 102). In addition, any functionality of the DRL model 124 and the content-based filter 126 is partially or fully implemented in hardware and / or the processor 102. For example, the functionality may be implemented using a dedicated integrated circuit, logic implemented in an embedded controller, or implemented in logic implemented in the processor 102. In some embodiments, the functionality of the recommendation system is implemented by logic, where, as referred to herein, the logic may include any appropriate hardware (e.g., a processor, etc.), software (e.g., an application, etc.), firmware, or any appropriate combination of hardware, software, and firmware.

[0021] Figure 2 200 is a block diagram illustrating an example recommendation system that uses a deep reinforcement learning (DRL) model and content-based filters to generate recommendations. Figure 1 The example computing system shown is implemented. In this embodiment, the process performed by the recommendation system starts with initial training of the DRL model 124 using domain knowledge 202. The DRL model 124 uses a deep neural network in conjunction with a reinforcement learning algorithm. A deep neural network is an artificial neural network that includes an input layer, an output layer, and two or more internal layers between the input and output layers. Each layer may include any appropriate number of nodes, and each node of a layer may be connected to each node of the previous layer and the subsequent (successive) layer. The connections between nodes may be referred to as edges. Nodes and edges may be associated with weights that are adjusted as the training process progresses. Prior to initialization, weights may be random or assigned arbitrary values.

[0022] The inputs to a trained neural network may be referred to as states, and the outputs of the neural network may be referred to as actions. The DRL model 124 identifies the state-action combinations that tend to provide the highest expected rewards. In the present disclosure, a state is a profile of a particular person describing the personal preferences of that person, and an action is one or more initial recommendation lists. The neural network models the interests of a large number of users in such a way that an input state (a particular person's preferences) will result in recommendations that are likely to be presented to that particular person.

[0023] Initial training of the DRL model 124 can be performed using a large collection of domain knowledge records that describe the user preferences of a large number of people. The domain knowledge 202 may include a list of items and a rating for each item in the list. For example, if the list of items is a list of products or services, the rating may be an average of the number of user reviews that the website can collect for the product or service. If the list is a list of media files such as songs or video clips, the rating may be calculated based on the number of times the item has been viewed or the number of likes and dislikes that the media file has received.

[0024] Each item in the list of items may be represented by a feature vector, which is used as an input to the neural network. A feature vector may be a string of characters or numbers representing characteristics of an item (e.g., price, product type, etc.). Any characteristic of an item may be represented as a feature in a feature vector. For example, if the item is a media file, characteristics that may be included as features in the feature vector may include the type of media, such as whether it is a movie, music video, television series, movie type, music type, media producer, actor or performer, movie director, etc. If the item is a retail product, characteristics may include price, manufacturer, specifications, etc. If the item is a service such as a restaurant, characteristics may include price, geographic location, food type, etc. A feature vector may include any suitable number of characteristics, including those listed above and others.

[0025] During initial training, feature vectors for some or all items may be input to a neural network to compute outputs. The outputs are compared to the ranks to compute a loss function, which represents the difference between the outputs of the neural network and the ranks. A training algorithm may be used to adjust the weights of the neural network so that the loss function (e.g., the difference between the outputs of the neural network and the ranks) is minimized. Any suitable DRL training algorithm may be used to adjust the weights, including Q-learning, policy gradient, etc.

[0026] The resulting trained neural network describes the importance of each feature in determining the user's interests. At this stage, the DRL model 124 has been trained using the domain knowledge of multiple users. The trained DRL model 124 can then be used to generate recommendations for target users. As used herein, a target user is a person that a recommendation is intended to attract. In order to generate recommendations, a list of items for multiple users and user data are input into the trained DRL model 124. The target list is a list of all or part of the potential items that can be recommended. For example, if the items are media files, the potential items may be a list of all items in the system database. The user data includes data about the preferences of the target user.

[0027] The neural network receives a list of items and user data and outputs a recommendation list that includes ranked items with corresponding scores. The goal of the reinforcement learning algorithm is to identify the combination / ranking of items that maximizes the objective / loss function, indicating that items with higher scores have a higher likelihood of attracting the user.

[0028] The items output by the DRL model 124 may be referred to as an initial recommended item list. The initial recommended item list has not yet been personalized for a specific user because the initial DRL model is trained using general domain knowledge rather than knowledge about specific user interests. The initial recommended item list may include any suitable number of items. Each item in the recommended item list may also include a list of attributes specific to each item. The attributes may be any of the attributes described above with respect to the training of the neural network. The initial recommended item list is then input into a content-based filter to perform personalized ranking of the initial recommended short list.

[0029] The content-based filter receives the initial list of recommended items and the user profile 130 of the user, and further filters the list based on the personal interests of the target user. The user profile 130 can be any suitable configuration for identifying the interests of the user. In some examples, the user profile 130 may include a list of features that can be ranked according to the user's preference level. For example, the user profile 130 may include a list of movie genres and a preference value indicating the relative interest level of each movie genre. The user profile 130 may also include demographic information, such as age, geographic location, etc. The user profile 130 can then be compared with the attributes of the items in the initial list of recommended items to rank the items, and some items may be excluded from the list to generate a smaller list of final recommended items. For example, if the initial list of recommended items is related to movies, and the user profile 130 indicates no interest in a particular genre, these items may be excluded from the list or given a low ranking. Conversely, if the user profile 130 indicates a high interest in a particular genre, those items in the initial list may be given a higher ranking. In addition, the ranking of each item may be increased based on the number of intersections between the item attributes and the user profile features. As used herein, the term intersection refers to item attributes that correspond to a feature of the user profile 130 that indicates a significant level of interest. For example, the user profile 130 may indicate a high interest in movies from a particular genre, movies with a particular actor, and movies made by a particular filmmaker. Those items in the initial recommendation list that have a greater number of attributes that correspond to highly ranked features of the user profile 130 will tend to be ranked higher than items in the initial list that have less or no intersection.

[0030] The resulting ranked list of final recommendations may then be sent to the user for display. In some embodiments, such as in a streaming media service, the recommended items may be displayed in a menu identified as recommended items. Each item in the display may also include a ranking assigned to each item to indicate to the user an estimated probability that the recommendation reflects the user's actual interests. In some embodiments, such as an advertising environment, a web page or application may be populated with advertisements corresponding to one or more of the highest ranked items. In an e-commerce marketplace, such as on an e-commerce website, the highest ranked items may be displayed as a list of products or services available for purchase.

[0031] As more data about the user's interests becomes available, as indicated by the user's activities (e.g., the user's viewing activity, purchasing activity, etc.), the user profile 130 may be updated. For example, in the case of a streaming service, a user's selection of a particular media such as a movie may cause the user profile 130 to be updated to indicate the user's interest in the attributes associated with the media file. In some embodiments, the user profile may be updated based on user rankings. For example, a user may visit a particular restaurant or purchase a particular product and then submit a review of the restaurant or product on the website. The user's review rating may be used to increase or decrease the corresponding features in the user profile related to the attributes of the restaurant or product. Over time, as more information is learned about the user's preferences, the user profile 130 will be updated and more accurate recommendations may be made.

[0032] In some embodiments, the recommendation system may include an exploration module 202, which can be used to collect more information about potential user interests that may not be included in the user profile 130. The exploration module 202 can select items from the initial recommendation short list and insert randomly selected items into the final recommended item list according to the exploration policy. For example, the exploration policy may indicate randomly selected items from the initial recommendation short list. Other exploration strategies are also possible. In this way, user feedback on items that the user may be interested in but the system previously did not have visibility for can be collected. For example, the randomly selected item can be a movie within a type that is not indicated by the user profile. If the user selects the randomly selected item, the user profile 130 can be updated to indicate the level of interest in the new movie type. In this way, the user profile 130 can be expanded to include user interests that were previously unknown and would not otherwise be recommended by the content-based filter 126.

[0033] Figure 3 is a block diagram illustrating another example recommendation system 300 that generates recommendations using a deep reinforcement learning (DRL) model and content-based filters. The recommendation system may be composed of Figure 1In this embodiment, the process performed by the recommendation system begins by enhancing domain knowledge 128 using content-based filters 126. Content-based filters 126 can be used to obtain a score for each item in a list of items based on preference data from a user profile 130. The score can be added to a feature vector for each item.

[0034] The enhanced list of items can then be used for initial training of the DRL model 124, which can be described above with reference to Figure 2 The trained DRL model 124 can then be used to make personalized recommendations for the DRL model. The input to the trained DRL model 124 is a list of items, and the output of the DRL model 124 is a ranked final recommendation list. The final recommendation can be as described above with respect to Figure 1 As described above, the user profile may be updated based on user activity indicative of user interests and the exploration module 202 may be used to gather more information about potential user interests that may not be included in the user profile 130 by randomly selecting items to present to the user from a list of potential items.

[0035] The above about Figure 2 and Figure 3 The recommendation system described is an example of a possible implementation of the present technology. Other implementations are also possible. For example, the recommendation system may include information about Figure 2 and Figure 3 In this hybrid system, Figure 2 The recommendation system can be used to generate a first recommendation list, Figure 3 The recommendation system may be used to generate the second recommendation list, and the final recommendation list may be selected from the highest ranked items in both the combined first and second lists. Other embodiments are also possible.

[0036] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0037] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0038] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0039] The computer program instructions for performing the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

[0040] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and / or block diagram and the combination of each box in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0041] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0042] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0043] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0044] Reference now Figure 4 , depicts a block diagram of an example of a tangible, non-transitory computer-readable medium on which instructions may be executed to generate a recommendation. A processor 402 may access a tangible, non-transitory computer-readable medium 400 via a computer interconnect 404 .

[0045] In addition, the tangible, non-transitory computer-readable medium 400 may include data and code to direct the processor 402 to perform the operations of the described technology. For example, the computer-readable medium 400 may include a DRL model 406 and a content-based filter 408 for generating recommendations as described above. The computer-readable medium 400 may include domain knowledge 410, which is used to train the DRL model during an initial training phase. The DRL model may also be trained using user profiles 412. User-specific recommendations may be calculated by the trained DRL model using a specific user profile in the user profiles 412. A final recommendation list may be selected from a list of items, item list 414, which represents all potential items that the recommendation system can recommend.

[0046] It should be understood that, depending on the specific application, Figure 4 Any number of additional software components not shown in the figure may be included in the tangible, non-transitory computer-readable medium 400. In addition, the tangible, non-transitory computer-readable medium 400 may include more than Figure 4 Fewer software components are shown.

[0047] Reference now Figure 5 , an illustrative cloud computing environment 500 is shown. As shown, the cloud computing environment 500 includes one or more cloud computing nodes 502, with which local computing devices used by cloud consumers (such as, for example, personal digital assistants (PDAs) or cellular phones 504A, desktop computers 504B, laptop computers 504C, and / or automotive computer systems 504N) can communicate. The cloud computing nodes 502 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks (such as the private clouds, community clouds, public clouds, or hybrid clouds described above) or a combination thereof. This allows the cloud computing environment 500 to provide infrastructure as a service, platform as a service, and / or software as a service, for which cloud consumers do not need to maintain resources on local computing devices. It should be understood that Figure 5 The types of computing devices 504A-N shown are merely exemplary, and cloud computing node 502 and cloud computing environment 500 may communicate with any type of computerized device over any type of network and / or network addressable connection (eg, using a web browser).

[0048] Reference now Figure 6 , showing a cloud computing environment 500 ( Figure 5 ) provides a set of functional abstraction layers. It should be understood in advance that Figure 6 The components, layers and functions shown in are only exemplary, and the embodiments of the present invention are not limited thereto. As shown in the figure, the following layers and corresponding functions are provided.

[0049] The hardware and software layer 600 includes hardware and software components. Examples of hardware components include: The host in the system; in an example IBM RISC (Reduced Instruction Set Computer) architecture based servers in the system; IBM System: IBM Blade systems; storage devices; network and networking components. Examples of software components include: In an example IBM Network application server software in application server software; and in an example IBM Database software in the database software. (IBM, zSeries, pSeries, xSeries, BladeCenter, WebSphere and DB2 are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide).

[0050] The virtualization layer 602 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers; virtual storage; virtual networks, including virtual private networks; virtual applications and operating systems; and virtual clients. In one example, the management layer 604 can provide the functions described below. The resource provisioning function provides dynamic acquisition of computing resources and other resources for performing tasks in a cloud computing environment. The metering and pricing function provides cost tracking of the use of resources within the cloud computing environment and can provide bills or invoices for the consumption of these resources. In one example, these resources can include application software licenses. The security function provides identity authentication for cloud consumers and tasks, and provides protection for data and other resources. The user portal function provides access to the cloud computing environment for consumers and system administrators. The service level management function provides cloud computing resource allocation and management to meet the required service levels. The service level agreement (SLA) planning and fulfillment function provides pre-arrangement and provision for future demand for cloud computing resources predicted according to the SLA.

[0051] The workload layer 606 provides examples of functions that may be implemented in a cloud computing environment. In this layer, examples of workloads or functions that may be provided include: mapping and navigation; software development and lifecycle management; virtual classroom teaching provision; data analysis processing; transaction processing; and natural language processing (NLP) technology.

[0052] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A system for generating personalized recommendations, comprising: A processor for: performing initial training for a deep reinforcement learning (DRL) model using domain knowledge, the domain knowledge comprising domain knowledge records describing user preferences of a plurality of persons; inputting the user data and the item lists of the plurality of persons into a trained DRL model to generate an initial recommended item list from the item lists; inputting the initial recommended item list from the DRL model and a user profile into a content-based filter to generate a final recommendation list for a target user; modifying the generated final recommendation list to include additional items selected from the initial list of recommended items generated by the trained DRL model; as well as The modified final recommendation list is sent to the target user for display.

2. The system according to claim 1, wherein: The domain knowledge includes user comments or other actions indicating user preferences of the plurality of persons corresponding to items in the list of items.

3. The system according to claim 1, wherein: The user profile includes a feature list, wherein each feature in the feature list includes a value indicating a user preference level corresponding to the feature.

4. The system according to claim 1, wherein: Each item in the list of items includes a feature vector describing properties of the item.

5. The system according to claim 1, wherein: The final recommendation list is ranked according to the probability that each item in the final recommendation list will be of interest to the target user.

6. The system according to claim 1, wherein: The processor is configured to insert random items from the initial recommended item list into the modified final recommended list according to an exploration strategy.

7. The system according to claim 1, wherein: The processor is configured to update the user profile based on a selection by the target user of one or more recommendation items in the modified final recommendation list.

8. The system according to claim 1, wherein: The item list includes a list of media files stored by the streaming service.

9. The system according to claim 1, wherein: The item list includes a list of products that can be purchased from the e-commerce website.

10. The system according to claim 1, wherein: The item list includes an advertisement list.

11. A method for generating personalized recommendations, comprising: performing initial training for a deep reinforcement learning (DRL) model using domain knowledge, the domain knowledge comprising domain knowledge records describing user preferences of a plurality of persons; inputting the user data and the item lists of the plurality of persons into a trained DRL model to generate an initial recommended item list from the item lists; inputting the initial list of recommended items from the trained DRL model and a user profile into a content-based filter to generate a final recommendation list for a target user; as well as The generated final recommendation list is modified to include additional items selected from the initial list of recommended items generated by the trained DRL model.

12. The method according to claim 11, wherein: The domain knowledge includes user comments or other actions indicating user preferences of the plurality of persons corresponding to items in the list of items.

13. The method according to claim 11, wherein: The user profile includes a list of features, wherein each feature in the list of features includes a value indicating a level of user interest corresponding to the feature.

14. The method according to claim 11, wherein: Each item in the list of items includes a feature vector describing properties of the item.

15. The method according to claim 11, comprising: The final recommendation list is ranked according to the probability that each item in the final recommendation list will be of interest to the target user.

16. The method according to claim 11, comprising: Random items from the initial recommended item list are inserted into the modified final recommended list according to an exploration strategy.

17. The method according to claim 11, comprising: The user profile is updated based on a selection by a target user of one or more recommended items in the modified final recommendation list.

18. The method according to claim 11, wherein: The item list includes a list of media files stored by the streaming service.

19. The method according to claim 11, wherein: The item list includes a list of products that can be purchased from the e-commerce website.

20. A computer program product for generating personalized recommendations, comprising a computer-readable storage medium, wherein the computer-readable storage medium embodies program instructions, wherein: The computer readable storage medium is not a transient signal itself, and wherein the program instructions are executable by a processor to cause the processor to: performing initial training for a deep reinforcement learning (DRL) model using domain knowledge, the domain knowledge comprising domain knowledge records describing user preferences of a plurality of persons; inputting the user data and the item lists of the plurality of persons into a trained DRL model to generate an initial recommended item list from the item lists; and inputting the initial list of recommended items from the trained DRL model and a user profile into a content-based filter to generate a final recommendation list for a target user; and The generated final recommendation list is modified to include additional items selected from the initial list of recommended items generated by the trained DRL model.

21. The computer program product of claim 20, wherein: The additional items are selected based on a heuristic strategy.

22. A computer system comprising modules respectively configured to execute the steps of the method according to any one of claims 11-19.

Images