An information recommendation method, device, equipment and computer readable storage medium

Abstract

The application provides an information recommendation method and device, equipment and a computer readable storage medium; related to artificial intelligence technology, the method comprises: obtaining a feedback information sequence corresponding to a recommendation instance sequence by performing information recommendation through an original recommendation model, each piece of feedback information in the feedback information sequence comprising a delayed feedback reward; obtaining a feedback correction parameter corresponding to each recommendation instance in the recommendation instance sequence, the feedback correction parameter being negatively correlated with a probability of receiving a delayed feedback of each recommendation instance and positively correlated with a probability of finally receiving a delayed feedback of each recommendation instance; based on the feedback correction parameter, correcting the delayed feedback reward corresponding to each recommendation instance to obtain a corrected feedback information sequence corresponding to the recommendation instance sequence; and updating the original recommendation model based on the recommendation instance sequence and the corrected feedback information sequence to obtain a recommendation model. Through the application, the accuracy of information recommendation can be improved.

Description

Technical Field

[0001] This application relates to artificial intelligence technology in the field of computer internet, and more particularly to an information recommendation method, apparatus, device and computer-readable storage medium. Background Technology

[0002] In information recommendation applications, recommendations are typically based on the correlation between user information and recommended information. This involves first acquiring historical recommendation data, including previously recommended information, user information, and user feedback; then training a recommendation model using this historical data; and finally, using the model to recommend information to the user. However, in this process, a delay exists between the completion of the recommendation and the conversion of the recommended information. Consequently, the acquired user feedback may contain missing or incorrect tags, resulting in poor predictive ability of the trained recommendation model and consequently, poor accuracy in information recommendations. Summary of the Invention

[0003] This application provides an information recommendation method, apparatus, device, and computer-readable storage medium, which can improve the accuracy of information recommendation.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides an information recommendation method, including:

[0006] Information recommendation is performed using the original recommendation model to obtain a feedback information sequence corresponding to the recommendation instance sequence, wherein each piece of feedback information in the feedback information sequence includes a delayed feedback reward.

[0007] Obtain the feedback correction parameter corresponding to each recommendation instance in the recommendation instance sequence, wherein the feedback correction parameter is negatively correlated with the probability of receiving delayed feedback from each recommendation instance at present and positively correlated with the probability of eventually receiving delayed feedback from each recommendation instance;

[0008] Based on the feedback correction parameters, the delayed feedback reward corresponding to each recommended instance is corrected to obtain a corrected feedback information sequence corresponding to the recommended instance sequence.

[0009] Based on the recommended instance sequence and the corrected feedback information sequence, the original recommendation model is updated to obtain a recommendation model, wherein the recommendation model is used for information recommendation.

[0010] This application provides an information recommendation device, including:

[0011] The data acquisition module is used to perform information recommendation through the original recommendation model and obtain the feedback information sequence corresponding to the recommendation instance sequence, wherein each piece of feedback information in the feedback information sequence includes a delayed feedback reward.

[0012] The parameter acquisition module is used to acquire the feedback correction parameter corresponding to each recommendation instance in the recommendation instance sequence, wherein the feedback correction parameter is negatively correlated with the probability of receiving delayed feedback from each recommendation instance at present and positively correlated with the probability of finally receiving delayed feedback from each recommendation instance.

[0013] The feedback correction module is used to correct the delayed feedback reward corresponding to each recommendation instance based on the feedback correction parameters, thereby obtaining a corrected feedback information sequence corresponding to the recommendation instance sequence;

[0014] The model update module is used to update the original recommendation model based on the recommended instance sequence and the corrected feedback information sequence to obtain a recommendation model, wherein the recommendation model is used for information recommendation.

[0015] In this embodiment, the parameter acquisition module is further configured to: acquire the collection time period corresponding to the recommended instance sequence, and determine the counterfactual time from the collection time period; select recommended instances from the recommended instance sequence between the start time of the collection time period and the counterfactual time to obtain an observation instance sequence; select recommended instances from the observation instance sequence whose delayed feedback reward is a delayed feedback reward threshold to obtain a training instance sequence; for each recommended action in the recommended action sequence corresponding to the recommended instance sequence, train the survival model parameters of each recommended action based on the training instance sequence to obtain a survival model parameter sequence corresponding to the recommended action sequence; and determine the feedback correction parameters based on the survival model parameter sequence and the recommended action corresponding to each recommended instance.

[0016] In this embodiment, each feedback message further includes a feedback start time and a churn duration; the parameter acquisition module is further configured to determine a feedback end time based on the feedback start time and the churn duration; determine a delayed feedback simulated reward corresponding to each training instance in the training instance sequence based on a comparison between the feedback end time and the counterfactual time; determine the duration between the counterfactual time and the feedback start time as the simulated feedback duration; determine the shortest duration between the simulated feedback duration and the churn duration as the simulated churn duration corresponding to each training instance, thereby obtaining a delayed feedback simulated reward and simulated churn duration pair corresponding to each training instance; and train the survival model parameters for each recommended action based on the sequence of delayed feedback simulated rewards and simulated churn duration pairs corresponding to the training instance sequence.

[0017] In this embodiment of the application, the parameter acquisition module is further configured to determine the target survival model parameters corresponding to the recommendation action for each recommendation instance from the survival model parameter sequence; determine the survival information of each recommendation instance through the target survival model parameters; and acquire the feedback correction parameters that are negatively correlated with the survival information.

[0018] In this embodiment of the application, the parameter acquisition module is further configured to determine the collection duration based on the collection time period; determine the simulated collection duration corresponding to the collection duration based on the counterfactual parameter; and determine the counterfactual time based on the start time of the collection time period and the simulated collection duration.

[0019] In this embodiment of the application, the parameter acquisition module is further configured to: determine the delayed feedback simulation reward corresponding to each training instance in the training instance sequence as a first reward when the comparison result between the feedback end time and the counterfactual time is that the feedback end time is earlier than the counterfactual time; and determine the delayed feedback simulation reward as a second reward when the comparison result is that the feedback end time is not earlier than the counterfactual time.

[0020] In this embodiment of the application, the model update module is further configured to update the original recommendation model based on the recommended instance sequence and the corrected feedback information sequence; perform information recommendation through the updated original recommendation model to obtain a new feedback information sequence corresponding to the new recommended instance sequence; correct the new feedback information sequence, and continue to iteratively update the updated original recommendation model based on the new recommended instance sequence and the corrected new feedback information sequence until the update cutoff condition is met, at which point the iterative update stops and the recommendation model is obtained.

[0021] In this embodiment of the application, the model update module is further configured to divide the recommendation instance sequence based on the recommendation actions in the recommendation instance sequence to obtain a recommendation instance subsequence corresponding to each recommendation action; obtain a corrected feedback information subsequence corresponding to the recommendation instance subsequence from the corrected feedback information sequence; and update the recommendation parameters of each recommendation action in the original recommendation model based on the recommendation instance subsequence and the corrected feedback information subsequence, thereby completing the update of the original recommendation model.

[0022] In this embodiment of the application, the model update module is further configured to: obtain the context space combination corresponding to the corrected recommendation instance subsequence; update the first sub-recommendation parameter of each recommendation action in the original recommendation model based on the context space combination; obtain the feedback reward combination corresponding to the corrected feedback information subsequence; and update the second sub-recommendation parameter of each recommendation action in the original recommendation model based on the feedback reward combination and the context space combination, wherein the recommendation parameter includes the first sub-recommendation parameter and the second sub-recommendation parameter.

[0023] In this embodiment of the application, the feedback information further includes an immediate feedback reward; the model update module is further configured to combine the immediate feedback reward and the modified delayed feedback reward into a feedback reward for each modified feedback information in the modified feedback information subsequence, thereby obtaining the feedback reward combination corresponding to the modified feedback information subsequence.

[0024] In this embodiment, the information recommendation device further includes a model application module, configured to, in response to an information recommendation request, acquire a target object and a target information sequence; extract features from the target object and the target information sequence to obtain a target recommendation context space; input the target recommendation context space into the recommendation model to determine an estimated feedback reward sequence corresponding to the target information sequence; and, in the target information sequence, select the target information corresponding to the maximum estimated feedback reward in the estimated feedback reward sequence as the target recommendation information and display the target recommendation information.

[0025] This application provides an information recommendation device, including:

[0026] Memory, used to store executable instructions;

[0027] The processor, when executing executable instructions stored in the memory, implements the information recommendation method provided in the embodiments of this application.

[0028] This application provides a computer-readable storage medium storing executable instructions for inducing a processor to execute and implement the information recommendation method provided in this application.

[0029] The embodiments of this application have at least the following beneficial effects: During the training of the original recommendation model, for the recommendation instance sequence of the original recommendation model, a corresponding feedback information sequence is collected, and the delayed feedback reward in the feedback information sequence is corrected based on the feedback correction parameter that is negatively correlated with the probability of receiving delayed feedback for each recommendation instance at present and positively correlated with the probability of receiving delayed feedback for each recommendation instance at the end. This improves the accuracy of the corrected delayed feedback reward, so when the original recommendation model is updated based on the corrected delayed feedback reward, the information recommendation accuracy of the obtained recommendation model is higher, thereby improving the accuracy of information recommendation. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of an exemplary information recommendation process;

[0031] Figure 2 This is a schematic diagram of an optional architecture of the information recommendation system provided in the embodiments of this application;

[0032] Figure 3 This is one of the embodiments provided in this application. Figure 2 A schematic diagram of the server's structure;

[0033] Figure 4 This is an optional flowchart illustrating the information recommendation method provided in the embodiments of this application;

[0034] Figure 5 This is another optional flowchart illustrating the information recommendation method provided in the embodiments of this application;

[0035] Figure 6 This is another optional flowchart of the information recommendation method provided in the embodiments of this application;

[0036] Figure 7 This is a flowchart illustrating an exemplary information recommendation method provided in an embodiment of this application;

[0037] Figure 8 This is an exemplary instance partitioning diagram provided in an embodiment of this application;

[0038] Figure 9 This is a flowchart illustrating another exemplary information recommendation method provided in an embodiment of this application;

[0039] Figure 10 This is a schematic diagram of an exemplary process for correcting delayed feedback rewards provided in an embodiment of this application;

[0040] Figure 11 This is an exemplary information recommendation result comparison diagram provided in an embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0043] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0045] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0046] 1) Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. Therefore, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. In other words, AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0047] Furthermore, artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning. Moreover, with the research and advancement of AI technology, it has been researched and applied in numerous fields; for example, common applications include smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. As technology develops, AI will be applied in even more fields and play an increasingly important role. The application of AI in the field of information recommendation described in this application will be explained later.

[0048] 2) Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills; and how to reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning typically includes techniques such as deep learning, artificial neural networks, belief networks, reinforcement learning, transfer learning, and inductive learning.

[0049] 3) Reinforcement Learning (RL), also known as reward learning, evaluation learning, or reinforcement learning, is one of the paradigms and methodologies of machine learning. It is used to describe and solve the problem of how an agent learns strategies to maximize rewards or achieve specific goals during its interaction with the environment.

[0050] 4) Counterfactual learning is a learning method that substitutes untrue conditions or possibilities.

[0051] 5) Deep learning is a type of machine learning, which is an essential path to achieving artificial intelligence. The concept of deep learning originated from the research of artificial neural networks; a multilayer perceptron with multiple hidden layers is a type of deep learning structure. Deep learning discovers distributed feature representations of data by combining low-level features to form more abstract high-level representations of attribute categories or features.

[0052] In information recommendation applications, recommendations are typically based on the correlation between user information and recommended information. By determining the optimal match between recommended information and the recommending user, efficient and accurate recommendations can be achieved. For example... Figure 1 , Figure 1 This is an exemplary diagram of an information recommendation process; such as Figure 1 As shown, historical recommendation data 1-1 is obtained, which includes recommended information, user information, and user feedback. Historical recommendation data 1-1 is used as a training sample, and model parameters are adjusted using deep learning and other machine learning methods to train recommendation model 1-2. When an information recommendation request is received, in response to the recommendation request, recommended user information 1-3 and the set of information to be recommended 1-4 are input into recommendation model 1-2, and target recommendation information 1-5 is output and recommended to the recommended user.

[0053] For example, user models can be built using neural networks (such as DNNs (Deep Neural Networks) and RNNs (Recurrent Neural Networks)) and historical user behavior data to accurately characterize user interests and behavioral patterns. This improves the predictive performance of click-through rates for recommended information while also increasing the click-through rate of exposed information. Alternatively, a relationship can be established between real-time user needs and the smallest unit of recommended information. By tracking user internet behavior and mining user needs in real time, all real-time needs can be modeled using deep neural networks based on the click-through rate ranking of the smallest unit of recommended information. This avoids the influence of work-related factors on users' basic interests and improves the accuracy of information recommendations. Other methods include information recommendation based on user mobile terminal geolocation data or information recommendation based on user blockchain identifiers.

[0054] However, in the aforementioned information recommendation process, due to the delay between completing the recommendation and converting the recommended information, the obtained user feedback may contain missing or incorrect tags. This results in poor recommendation capabilities of the trained recommendation model, leading to low accuracy in information recommendations. Furthermore, the historical recommendation data is passively acquired in this process, meaning that the collected historical data is not always essential for updating the recommendation model. This affects the real-time nature and accuracy of capturing user interests and behaviors from historical data, further contributing to the low accuracy of the recommendation model's information recommendations.

[0055] Based on this, embodiments of this application provide an information recommendation method, apparatus, device, and computer-readable storage medium, which can improve the accuracy of information recommendation. The exemplary application of the information recommendation device provided in this application embodiment is described below. The information recommendation device provided in this application embodiment can be implemented as various types of user terminals such as laptops, tablets, desktop computers, set-top boxes, and mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), or as a server. The exemplary application of the information recommendation device when implemented as a server will be described below.

[0056] See Figure 2 , Figure 2 This is a schematic diagram of an optional architecture of the information recommendation system provided in the embodiments of this application; as shown... Figure 1 As shown, to support an information recommendation application, in the information recommendation system 100, a terminal 400 (terminals 400-1 and 400-2 are shown as examples) connects to a server 200 (information recommendation device) via a network 300. The network 300 can be a wide area network (WAN), a local area network (LAN), or a combination of both. Additionally, the information recommendation system 100 also includes a database 500, used to provide data support to the server 200 when the server 200 executes the information recommendation method provided in this embodiment.

[0057] Terminal 400 is used to send objects to be recommended to server 200 via network 300; it is also used to receive target recommendation information sent by server 200 via network 300 and display the target recommendation information.

[0058] Server 200 is used to recommend information using an original recommendation model, obtain a feedback information sequence corresponding to a sequence of recommended instances, wherein each piece of feedback information in the feedback information sequence includes a delayed feedback reward; obtain feedback correction parameters corresponding to each recommended instance in the sequence of recommended instances, wherein the feedback correction parameters are negatively correlated with the probability of receiving delayed feedback for each recommended instance currently, and positively correlated with the probability of receiving delayed feedback for each recommended instance eventually; correct the delayed feedback reward corresponding to each recommended instance based on the feedback correction parameters, thereby obtaining a corrected feedback information sequence corresponding to the sequence of recommended instances; update the original recommendation model based on the sequence of recommended instances and the corrected feedback information sequence to obtain a recommendation model, wherein the recommendation model is used for information recommendation. Server 200 is also used to receive objects to be recommended from terminal 400 via network 300, obtain a sequence of information to be recommended from database 500, determine the target recommendation information for the objects to be recommended from the sequence of information to be recommended, and send the target recommendation information to terminal 400 via network 300.

[0059] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Terminal 400 may be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0060] See Figure 3 , Figure 3 This is one of the embodiments provided in this application. Figure 2 A schematic diagram of the server's structure. Figure 3 The server 200 shown includes at least one processor 210, memory 250, at least one network interface 220, and a user interface 230. The various components in server 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to implement communication between these components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 3 The general labeled all buses as Bus System 240.

[0061] Processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0062] User interface 230 includes one or more output devices 231 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 230 also includes one or more input devices 232, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0063] The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 250 may optionally include one or more storage devices physically located away from the processor 210.

[0064] The memory 250 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 250 described in this application embodiment is intended to include any suitable type of memory.

[0065] In some embodiments, memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0066] Operating system 251 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0067] The network communication module 252 is used to reach other computing devices via one or more (wired or wireless) network interfaces 220, such as Bluetooth, Wi-Fi, and Universal Serial Bus (USB).

[0068] Presentation module 253 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 231 (e.g., a display screen, a speaker, etc.) associated with user interface 230;

[0069] The input processing module 254 is used to detect and translate one or more user inputs or interactions from one or more input devices 232.

[0070] In some embodiments, the information recommendation device provided in this application can be implemented in software. Figure 3 An information recommendation device 255 stored in memory 250 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a data acquisition module 2551, a parameter acquisition module 2552, a feedback correction module 2553, a model update module 2554, and a model application module 2555. These modules are logically connected and can therefore be arbitrarily combined or further divided according to their implemented functions. The functions of each module will be described below.

[0071] In other embodiments, the information recommendation device provided in this application can be implemented in hardware. As an example, the information recommendation device provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the information recommendation method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0072] The information recommendation method provided in this application will be described below with reference to exemplary applications and implementations of the server provided in the embodiments of this application.

[0073] See Figure 4 , Figure 4 This is an optional flowchart illustrating the information recommendation method provided in the embodiments of this application, which will be combined with... Figure 4 The steps shown are explained.

[0074] S401. Recommend information using the original recommendation model to obtain the feedback information sequence corresponding to the recommended instance sequence.

[0075] In this embodiment, when the information recommendation device performs information recommendation using the original recommendation model, it obtains corresponding information recommendation data, thus acquiring a recommendation instance sequence. This recommendation instance sequence includes at least one recommendation instance, and each recommendation instance includes at least one of the following: a recommendation object, a recommendation information sequence, and a recommendation action corresponding to the determined recommendation information. After completing information recommendation using the original recommendation model and obtaining the recommendation instance sequence, the information recommendation device obtains corresponding feedback information for each recommendation instance in the recommendation instance sequence, thus obtaining a feedback information sequence corresponding to the recommendation instance sequence. Here, the recommendation instance sequence and the feedback information sequence correspond one-to-one.

[0076] It should be noted that each piece of feedback information in the feedback information sequence includes a delayed feedback reward. The delayed feedback reward is a reward determined based on whether delayed feedback has been received, such as a reward determined based on whether delayed feedback has been received at the current time. Here, delayed feedback is a type of feedback with a delay, where the time between the information recommendation time and the time of receiving the feedback is greater than a time threshold (e.g., 5 minutes, 1 hour, 1 day, or 7 days). For example, in the information recommendation of consumer vouchers, the time between the time of completing the information recommendation of the consumer voucher and the time of the user using the consumer voucher is greater than the time threshold.

[0077] S402. Obtain the feedback correction parameters corresponding to each recommended instance in the recommended instance sequence.

[0078] It should be noted that since delayed feedback rewards are determined based on the receipt of delayed feedback, when the received delayed feedback reward is based on the absence of delayed feedback, there may be cases where users have not yet converted based on the recommended information but will convert later, or vice versa; therefore, the authenticity of delayed feedback rewards is relatively low. Here, to improve the authenticity of delayed feedback rewards, the information recommendation device obtains a feedback correction parameter for each recommendation instance in the recommendation instance sequence. This parameter is negatively correlated with the probability of receiving delayed feedback from each recommendation instance at the current time (the time feedback information is received) and positively correlated with the probability of eventually receiving delayed feedback from each recommendation instance. This feedback correction parameter is used to improve the authenticity of delayed feedback rewards.

[0079] For example, the feedback correction parameter can be expressed by equation (0):

[0080]

[0081] Where w represents the feedback correction parameter, S is the context space corresponding to the recommended object and the recommended information sequence, Pr{Y=1|S} represents the probability of receiving delayed feedback for each recommended instance at present, and Pr{V=1|S} represents the probability of finally receiving delayed feedback for each recommended instance.

[0082] Understandably, since the feedback correction parameter is negatively correlated with the probability of receiving delayed feedback for each recommended instance at present and positively correlated with the probability of eventually receiving delayed feedback for each recommended instance, w is higher for recommended instances that currently receive delayed feedback and for recommended instances that will eventually receive delayed feedback, while w is lower for recommended instances that currently do not receive delayed feedback.

[0083] S403. Based on the feedback correction parameters, the delayed feedback reward corresponding to each recommended instance is corrected to obtain the corrected feedback information sequence corresponding to the recommended instance sequence.

[0084] In this embodiment, after obtaining the feedback correction parameters corresponding to each recommendation instance, the information recommendation device uses these parameters as weights to fuse with the delayed feedback reward, thus completing the correction of the delayed feedback reward and consequently the correction of the feedback information, obtaining the corrected feedback information. Here, after correcting the delayed feedback rewards for all recommendation instances in the recommendation instance sequence, a corrected feedback information sequence corresponding to the recommendation instance sequence is obtained. The corrected feedback information sequence corresponds one-to-one with the original feedback information sequence, and the recommendation instance sequence also corresponds one-to-one with the corrected feedback information sequence.

[0085] It should be noted that the information recommendation device can multiply the feedback correction parameter with the delayed feedback reward to correct the delayed feedback reward; other methods can also be used to combine the feedback correction parameter with the delayed feedback reward, and this application embodiment does not specifically limit this.

[0086] S404. Based on the recommended instance sequence and the corrected feedback information sequence, update the original recommendation model to obtain the recommendation model.

[0087] In this embodiment of the application, after the information recommendation device obtains the recommended instance sequence and the corrected feedback information sequence, it uses the recommended instance sequence and the corrected feedback information sequence as training samples to train the original recommendation model, so as to update the original recommendation model. After the update is completed, the recommendation model is obtained. It is easy to see that the recommendation model is used for information recommendation.

[0088] It is understandable that by obtaining feedback correction parameters to adjust the delayed feedback reward, the authenticity of the adjusted delayed feedback reward is improved, and thus the information recommendation of the recommendation model updated based on the adjusted delayed feedback reward has higher accuracy.

[0089] See Figure 5 , Figure 5 This is another optional flowchart illustrating the information recommendation method provided in the embodiments of this application; such as Figure 5 As shown in the embodiment of this application, S402 can be implemented through S4021-S4025; that is, the information recommendation device obtains the feedback correction parameters corresponding to each recommendation instance in the recommendation instance sequence, including S4021-S4025. The steps are described below.

[0090] S4021. Obtain the collection time period corresponding to the recommended instance sequence, and determine the counterfactual time from the collection time period.

[0091] In this embodiment, the information recommendation device can use a counterfactual importance sampling method to determine the feedback correction parameters corresponding to each recommendation instance for the recommendation instance sequence and the feedback information sequence. First, the information recommendation device obtains the start and end times of the collection of the recommendation instance sequence, thus obtaining the collection time period; then, the information recommendation device determines a time from the collection time period, thus obtaining the counterfactual time.

[0092] It should be noted that the counterfactual time is located between the collection periods and is used to simulate the collection deadline for recommended instances.

[0093] S4022. From the recommended instance sequence, select recommended instances between the start time and the counterfactual time of the collection period to obtain the observation instance sequence.

[0094] In this embodiment of the application, the information recommendation device divides the recommended instance sequence into two parts using the counterfactual time as the dividing point. The first part is the observation instance sequence; that is, the observation instance sequence is the sequence of recommended instances between the start time of the collection period and the counterfactual time.

[0095] S4023. From the observed instance sequence, select recommended instances with delayed feedback reward as the delayed feedback reward threshold to obtain the training instance sequence.

[0096] It should be noted that the delayed feedback reward threshold refers to the reward corresponding to the delayed feedback of the recommended instance that has received delayed feedback; here, the training instance sequence includes at least one recommended instance whose delayed feedback reward is equal to the delayed feedback reward threshold.

[0097] S4024. For each recommended action in the recommended action sequence corresponding to the recommended instance sequence, based on the training instance sequence, train the survival model parameters for each recommended action, thereby obtaining the survival model parameter sequence corresponding to the recommended action sequence.

[0098] In this embodiment, the information recommendation device uses the collection time period as a simulated feedback time interval for information recommendation and the counterfactual time as a simulated deadline for collecting recommendation instances. This is equivalent to simulating a scenario of collecting feedback information for recommendation instances under the condition that delayed feedback has been received. Based on this scenario, the survival model parameters for each recommendation action are learned. Here, once the information recommendation device obtains the survival model parameters corresponding to all recommendation information for the recommendation action sequence, it also obtains the survival model parameter sequence corresponding to the recommendation information sequence.

[0099] It should be noted that the survival model parameters are used to predict the feedback correction parameters for each recommendation instance. Each recommendation instance includes a corresponding recommendation action, which refers to the process of recommending a piece of information; the recommendation action sequence is the sequence of recommendation actions formed by the recommendation instance sequence.

[0100] S4025. Based on the survival model parameter sequence and the recommendation action corresponding to each recommendation instance, determine the feedback correction parameters.

[0101] In this embodiment, the information recommendation device determines the corresponding survival model parameters based on the recommendation action corresponding to each recommendation instance, and then estimates the feedback correction parameters of the recommendation instance based on the corresponding survival model parameters.

[0102] Understandably, since the probability of ultimately receiving delayed feedback for each recommended instance is unpredictable, predicting the feedback correction parameters for each recommended instance through counterfactual learning and importance sampling methods provides a feasible way to obtain these parameters.

[0103] In this embodiment of the application, the information recommendation device in S4021 determines the counterfactual time from the collection time period, including S40211 to S40213. Each step is described below.

[0104] S40211. Determine the collection duration based on the collection time period.

[0105] It should be noted that the collection duration refers to the duration corresponding to the collection period.

[0106] S40212. Based on counterfactual parameters, determine the simulated collection duration corresponding to the collection duration.

[0107] In this embodiment, the information recommendation device can obtain a preset counterfactual parameter, which is used to simulate the collection duration corresponding to a training instance sequence based on the collection duration, i.e., the simulated collection duration. Here, the information recommendation device can determine the simulated collection duration based on the fusion result of the counterfactual parameter and the collection duration. For example, the simulated collection duration can be the product of the counterfactual parameter and the collection duration.

[0108] S40213. Determine the counterfactual time based on the start time of the collection period and the simulated collection duration.

[0109] In this embodiment of the application, the information recommendation device determines the time from the start time of the collection period to the simulated collection duration, which is the counterfactual time; that is, the sum of the start time of the collection period and the simulated collection duration is the counterfactual time.

[0110] It should be noted that the larger the counterfactual parameter, the later the counterfactual time; the smaller the counterfactual parameter, the earlier the counterfactual time.

[0111] In this embodiment, each feedback message also includes a feedback start time and a lapse duration. The feedback start time can be, for example, the information recommendation time, the time of receiving immediate feedback, or the time of collecting each recommendation instance. Here, immediate feedback and delayed feedback are relative concepts. The lapse duration refers to the time between the feedback start time and the observation time, which is the earliest of the data collection time and the time of receiving delayed feedback. The data collection time can be the end time of the collection period. Furthermore, the start time of the collection period can be the feedback start time of the first recommendation instance. In this case, S4024 can be implemented through S40241 to S40245. That is, the information recommendation device trains the survival model parameters for each recommendation action based on the training instance sequence, including S40241 to S40245. Each step is described below.

[0112] S40241. Determine the feedback end time based on the feedback start time and churn duration.

[0113] In this embodiment of the application, the information recommendation device uses the feedback start time as the start time and the time determined after the elapsed period as the feedback end time; that is, the feedback end time is the sum of the feedback start time and the elapsed period.

[0114] S40242. Based on the comparison results between the feedback end time and the counterfactual time, determine the delayed feedback simulated reward corresponding to each training instance in the training instance sequence.

[0115] It should be noted that when the comparison between the feedback end time and the counterfactual time shows that the feedback end time is earlier than the counterfactual time, the simulated delayed feedback reward for each training instance in the training instance sequence is determined as the first reward, for example, 1, where the first reward represents the reward corresponding to when delayed feedback has been received; while when the comparison shows that the feedback end time is not earlier than the counterfactual time, the simulated delayed feedback reward is determined as the second reward, for example, 0, where the second reward represents the reward corresponding to when delayed feedback has not been received.

[0116] S40243. The duration between the counterfactual time and the feedback start time is determined as the simulated feedback duration.

[0117] In other words, the simulated feedback duration is the time between the counterfactual time and the feedback start time.

[0118] S40244. Determine the shortest duration between the simulated feedback duration and the churn duration as the simulated churn duration for each training instance, thereby obtaining a pair of delayed feedback simulated reward and simulated churn duration for each training instance.

[0119] In this embodiment of the application, after the information recommendation device obtains the simulated feedback duration, it compares the simulated feedback duration with the churn duration and determines the shortest duration between the simulated feedback duration and the churn duration as the simulated churn duration corresponding to each training instance; at this time, for each training instance, a corresponding delayed feedback simulated reward and simulated churn duration pair is obtained.

[0120] S40245. Based on the sequence of delayed feedback simulated reward and simulated churn duration corresponding to the training instance sequence, train the survival model parameters for each recommended action.

[0121] It should be noted that once the corresponding delayed feedback simulated reward and simulated churn duration pairs are obtained for all training instances in the training instance sequence, a sequence of delayed feedback simulated reward and simulated churn duration pairs corresponding to the training instance sequence is also obtained. At this point, the information recommendation device performs maximum likelihood estimation based on each delayed feedback simulated reward and simulated churn duration pair in the delayed feedback simulated reward and simulated churn duration pair sequence to train the survival model parameters for each recommendation action.

[0122] In this embodiment, S4025 can be implemented through S40251 to S40253; that is, the information recommendation device determines the feedback correction parameters based on the survival model parameter sequence and the recommendation action corresponding to each recommendation instance, including S40251 to S40253. Each step is described below.

[0123] S40251. From the survival model parameter sequence, determine the target survival model parameters corresponding to the recommended action for each recommended instance.

[0124] It should be noted that each survival model parameter in the survival model parameter sequence corresponds to a recommendation action, and each recommendation instance includes a corresponding recommendation action. Thus, the information recommendation device determines the survival model parameter corresponding to the recommendation action that matches the recommendation action corresponding to each recommendation instance from the survival model parameter sequence, thereby obtaining the target survival model parameter.

[0125] S40252. Determine the survival information of each recommended instance using the target survival model parameters.

[0126] In this embodiment, the information recommendation device solves for the survival function based on the target survival model parameters, thereby determining the survival information of each recommended instance. Here, the information recommendation device can calculate the value of the survival function based on the target survival model parameters and the context space formed by the recommended object and the recommended information sequence corresponding to each recommended instance, to obtain the survival information.

[0127] S40253, Obtain feedback correction parameters that are negatively correlated with survival information.

[0128] In this embodiment of the application, after the information recommendation device obtains the survival information, it can determine the corresponding feedback correction parameters based on the survival information; here, the feedback correction parameters are negatively correlated with the survival information.

[0129] It should be noted that when each recommended instance includes churn duration, the information recommendation device can also obtain feedback correction parameters based on survival information and churn duration; in this case, the feedback correction parameters are negatively correlated with both survival information and churn duration.

[0130] In this embodiment, S404 can be implemented through S4041 to S4043; that is, the information recommendation device updates the original recommendation model based on the recommendation instance sequence and the corrected feedback information sequence to obtain the recommendation model, including S4041 to S4043. Each step is described below.

[0131] S4041. Update the original recommendation model based on the recommended instance sequence and the corrected feedback information sequence.

[0132] In this embodiment, the information recommendation device uses the recommended instance sequence and the corrected feedback information sequence as training samples to update the original recommendation model in batches, thus obtaining the updated original recommendation model.

[0133] S4042. Recommend information using the updated original recommendation model to obtain a new feedback information sequence corresponding to the new recommendation instance sequence.

[0134] It should be noted that after obtaining the updated original recommendation model, the information recommendation device performs information recommendation using the updated original recommendation model. At this time, the information recommendation device acquires the information recommendation data corresponding to the information recommendation performed by the updated original recommendation model, thus obtaining a new recommendation instance sequence. The training of this new recommendation instance includes at least one new recommendation instance, and each new recommendation instance includes at least one of the following: a new recommendation object, a new recommendation information sequence, and a recommendation action corresponding to the newly determined recommendation information. After the updated original recommendation model completes information recommendation and obtains the new recommendation instance sequence, the information recommendation device acquires the corresponding new feedback information for each new recommendation instance in the new recommendation instance sequence, thus obtaining a new feedback information sequence corresponding to the new recommendation instance sequence. Here, the new recommendation instance sequence and the new feedback information sequence correspond one-to-one, and each new feedback information in the new feedback information sequence includes a new delayed feedback reward.

[0135] S4043. Based on the new recommended instance sequence and the corrected new feedback information sequence, continue to iteratively update the updated original recommendation model until the update deadline is met, then stop iterative updating and obtain the recommendation model.

[0136] It should be noted that by continuously iterating and updating the original recommendation model, the accuracy of the information recommendation obtained by the model can be improved. Here, the update cutoff condition can be reaching the required number of iterations, or the recommendation metric corresponding to the obtained model reaching a threshold, etc., and this application does not specifically limit it in this way.

[0137] In this embodiment, S4041 can be implemented through S40411 to S40413; that is, the information recommendation device updates the original recommendation model based on the recommendation instance sequence and the corrected feedback information sequence, including S40411 to S40413. Each step is described below.

[0138] S40411. Divide the recommendation instance sequence based on the recommendation actions in the recommendation instance sequence to obtain the recommendation instance subsequence corresponding to the recommendation action of each recommendation information.

[0139] It should be noted that since each recommendation instance in each recommendation instance sequence includes a recommendation action, the information recommendation device can divide the recommendation instance sequence based on the recommendation action, thereby obtaining the recommendation instance subsequence corresponding to each recommendation action. It is easy to see that all recommendation instance subsequences corresponding to all recommendation actions constitute the recommendation instance sequence.

[0140] S40412. Obtain the corrected feedback information subsequence corresponding to the recommended instance subsequence from the corrected feedback information sequence.

[0141] In this embodiment, after the information recommendation device obtains the recommended instance subsequence, since each recommended instance corresponds to a modified feedback information in the modified feedback information sequence, the information recommendation device can obtain the modified feedback information subsequence corresponding to the recommended instance subsequence from the modified feedback information sequence. Here, the recommended instance subsequence and the modified feedback information subsequence correspond one-to-one.

[0142] S40413. Based on the recommended instance subsequence and the corrected feedback information subsequence, update the recommendation parameters of each recommendation action in the original recommendation model to complete the update of the original recommendation model.

[0143] It should be noted that during the batch update of the original recommendation model by the information recommendation device, each recommendation action is updated separately. Here, for each recommendation action, the information recommendation device updates the recommendation parameters of that recommendation action in the original recommendation model based on the recommendation instance subsequence and the corrected feedback information subsequence. When the information recommendation device has completed updating the recommendation parameters of all recommendation actions, it has also completed updating the original recommendation model.

[0144] In this embodiment, the process by which the information recommendation device updates the original recommendation model based on the new recommendation instance sequence and the corrected new feedback information sequence in S4043 is similar to the update process described in S40411 to S40413, and will not be repeated here.

[0145] In this embodiment of the application, in S40413, the information recommendation device updates the recommendation parameters of each recommendation action in the original recommendation model based on the recommendation instance subsequence and the corrected feedback information subsequence, including S404131 to S404134. Each step is described below.

[0146] S404131. Obtain the context space combination corresponding to the corrected recommended instance subsequence.

[0147] It should be noted that when the feedback instance includes a recommended object and a sequence of recommended information, the information recommendation device extracts the features of the recommended object and the features of the recommended information sequence, thus obtaining the corresponding context space. Here, the information recommendation device combines the context spaces corresponding to each recommended instance in the corrected feedback instance subsequence, thus obtaining a context space combination. This context space combination can be in matrix form, set form, etc., and this application embodiment does not specifically limit it.

[0148] S404132. Based on context space combination, update the first sub-recommendation parameter of each recommendation action in the original recommendation model.

[0149] In this embodiment, the information recommendation device integrates the context space combination with the first sub-recommendation parameter of each recommendation action in the original recommendation model, thus updating the first sub-recommendation parameter and obtaining the updated first sub-recommendation parameter. Here, the first sub-recommendation parameter is a parameter of the recommendation strategy corresponding to each recommendation action, such as the covariance matrix of each recommendation action.

[0150] S404133, Obtain the feedback reward combination corresponding to the corrected feedback information subsequence.

[0151] It should be noted that since each feedback message includes a delayed feedback reward, the corrected feedback message also includes a corrected delayed feedback reward. The information recommendation device combines the corrected delayed feedback rewards included in each corrected feedback message in the corrected feedback message subsequence to obtain the feedback reward combination corresponding to the corrected feedback message subsequence.

[0152] S404134. Based on the combination of feedback reward and recommendation context space, update the second sub-recommendation parameters of each recommendation action in the original recommendation model.

[0153] In this embodiment, the information recommendation device integrates the feedback reward combination and the recommendation context space, and then integrates them with the second sub-recommendation parameters of each recommendation action in the original recommendation model. This completes the update of the second sub-recommendation parameters, resulting in the updated second sub-recommendation parameters. Here, the second sub-recommendation parameter is another parameter of the recommendation strategy corresponding to each recommendation action.

[0154] It should be noted that the recommended parameters include the first sub-recommendation parameter and the second sub-recommendation parameter, and the updated recommended parameters include the updated first sub-recommendation parameter and the updated second sub-recommendation parameter.

[0155] In this embodiment, the feedback information also includes an instant feedback reward, which is a reward determined based on the received instant feedback, such as a reward determined based on whether instant feedback has been received at the current time. Here, instant feedback is feedback that occurs immediately, where the time between the information recommendation time and the time the feedback is received is less than a time threshold (e.g., 5 minutes, 1 hour, 1 day, or 7 days). For example, in the information recommendation of a consumer voucher, the user's action of clicking to claim the consumer voucher at the time the information recommendation is completed is an instant feedback. At this time, the information recommendation device in S404133 obtains the feedback reward combination corresponding to the corrected feedback information subsequence through the following processing: For each corrected feedback information in the corrected feedback information subsequence, the information recommendation device combines the instant feedback reward and the corrected delayed feedback reward into a feedback reward, thereby obtaining the feedback reward combination corresponding to the corrected feedback information subsequence.

[0156] See Figure 6 , Figure 6 This is another optional flowchart illustrating the information recommendation method provided in the embodiments of this application; such as Figure 6 As shown in the embodiment of this application, S404 is followed by S405 to S408; that is, after the information recommendation device obtains the recommendation model, the information recommendation method further includes S405 to S408. Each step is described below.

[0157] S405. In response to the information recommendation request, obtain the object to be recommended and the sequence of information to be recommended.

[0158] In this embodiment, when an information recommendation event is triggered, an information recommendation request is generated. This request can be received by the information recommendation device or generated by the device itself; this embodiment does not specifically limit this. The information recommendation device responds to the request, thus obtaining the object to be recommended and the sequence of information to be recommended.

[0159] It should be noted that the information recommendation event can be triggered by receiving user actions, such as when a user clicks the "View More" button on the page; it can also be triggered by meeting pre-set conditions, such as when the video playback progress is nearing its end; etc., this application embodiment does not specifically limit this. Here, the object to be recommended is the object to be recommended information, and the sequence of information to be recommended is the set of candidate recommendation information to be recommended to the object to be recommended.

[0160] S406. Extract the features of the object to be recommended and the information sequence to be recommended respectively to obtain the context space to be recommended.

[0161] In this embodiment of the application, the information recommendation device extracts the features of the object to be recommended and the features of each piece of information to be recommended in the information sequence to be recommended. The extracted features of the object to be recommended and the features of each piece of information to be recommended in the information sequence to be recommended are combined to obtain the context space to be recommended.

[0162] S407. Input the context space to be recommended into the recommendation model to determine the estimated feedback reward sequence corresponding to the information sequence to be recommended.

[0163] In this embodiment, after obtaining the context space to be recommended, the information recommendation device inputs the context space into the recommendation model. Based on the features of the objects to be recommended in the context space and the features of the information sequence to be recommended, the recommendation model calculates the score corresponding to each piece of information in the information sequence to be recommended, thus obtaining the estimated feedback reward for each piece of information. Here, when the information recommendation device has completed the calculation of the estimated feedback rewards for all the information in the information sequence to be recommended, it obtains the estimated feedback reward sequence corresponding to the information sequence to be recommended. The estimated feedback reward sequence corresponds one-to-one with the information sequence to be recommended.

[0164] S408. In the sequence of information to be recommended, the information to be recommended corresponding to the maximum estimated feedback reward in the estimated feedback reward sequence is taken as the target recommendation information and displayed.

[0165] In this embodiment of the application, the information recommendation device selects the maximum estimated feedback reward from the obtained estimated feedback reward sequence, and determines the information to be recommended that matches the maximum estimated feedback reward from the information to be recommended sequence, thereby obtaining the target recommendation information.

[0166] It should be noted that when the information recommendation device displays target recommendation information, it may display the target recommendation information on the information recommendation device itself, or it may send the target recommendation information to other devices and display the target recommendation information on other devices; etc., the embodiments of this application do not specifically limit this.

[0167] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0168] See Figure 7 , Figure 7 This is a flowchart illustrating an exemplary information recommendation method provided in an embodiment of this application; as shown below. Figure 7 As shown, when the recommended information is a product, this exemplary information recommendation method includes:

[0169] S701. Initialize the recommendation model to be trained (original recommendation model).

[0170] It should be noted that when the recommendation model to be trained is a strategy without recommendation experience, the initialization process can randomly select products from the candidate product set (recommendation information sequence) for recommendation.

[0171] Here, corresponding to the M items in the candidate item set, the action space (recommendation action sequence) can be determined. Where j represents the j-th item out of M items, and the action space... Each action (each recommended action) corresponds to a candidate product. Selection Action A in the equation indicates the recommendation of the corresponding product; therefore, for the recommendation strategy, i.e. the recommendation model π0 to be trained, the corresponding initialization process can be expressed as: π0←1 / M.

[0172] S702. Obtain batch recommendation instances (recommendation instance sequence) when the recommendation model to be trained performs information recommendation.

[0173] It should be noted that the batch recommendation instances here include B recommendation instances, and the data records corresponding to these B recommendation instances can be represented as follows: Where b represents the b-th recommended instance out of B recommended instances; S 0,b Let A represent the context space corresponding to the b-th recommendation instance obtained by information recommendation based on the recommendation model π0 to be trained. The context space includes user information (recommendation object) and vector features of the candidate item set; 0,b C represents the recommendation action corresponding to the b-th recommendation instance obtained based on the recommendation model π0 to be trained; 0,b Let Y represent the immediate feedback variable corresponding to the b-th recommendation instance obtained based on the recommendation model π0 to be trained. The immediate feedback variable represents whether immediate feedback has been given, such as a click on a product. If it is, the value is 1, and if it is not, the value is 0. Therefore, the immediate feedback variable C∈{0,1}. Here, the immediate feedback reward corresponds to the immediate feedback variable. 0,b Let represent the delayed feedback variable corresponding to the b-th recommendation instance obtained based on the recommendation model π0 to be trained. The delayed feedback variable characterizes whether delayed feedback has occurred before the data collection time, such as the conversion of goods (use of vouchers, purchase of goods, etc.). If it has, it is 1; otherwise, it is 0. Therefore, the immediate feedback variable Y∈{0,1}, where the immediate feedback reward corresponds to the immediate feedback variable. Obviously, if C=0, then Y=0, but Y=0 does not mean that delayed feedback has not occurred, because delayed feedback may occur after the data collection time. Therefore, we set the variable V∈{0,1} to indicate whether delayed feedback will eventually occur. Since the value of V may be unobservable, we can set the variable D∈{0,1} to indicate whether delayed feedback can be observed.0,b Indicates the time of immediate feedback (feedback start time); e 0,b The churn duration refers to the time between the immediate feedback time and the observation time, which is the earliest of the data collection time and the delayed feedback time. The churn duration is obtained here because the delayed duration (the time between the immediate feedback time and the delayed feedback time) is not visible.

[0174] Here, based on the set sample size T, the vector dimension d of the context space, and the constant C... B (For example, C) B ∈[75,80]), determine the batch size B (the number of recommended instances in the recommended instance sequence), as shown in equation (1):

[0175]

[0176] S703. Based on batch recommendation instances, an iterative reinforcement learning approach is used to train the recommendation model to obtain the recommendation model.

[0177] It should be noted that the number of episodes N can be determined based on the sample size T and the batch size B, as shown in equation (2):

[0178] N = T / B (2)

[0179] Here, the number of episodes N refers to the number of iterations of reinforcement learning. In each episode, the recommendation strategy is updated based on the data of B collected recommendation instances, and then the updated recommendation strategy is used to recommend information.

[0180] The following section explains the adjustments to rewards and updates to the recommendation strategy for a specific scene.

[0181] In this embodiment of the application, the instant feedback reward can be determined based on the observed value C of the obtained instant feedback. As shown in equation (3):

[0182]

[0183] Based on the observed value Y of the delayed feedback, the delayed feedback reward can be determined. As shown in equation (4):

[0184]

[0185] Thus, a feedback reward R is obtained, which is an immediate feedback reward. and delayed feedback rewards The linear combination of is shown in equation (5):

[0186]

[0187] Where λ∈[0,1) is the weight parameter, which can be obtained through equation (6), which is:

[0188]

[0189] Among them, C λ This is the estimated CVR (Conversion Rate) value.

[0190] It should be noted that, due to the instant feedback reward It is observable and accurate; while delayed feedback rewards Delayed feedback may be observable (occurring before data collection) or it may not be observable (occurring after data collection), therefore, rewards are needed for delayed feedback. Corrections are made to obtain training samples with accurate feedback information. Therefore, it is desired to obtain an importance weight w (feedback correction parameter) to enhance the reward of identified and more likely delayed feedback recommendation instances, thereby correcting the delayed feedback reward.

[0191] Here, the enhancement process can be achieved through equation (7), which is shown below:

[0192]

[0193] in, This is the revised delayed feedback reward; therefore, the revised feedback reward R mod This can be expressed as equation (8), which is:

[0194]

[0195] It should also be noted that, since V is an unobservable variable, based on the principle corresponding to equation (0), a counterfactual learning method can be used to determine the importance weight w. First, determine the data corresponding to the B collected recommendation instances. Collection start time (Start time of the collection period) and end time of the collection period (The deadline for collecting data); where n represents the nth act in N acts, and when n is 1, Right now Collection start time Indicates the start of collection Timestamp, collection deadline Indicates completion The timestamps collected.

[0196] Secondly, based on the set counterfactual parameter C ξ and the start time of collection and collection deadline The counterfactual time ξ is determined, and the process of determining it can be achieved through equation (9), which is:

[0197]

[0198] Here, by using counterfactual time ξ, Divided into two sets of recommended instances: the observation set and the reserved instance set; see also Figure 8 , Figure 8 This is an exemplary instance partitioning diagram provided in an embodiment of this application; as shown... Figure 8 As shown, the counterfactual time ξ will be the collection start time. and collection deadline The recommended instance 8-1 is divided into an observation instance set 8-11 and a retention instance set 8-12. The observation instance set 8-11 includes data for q recommended instances. to Reserve the data for instance set 8-12, which includes B-q+1 recommended instances: to

[0199] Next, recommended instances with Y=1 (recommended instances whose delayed feedback reward is the delayed feedback reward threshold) are selected from the set of observed instances as training instances, and the counterfactual time ξ is determined as the simulated data collection time. Thus, when the immediate feedback time c (feedback start time) and the time determined by the churn duration e (feedback end time) are before the counterfactual time ξ, the simulated immediate feedback variable Y is determined. obs (Corresponding to the delayed feedback simulation reward) is 1, and when the time for determining the immediate feedback time and the churn duration is not before the counterfactual time ξ, the simulation's immediate feedback variable Y is determined. obs A value of 0 corresponds to the instantaneous feedback variable Y in the simulation. obs The process can be achieved through equation (10), which is:

[0200]

[0201] Furthermore, when the counterfactual time ξ is defined as the simulated data collection time, the duration between the counterfactual time ξ and the immediate feedback time is compared with the churn duration e, and the shorter duration is selected as the model's churn duration e. obs (Simulated churn duration); the corresponding simulated churn duration e is determined. obs The process can be achieved through equation (11), which is:

[0202] e obs =min{ξ-c,e} (11)

[0203] Then, for the recommendation action of each item in the candidate item set, maximum likelihood estimation is used, based on the simulated immediate feedback variable Y. obs And the simulated lapse time e obs Determine the parameterized prediction model β with the corresponding estimated importance weights. A (Survival model parameters); the corresponding prediction model β that determines the recommendation action for a product. A The process can be achieved through equation (12), which is:

[0204]

[0205] in, This represents the recommended instance corresponding to recommended action A in the training instances; h k for The survival function corresponding to the k-th recommended instance is shown in Equation (13):

[0206] h k =exp(<β) A ,S k >) (13)

[0207] at last, The importance weight w for each recommended instance in the list i This can be achieved through equation (14), which is:

[0208]

[0209] At this point, the feedback reward has been corrected, and the corrected feedback reward R has been obtained. mod At this point, batch updates of the strategy begin. First, based on recommended action A... The process involves partitioning the data to obtain each recommendation instance (a subsequence of recommendation instances) corresponding to each recommendation action A. Based on the context spaces S of all recommendation instances corresponding to each recommendation action A, a context space matrix is ​​obtained. (Contextual space composition), it is easy to see, The dimension is and The number of times action A is performed in each recommended instance; it can also obtain a reward vector based on all feedback rewards R of each recommended instance corresponding to action A. (Feedback reward combination), easy to understand, The dimension is also

[0210] Then, based on the context space matrix Update the covariance matrix for each recommended action A. (First sub-recommendation parameter), obtain the updated As shown in equation (15):

[0211]

[0212] And, according to the context space matrix and reward vector Update another recommendation parameter corresponding to each recommended action A. (Second sub-recommendation parameter), to obtain the updated As shown in equation (16):

[0213]

[0214] Furthermore, based on the updated and recommended parameters Obtain the closed-form solution for each recommended action A. (The updated recommended parameters) are shown in equation (17):

[0215]

[0216] Finally, based on each recommended action A, the corresponding recommendation strategy is obtained as shown in equation (18):

[0217]

[0218] Where μ is the regularization parameter, and its value range can be [0.5:+0.1:2].

[0219] Next, based on equation (18), product recommendations are made to obtain a definite recommendation action A', as shown in equation (19):

[0220]

[0221] Among them, S n+1 For the context space to be recommended;

[0222] S704. Based on the recommendation model, information is recommended.

[0223] based on Figure 7 For descriptions of S701 to S704, see [link to S704]. Figure 9 , Figure 9 This is a flowchart illustrating another exemplary information recommendation method provided in this application embodiment; as follows: Figure 9As shown, firstly, information is recommended to user 9-2 based on recommendation model 9-1, and information recommendation data 9-3 is collected, including recommendation decisions, immediate feedback, and delayed feedback; then, the information recommendation data 9-3 is corrected to obtain batch training data 9-4, and the recommendation model 9-1 is incrementally updated using the batch training data 9-4 until the recommendation model 9-1 has been updated N times.

[0224] based on Figure 7 For descriptions of S701 to S704, see [link to S704]. Figure 10 , Figure 10 This is a schematic diagram illustrating an exemplary process for correcting delayed feedback rewards provided in an embodiment of this application; as shown... Figure 10 As shown, based on the recommendation strategy 10-1(π) n (S) When making information recommendations to user 10-2, data is collected for the information recommendation processing of user 10-2, and a data buffer 10-3 is obtained. By employing a counterfactual importance sampling method to correct for delayed feedback rewards, a counterfactual buffer of 10⁻⁴ can be obtained. Finally, policy 10-1 is updated using counterfactual buffer 10-4.

[0225] It should be noted that, see Figure 11 , Figure 11 This is an exemplary diagram illustrating the comparison of information recommendation results provided in an embodiment of this application; as shown below. Figure 11 As shown, when the recommended information is a coupon, the sample size T is 216 or 568, the vector dimension d is 50, the batch size B is 5000, and the number of coupon types is 5, the recommendation model in the information recommendation method provided in this application embodiment is used for information recommendation. Compared with existing recommendation algorithms, curve 11-1 corresponds to the result of information recommendation using the recommendation model in the information recommendation method provided in this application embodiment, curve 11-2 corresponds to the result of information recommendation using the Naive Online Algorithm (SBUCB), curve 11-3 corresponds to the result of information recommendation using the batch version of the offline algorithm (DFM-S), curve 11-4 corresponds to the result of information recommendation using the Exponential Strategy Online Algorithm (EXP3-B), and curve 11-5 corresponds to the result of information recommendation using the Discard Strategy Online Algorithm (SBUCB-D). The horizontal axis represents the number of episodes, and the vertical axis represents the feedback reward. It is easy to see that the result of information recommendation using the recommendation model in the information recommendation method provided in this application embodiment is better than the result of information recommendation using all baseline algorithms.

[0226] In addition, Table 1 describes the results of information recommendation by the recommendation model in the information recommendation method provided in the embodiments of this application, as well as the results of information recommendation by other baseline algorithms, as shown in Table 1:

[0227] Table 1

[0228]

[0229] As shown in Table 1, the recommendation model in this embodiment improves the CVR and CTCVR (CTR (click pass rate) * CVR) by 3.86% and 2.39% respectively compared to the suboptimal algorithm; in addition, it is much more efficient than the batch version of the offline algorithm, and the time cost is comparable to the naive online algorithm; thus, it further demonstrates that the recommendation model obtained by the information recommendation method provided in this embodiment has high accuracy and efficiency.

[0230] The following description continues to illustrate the exemplary structure of the information recommendation device 255 provided in the embodiments of this application as a software module. In some embodiments, such as... Figure 3 As shown, the software module stored in the information recommendation device 255 of the memory 250 may include:

[0231] The data acquisition module 2551 is used to perform information recommendation through the original recommendation model and obtain the feedback information sequence corresponding to the recommendation instance sequence, wherein each piece of feedback information in the feedback information sequence includes a delayed feedback reward.

[0232] The parameter acquisition module 2552 is used to acquire the feedback correction parameter corresponding to each recommendation instance in the recommendation instance sequence, wherein the feedback correction parameter is negatively correlated with the probability of receiving delayed feedback from each recommendation instance at present and positively correlated with the probability of finally receiving delayed feedback from each recommendation instance.

[0233] The feedback correction module 2553 is used to correct the delayed feedback reward corresponding to each recommended instance based on the feedback correction parameters, thereby obtaining a corrected feedback information sequence corresponding to the recommended instance sequence;

[0234] The model update module 2554 is used to update the original recommendation model based on the recommended instance sequence and the corrected feedback information sequence to obtain a recommendation model, wherein the recommendation model is used for information recommendation.

[0235] In this embodiment, the parameter acquisition module 2552 is further configured to: acquire the collection time period corresponding to the recommended instance sequence; determine the counterfactual time from the collection time period; select recommended instances from the recommended instance sequence between the start time of the collection time period and the counterfactual time to obtain an observation instance sequence; select recommended instances from the observation instance sequence whose delayed feedback reward is a delayed feedback reward threshold to obtain a training instance sequence; for each recommended action in the recommended action sequence corresponding to the recommended instance sequence, train the survival model parameters of each recommended action based on the training instance sequence to obtain a survival model parameter sequence corresponding to the recommended action sequence; and determine the feedback correction parameters based on the survival model parameter sequence and the recommended action corresponding to each recommended instance.

[0236] In this embodiment, each feedback message further includes a feedback start time and a churn duration; the parameter acquisition module 2552 is further configured to determine a feedback end time based on the feedback start time and the churn duration; determine a delayed feedback simulated reward corresponding to each training instance in the training instance sequence based on a comparison between the feedback end time and the counterfactual time; determine the duration between the counterfactual time and the feedback start time as the simulated feedback duration; determine the shortest duration between the simulated feedback duration and the churn duration as the simulated churn duration corresponding to each training instance, thereby obtaining a delayed feedback simulated reward and simulated churn duration pair corresponding to each training instance; and train the survival model parameters for each recommended action based on the sequence of delayed feedback simulated rewards and simulated churn duration pairs corresponding to the training instance sequence.

[0237] In this embodiment of the application, the parameter acquisition module 2552 is further configured to determine the target survival model parameters corresponding to the recommendation action for each recommended instance from the survival model parameter sequence; determine the survival information of each recommended instance through the target survival model parameters; and acquire the feedback correction parameters that are negatively correlated with the survival information.

[0238] In this embodiment of the application, the parameter acquisition module 2552 is further configured to determine the collection duration based on the collection time period; determine the simulated collection duration corresponding to the collection duration based on the counterfactual parameter; and determine the counterfactual time based on the start time of the collection time period and the simulated collection duration.

[0239] In this embodiment of the application, the parameter acquisition module 2552 is further configured to: when the comparison result of the feedback end time and the counterfactual time is that the feedback end time is earlier than the counterfactual time, determine the delayed feedback simulation reward corresponding to each training instance in the training instance sequence as a first reward, wherein the first reward represents the reward for having received delayed feedback; and when the comparison result is that the feedback end time is not earlier than the counterfactual time, determine the delayed feedback simulation reward as a second reward, wherein the second reward represents the reward for not having received delayed feedback.

[0240] In this embodiment of the application, the model update module 2554 is further configured to update the original recommendation model based on the recommended instance sequence and the corrected feedback information sequence; perform information recommendation through the updated original recommendation model to obtain a new feedback information sequence corresponding to the new recommended instance sequence; correct the new feedback information sequence, and continue to iteratively update the updated original recommendation model based on the new recommended instance sequence and the corrected new feedback information sequence until the update cutoff condition is met, at which point the iterative update stops and the recommendation model is obtained.

[0241] In this embodiment of the application, the model update module 2554 is further configured to divide the recommendation instance sequence based on the recommendation actions in the recommendation instance sequence to obtain a recommendation instance subsequence corresponding to each recommendation action; obtain a corrected feedback information subsequence corresponding to the recommendation instance subsequence from the corrected feedback information sequence; and update the recommendation parameters of each recommendation action in the original recommendation model based on the recommendation instance subsequence and the corrected feedback information subsequence, thereby completing the update of the original recommendation model.

[0242] In this embodiment of the application, the model update module 2554 is further configured to obtain the context space combination corresponding to the corrected recommendation instance subsequence; update the first sub-recommendation parameter of each recommendation action in the original recommendation model based on the context space combination; obtain the feedback reward combination corresponding to the corrected feedback information subsequence; and update the second sub-recommendation parameter of each recommendation action in the original recommendation model based on the feedback reward combination and the context space combination, wherein the recommendation parameter includes the first sub-recommendation parameter and the second sub-recommendation parameter.

[0243] In this embodiment of the application, the feedback information further includes an immediate feedback reward; the model update module 2554 is further configured to combine the immediate feedback reward and the modified delayed feedback reward into a feedback reward for each modified feedback information in the modified feedback information subsequence, thereby obtaining the feedback reward combination corresponding to the modified feedback information subsequence.

[0244] In this embodiment, the information recommendation device further includes a model application module 2555, configured to, in response to an information recommendation request, acquire a target object and a target information sequence; extract features from the target object and the target information sequence to obtain a target recommendation context space; input the target recommendation context space into the recommendation model to determine an estimated feedback reward sequence corresponding to the target information sequence; and, in the target information sequence, select the target information corresponding to the maximum estimated feedback reward in the estimated feedback reward sequence as the target recommendation information and display the target recommendation information.

[0245] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the information recommendation method described in this application.

[0246] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to perform the method provided in this application, for example... Figure 4 The information recommendation method is shown.

[0247] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0248] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0249] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0250] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0251] In summary, through the embodiments of this application, during the training of the original recommendation model, corresponding feedback information sequences are collected for the recommendation instance sequences of the original recommendation model. Based on feedback correction parameters that are negatively correlated with the probability of delayed feedback received for each current recommendation instance and positively correlated with the probability of delayed feedback received for each final recommendation instance, the delayed feedback reward in the feedback information sequence is corrected. This improves the accuracy of the corrected delayed feedback reward. Consequently, when updating the original recommendation model based on the corrected delayed feedback reward, the information recommendation accuracy of the obtained recommendation model is higher, thus improving the overall information recommendation accuracy. Furthermore, when updating the original recommendation model, an iterative reinforcement learning method is used to actively collect recommendation instance sequences and corresponding feedback information sequences, enabling real-time updates to the original recommendation model and further improving the information recommendation accuracy of the obtained recommendation model.

[0252] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. An information recommendation method, characterized in that, include: Information recommendation is performed using the original recommendation model to obtain a feedback information sequence corresponding to the recommendation instance sequence, wherein each piece of feedback information in the feedback information sequence includes a delayed feedback reward. Obtain the feedback correction parameter corresponding to each recommendation instance in the recommendation instance sequence, wherein the feedback correction parameter is negatively correlated with the probability of receiving delayed feedback from each recommendation instance at present and positively correlated with the probability of eventually receiving delayed feedback from each recommendation instance; Based on the feedback correction parameters, the delayed feedback reward corresponding to each recommended instance is corrected to obtain a corrected feedback information sequence corresponding to the recommended instance sequence. Based on the recommended instance sequence and the corrected feedback information sequence, the original recommendation model is updated to obtain a recommendation model, wherein the recommendation model is used for information recommendation.

2. The method according to claim 1, characterized in that, The step of obtaining the feedback correction parameters corresponding to each recommended instance in the recommended instance sequence includes: Obtain the collection time period corresponding to the recommended instance sequence, and determine the counterfactual time from the collection time period; From the recommended instance sequence, recommended instances are selected between the start time of the collection period and the counterfactual time to obtain the observation instance sequence; From the observed instance sequence, recommended instances with the delayed feedback reward set as the delayed feedback reward threshold are selected to obtain a training instance sequence; For each recommended action in the recommended action sequence corresponding to the recommended instance sequence, survival model parameters for each recommended action are trained based on the training instance sequence, thereby obtaining a survival model parameter sequence corresponding to the recommended action sequence. The feedback correction parameters are determined based on the survival model parameter sequence and the recommendation action corresponding to each recommendation instance.

3. The method according to claim 2, characterized in that, Each feedback message also includes the feedback start time and the duration of the interruption; The step of training the survival model parameters for each recommended action based on the training instance sequence includes: The feedback end time is determined based on the feedback start time and the churn duration; Based on the comparison between the feedback end time and the counterfactual time, the delayed feedback simulated reward corresponding to each training instance in the training instance sequence is determined; The duration between the counterfactual time and the feedback start time is determined as the simulated feedback duration; The shortest duration between the simulated feedback duration and the churn duration is determined as the simulated churn duration corresponding to each training instance, thereby obtaining the delayed feedback simulated reward and the simulated churn duration pair corresponding to each training instance; Based on the sequence of delayed feedback simulated rewards and simulated churn durations corresponding to the training instance sequence, the survival model parameters for each recommended action are trained.

4. The method according to claim 2 or 3, characterized in that, The step of determining the feedback correction parameters based on the survival model parameter sequence and the recommendation action corresponding to each recommendation instance includes: From the survival model parameter sequence, determine the target survival model parameters corresponding to the recommendation action for each recommendation instance; The survival information of each recommended instance is determined using the target survival model parameters. Obtain the feedback correction parameter that is negatively correlated with the survival information.

5. The method according to claim 2 or 3, characterized in that, Determining the counterfactual time from the collection period includes: Based on the aforementioned collection period, the collection duration is determined; Based on counterfactual parameters, the simulated collection duration corresponding to the collection duration is determined; The counterfactual time is determined based on the start time of the collection period and the duration of the simulated collection.

6. The method according to claim 3, characterized in that, The step of determining the delayed feedback simulated reward for each training instance in the training instance sequence based on the comparison result between the feedback end time and the counterfactual time includes: When the comparison result between the feedback end time and the counterfactual time is that the feedback end time is earlier than the counterfactual time, the delayed feedback simulation reward corresponding to each training instance in the training instance sequence is determined as the first reward, wherein the first reward represents the reward for having received delayed feedback. When the comparison result indicates that the feedback end time is not earlier than the counterfactual time, the delayed feedback simulation reward is determined to be the second reward, wherein the second reward represents the reward for not receiving delayed feedback.

7. The method according to any one of claims 1 to 3, characterized in that, The step of updating the original recommendation model based on the recommended instance sequence and the corrected feedback information sequence to obtain a recommendation model includes: The original recommendation model is updated based on the recommended instance sequence and the corrected feedback information sequence; Information recommendation is performed using the updated original recommendation model to obtain a new feedback information sequence corresponding to the new recommendation instance sequence; The new feedback information sequence is corrected, and based on the new recommended instance sequence and the corrected new feedback information sequence, the updated original recommendation model is iteratively updated until the update deadline is met, at which point the iterative update stops, and the recommendation model is obtained.

8. The method according to claim 7, characterized in that, The step of updating the original recommendation model based on the recommended instance sequence and the corrected feedback information sequence includes: The recommended instance sequence is divided based on the recommended actions in the recommended instance sequence to obtain a recommended instance subsequence corresponding to each recommended action; From the corrected feedback information sequence, obtain the corrected feedback information subsequence corresponding to the recommended instance subsequence; Based on the recommended instance subsequence and the corrected feedback information subsequence, the recommendation parameters of each recommendation action in the original recommendation model are updated, thus completing the update of the original recommendation model.

9. The method according to claim 8, characterized in that, The step of updating the recommendation parameters for each recommendation action in the original recommendation model based on the recommended instance subsequence and the corrected feedback information subsequence includes: Obtain the context space combination corresponding to the corrected recommended instance subsequence; Based on the context space combination, update the first sub-recommendation parameter of each recommendation action in the original recommendation model; Obtain the feedback reward combination corresponding to the corrected feedback information subsequence; Based on the feedback reward combination and the context space combination, the second sub-recommendation parameter of each recommendation action in the original recommendation model is updated, wherein the recommendation parameter includes the first sub-recommendation parameter and the second sub-recommendation parameter.

10. The method according to claim 9, characterized in that, The feedback information also includes immediate feedback rewards; The step of obtaining the feedback reward combination corresponding to the corrected feedback information subsequence includes: For each corrected feedback message in the corrected feedback information subsequence, the immediate feedback reward and the corrected delayed feedback reward are combined into a feedback reward, thereby obtaining the feedback reward combination corresponding to the corrected feedback information subsequence.

11. The method according to any one of claims 1 to 3, characterized in that, After obtaining the recommendation model, the method further includes: In response to an information recommendation request, obtain the object to be recommended and the sequence of information to be recommended; The features of the object to be recommended and the information sequence to be recommended are extracted respectively to obtain the context space for recommendation. The context space to be recommended is input into the recommendation model to determine the estimated feedback reward sequence corresponding to the information sequence to be recommended. In the sequence of information to be recommended, the information corresponding to the maximum estimated feedback reward in the sequence of estimated feedback rewards is taken as the target recommended information and displayed.

12. An information recommendation device, characterized in that, include: The data acquisition module is used to perform information recommendation through the original recommendation model and obtain the feedback information sequence corresponding to the recommendation instance sequence, wherein each piece of feedback information in the feedback information sequence includes a delayed feedback reward. The parameter acquisition module is used to acquire the feedback correction parameter corresponding to each recommendation instance in the recommendation instance sequence, wherein the feedback correction parameter is negatively correlated with the probability of receiving delayed feedback from each recommendation instance at present and positively correlated with the probability of finally receiving delayed feedback from each recommendation instance. The feedback correction module is used to correct the delayed feedback reward corresponding to each recommendation instance based on the feedback correction parameters, thereby obtaining a corrected feedback information sequence corresponding to the recommendation instance sequence; The model update module is used to update the original recommendation model based on the recommended instance sequence and the corrected feedback information sequence to obtain a recommendation model, wherein the recommendation model is used for information recommendation.

13. An information recommendation device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, It stores executable instructions for implementing the method of any one of claims 1 to 11 when executed by a processor.

15. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the method according to any one of claims 1 to 11.

Images