Method, device, electronic device and storage medium for pushing recommended content

Abstract

The present application provides a method, device, electronic device and storage medium for pushing recommended content, the method comprising: obtaining the current label of the current content in the current display interface; analyzing the current label through a target recommendation model to obtain at least one label score corresponding to the current label, wherein the target recommendation model is obtained based on a Q matrix; selecting the recommendation score with the largest score from the at least one label score; determining the recommended label corresponding to the recommendation score of the current label according to the correspondence between the score and the label in the Q matrix, wherein the recommended label is the label of the content to be watched by the user; and selecting the recommended content corresponding to the recommended label from a database for pushing. The recommendation logic of the present application is simple and reduces the amount of calculation.

Description

Technical Field

[0001] The present application relates to the field of recommendation technology, and in particular to a method, device, electronic device and storage medium for pushing recommended content. Background Art

[0002] With the development of the Internet, platforms can use recommendation strategies to recommend content of interest to users. Recommendation strategies play an extremely important role on major platforms. For example, video platforms can recommend videos of interest to users based on their video viewing records, and shopping platforms can recommend products of interest to users based on their browsing records.

[0003] The current recommendation situation is that each platform has its own recommendation strategy. Some platforms use traditional recommendation algorithms, some platforms use deep learning recommendation algorithms; and some platforms use a mixture of multiple recommendation algorithms.

[0004] The calculation logic of the existing recommendation algorithm is too complicated, resulting in a large amount of calculation. There is currently no good solution to the complexity of the current recommendation logic. Summary of the invention

[0005] In order to solve the above technical problem or at least partially solve the above technical problem, the present application provides a method, device, electronic device and storage medium for pushing recommended content.

[0006] In a first aspect, the present application provides a method for pushing recommended content, the method comprising:

[0007] Get the current label of the current content in the current display interface;

[0008] Analyze the current tag through a target recommendation model to obtain at least one tag score corresponding to the current tag, wherein the target recommendation model is obtained based on a Q matrix;

[0009] Selecting the recommendation score with the largest score from the at least one tag score;

[0010] Determine a recommended tag corresponding to the recommendation score of the current tag according to the correspondence between the score and the tag in the Q matrix, wherein the recommended tag is a tag of the content to be watched by the user;

[0011] The recommended content corresponding to the recommended tag is selected from the database for push.

[0012] Optionally, after determining the recommended tag corresponding to the recommendation score of the current tag, the method further includes:

[0013] Using the current label, the recommended label and the recommended content as training samples, updating the reward matrix, wherein the reward value in the reward matrix is ​​determined based on the sample viewing label, the sample recommendation label and the sample recommended content corresponding to the sample recommendation label;

[0014] Generate an updated Q matrix based on the updated reward matrix;

[0015] According to the updated Q matrix, an updated target recommendation model is obtained;

[0016] Based on the updated target recommendation model, new recommendation tags and new recommended content are determined for the new current content.

[0017] Optionally, before analyzing the current tag by using the target recommendation model, the method further includes:

[0018] Establishing an initial Q matrix and an initial reward matrix, wherein the initial Q matrix and the initial reward matrix each include N×N element values ​​initialized to 0, where N is the number of tags in the tag library;

[0019] Based on the initial reward matrix, determining a reward matrix through sample viewing labels, sample recommendation labels and sample recommendation content;

[0020] For the viewing order of sample contents in the historical database, construct a Q matrix based on the initial Q matrix and the reward matrix, wherein each row in the Q matrix indicates a label, and each column in the Q matrix indicates an action of jumping from a current label to a next label;

[0021] The target recommendation model is generated by using the Q matrix.

[0022] Optionally, the determining the reward matrix based on the initial reward matrix by using the sample viewing tags, the sample recommendation tags and the sample recommended content includes:

[0023] Determine a sample viewing label and a sample recommendation label, wherein the sample recommendation label is the next label of the sample viewing label in the initial reward matrix;

[0024] Determining the viewing time of the sample recommended content corresponding to the sample recommended tag;

[0025] When the viewing time is greater than or equal to a preset time threshold, determining the reward value of the sample viewing tag as a reward value;

[0026] When the viewing time is less than a preset time threshold, determining the reward value of the sample viewing tag as a penalty value;

[0027] The reward matrix is ​​constructed based on the reward value and the penalty value.

[0028] Optionally, determining the sample viewing tag and the sample recommendation tag includes:

[0029] Acquire sample viewing content and sample recommended content in a history database, wherein the sample recommended content is the next display content of the sample viewing content;

[0030] According to the tag library, the sample viewing tags of the sample viewing content and the sample recommendation tags of the sample recommended content are determined.

[0031] Optionally, constructing a Q matrix based on the initial Q matrix and the reward matrix for the viewing order of sample contents in the historical database includes:

[0032] Each sample content in the historical database is traversed in the following manner:

[0033] Determine sample viewing content and sample recommended content according to the viewing order of sample content in the historical database, wherein the sample viewing content corresponds to a sample viewing tag, and the sample recommended content corresponds to a sample recommendation tag;

[0034] Determine the matrix value of the sample viewing label based on the reward value of the sample viewing label in the reward matrix and the matrix value of the sample recommendation label in the initial Q matrix;

[0035] The Q matrix is ​​constructed based on each matrix value.

[0036] Optionally, analyzing the current tag by using a target recommendation model to obtain at least one tag score corresponding to the current tag includes:

[0037] Determine at least one action corresponding to the current label in the Q matrix, wherein each action indicates jumping from the current label to a candidate label in the Q matrix;

[0038] Determine the matrix value of jumping from the current label to each label to be selected;

[0039] The matrix values ​​are used as label scores.

[0040] In a second aspect, a device for pushing recommended content is provided, the device comprising:

[0041] The acquisition module is used to obtain the current label of the current content in the current display interface;

[0042] An analysis module, configured to analyze the current tag through a target recommendation model to obtain at least one tag score corresponding to the current tag, wherein the target recommendation model is obtained based on a Q matrix;

[0043] A first selection module, configured to select a recommendation score with the largest score from the at least one tag score;

[0044] A determination module, configured to determine a recommended tag corresponding to the recommendation score of the current tag according to the correspondence between the score and the tag in the Q matrix, wherein the recommended tag is a tag of the content to be watched by the user;

[0045] The second selection module is used to select the recommended content corresponding to the recommendation tag from the database for push.

[0046] In a third aspect, the present application provides an electronic device, characterized in that it includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;

[0047] Memory, used to store computer programs;

[0048] The processor is used to implement the steps of the method for pushing any recommended content when executing the program stored in the memory.

[0049] In a fourth aspect, the present application provides a computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of a method for pushing any recommended content are implemented.

[0050] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art:

[0051] In the method provided by the embodiment of the present application, the server can obtain the target recommendation model according to the Q matrix, thereby obtaining at least one tag score corresponding to the current tag through the target recommendation model, and then determine the recommended tag corresponding to the recommendation score with the largest score according to the Q matrix, and finally push the recommended content corresponding to the recommended tag. The present application uses the target recommendation model generated by the Q matrix to determine the recommended content, which has a simple recommendation logic and reduces the amount of calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0054] Figure 1 A flowchart of a method for pushing recommended content provided in an embodiment of the present application;

[0055] Figure 2 A flowchart of a method for pushing recommended content provided in an embodiment of the present application;

[0056] Figure 3 A schematic diagram of the structure of a device for pushing recommended content provided in an embodiment of the present application;

[0057] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0059] An embodiment of the present application provides a method for determining recommended content, which can be applied to a server to determine recommended content that a user is interested in based on a current tag of the current content.

[0060] The following will describe in detail a method for pushing recommended content provided by an embodiment of the present application in conjunction with a specific implementation method. Figure 1 As shown, the specific steps are as follows:

[0061] Step 101: Get the current label of the current content in the current display interface.

[0062] In an embodiment of the present application, there is display content in the current display interface of the terminal, that is, the current content. The server obtains the tag corresponding to the current content and records the tag corresponding to the current content as the current tag, wherein each display content has a corresponding tag, and the same tag can correspond to at least one display content.

[0063] The displayed content can be videos, products, texts or pictures, etc. For example, the tags of videos include fitness, children's fun, travel, pets, etc.; the tags of products include toys, beauty, clothes, etc.

[0064] Step 102: Analyze the current tag through the target recommendation model to obtain at least one tag score corresponding to the current tag.

[0065] Among them, the target recommendation model is obtained based on the Q matrix.

[0066] In the embodiment of the present application, each row in the Q matrix indicates a label, and each column in the Q matrix indicates an action to jump from the current label to the next label. The server determines at least one action corresponding to the current label in the Q matrix, wherein each action indicates jumping from the current label to a label to be selected in the Q matrix, and the action to jump from the current label to the label to be selected corresponds to a matrix value. Since the current label corresponds to at least one action, the current label corresponds to at least one matrix value, and the server uses the matrix value as the label score.

[0067] If the current label corresponds to N actions in the Q matrix, each action indicates the next label (to-be-selected label) to which the current label can jump, and a label score corresponding to the current label can be obtained according to each to-be-selected label, then N label scores can be obtained for the current label.

[0068] After the server generates a target recommendation model according to the Q matrix, the target recommendation model analyzes the current tag to obtain at least one tag score corresponding to the current tag.

[0069] Step 103: Select a recommendation score with the largest score from at least one tag score.

[0070] In an embodiment of the present application, the server selects the maximum score from at least one tag score as the recommended score.

[0071] Step 104: Determine the recommended tag corresponding to the recommendation score of the current tag according to the correspondence between the scores and the tags in the Q matrix, wherein the recommended tag is the tag of the content to be viewed by the user.

[0072] In an embodiment of the present application, each action corresponding to each tag in the Q matrix corresponds to a tag score. After the server determines the recommendation score of the current tag, the tag corresponding to the column where the recommendation score is located in the Q matrix can be used as a recommended tag. The recommended tag is the tag of the next content that the user is about to watch.

[0073] Step 105: Select recommended content corresponding to the recommended tag from the database for display.

[0074] In an embodiment of the present application, a plurality of recommended contents are stored in a database, each of which has a corresponding label. The server selects corresponding recommended contents from the database for display according to the recommended label. The recommended contents may be any one of the contents under the recommended label, or the first content under the recommended label. The present application does not limit the selection method.

[0075] In this application, the server can obtain the target recommendation model according to the Q matrix, thereby obtaining at least one tag score corresponding to the current tag through the target recommendation model, and then determine the recommended tag corresponding to the recommendation score with the largest score according to the Q matrix, and finally push the recommended content corresponding to the recommended tag. This application uses the target recommendation model generated by the Q matrix to determine the recommended content, which has a simple recommendation logic and reduces the amount of calculation.

[0076] As an optional implementation, after determining the recommendation score corresponding to the recommendation label, the method also includes: using the current label, recommended label and recommended content as training samples, updating the reward matrix, wherein the reward value in the reward matrix is ​​determined based on the sample viewing label, the sample recommendation label and the sample recommendation content corresponding to the sample recommendation label; generating an updated Q matrix according to the updated reward matrix; obtaining an updated target recommendation model according to the updated Q matrix; and determining a new recommendation label for the new current content based on the updated target recommendation model.

[0077] In an embodiment of the present application, the reward value in the reward matrix is ​​determined based on the sample viewing label, the sample recommendation label, and the sample recommendation content corresponding to the sample recommendation label. The server uses the current label, the recommendation label, and the recommended content as new training samples to update the reward matrix. For the same current label, the corresponding recommendation label may include N types. If the probability of a recommended label appearing is high, it means that the user is more interested in the recommended content corresponding to the recommended label. As the number of training times increases, the number of times the reward values ​​generated based on the current label and the recommended label appear in the reward matrix is ​​large, that is, the number of times the reward values ​​are accurate will increase. Since the Q matrix is ​​generated based on the reward matrix, as the number of training times increases, the matrix value of the Q matrix will also be more accurate. The target recommendation model obtained according to the updated Q matrix will also output the recommended labels more accurately, thereby obtaining the recommended content that the user is interested in and improving the user experience.

[0078] In the embodiment of the present application, after the server obtains the updated target recommendation model, it processes the current tag of the new current content to obtain the recommendation score with the maximum score, thereby determining the recommendation tag corresponding to the recommendation score, and then selects the recommended content corresponding to the recommendation tag for push. The server continues to update the reward matrix based on the obtained current tag, recommended tag, and recommended content to obtain the updated target recommendation model again, and the above method is continuously cycled to continuously update the target recommendation model.

[0079] The server can update the reward matrix once each time it obtains a recommendation tag and a recommendation content according to the target recommendation model. In this way, the accuracy of the target recommendation model can be continuously improved. The reward matrix can also be updated after obtaining at least two recommendation tags and recommended contents according to the target recommendation model. In this way, the number of times the target recommendation model is updated can be reduced.

[0080] This application adopts a reinforcement learning algorithm. Reinforcement learning includes three elements: environment, agent, and control strategy. The purpose of reinforcement learning is to construct a control strategy to maximize the possible performance of agent behavior prediction, thereby analyzing and predicting the target behavior through learning.

[0081] In this application, the server updates the reward matrix by taking the input (current label) and output (recommended label, recommended content) of the target recommendation model as training samples, thereby obtaining an updated target recommendation model, which can make the recommended labels output by the updated target recommendation model more accurate. This application uses reinforcement learning to continuously improve the accuracy of the target recommendation model with the latest training samples, thereby improving the accuracy of the detected recommended labels.

[0082] As an optional implementation, before analyzing the current label through the target recommendation model, the method also includes: establishing an initial Q matrix and an initial reward matrix, wherein the initial Q matrix and the initial reward matrix both include N×N element values ​​initialized to 0, and N is the number of labels in the label library; based on the initial reward matrix, determining the reward matrix through sample viewing labels, sample recommendation labels and sample recommendation content; constructing a Q matrix based on the initial Q matrix and the reward matrix for the viewing order of sample content in the historical database, wherein each row in the Q matrix indicates a label, and each column in the Q matrix indicates an action of jumping from the current label to the next label; generating a target recommendation model through the Q matrix.

[0083] In the embodiment of the present application, the server first establishes an initial Q matrix and an initial R matrix, wherein the initial Q matrix and the initial R matrix both include N×N element values ​​initialized to 0. Then, the reward values ​​in the reward matrix (R matrix) are obtained through the sample viewing labels, sample recommendation labels, and sample recommendation contents in the training samples, and then the Q matrix is ​​obtained based on the initial Q matrix and the R matrix, and finally the target recommendation model is generated according to the Q matrix.

[0084] Specifically, the server first determines N based on the number of tags in the tag library, where the tags in the tag library are pre-set, and different types of viewing content have their corresponding tag libraries. The server can determine the corresponding tag library based on the type of content viewed by the user. For example, if the content viewed by the user is a short video, then the tag library is a video tag library, including tags such as fitness, children's fun, travel, and pets. If the content viewed by the user is a commodity, then the tag library is a commodity tag library, including tags such as toys, beauty, and clothing. The tag library contains as many tags as possible, so that the tags in the initial Q matrix can be as many as possible, and can cover any type of content.

[0085] Among them, the order of tags in the tag library is determined according to a preset method, which can be determined randomly or by sorting by the first letters of the pinyin of the tags. This application does not make any specific restrictions on the preset method.

[0086] The server establishes an N×N initial Q matrix Q(s, a). The labels in the initial Q matrix come from the label library and are arranged in a preset manner. Each row of the initial Q matrix represents a label Is, and each column of the initial Q matrix represents an action corresponding to each label, jumping to any of the other (N-1) labels or staying where it is. Action a represents the action of jumping from label Is to any label Is'.

[0087] In this way, we construct an N×N matrix Q and initialize the matrix values ​​in the matrix Q to 0, as follows.

[0088]

[0089] In an embodiment of the present application, the jump order is determined based on the viewing order of the sample content in the historical database. Then, during the jump process, action a represents the action of jumping from the sample viewing label Is corresponding to the sample viewing content to the sample recommendation label Is' corresponding to the sample recommended content, where the sample recommended content is the next display content of the sample viewing content.

[0090] For example, the arrangement in the initial Q matrix is ​​shown in Table 1. It can be seen that in the initial Q matrix, it is possible to jump from the sample viewing label to any label. If the order of the sample content in the historical database is xyz, then in the initial Q matrix, the matrix value of the label of x jumping to the label of y and the matrix value of the label of y jumping to the label of z can be determined (the matrix value is represented by r in Table 1).

[0091] X's Label Label of y Tags of z x's label 0 r 0 Label of y 0 0 r Tags of z 0 0 0

[0092] Table 1

[0093] The server establishes an initial R matrix, and the matrix values ​​in the initial R matrix are initialized to 0. The server obtains the reward value through the sample viewing label, sample recommendation label and sample recommendation content in the training sample, thereby obtaining the R matrix R(s, a).

[0094] In the embodiment of the present application, the labels in the initial R matrix are the same as the labels in the initial Q matrix, and the arrangement order of the labels in the initial R matrix and the R matrix is ​​the same, only the reward values ​​are not exactly the same. The reward values ​​in the initial R matrix are all 0, and the present application determines the reward values ​​in the reward matrix through sample viewing labels, sample recommendation labels, and sample recommendation content.

[0095] The server selects sample viewing content and sample recommended content from the history database, wherein the sample recommended content is the next display content of the sample viewing content in the history database, and then determines a sample viewing tag of the sample viewing content and a sample recommendation tag of the sample recommended content.

[0096] The server determines the viewing time of the sample recommended content. If the viewing time is greater than or equal to the preset time threshold, indicating that the user is interested in the sample recommended content, the server determines the reward value of the sample viewing tag as a reward value, exemplarily, R(s, a) = 1; if the viewing time is less than the preset time threshold, indicating that the user is not interested in the sample recommended content, the server determines the reward value of the sample viewing tag as a penalty value, exemplarily, R(s, a) = -1. The preset time threshold may be half of the total time of the sample recommended content, and this application does not impose any specific restrictions on the preset time threshold.

[0097] Exemplarily, the historical database includes three consecutive display contents xyz. If the viewing time of y is less than the preset time threshold, then the reward value corresponding to the action of jumping from the label of x to the label of y is -1. If the viewing time of z is greater than or equal to the preset time threshold, then the reward value corresponding to the action of jumping from the label of y to the label of z is 1, as shown in Table 2.

[0098] x's label Label of y Tags of z x's label 0 -1 0 Label of y 0 1 Tags of z 0 0 0

[0099] Table 2

[0100] The server obtains each reward value in the above manner, and then constructs an N×N reward matrix R(s, a) based on the reward value.

[0101] The server obtains the Q matrix according to the Q-learning state transfer equation, where the formula of the Q-learning state transfer equation is:

[0102]

[0103] Among them, γ is the greed factor (0<γ<1), Is represents the current state, a represents the current action, and Is' represents the next state. Indicates the next action.

[0104] The learning process of the Q matrix is ​​as follows

[0105] For the sample content in the historical database, perform steps 1 to 6.

[0106] Step 1: Set up sample viewing content;

[0107] Step 2: Determine behavior a according to the sample viewing content, where behavior a indicates jumping to the sample recommended content, and the sample recommended content is the next display content of the sample viewing content in the historical database;

[0108] Step 3: Using the specified behavior a, sample recommended content is obtained;

[0109] Step 4: Determine the sample viewing label Is corresponding to the sample viewing content and the sample recommendation label Is' corresponding to the sample recommendation content;

[0110] Step 5: Calculate according to the above Q-learning state transfer equation to obtain the reward value of the sample viewing label Is and the matrix value of the sample recommendation label Is' in the initial Q matrix, and then use the sum of the reward value and the matrix value of Is' as the new value of Q(s, a) of the sample viewing label Is, so that a Q value in the Q matrix is ​​obtained.

[0111] Step 6: Starting from the sample recommendation content, calculate according to steps 2 to 5 for each sample content in the historical database to obtain N×N Q values ​​(matrix values) in the Q matrix.

[0112] For example, the matrix Q is as follows

[0113]

[0114] Based on the same technical concept, the embodiment of the present application also provides a flowchart of a method for pushing recommended content, such as Figure 2As shown, the flow chart includes:

[0115] Step 201: Establish an initial Q matrix.

[0116] Step 202: Establish an initial R matrix (initial reward matrix).

[0117] Step 203: Obtain an R matrix (reward matrix) through the sample viewing labels, sample recommendation labels and sample recommendation contents in the training samples.

[0118] Step 204: Obtain a Q matrix according to the initial Q matrix and the R matrix, and generate a target recommendation model.

[0119] Step 205: Input the current tag into the target recommendation model and output the recommended tag.

[0120] Step 206: Determine the recommended content corresponding to the recommended tag.

[0121] Step 207: Use the current label, recommended label and recommended content as training samples and return to step 203.

[0122] There is no particular order in which step 201 and step 202 are performed.

[0123] Based on the same technical concept, the embodiment of the present application also provides a device for pushing recommended content, such as Figure 3 As shown, the device comprises:

[0124] The acquisition module 301 is used to acquire the current label of the current content in the current display interface;

[0125] An analysis module 302 is used to analyze the current tag through a target recommendation model to obtain at least one tag score corresponding to the current tag, wherein the target recommendation model is obtained based on a Q matrix;

[0126] A first selection module 303, configured to select a recommendation score with the largest score from at least one tag score;

[0127] A first determination module 304 is used to determine a recommended tag corresponding to the recommendation score of the current tag according to the correspondence between the score and the tag in the Q matrix, wherein the recommended tag is a tag of the content to be watched by the user;

[0128] The second selection module 305 is used to select recommended content corresponding to the recommended tag from the database for push.

[0129] Optionally, the device is also used for:

[0130] The current label, the recommended label and the recommended content are used as training samples to update the reward matrix, wherein the reward value in the reward matrix is ​​determined based on the sample viewing label, the sample recommendation label and the sample recommendation content corresponding to the sample recommendation label;

[0131] Generate an updated Q matrix based on the updated reward matrix;

[0132] According to the updated Q matrix, an updated target recommendation model is obtained;

[0133] Based on the updated target recommendation model, new recommendation tags and new recommended content are determined for the new current content.

[0134] Optionally, the device further comprises:

[0135] Establishing a module, used to establish an initial Q matrix and an initial reward matrix, wherein the initial Q matrix and the initial reward matrix each include N×N element values ​​initialized to 0, where N is the number of tags in the tag library;

[0136] A second determination module is used to determine the reward matrix based on the initial reward matrix through the sample viewing label, the sample recommendation label and the sample recommendation content;

[0137] A construction module, for constructing a Q matrix based on the initial Q matrix and the reward matrix for the viewing order of sample contents in the historical database, wherein each row in the Q matrix indicates a label, and each column in the Q matrix indicates an action of jumping from a current label to a next label;

[0138] The generation module is used to generate a target recommendation model through the Q matrix.

[0139] Optionally, the second determining module is used to:

[0140] Determine a sample viewing label and a sample recommendation label, wherein the sample recommendation label is the next label of the sample viewing label in the initial reward matrix;

[0141] Determine the viewing time of the sample recommended content corresponding to the sample recommended label;

[0142] When the viewing time is greater than or equal to a preset time threshold, determining the reward value of the sample viewing label as a reward value;

[0143] When the viewing time is less than a preset time threshold, the reward value of the sample viewing label is determined to be a penalty value;

[0144] Construct a reward matrix based on reward values ​​and penalty values.

[0145] Optionally, the second determining module is further used for:

[0146] Obtaining sample viewing content and sample recommended content in a history database, wherein the sample recommended content is the next display content of the sample viewing content;

[0147] According to the tag library, sample viewing tags of sample viewing content and sample recommendation tags of sample recommended content are determined.

[0148] Optionally, building blocks are used to:

[0149] Use the following method to traverse each sample content in the historical database:

[0150] Determine sample viewing content and sample recommended content according to the viewing order of sample content in the historical database, wherein the sample viewing content corresponds to the sample viewing label, and the sample recommended content corresponds to the sample recommendation label;

[0151] Determine the matrix value of the sample viewing label based on the reward value of the sample viewing label in the reward matrix and the matrix value of the sample recommendation label in the initial Q matrix;

[0152] Construct a Q matrix based on each matrix value.

[0153] Optionally, the analysis module 302 is used to:

[0154] Determine at least one action corresponding to the current label in the Q matrix, wherein each action indicates a jump from the current label to a candidate label in the Q matrix;

[0155] Determine the matrix value to jump from the current label to each candidate label;

[0156] Use the matrix values ​​as label scores.

[0157] Based on the same technical concept, an embodiment of the present invention further provides an electronic device, as shown in FIG. *, including a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404.

[0158] Memory 403, used for storing computer programs;

[0159] The processor 401 is used to implement the above steps when executing the program stored in the memory 403.

[0160] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0161] The communication interface is used for communication between the above electronic device and other devices.

[0162] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0163] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0164] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0165] In another embodiment of the present invention, a computer program product including instructions is provided, which enables the computer to execute any one of the methods in the above embodiments when the computer program product is executed on the computer.

[0166] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidStateDisk (SSD)), etc.

[0167] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0168] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for pushing recommended content, characterized in that: The method comprises: Get the current label of the current content in the current display interface; Analyze the current tag through a target recommendation model to obtain at least one tag score corresponding to the current tag, wherein the target recommendation model is obtained based on a Q matrix; Selecting the recommendation score with the largest score from the at least one tag score; Determine a recommended tag corresponding to the recommendation score of the current tag according to the correspondence between the score and the tag in the Q matrix, wherein the recommended tag is a tag of the content to be watched by the user; Selecting recommended content corresponding to the recommended tag from a database for push; Before analyzing the current tag by the target recommendation model, the method further includes: Establish an initial Q matrix and an initial reward matrix, wherein the initial Q matrix and the initial reward matrix both include N N element values ​​initialized to 0, where N is the number of tags in the tag library; Based on the initial reward matrix, determining a reward matrix through sample viewing labels, sample recommendation labels and sample recommendation content; For the viewing order of sample contents in the historical database, construct a Q matrix based on the initial Q matrix and the reward matrix, wherein each row in the Q matrix indicates a label, and each column in the Q matrix indicates an action of jumping from a current label to a next label; Generate the target recommendation model through the Q matrix; Wherein, determining the reward matrix based on the initial reward matrix through the sample viewing tags, the sample recommendation tags and the sample recommended content includes: Determine a sample viewing label and a sample recommendation label, wherein the sample recommendation label is the next label of the sample viewing label in the initial reward matrix; Determining the viewing time of the sample recommended content corresponding to the sample recommended tag; When the viewing time is greater than or equal to a preset time threshold, determining the reward value of the sample viewing tag as a reward value; When the viewing time is less than a preset time threshold, determining the reward value of the sample viewing tag as a penalty value; The reward matrix is ​​constructed based on the reward value and the penalty value.

2. The method according to claim 1, characterized in that After determining the recommended tag corresponding to the recommendation score of the current tag, the method further includes: Using the current label, the recommended label and the recommended content as training samples, updating the reward matrix, wherein the reward value in the reward matrix is ​​determined based on the sample viewing label, the sample recommendation label and the sample recommended content corresponding to the sample recommendation label; Generate an updated Q matrix based on the updated reward matrix; According to the updated Q matrix, an updated target recommendation model is obtained; Based on the updated target recommendation model, new recommendation tags and new recommended content are determined for the new current content.

3. The method according to claim 1, characterized in that Determining the sample viewing label and the sample recommendation label includes: Acquire sample viewing content and sample recommended content in a history database, wherein the sample recommended content is the next display content of the sample viewing content; According to the tag library, the sample viewing tags of the sample viewing content and the sample recommendation tags of the sample recommended content are determined.

4. The method according to claim 3, characterized in that The constructing of a Q matrix based on the initial Q matrix and the reward matrix for the viewing order of sample contents in the historical database comprises: Each sample content in the historical database is traversed in the following manner: Determine sample viewing content and sample recommended content according to the viewing order of sample content in the historical database, wherein the sample viewing content corresponds to a sample viewing tag, and the sample recommended content corresponds to a sample recommendation tag; Determine the matrix value of the sample viewing label based on the reward value of the sample viewing label in the reward matrix and the matrix value of the sample recommendation label in the initial Q matrix; The Q matrix is ​​constructed based on each matrix value.

5. The method according to claim 1, characterized in that The analyzing the current tag by the target recommendation model to obtain at least one tag score corresponding to the current tag includes: Determine at least one action corresponding to the current label in the Q matrix, wherein each action indicates jumping from the current label to a candidate label in the Q matrix; Determine the matrix value of jumping from the current label to each label to be selected; The matrix values ​​are used as label scores.

6. A device for pushing recommended content, characterized in that: The device comprises: The acquisition module is used to obtain the current label of the current content in the current display interface; An analysis module, configured to analyze the current tag through a target recommendation model to obtain at least one tag score corresponding to the current tag, wherein the target recommendation model is obtained based on a Q matrix; A first selection module, configured to select a recommendation score with the largest score from the at least one tag score; A determination module, configured to determine a recommended tag corresponding to the recommendation score of the current tag according to the correspondence between the score and the tag in the Q matrix, wherein the recommended tag is a tag of the content to be watched by the user; A second selection module is used to select recommended content corresponding to the recommendation tag from the database for push; Wherein, the device is also used for: Establish an initial Q matrix and an initial reward matrix, wherein the initial Q matrix and the initial reward matrix both include N N element values ​​initialized to 0, where N is the number of tags in the tag library; Based on the initial reward matrix, determining a reward matrix through sample viewing labels, sample recommendation labels and sample recommendation content; For the viewing order of sample contents in the historical database, construct a Q matrix based on the initial Q matrix and the reward matrix, wherein each row in the Q matrix indicates a label, and each column in the Q matrix indicates an action of jumping from a current label to a next label; Generate the target recommendation model through the Q matrix; Wherein, the device is also used for: Determine a sample viewing label and a sample recommendation label, wherein the sample recommendation label is the next label of the sample viewing label in the initial reward matrix; Determining the viewing time of the sample recommended content corresponding to the sample recommended tag; When the viewing time is greater than or equal to a preset time threshold, determining the reward value of the sample viewing tag as a reward value; When the viewing time is less than a preset time threshold, determining the reward value of the sample viewing tag as a penalty value; The reward matrix is ​​constructed based on the reward value and the penalty value.

7. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor is used to implement any method step of claims 1-5 when executing a program stored in a memory.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps of any one of claims 1 to 5 are implemented.

Images