Push discrimination model training method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention relates to a training method and device for a push discrimination model, electronic equipment and a storage medium. The method comprises the following steps: acquiring user behavior information recorded by a behavior of a target user on target display information; based on a preset reward function, performing reward value calculation on the user behavior information to obtain a reward value representing an interaction degree between the target user and the target display information; based on the user behavior information, historical state information of the target user is updated, and the historical state information is information recorded when the target user interacts with display information of a preset type; and training based on the updated historical state information and the reward value to obtain a push discrimination model. According to the embodiment of the invention, the method achieves the continuous updating of the historical state information, achieves the training of the push discrimination model, achieves the continuous optimization of the prediction precision of the push discrimination model, and facilitates the improvement of the precision of pushing information to a user.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a training method, device, electronic device and computer-readable storage medium for a push discriminant model. Background Art

[0002] Currently, when pushing various information to users, hard rules are generally used to control the frequency of users viewing information (such as advertisements, videos, etc.). In most scenarios, this can improve the accuracy of push, but it cannot fully meet the personalized needs of users.

[0003] Different users may have different sensitivities to different information at different time periods. For example, if a user has recently clicked and converted multiple times on a certain type of information, it means that the user is interested in this type of information and has actual needs. The publisher of the information should relax the control over the frequency of this type of information to allow users to read more of this type of information. Therefore, in order to fully explore the push demand patterns behind a large number of users, a method is needed to more accurately determine what information to push to users. Summary of the invention

[0004] In view of this, in order to solve some or all of the above-mentioned technical problems, the embodiments of the present application provide a training method, device, electronic device and computer-readable storage medium for a push discrimination model.

[0005] In a first aspect, an embodiment of the present application provides a method for training a push discrimination model, the method comprising: obtaining user behavior information recorded on the target user's behavior on target display information; calculating a reward value for the user behavior information based on a preset reward function to obtain a reward value representing the degree of interaction between the target user and the target display information; updating the target user's historical status information based on the user behavior information, wherein the historical status information is information recorded on the interaction between the target user and display information of a preset type; and training a push discrimination model based on the updated historical status information and the reward value.

[0006] In one possible implementation, a reward value is calculated for user behavior information based on a preset reward function to obtain a reward value representing the degree of interaction between the target user and the target display information, including: determining the type of operation performed by the target user on the target display information; and determining the reward value corresponding to the operation type based on the reward function.

[0007] In one possible implementation, the operation types include click operations and conversion operations; based on the reward function, determining the reward value corresponding to the operation type includes: based on the reward function, when the operation type is a click operation, determining the corresponding reward value to be a first value, and when the operation type is a conversion operation, determining the corresponding reward value to be a second value.

[0008] In one possible implementation, a push discrimination model is trained based on updated historical status information and reward values, including: storing the updated historical status information, user behavior information and reward values ​​as training samples into a preset cache; in response to the number of training samples in the preset cache reaching a preset number, extracting a training sample set from the preset cache; and using the training sample set to train an initial push discrimination model to obtain a push discrimination model.

[0009] In one possible implementation, after training a push discrimination model based on the updated historical status information and reward value, the method further includes: determining the display information to be pushed from a preset display information set, where the type of each display information in the display information set is a preset type; using the push discrimination model, analyzing the historical status information to determine whether the display information to be pushed meets the push conditions; if so, pushing the display information to be pushed to the target user.

[0010] In one possible implementation, a push discrimination model is used to analyze historical status information to determine whether the display information to be pushed meets the push conditions, including: inputting the historical status information into the push discrimination model to obtain the predicted push frequency of the display information to be pushed; if the predicted push frequency does not exceed the preset push frequency threshold, it is determined that the display information to be pushed meets the push conditions.

[0011] In a possible implementation, the historical status information includes at least one of the following: user attribute information of the target user, basic information of the display information browsed by the target user, behavior information of the target user on the display information, and display time of the display information.

[0012] In a second aspect, an embodiment of the present application provides a training device for a push discrimination model, the device comprising: a first acquisition module, used to obtain user behavior information recorded by a target user on target display information; a calculation module, used to calculate a reward value for the user behavior information based on a preset reward function, to obtain a reward value representing the degree of interaction between the target user and the target display information; an update module, used to update the historical status information of the target user based on the user behavior information, wherein the historical status information is information recorded on the interaction between the target user and display information of a preset type; a training module, used to train a push discrimination model based on the updated historical status information and the reward value.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory for storing a computer program; a processor for executing the computer program stored in the memory, and when the computer program is executed, the method of any embodiment of the method for training a push discrimination model of the above-mentioned first aspect of the present application is implemented.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method of any embodiment of the method for training a push discrimination model as described in the first aspect above is implemented.

[0015] In a fifth aspect, an embodiment of the present application provides a computer program, which includes a computer-readable code. When the computer-readable code runs on a device, the processor in the device implements a method as in any embodiment of the method for training a push discrimination model of the first aspect described above.

[0016] The training method, device, electronic device and computer-readable storage medium of the push discrimination model provided in the embodiments of the present application obtain user behavior information recorded by the target user's behavior on the target display information, and then calculate the reward value for the user behavior information based on a preset reward function to obtain a reward value representing the degree of interaction between the target user and the target display information, and then update the historical status information of the target user based on the user behavior information, and train the push discrimination model based on the updated historical status information and the reward value. The embodiments of the present application realize the calculation of the reward value representing the degree of interaction between the target user and the target display information, and use a reinforcement learning algorithm to continuously update the historical status information and train the push discrimination model, thereby realizing the continuous optimization of the prediction accuracy of the push discrimination model, thereby helping to improve the accuracy of information pushed to users. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0020] Figure 1 A flowchart of a method for training a push discrimination model provided in an embodiment of the present application;

[0021] Figure 2 A flowchart of another method for training a push discrimination model provided in an embodiment of the present application;

[0022] Figure 3 A flowchart of another method for training a push discrimination model provided in an embodiment of the present application;

[0023] Figure 4 A flowchart of another method for training a push discrimination model provided in an embodiment of the present application;

[0024] Figure 5 A flowchart of another method for training a push discrimination model provided in an embodiment of the present application;

[0025] Figure 6 A schematic diagram of the structure of a training device for a push discrimination model provided in an embodiment of the present application;

[0026] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. It should be noted that unless otherwise specifically stated, the relative arrangement of the components and steps, the numerical expressions and the numerical values ​​described in these embodiments do not limit the scope of the present application.

[0028] Those skilled in the art will understand that the terms "first" and "second" in the embodiments of the present application are only used to distinguish between different steps, devices, modules and other objects, and do not represent any specific technical meanings, nor do they indicate the logical order between them.

[0029] It should also be understood that in this embodiment, “plurality” may refer to two or more than two, and “at least one” may refer to one, two or more than two.

[0030] It should also be understood that any component, data or structure mentioned in the embodiments of the present application can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0031] In addition, the term "and / or" in this application is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this application generally indicates that the associated objects before and after are in an "or" relationship.

[0032] It should also be understood that the description of the various embodiments in this application focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0033] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or uses.

[0034] Technologies, circuits, and devices known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the above-mentioned technologies, circuits, and devices should be considered as part of the specification.

[0035] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0036] It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. To facilitate the understanding of the embodiments of the present application, the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0037] In order to solve the technical problem that the prior art is not sufficiently targeted in pushing information to users, the present application provides a method for training a push discrimination model, which can push information to users more accurately and meet the personalized needs of users.

[0038] Figure 1 A flow chart of a method for training a push discrimination model provided for an embodiment of the present application. This method can be applied to a server, where the server uses the user's historical status information and user behavior information to train a push discrimination model for discriminating push conditions for display information. In addition, the execution subject of this method can be hardware or software. When the above-mentioned execution subject is hardware, the execution subject can be one or more of the above-mentioned electronic devices. For example, a single electronic device can execute this method, or multiple electronic devices can cooperate with each other to execute this method. When the above-mentioned execution subject is software, this method can be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made here.

[0039] like Figure 1 As shown, the method specifically includes:

[0040] Step 101: Obtain user behavior information recorded based on the target user's behavior on the target display information.

[0041] In some embodiments, the target display information is a specific type of information pushed to the user, such as advertisements, videos, product descriptions, etc. The user's behavior refers to various types of operations performed on the target display information by the user after viewing the target display information. For example, the user behavior information may indicate various operations performed by the user on the target display information, such as clicking, converting (e.g., converting to purchase), etc.

[0042] Step 102: Based on a preset reward function, a reward value is calculated for the user behavior information to obtain a reward value representing the degree of interaction between the target user and the target display information.

[0043] In some embodiments, the reward value represents the degree of interaction between the target user and the target display information. The higher the degree of interaction, the higher the reward value, the higher the matching degree between the target display information and the target user, and the higher the target user's demand for the target display information. The reward function can be set arbitrarily according to actual needs.

[0044] Step 103: Update the historical status information of the target user based on the user behavior information.

[0045] In some embodiments, the historical status information is information recorded about the target user's interaction with the display information of a preset type. The display information of the preset type may be display information of various set types, for example, it may be information of types such as advertisements, videos, and texts; the preset type may also be a subtype under a certain main type, for example, if the main type is advertisement, the preset type may be an e-commerce advertisement subtype under the advertisement type.

[0046] The above historical status information is various information recorded when the target user interacts with the display information of the preset type. Generally, the historical status information can be obtained from the historical log.

[0047] Optionally, the historical status information includes but is not limited to at least one of the following: user attribute information of the target user, basic information of the display information browsed by the target user, behavior information of the target user on the display information, and display time of the display information. The user attribute information may include the user's preference for information such as the classification and display form of the historical display information. The basic information of the display information browsed by the target user may include information such as the type, name, and style of the display information. The behavior information of the target user on the display information may include information indicating that the user has performed operations such as clicking and purchasing on the display information. The display time of the display information may include the specific time displayed to the user, the time period to which it belongs, the time when the user clicks, etc.

[0048] By obtaining various forms of historical status information, the model can fully explore the target users' preferences for displayed information, which is conducive to pushing information to users more accurately.

[0049] The electronic device records the target user's operation on the target display information this time in the historical status information, thereby completing the update of the historical status information.

[0050] Step 104: Based on the updated historical state information and reward value, a push discrimination model is trained.

[0051] In some embodiments, the trained push discrimination model can be used to predict the push frequency of a certain display information. The above push frequency is the number of times the display information is pushed to the target user within a certain period of time. Since the display signal is displayed on the terminal device of the target user after being pushed to the target user, the push frequency can also be called the exposure frequency.

[0052] The push discrimination model can be a deep reinforcement learning (DRL) model. The model structure can be various types of deep reinforcement learning model structures, such as a DQN (Deep Q-Network) model. The DQN model can execute the Q-learning algorithm, which is a classic reinforcement learning algorithm used to solve the control problem in the Markov decision process. It is based on the idea of ​​value iteration and guides the agent to select the best action in each state by estimating the value function Q value of each state-action pair.

[0053] In the Q-learning algorithm, the value judgment function Q(s,a) is used to quantify the value of the current behavior. Its specific update expression is shown in the following formula (1):

[0054]

[0055] Among them, Q(s,a) is the expected total reward obtained after taking action a in the current state s until the final state. In this embodiment, state s is the above-mentioned historical state information, and action a is the predicted information output by the model, that is, the predicted push frequency. R(s,a) is the reward value (Reward) obtained after taking action a in the current state s, that is, the reward value obtained in step 1043. It represents the subsequent state after taking action a in the current state s. Indicates that Therefore, It represents the value of the optimal action in the subsequent state. γ (0≤γ<1) represents the discount coefficient, that is, the benefit reduction coefficient considering the future. When γ is small, the model will mainly use R (s, a), that is, the current value, when updating Q (s, a); otherwise, it will use the long-term value. α is the learning rate.

[0056] Based on the above formula (1), the updated historical state information reward value can be used to maximize the Q value as the training goal. In the process of users continuously clicking on the displayed information, converting and other operations, the parameters of the model are continuously adjusted, so that the prediction accuracy of the model is continuously improved.

[0057] The training method of the push discrimination model provided in the embodiment of the present application obtains the user behavior information recorded by the target user's behavior on the target display information, and then calculates the reward value for the user behavior information based on a preset reward function to obtain a reward value representing the degree of interaction between the target user and the target display information, and then updates the historical status information of the target user based on the user behavior information, and trains the push discrimination model based on the updated historical status information and the reward value. The embodiment of the present application realizes the calculation of the reward value representing the degree of interaction between the target user and the target display information, uses a reinforcement learning algorithm to continuously update the historical status information and train the push discrimination model, and realizes the continuous optimization of the prediction accuracy of the push discrimination model, thereby helping to improve the accuracy of information pushed to users.

[0058] In some optional implementations, such as Figure 2 As shown, step 102 includes:

[0059] Step 1021, determining the operation type of the target user on the target display information.

[0060] The operation type can be set according to the actual application scenario. For example, when the target display information is an advertisement, the operation type can include click, conversion to purchase, etc.

[0061] Step 1022: Determine the reward value corresponding to the operation type based on the reward function.

[0062] Specifically, different operation types may correspond to different reward value calculation methods. For example, for a click operation, a smaller reward value may be calculated based on a reward function, and for a conversion to a purchase operation, a larger reward value may be calculated based on a reward function.

[0063] This embodiment generates different reward values ​​based on the different types of operations performed by the target user on the target display information, thereby making more targeted use of user feedback information to optimize the model when training the model, thereby helping to improve the prediction accuracy of the trained model.

[0064] In some optional implementations, the operation types include click operations and conversion operations. A click operation is an operation in which a target user clicks on display information, and a conversion operation is an operation in which a target user successfully performs a purchase, redemption, or other type of operation through the target display information.

[0065] Step 1022 may be performed as follows:

[0066] Based on the reward function, when the operation type is a click operation, the corresponding reward value is determined to be a first value, and when the operation type is a conversion operation, the corresponding reward value is determined to be a second value.

[0067] As an example, the reward function of this embodiment is shown in the following formula (2):

[0068]

[0069] That is, when the target user clicks on the target display information, the reward value is 1 (first value), and when the target user performs a conversion operation through the target display information, the reward value is 10 (second value).

[0070] This embodiment achieves simple and efficient model optimization of the feedback made by users using target display information by setting reward values ​​corresponding to click operations and conversion operations respectively, thereby improving the efficiency of model training.

[0071] In some optional implementations, such as Figure 3 As shown, step 104 includes:

[0072] Step 1041, storing the updated historical status information, user behavior information and reward value as training samples in a preset cache.

[0073] Specifically, according to the training method of deep reinforcement learning, the historical state information is used as the state s, and the actual action a can be obtained according to the user behavior information, and is stored in a preset cache together with the reward value R, waiting for the training opportunity to arrive.

[0074] Step 1042: In response to the number of training samples in the preset cache reaching a preset number, extract a training sample set from the preset cache.

[0075] That is, when a sufficient number of training samples are stored in the preset cache, the training process can be started. At this time, a batch of training samples can be extracted (for example, randomly extracted) from the preset cache for model training.

[0076] Step 1043: Use the training sample set to train the push discrimination model to obtain the push discrimination model.

[0077] Specifically, according to the above formula (1), the parameters of the initial push discrimination model can be updated with the maximum Q value as the training goal. The initial push discrimination model is the push discrimination model after the last training. When the training end condition is met, the training is stopped, and the trained push discrimination model can continue to analyze the historical status information of each user.

[0078] This embodiment stores training samples by setting up a cache. In the process of continuously pushing display information to users, if a certain number of training samples are collected, the model parameters are updated, thereby realizing online learning of the model, so that it can continuously adapt to environmental changes in actual scenarios and improve the prediction accuracy of the model.

[0079] In some optional implementations, such as Figure 4 As shown, after the above step 104, the method further includes:

[0080] Step 105: Determine target display information from a preset display information set.

[0081] In some embodiments, the type of each display information in the display information set may be consistent with the above-mentioned preset type. The electronic device may determine the display information to be pushed from the display information set in various ways, such as random selection, sequential selection, selection based on the degree of matching between the display information and the above-mentioned historical status information, etc.

[0082] Step 106: Analyze the historical status information using the push discrimination model to determine whether the information to be pushed meets the push conditions.

[0083] In some embodiments, the push discrimination model is a model pre-trained using the above embodiments. The model can process the input historical status information and output a discrimination result, which indicates whether the display information to be pushed meets the push conditions. Optionally, the push discrimination model can predict the push frequency of the display information to be pushed. If the output push frequency is small (i.e., less than the push frequency threshold), it means that the user can click, convert, and perform other operations with a small number of views, and it is judged that the push conditions are met. Optionally, the push discrimination model can also analyze the match between the historical status information of the target user and the display information to be pushed. If the match is high (i.e., exceeds the match threshold), it is judged that the push conditions are met.

[0084] Step 107: If the conditions are met, the above-mentioned information to be pushed and displayed is pushed to the target user.

[0085] In some embodiments, the display information to be pushed can be sent to the terminal device used by the target user in real time, and the terminal device can display the received display information in real time, or record the corresponding relationship between the target user and the received display information. After the terminal device used by the target user opens the page corresponding to the display information to be pushed, the display information to be pushed is sent to the terminal device used by the target user. The target user can use the terminal device to click on the received display information, convert (i.e. successfully purchase the product), and other operations.

[0086] The training method of the push discrimination model provided in the embodiment of the present application uses the push discrimination model to accurately analyze the historical status information of the target user and accurately determine whether the user has a demand for specific display information, so that information can be pushed to the user in a more targeted manner to meet the user's actual needs.

[0087] In some optional implementations, such as Figure 5 As shown, step 106 includes:

[0088] Step 1061, input the historical status information into a pre-trained push discrimination model to obtain the predicted push frequency of the information to be pushed.

[0089] Among them, the push discrimination model can be pre-trained to predict the predicted push frequency of the display information to be pushed to the target user. The above push frequency is the number of times the display information to be pushed is pushed to the target user within a certain period of time. Since the display signal to be pushed is displayed on the terminal device of the target user after being pushed to the target user, the push frequency can also be called the exposure frequency.

[0090] Step 1062: If the predicted push frequency does not exceed the preset push frequency threshold, it is determined that the display information to be pushed meets the push condition.

[0091] Specifically, the predicted push frequency output by the push discrimination model refers to the frequency corresponding to the target user performing a specific operation (such as a click operation, conversion to a purchase operation, etc.) on the target display information after viewing the target display information several times. The higher the push frequency, the more it is predicted that the user will perform a specific operation after viewing the display information to be pushed multiple times; the lower the push frequency, the more it means that the user will perform a specific operation after viewing the target display information fewer times.

[0092] Therefore, in this embodiment, when the predicted push frequency does not exceed the preset push frequency threshold, the probability that the user performs a specific operation on the target display information is relatively high, and it is determined that the push condition is met at this time.

[0093] This embodiment predicts the push frequency of target display information by using a push discrimination model, and determines whether the push conditions are met based on the push frequency. This enables the push discrimination model to fully explore the push frequency control rules of a large number of users, thereby improving the pertinence of information pushed to users.

[0094] Figure 6A schematic diagram of the structure of a training device for a push discrimination model provided in an embodiment of the present application. Specifically comprising: an acquisition module 601, used to obtain user behavior information recorded by the target user's behavior on the target display information; a calculation module 602, used to calculate the reward value of the user behavior information based on a preset reward function, and obtain a reward value representing the degree of interaction between the target user and the target display information; an update module 603, used to update the historical status information of the target user based on the user behavior information, wherein the historical status information is the information recorded for the interaction between the target user and the display information of the preset type; a training module 604, used to train the push discrimination model based on the updated historical status information and the reward value.

[0095] In some optional implementations, the calculation module includes: a first determination unit, used to determine the operation type of the target user on the target display information; and a second determination unit, used to determine the reward value corresponding to the operation type based on the reward function.

[0096] In some optional implementations, the operation types include click operations and conversion operations; the second determination unit is further used to: based on the reward function, when the operation type is a click operation, determine the corresponding reward value to be a first value; when the operation type is a conversion operation, determine the corresponding reward value to be a second value.

[0097] In some optional implementations, the training module includes: a storage unit, used to store the updated historical status information, user behavior information and reward value as training samples into a preset cache; an extraction unit, used to extract a training sample set from the preset cache in response to the number of training samples in the preset cache reaching a preset number; a training unit, used to train the initial push discrimination model using the training sample set to obtain a push discrimination model.

[0098] In some optional implementations, the device also includes: a first determination module, used to determine the display information to be pushed from a preset display information set, wherein the type of each display information in the display information set is a preset type; a second determination module, used to use a push discrimination model to analyze the historical status information to determine whether the display information to be pushed meets the push conditions; and a push module, used to push the display information to be pushed to the target user if it meets the conditions.

[0099] In some optional implementations, the second determination module includes: a prediction unit, used to input historical status information into a push discrimination model to obtain a predicted push frequency of the display information to be pushed; and a third determination unit, used to determine that the display information to be pushed meets the push conditions if the predicted push frequency does not exceed a preset push frequency threshold.

[0100] In some optional implementations, the historical status information includes at least one of the following: user attribute information of the target user, basic information of the display information browsed by the target user, behavior information of the target user on the display information, and display time of the display information.

[0101] The training device for the push discrimination model provided in this embodiment can be as follows: Figure 6 The training device of the push discrimination model shown in can execute all the steps of the above training methods of the push discrimination models, and thus achieve the technical effects of the above training methods of the push discrimination models. For details, please refer to the above related description. For the sake of brevity, it will not be repeated here.

[0102] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, Figure 7 The electronic device 700 shown includes: at least one processor 701, a memory 702, at least one network interface 704 and other user interfaces 703. The various components in the electronic device 700 are coupled together via a bus system 705. It is understood that the bus system 705 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 705 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 705 is not described in detail. Figure 7 Various buses are labeled as bus system 705 .

[0103] The user interface 703 may include a display, a keyboard, or a pointing device (eg, a mouse, a trackball, a touch pad, or a touch screen).

[0104] It can be understood that the memory 702 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct RAM bus random access memory (DRRAM). The memory 702 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0105] In some implementations, the memory 702 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system 7021 and an application 7022 .

[0106] The operating system 7021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 7022 includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present application can be included in the application 7022.

[0107] In this embodiment, by calling the program or instruction stored in the memory 702, specifically, the program or instruction stored in the application 7022, the processor 701 is used to execute the method steps provided by each method embodiment, for example, including:

[0108] Obtain user behavior information recorded on the target user's behavior on the target display information; calculate the reward value for the user behavior information based on a preset reward function to obtain a reward value representing the degree of interaction between the target user and the target display information; update the target user's historical status information based on the user behavior information, wherein the historical status information is information recorded on the target user's interaction with the display information of a preset type; and train a push discrimination model based on the updated historical status information and the reward value.

[0109] The method disclosed in the above embodiment of the present application can be applied to the processor 701, or implemented by the processor 701. The processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 701 or the instruction in the form of software. The above processor 701 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software units in the decoding processor can be executed. The software unit can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 702, and the processor 701 reads the information in the memory 702 and completes the steps of the above method in combination with its hardware.

[0110] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPDevice, DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the above functions of the present application, or a combination thereof.

[0111] For software implementation, the technology described above in this article can be implemented by a unit that performs the functions described above in this article. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0112] The electronic device provided in this embodiment may be Figure 7 The electronic device shown in can execute all steps of the training methods of the push discrimination models described above, thereby achieving the technical effects of the training methods of the push discrimination models described above. Please refer to the above related description for details. For the sake of brevity, it will not be repeated here.

[0113] The embodiment of the present application also provides a storage medium (computer-readable storage medium). The storage medium here stores one or more programs. The storage medium may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory may also include a combination of the above-mentioned types of memory.

[0114] When one or more programs in the storage medium can be executed by one or more processors, the training method of the push discrimination model executed on the electronic device side can be implemented.

[0115] The processor is used to execute the program stored in the memory to implement the following steps of the training method of the push discrimination model executed on the electronic device side:

[0116] Obtain user behavior information recorded on the target user's behavior on the target display information; calculate the reward value for the user behavior information based on a preset reward function to obtain a reward value representing the degree of interaction between the target user and the target display information; update the target user's historical status information based on the user behavior information, wherein the historical status information is information recorded on the target user's interaction with the display information of a preset type; and train a push discrimination model based on the updated historical status information and the reward value.

[0117] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different circuits to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0118] The steps of the circuits or algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, software modules executed by a processor, or a combination of the two. The software modules may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0119] It should be understood that the terms used in the text are only for the purpose of describing specific example embodiments, and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used in the text may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The steps, processes, and operations described in the text are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0120] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for training a push discrimination model is characterized by: The method comprises: Obtain user behavior information recorded based on the target user's actions on the target displayed information; Based on a preset reward function, a reward value is calculated for the user behavior information to obtain a reward value representing the degree of interaction between the target user and the target display information; Based on the user behavior information, updating the historical status information of the target user, wherein the historical status information is information recorded about the interaction between the target user and the display information of a preset type; Based on the updated historical state information and the reward value, a push discrimination model is trained.

2. The method according to claim 1, characterized in that The step of calculating a reward value for the user behavior information based on a preset reward function to obtain a reward value representing the degree of interaction between the target user and the target display information includes: Determining an operation type of the target user on the target display information; Based on the reward function, a reward value corresponding to the operation type is determined.

3. The method according to claim 2, characterized in that The operation types include click operations and conversion operations; The determining, based on the reward function, a reward value corresponding to the operation type includes: Based on the reward function, when the operation type is a click operation, the corresponding reward value is determined to be a first value, and when the operation type is a conversion operation, the corresponding reward value is determined to be a second value.

4. The method according to claim 1, characterized in that The training of a push discrimination model based on the updated historical state information and the reward value includes: Storing the updated historical status information, the user behavior information and the reward value as training samples in a preset cache; In response to the number of training samples in the preset cache reaching a preset number, extracting a training sample set from the preset cache; The initial push discrimination model is trained using the training sample set to obtain the push discrimination model.

5. The method according to claim 1, characterized in that After training the push discrimination model based on the updated historical state information and the reward value, the method further includes: Determine the display information to be pushed from a preset display information set, where the type of each display information in the display information set is the preset type; Analyze the historical status information using the push discrimination model to determine whether the to-be-pushed display information meets the push condition; If it is consistent, the to-be-pushed display information is pushed to the target user.

6. The method according to claim 5, characterized in that The using the push discrimination model to analyze the historical status information to determine whether the to-be-pushed display information meets the push condition includes: Inputting the historical status information into the push discrimination model to obtain the predicted push frequency of the display information to be pushed; If the predicted push frequency does not exceed the preset push frequency threshold, it is determined that the display information to be pushed meets the push condition.

7. The method according to any one of claims 1 to 6, characterized in that: The historical status information includes at least one of the following: user attribute information of the target user, basic information of the display information browsed by the target user, behavior information of the target user on the display information, and display time of the display information.

8. A training device for a push discrimination model, characterized in that: The device comprises: The first acquisition module is used to acquire user behavior information recorded by the target user's behavior on the target display information; A calculation module, configured to calculate a reward value for the user behavior information based on a preset reward function, and obtain a reward value representing the degree of interaction between the target user and the target display information; An updating module, configured to update the historical status information of the target user based on the user behavior information, wherein the historical status information is information recorded about the interaction between the target user and the display information of a preset type; A training module is used to train a push discrimination model based on the updated historical state information and the reward value.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor is used to execute the computer program stored in the memory, and when the computer program is executed, the training method of the push discrimination model described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the training method of the push discrimination model described in any one of claims 1 to 7 is implemented.

Images