A method and system for predicting ad click rate based on user behavior

Abstract

The application discloses a kind of based on user behavior's advertisement click rate prediction method and system, it is related to click prediction field, including, collection user behavior data and space-time trajectory data, constructs user mobile graph structure and then extracts space embedding vector by graph neural network, simultaneously extracts time embedding vector by user behavior data, according to space embedding vector and time embedding vector, generate space-time behavior vector, input space-time behavior vector into generative adversarial network, separate out short-term interest vector and long-term interest vector by generator, and construct bayesian network based on historical behavior data, obtain interest state transition probability matrix, based on short-term interest vector, long-term interest vector and interest transition probability matrix, user behavior link probability is predicted using bayesian-markov model, generates preliminary click rate prediction probability;The application predicts user behavior link probability by bayesian-markov model and variational autoencoder, realizes dynamic probability modeling.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of click prediction, in particular to an advertisement click rate prediction method and system based on user behavior. BACKGROUND

[0002] In recent years, with the rapid development of the digital advertising industry, advertisement click rate prediction technology has played an important role in improving advertising efficiency, optimizing user experience and maximizing business value. Existing click prediction technology is mainly based on machine learning and deep learning methods, by analyzing user behavior data, context information and advertisement characteristics, constructing a prediction model to estimate the probability of user clicking on an advertisement. Traditional methods include logistic regression, gradient boosting decision tree and deep neural network, which use user historical click records, browsing time and search keywords and other features, combined with context data for prediction.

[0003] There is still room for improvement in existing click prediction technology, for example, existing click prediction technology usually regards user behavior as a unified interest representation, ignoring the difference between short-term impulsive interest and long-term stable interest, and failing to effectively capture the dynamic probability of behavior link or spatio-temporal correlation. Generative adversarial networks, Bayesian networks, hidden Markov models, graph neural networks and temporal convolution networks have applications in behavior modeling, but have not been integrated for click prediction. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides an advertisement click rate prediction method based on user behavior to solve the problem of failing to effectively capture the spatio-temporal correlation of behavior link and the model not being integrated.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides an advertisement click rate prediction method based on user behavior, comprising,

[0008] Collecting user behavior data and spatio-temporal trajectory data, constructing a user movement graph structure, extracting a spatial embedding vector through a graph neural network, and simultaneously extracting a temporal embedding vector through user behavior data, generating a spatio-temporal behavior vector according to the spatial embedding vector and the temporal embedding vector;

[0009] Inputting the spatio-temporal behavior vector into a generative adversarial network, separating a short-term interest vector and a long-term interest vector through a generator, and constructing a Bayesian network based on historical behavior data to obtain an interest state transition probability matrix;

[0010] Based on the short-term interest vector, the long-term interest vector and the interest transition probability matrix, a user behavior link probability is predicted by using a Bayesian-Markov model to generate a preliminary click rate prediction probability;

[0011] The preliminary click rate prediction probability is adjusted by using reinforcement learning to generate a final advertisement click rate prediction probability.

[0012] As a preferred scheme of the advertisement click rate prediction method based on user behavior, the generation of the space-time behavior vector comprises the following steps,

[0013] User behavior data, space-time trajectory data, context data and advertisement metadata are collected and cleaned, and the GPS trajectory in the cleaned space-time trajectory data is mapped to a space-time grid, a unique grid ID is assigned, and a timestamp is encoded into a periodic time feature vector;

[0014] The keyword vector and the jump vector are extracted from the cleaned user behavior data, and the grid ID, the periodic time feature vector, the keyword vector and the jump vector are spliced into a space-time grid vector;

[0015] Based on the grid ID in the space-time grid vector, a graph structure is constructed, the grid ID is used as a node, the user grid movement is used as an edge, and the edge weight is the movement frequency; based on the graph structure, a GraphSAGE model is used to output a spatial embedding vector, and a time convolution network is used to generate a time embedding vector through user behavior data;

[0016] The spatial embedding vector and the time embedding vector are spliced into a fusion vector through a full connection layer, and a quantized space-time behavior vector is generated by clustering.

[0017] As a preferred scheme of the advertisement click rate prediction method based on user behavior, the space-time behavior vector is input into a generative adversarial network, the short-term interest vector and the long-term interest vector are separated out by a generator, and a Bayesian network is constructed based on historical behavior data to obtain an interest state transition probability matrix, and the method comprises the following steps,

[0018] The historical user behavior data is extracted, the short-term real interest vector is generated by Word2Vec based on J-day click sequences, the long-term real interest vector is generated based on L-month click preferences, and the real interest vector is spliced;

[0019] The real interest vector is used to train the generative adversarial network, the generative adversarial network is composed of a generator and a discriminator, the generator adopts a multilayer perceptron to generate the short-term interest vector and the long-term interest vector, the discriminator adopts a convolutional neural network to output a binary classification probability, and the training adopts an adversarial loss, and the trained generative adversarial network generator is obtained when the discriminator loss no longer decreases;

[0020] Infer the quantumized space-time behavior vector using the trained generative adversarial network generator, and generate the short-term interest vector and the long-term interest vector;

[0021] Define M interest states based on the historical click sequence, construct a Bayesian network using the Pyro library, calculate the conditional transition probability, and obtain the interest transition probability matrix based on the statistical likelihood probability and the initial transition probability.

[0022] As a preferred scheme of the advertisement click rate prediction method based on user behavior, wherein: based on the short-term interest vector, the long-term interest vector and the interest transition probability matrix, the user behavior link probability is predicted by using the Bayesian-Markov model, and the method comprises the following steps,

[0023] Extract the user behavior sequence from the distributed file unit;

[0024] Abstract the user behavior sequence into a hidden Markov chain, define the observation state, and the N interest levels are hidden states; the interest transition probability matrix is reduced to a state transition matrix by average pooling; and the initial state distribution is generated based on the user portrait data;

[0025] Calculate the conditional transition probability of the hidden state by using the Bayesian network, and generate a state transition probability vector;

[0026] Based on the user behavior sequence, the current hidden state is calculated by the hidden Markov chain, the current hidden state and the state transition probability vector are input into the variational autoencoder, the hidden state parameters are inferred and generated, and the user behavior link probability is calculated by sampling.

[0027] As a preferred scheme of the advertisement click rate prediction method based on user behavior, wherein: the generation of the preliminary click rate prediction probability refers to inputting the short-term interest vector, the long-term interest vector and the user behavior link probability into the Transformer model, and generating the preliminary click rate prediction probability by using the attention mechanism and the Sigmoid function.

[0028] As a preferred scheme of the advertisement click rate prediction method based on user behavior, wherein: the preliminary click rate prediction probability is adjusted by using reinforcement learning to generate the final advertisement click rate prediction probability, and the method comprises the following steps,

[0029] Splice the short-term interest vector, the long-term interest vector and the preliminary click rate prediction probability to generate a user feature vector, define a reinforcement learning framework, the reinforcement learning state is the user feature vector, the action is a prediction adjustment factor, and the reward is prediction accuracy;

[0030] The Deep-Q-Network adopts a multilayer perceptron, initializes a main network and a target network, verifies the Deep-Q-Network by using a mean square error as a loss function, completes training after the loss function reaches a specified requirement, and obtains the trained Deep-Q-Network;

[0031] The user feature vector is input into the trained Deep-Q-Network, an optimal prediction adjustment factor is selected, adjustment is performed based on the preliminary click rate prediction probability, and a final advertisement click rate prediction probability is generated through Sigmoid function normalization.

[0032] As a preferred scheme of the advertisement click rate prediction method based on user behavior, the current hidden state is statistically generated by using a hidden Markov chain according to the user behavior sequence, in combination with a state transition matrix and an initial state distribution.

[0033] In a second aspect, the application provides an advertisement click rate prediction system based on user behavior, comprising,

[0034] The fusion module collects user behavior data and space-time trajectory data, extracts a space embedding vector through a graph neural network after constructing a user movement graph structure, simultaneously extracts a time embedding vector through the user behavior data, generates a space-time behavior vector according to the space embedding vector and the time embedding vector, and inputs the space-time behavior vector into the generation adversarial network.

[0035] The construction module separates a short-term interest vector and a long-term interest vector through a generator by inputting the space-time behavior vector into the generation adversarial network, constructs a Bayesian network based on historical behavior data, and obtains an interest state transition probability matrix.

[0036] The preliminary prediction module predicts a user behavior link probability by using a Bayesian-Markov model based on the short-term interest vector, the long-term interest vector and the interest transition probability matrix, and generates a preliminary click rate prediction probability.

[0037] The optimization module adjusts the preliminary click rate prediction probability by using reinforcement learning, and generates a final advertisement click rate prediction probability.

[0038] In a third aspect, the application provides a computer device comprising a memory and a processor, and the memory stores a computer program.

[0039] In a fourth aspect, the present application provides a computer readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements any step of the user behavior based advertisement click rate prediction method according to the first aspect of the present application.

[0040] The present application has the following advantages: by collecting user behavior data and space-time trajectory data, space embedding vectors and time embedding vectors are generated, and quantumized space-time behavior vectors are generated by splicing and clustering, thereby realizing space-time characteristic modeling of user behavior, capturing spatial correlation and temporal dynamics, separating short-term interest vectors and long-term interest vectors using a generative adversarial network, generating an interest transfer probability matrix using a Bayesian network, realizing the differentiation and dynamic transfer modeling of short-term impulsive interest and long-term stable interest, and solving the limitations of unified interest representation; in addition, by using a Bayesian-Markov model and a variational autoencoder, the hidden state is counted based on the user behavior sequence, and the user behavior link probability is predicted, thereby realizing dynamic probability modeling, making up for the lack of behavior link dynamics, improving the accuracy and personalization level of click rate prediction, and enhancing the adaptability to complex behavior patterns. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0042] Fig. 1 The flowchart of the user behavior based advertisement click rate prediction method.

[0043] Fig. 2 The flowchart of interest modeling and transfer prediction.

[0044] Fig. 3 The flowchart of behavior link prediction.

[0045] Fig. 4 The flowchart of reinforcement learning optimization. DETAILED DESCRIPTION

[0046] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification.

[0047] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the concept of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0048] Second, the term "one embodiment" or "an embodiment" as may be used herein means a specific implementation, or example, that can include features that are, but are not required to be, included in at least one implementation of the present disclosure. This term does not mean that all embodiments will include the discussed feature, advantage or property. Use of the term "in one embodiment" or "in an embodiment" in different places in the specification are not necessarily referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.

[0049] Referring to Figs. 1-4 For one embodiment of the present disclosure, the embodiment provides a user behavior based advertisement click rate prediction method, comprising the following steps:

[0050] S1: Collect user behavior data and space-time trajectory data, construct a user movement graph structure, extract a space embedding vector through a graph neural network, extract a time embedding vector through user behavior data, generate a space-time behavior vector according to the space embedding vector and the time embedding vector;

[0051] Specifically, the following steps are included,

[0052] Collect user behavior data from the real-time log unit of the advertisement platform, including click records, browsing time, search keywords and page jump sequences; collect space-time trajectory data, including GPS trajectory and time stamp; collect context data, including device type and page type; collect advertisement metadata, including advertisement ID, category and content.

[0053] Clean the original user behavior data, space-time trajectory data, context data and advertisement metadata, remove abnormal records, including records with a click interval less than 0.1 seconds in the click records, and coordinate points with longitude and latitude exceeding the reasonable range in the GPS trajectory; complete the missing values, for example, the unrecorded device type is filled by the nearest neighbor interpolation method based on the historical data of the same user, the missing page type is filled with a default value; the time stamp is standardized to UTC format, the numerical features such as browsing time are processed by linear normalization to the interval of 0 to 1, and the cleaned data is stored in a distributed file unit in JSON format.

[0054] The GPS trajectory in the cleaned spatio-temporal trajectory data is mapped to the spatio-temporal grid, and a unique grid ID is assigned to each spatio-temporal grid. The grid ID is represented in the form of a string. The cleaned timestamp is encoded as a periodic feature. Specifically, the hour part is converted into a 10-dimensional periodic time feature vector using the sine and cosine functions, which correspond to the periodic representation of hours and cover a 24-hour period. The search keywords in the cleaned user behavior data are extracted by the TF-IDF method to generate a 10-dimensional keyword vector. The page jump sequence is encoded by a sequence to generate a 5-dimensional jump vector. The click record and the browsing time are directly used as scalar features. The grid ID, the periodic time feature vector, the keyword vector, and the jump vector are concatenated to generate a 10-dimensional spatio-temporal grid vector, which is stored in a temporary database.

[0055] Based on the grid ID in the spatio-temporal grid vector, a graph structure is constructed. The grid ID is used as a node, and the movement of the user between grids is used as an edge. The edge weight is the frequency of the user moving from one grid to another within 24 hours, which is calculated by counting the number of transitions of the user's GPS trajectory. The GraphSAGE model is used to process the graph structure. The GraphSAGE model is configured to have 2 layers, and the aggregation function is mean pooling. The output dimension of each layer is 25, and finally a 50-dimensional spatial embedding vector is generated to capture the spatial correlation between grids. Based on the page jump sequence, the click record, and the browsing time in the cleaned user behavior data, a time convolution network is used to process the behavior sequence. The time convolution network is configured to have 3 layers of convolution, and the convolution kernel size is 3. A 30-dimensional time embedding vector is output to capture the periodicity of the behavior sequence. The spatial embedding vector and the time embedding vector are stored in a temporary database.

[0056] It is particularly pointed out that the GraphSAGE model needs to be trained with the time convolution network, and the training process of the GraphSAGE model is as follows: (1) using historical user behavior data and historical spatio-temporal trajectory data covering records in the past 7 days, generate graph structure data and node features for GraphSAGE model training. The historical user behavior data includes click records, browsing time, search keywords, page jump sequence, and the historical spatio-temporal trajectory data includes GPS trajectory and timestamp. Based on the historical GPS trajectory data, the frequency of user movement from one grid to another within 24 hours is counted to generate an edge weight matrix; the graph structure takes grid ID as the node, the user movement between grids as the edge, and the edge weight as the movement frequency; the node feature is a 10-dimensional spatio-temporal grid vector generated by cleaning historical data. The training data includes graph structure and node features, which are divided into training set and validation set. (2) The GraphSAGE model is configured to have two layers, the first layer inputs a 10-dimensional spatio-temporal grid vector and outputs a 25-dimensional vector, and the second layer inputs a 25-dimensional vector and outputs a 25-dimensional vector, finally splicing to generate a 50-dimensional spatial embedding vector. Each layer uses mean pooling to aggregate neighbor node features, and the activation function is ReLU. The loss function uses mean square error, and the goal is to minimize the error between the predicted spatial embedding vector and the true spatial relationship, and the true spatial relationship is represented by the weighted average of the movement frequency. (3) Initialize the GraphSAGE model parameters, randomly set the weights, and the bias is zero. The batch size is 128, the learning rate is 0.001, and the Adam optimizer is used. The training process iterates 1000 times, and in each round, a batch of graph structure data and node features are sampled from the training set; for each node, the GraphSAGE model aggregates the features of one-hop and two-hop neighbors of each node, calculates the mean pooling result, and generates an intermediate representation through ReLU activation. The first layer outputs a 25-dimensional vector, and the second layer further aggregates to output a 25-dimensional vector, and splicing generates a 50-dimensional spatial embedding vector. Calculate the mean square error between the predicted spatial embedding vector and the true spatial relationship, update the GraphSAGE model parameters, and evaluate the validation set every 10 rounds to monitor the mean square error loss. If the validation set loss does not decrease for 50 consecutive rounds, stop training early, and get the trained GraphSAGE model. (4) Use the trained GraphSAGE model to infer real-time graph structure data: input is a 10-dimensional spatio-temporal grid vector and graph structure of the current grid ID, for each grid ID, the GraphSAGE model aggregates neighbor node features to generate a 50-dimensional spatial embedding vector.

[0057] The training process of the time convolution network is as follows: (1) Extract historical user behavior data covering the records of the past 7 days, extract page jump sequence, click record and browsing time, construct behavior sequence data for time convolution network model training. The training data is behavior sequence data and corresponding click label (0 for not clicking, 1 for clicking), which is divided into training set, validation set and test set. (2) The time convolution network model is configured as 3 layers of convolution, each layer of convolution kernel size is 3, expansion factor is 1, 2 and 4 respectively, and output channel number is 10, 10 and 10 respectively. Finally, a 30-dimensional time embedding vector is output through a fully connected layer. Each layer uses a ReLU activation function and adds a residual connection to avoid gradient disappearance. The loss function uses mean square error, and the goal is to minimize the error between the predicted time embedding vector and the real behavior sequence periodic feature. The real periodic feature is represented by the frequency components of the sequence extracted by Fourier transform. (3) Initialize the parameters of the time convolution network model, randomly set the weights, and the bias is zero. Each round samples behavior sequence data from the training set. For each behavior sequence data, the time convolution network model is processed by 3 layers of convolution, and the expansion factors 1, 2 and 4 are applied in turn to capture short period (e.g. every hour) and long period (e.g. every day) rules. The first layer of convolution outputs 10-dimensional features, and the second and third layers further extract high-order features, outputting 10-dimensional features, and the fully connected layer fuses into a 30-dimensional time embedding vector. Calculate the mean square error between the predicted time embedding vector and the real periodic feature, and update the time convolution network parameters. The validation set is evaluated every 5 rounds, and the mean square error loss is monitored. If the validation set loss does not decrease for 20 consecutive rounds, the training is stopped in advance, and the trained time convolution network is obtained. (4) Use the trained time convolution network model to infer real-time behavior sequence data and generate a 30-dimensional time embedding vector.

[0058] The spatial embedding vector and the time embedding vector are spliced into an 80-dimensional combined vector through a fully connected layer, the number of neurons in the fully connected layer is 128, the activation function is ReLU, and an 80-dimensional fusion vector is output, which fuses the spatial and temporal feature representations. The fully connected layer is trained jointly with the GraphSAGE model and the time convolution network. Apply K-Means clustering to the fusion vector, set the number of cluster centers K to 10, generate 10 discretized cluster labels, and each label corresponds to a quantized state. Map the fusion vector to the nearest cluster center to generate an 80-dimensional quantized spatio-temporal behavior vector.

[0059] Further explanation, through the spatio-temporal grid mapping and graph structure modeling, the sensitivity of the feature representation to the user's geographic location and behavior time is enhanced, thereby improving the ability to describe the user's behavior patterns, providing more accurate input for subsequent interest decoupling and click-through rate prediction.

[0060] S2: inputting the spatio-temporal behavior vector into a generative adversarial network, separating a short-term interest vector and a long-term interest vector through a generator, and constructing a Bayesian network based on historical behavior data to obtain an interest state transition probability matrix;

[0061] Specifically, the method comprises the following steps,

[0062] The historical user behavior data is extracted, covering a click sequence in the past 7 days and a click preference in the past 3 months. Based on the click sequence in the past 7 days, word embedding is trained by a Word2Vec method to generate a 50-dimensional short-term real interest vector, capturing short-term interest features. Based on the click preference in the past 3 months, word embedding is trained by a Word2Vec method to generate a 50-dimensional long-term real interest vector, capturing long-term interest features. The 50-dimensional short-term real interest vector and the 50-dimensional long-term real interest vector are spliced to generate a 100-dimensional real interest vector.

[0063] S2.1: training a generative adversarial network generator and a discriminator: using the historical quantized spatio-temporal behavior vector as the input of the generative adversarial network generator, the generator adopts a multi-layer perception, configured as 3 layers, with neuron numbers of 256, 128 and 64 respectively, combined with a self-attention mechanism, outputting a 40-dimensional short-term interest vector and a 40-dimensional long-term interest vector, spliced into an 80-dimensional generated interest vector. The discriminator adopts a convolutional neural network, configured as 2 layers of convolution, with a convolution kernel size of 3, a stride of 1, and output channel numbers of 32 and 64, connected to 2 layers of fully connected layers with neuron numbers of 128 and 1 respectively, inputting a 100-dimensional real historical interest vector and outputting a binary classification probability (real or generated). The generative adversarial network training adopts an adversarial loss, aiming to generate realistic interest vectors by the generator and to distinguish real and generated vectors by the discriminator. The training is carried out on a GPU cluster, with a batch size of 128 and a learning rate of 0.0002, using an Adam optimizer. The training process iterates 1000 rounds, in each round a batch of historical quantized spatio-temporal behavior vectors and 100-dimensional real historical interest vectors are sampled from a temporary database, the generator generates an 80-dimensional generated interest vector, the discriminator calculates a binary classification probability, and the parameters of the generator and the discriminator are updated alternately. 10% of the historical data is used as a validation set, and the discriminator loss is monitored by evaluating every 10 rounds. If the discriminator loss is stable for 50 consecutive rounds, the training is stopped, and the trained generative adversarial network generator is obtained.

[0064] The trained generative adversarial network generator is used to generate real-time quantized spatio-temporal behavior vectors, with an input of 80-dimensional quantized spatio-temporal behavior vectors. The generator is processed by a multi-layer perceptron and a self-attention mechanism, and outputs a 40-dimensional short-term interest vector and a 40-dimensional long-term interest vector, which respectively capture the user's recent promotional interest and long-term brand preference. Specifically, (1) the first layer of the multi-layer perceptron receives the 80-dimensional quantized spatio-temporal behavior vector, maps it to a 256-dimensional intermediate representation through linear transformation, applies a ReLU activation function to introduce nonlinearity, and generates a 256-dimensional feature vector. The second layer of the multi-layer perceptron receives the 256-dimensional feature vector, maps it to a 128-dimensional intermediate representation through linear transformation, and applies a ReLU activation function again to generate a 128-dimensional feature vector. The third layer of the multi-layer perceptron receives the 128-dimensional feature vector, maps it to a 64-dimensional intermediate representation through linear transformation, and applies a ReLU activation function to generate a 64-dimensional feature vector. The multi-layer perceptron extracts high-order features in the 80-dimensional quantized spatio-temporal behavior vector through layer-by-layer dimension reduction and nonlinear transformation, enhancing the representation ability of user interest.

[0065] ;

[0066] wherein, represents the attention score of the i-th attention head, represents the dimension of the key vector of each attention head represents the dimension of the key vector of each attention head represents the query matrix of the i-th attention head, represents the transpose of the key matrix of the i-th attention head.

[0067] ​​​The attention score is normalized by applying a Softmax function to generate attention weights, and the value matrix is weighted and summed using the attention weights to generate an output vector for each attention head. The outputs of the 4 attention heads are spliced into a 64-dimensional feature vector, which is fused through linear transformation to generate a 64-dimensional self-attention feature vector. The self-attention mechanism captures the dependency between different dimensions in the 80-dimensional quantized space-time behavior vector, highlighting key features related to short-term interest (e.g., promotion clicks) and long-term interest (e.g., brand preference).

[0068] S2.2: Extract the click sequence from the historical user behavior data, define 10 interest states (e.g., promotion interest, brand interest, entertainment interest, etc.) as nodes of the Bayesian network, and the edges represent the transition probabilities between interest states. Based on user behavior, such as clicking on the category of the ad, implement the Bayesian network using the Pyro library to calculate the conditional probability, expressed as:

[0069] ;

[0070] wherein, represents the conditional probability, indicating the probability of transitioning to the current interest state from the previous interest state and user behavior , represents the likelihood probability, indicating the probability of the user performing user behavior given the current interest state and the previous interest state , represents the transition probability, indicating the probability of transitioning from the previous interest state to the current interest state , represents the marginal probability, which serves as a normalization factor.

[0071] The frequency of user behavior under each pair of interest states is calculated by statistical analysis of historical click sequences to estimate the likelihood probability, for example, if the frequency of a user clicking on a promotion ad in the promotion interest state is 0.8, then the likelihood probability is 0.8. The initial transition probability is calculated as the number of transitions from the previous state to the current state divided by the total number of times in the previous state, formula:

[0072] ;

[0073] wherein, denotes the initial transition probability from an interest state to an interest state , denotes that the current interest state is the th state, denotes that the last interest state is the th state, denotes the number of times from an interest state to an interest state in the historical click sequence, denotes the total number of times of the interest state appearing in the historical click sequence.

[0074] Based on the historical click sequence and the 40-dimensional short-term interest vector, the 40-dimensional long-term interest vector, the conditional probability is optimized by using the variational inference of the Pyro library, the likelihood estimation of the historical data is maximized, the Bayesian network training parameters are batch size 64, learning rate 0.0001, and iteration 500 rounds. After optimization, the transition probability is extracted, and a 10*10-dimensional interest transition probability matrix is generated.

[0075] The 40-dimensional short-term interest vector, the 40-dimensional long-term interest vector and the 10*10-dimensional interest transition probability matrix are obtained from the temporary database to verify their effectiveness. The cosine similarity of the 40-dimensional short-term interest vector and the 7-day click sequence is compared using the click sequence in the historical user behavior data, and the target similarity is greater than 0.8. The cosine similarity of the 40-dimensional long-term interest vector and the 3-month click preference is compared, and the target similarity is greater than 0.7. The 10*10-dimensional interest transition probability matrix is verified, and the next interest state is predicted by the Bayesian network. Compared with the interest state of the actual click sequence, the target accuracy is greater than 85%. If the verification fails, adjust the parameters of the generative adversarial network generator or the Bayesian network and regenerate. After the verification is passed, the 40-dimensional short-term interest vector, the 40-dimensional long-term interest vector and the 10*10-dimensional interest transition probability matrix are stored in the Redis cache.

[0076] Further, the generative adversarial network significantly improves the expression ability of interest features through layer-by-layer dimension reduction of the multi-layer perception and dimension-dependent capture of the self-attention mechanism, and provides a more fine-grained user motivation representation for subsequent behavior prediction.

[0077] S3: Based on the short-term interest vector, the long-term interest vector and the interest transition probability matrix, the user behavior link probability is predicted by using the Bayesian-Markov model to generate a preliminary click rate prediction probability;

[0078] Specifically, the method comprises the following steps,

[0079] The 40-dimensional short-term interest vector, the 40-dimensional long-term interest vector and the 10*10-dimensional interest transition probability matrix are obtained from the Redis cache, the user behavior sequence, the user portrait data and the context data are extracted from the distributed file unit, and the user portrait data includes user age and gender data.

[0080] The user behavior sequence is abstracted into a hidden Markov chain, the state node is defined as exposure, click and conversion, the hidden state is five interest levels, and the five interest levels are high interest, medium-high interest, medium interest, medium-low interest and low interest. The state transition matrix of the hidden Markov chain is initialized by the 10*10-dimensional interest transition probability matrix, the 10 interest states are mapped to 5 hidden states by average pooling dimension reduction, for example, the high-frequency transition probability of promotion interest and entertainment interest is combined into a high interest state. The initial state distribution is estimated based on the user portrait data, and a 5-dimensional probability vector is generated by counting the historical behavior frequency corresponding to the age and gender.

[0081] The conditional transition probability of the hidden state of the hidden Markov chain is calculated by combining the context data and the user portrait data using the Bayesian network: the context data is encoded into a 10-dimensional context feature vector through the timestamp and the device type, the user portrait data is encoded into a 5-dimensional user portrait embedding vector through the age segmentation and the gender, and the conditional probability formula is:

[0082] ;

[0083] wherein, represents the probability that the previous hidden state , the context feature vector and the user portrait embedding vector transits to the current hidden state , represents the current hidden state, represents the previous hidden state, represents the context feature vector, represents the user portrait embedding vector, represents the likelihood probability, represents the hidden state transition probability, which is directly obtained from the state transition matrix of the hidden Markov chain, represents the normalization factor.

[0084] The 5-dimensional hidden state and the 25-dimensional state transition probability vector are obtained, and the state transition probability vector is obtained by flattening the state transition matrix of the hidden Markov chain. The hidden state is obtained based on the user behavior sequence by the hidden Markov chain. The historical user behavior sequence is extracted and divided into a training set and a validation set. The variational autoencoder is implemented by using PyTorch, and is configured as a two-layer multilayer perceptron. The first layer receives the 5-dimensional hidden state and the 25-dimensional state transition probability vector as input, and is mapped to a 64-dimensional intermediate representation through linear transformation. The ReLU activation function is applied to generate a 64-dimensional feature vector. The second layer receives the 64-dimensional feature vector, and is mapped to a 32-dimensional output through linear transformation, representing the parameters of the hidden state posterior distribution. The weights of the variational autoencoder are initialized as random values, and the bias is zero. The training process is as follows: the variational inference is implemented by using the Pyro library, the evidence lower bound (ELBO) loss function is optimized, the training data is the training set, the batch size is 64, the learning rate is 0.0001, the optimizer is Adam, and the iteration is 500 rounds. The variational autoencoder outputs the 32-dimensional hidden state parameters every batch, the Pyro library calculates the ELBO loss, and the parameters of the variational autoencoder are updated through gradient descent. After every 10 rounds of training, the validation set is used to evaluate the variational autoencoder, and the ELBO loss of the last 50 rounds is recorded. If the loss change is less than 0.001 (regarded as stable), the training is stopped, and the iteration is terminated in advance. The trained variational autoencoder is obtained.

[0085] The 5-dimensional hidden state and the 25-dimensional state transition probability vector are input into the trained variational autoencoder to obtain the 32-dimensional hidden state parameters. The hidden state parameters represent the parameters of the posterior distribution of the current hidden state of the hidden Markov chain. The Pyro library samples the hidden state from the posterior distribution, and defines the classification distribution using the 32-dimensional hidden state parameters. For example, the non-normalized scores of the 5 kinds of hidden states are normalized by the Softmax function to obtain the probabilities. The user behavior link probability is calculated. The user behavior link probability is approximated by the conditional probability formula:

[0086] ;

[0087]

[0088] ​​​​​​​​​​​​S3.1: Concatenate the 40-dimensional short-term interest vector, the 40-dimensional long-term interest vector, the 10-dimensional context feature vector, the 1-dimensional user behavior link probability, and the 20-dimensional ad feature vector to generate a 111-dimensional combined vector. The 20-dimensional ad feature vector is encoded by the ad category and keywords, and the 1-dimensional user behavior link probability is used as a prior probability to enhance the behavior feature representation. The 111-dimensional combined vector is input into the Transformer model, which is configured with 2 layers, 4 attention heads, and a hidden layer dimension of 128. The Transformer model focuses on the behavior features related to the ad through the attention mechanism. Specifically, the 111-dimensional combined vector generates query vectors, key vectors, and value vectors through linear transformation. The attention score is calculated by the dot product of the query vector and the key vector, which is normalized as a weight representing the importance of each feature to other features. For example, a high correlation between promotion interest and discount ad generates a high weight, and the value vector is weighted and summed according to the attention weight to generate a 32-dimensional output. The 4 heads are concatenated into a 128-dimensional intermediate representation. The attention mechanism emphasizes the features related to the ad by high weights, such as assigning high weights to promotion interest and discount ad if the user's short-term interest vector represents promotion interest and the ad feature vector represents discount ad, thereby enhancing the association between promotion interest and discount ad. The context feature vector further strengthens this association, such as higher click-through rate for night promotion ads, which is captured by the attention mechanism through weights. The first layer of the Transformer processes the 111-dimensional input to generate a 128-dimensional intermediate representation, and the second layer further extracts high-order features to output a 128-dimensional feature vector. The 128-dimensional feature vector is mapped to a scalar value through a fully connected layer (64 neurons, ReLU activation), normalized by the Sigmoid function, and generates a preliminary click-through rate prediction probability (0 to 1).

[0089] Further explanation, the training data of the Transformer model is the historical user behavior sequence and the click label (0 for not clicked, 1 for clicked), with a batch size of 64, a learning rate of 0.0001, 1000 iterations, using the Adam optimizer, and the loss function is binary cross-entropy. The validation set is evaluated every 10 rounds, and the loss is monitored. If the loss is stable for 50 consecutive rounds, the training of the Transformer model is completed, and the preliminary click-through rate prediction probability is used as the prediction result for subsequent verification.

[0090] Preferably, the Bayesian-Markov model significantly improves the prediction accuracy of user click behavior by dynamically transferring states and fusing context features, providing a fine-grained behavior link representation for ad click-through rate prediction.

[0091] S4: Adjust the preliminary click-through rate prediction probability using reinforcement learning to generate the final ad click-through rate prediction probability.

[0092] Specifically, the following steps are included,

[0093] The 40-dimensional short-term interest vector, the 40-dimensional long-term interest vector, the 10-dimensional context feature vector and the user behavior link probability are spliced into a user feature vector as a reinforcement learning state.

[0094] A reinforcement learning framework is defined: the state is the user feature vector, the action is defined as a predicted adjustment factor, which is discretized into 5 values, for adjusting the preliminary click rate prediction probability; the reward is the prediction accuracy. The historical user behavior sequence is extracted from the distributed file unit as training data, which is divided into a training set and a validation set.

[0095] Deep-Q-Network is implemented by PyTorch, and the Q function is approximated by a 3-layer multi-layer perceptron with neuron numbers of 128, 64 and 32, respectively. The input is the user feature vector and the action, and the output is a scalar Q value representing the expected reward of the state-action pair. Two sets of Deep-Q-Network parameters are initialized: the main network is used for action selection, and the target network is used for stable training. The initial weights are random, and the bias is zero. An experience replay buffer is set to store the state-action-reward-next state four-tuple, and the initialization of Deep-Q-Network is completed.

[0096] The Deep Q-Network is trained to optimize the Q function, and the process is as follows: 32 four-tuples are randomly sampled from the experience replay buffer, the main network selects the action according to the current state and calculates the Q value; the target network calculates the optimal Q value of the next state, the loss function is the mean square error, and the main network parameters are updated by gradient descent. The main network parameters are copied to the target network every 100 steps. Train for 10,000 steps, and evaluate the mean square error of Q value prediction every 10 steps using the validation set. If the error changes less than 0.001 for 100 consecutive steps, the trained Deep-Q-Network is obtained.

[0097] The trained Deep-Q-Network is used to infer real-time data, and the input is the user feature vector. The Deep-Q-Network selects the optimal action, calculates the adjusted prediction probability, and normalizes it through the Sigmoid function:

[0098] ;

[0099] wherein, represents the final advertisement click rate prediction probability, ranging from 0 to 1, represents the predicted adjustment factor, represents the preliminary click rate prediction probability.

[0100] The embodiment also provides an advertisement click rate prediction system based on user behavior, comprising:

[0101] The fusion module collects user behavior data and spatio-temporal trajectory data, constructs a user movement graph structure, extracts a spatial embedding vector through a graph neural network, extracts a temporal embedding vector through the user behavior data, generates a spatio-temporal behavior vector according to the spatial embedding vector and the temporal embedding vector, and generates a spatio-temporal behavior vector;

[0102] The construction module inputs the spatio-temporal behavior vector into a generative adversarial network, separates a short-term interest vector and a long-term interest vector through a generator, constructs a Bayesian network based on historical behavior data, and obtains an interest state transition probability matrix.

[0103] The preliminary prediction module predicts a user behavior link probability by using a Bayesian-Markov model based on the short-term interest vector, the long-term interest vector, and the interest transition probability matrix, and generates a preliminary click rate prediction probability.

[0104] The optimization module adjusts the preliminary click rate prediction probability by using reinforcement learning, and generates a final advertisement click rate prediction probability.

[0105] The embodiment also provides a computer device suitable for the case of the advertisement click rate prediction method based on user behavior, which comprises a memory and a processor.

[0106] The computer device can be a terminal, and the computer device comprises a processor, a memory, a communication interface, a display screen, and an input device which are connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved through WIFI, an operator network, NFC (near field communication), or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, a trackball, or a touchpad arranged on the shell of the computer device. In addition, the input device can be an external keyboard, a touchpad, a mouse, or the like.

[0107] The embodiment also provides a storage medium on which a computer program is stored, the program being executed by a processor to implement the method for predicting an advertisement click rate based on user behavior proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk, or an optical disk.

[0108] To sum up, the application achieves the spatio-temporal characteristics modeling of user behavior by collecting user behavior data and spatio-temporal trajectory data, generating a space embedding vector, a time embedding vector, splicing and clustering to generate a quantized spatio-temporal behavior vector, captures the spatial correlation and temporal dynamics, uses a generative adversarial network to separate a short-term interest vector and a long-term interest vector, combines a Bayesian network to generate an interest transfer probability matrix, realizes the differentiation and dynamic transfer modeling of short-term impulsive interest and long-term stable interest, and solves the limitation of unified interest representation; in addition, the dynamic probability modeling is realized by using a Bayesian-Markov model and a variational autoencoder to statistically determine a hidden state based on a user behavior sequence and predict a user behavior link probability, the deficiency of behavior link dynamics is made up, the precision and personalization level of the click rate prediction are improved, and the adaptability to complex behavior patterns is enhanced.

[0109] It should be noted that the above embodiments are only used to illustrate the technical solutions of the application rather than limit the application, although the application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the application, and all of them should be covered in the scope of the claims of the application.

Claims

1. A method for predicting click-through rate of an advertisement based on user behavior, the method comprising: Comprising, Collect user behavior data and spatio-temporal trajectory data, construct user movement graph structure, extract spatial embedding vector through graph neural network, extract temporal embedding vector through user behavior data, generate spatio-temporal behavior vector according to spatial embedding vector and temporal embedding vector, specifically comprising the following steps, Collect user behavior data, spatio-temporal trajectory data, context data and advertisement metadata, and perform data cleaning, map the GPS trajectory in the cleaned spatio-temporal trajectory data to the spatio-temporal grid, assign a unique grid ID, and encode the timestamp into a periodic time feature vector; Extract keyword vector and jump vector from cleaned user behavior data, and concatenate grid ID, periodic time feature vector, keyword vector and jump vector into spatio-temporal grid vector; Based on the grid ID in the spatio-temporal grid vector, a graph structure is constructed, the grid ID is the node, the user grid movement is the edge, and the edge weight is the movement frequency. Based on the graph structure, the GraphSAGE model is used to output the spatial embedding vector, and the time convolution network is used to generate the temporal embedding vector through the user behavior data; Concatenate the spatial embedding vector and the temporal embedding vector into a fusion vector through a fully connected layer, and cluster to generate a quantized spatio-temporal behavior vector; Input the spatio-temporal behavior vector into the generative adversarial network, separate the short-term interest vector and the long-term interest vector through the generator, and construct a Bayesian network based on the historical behavior data to obtain an interest state transition probability matrix, specifically comprising the following steps, Extract historical user behavior data, generate a short-term real interest vector based on J-day click sequence through Word2Vec, generate a long-term real interest vector based on L-month click preference, and concatenate them into a real interest vector; Train the generative adversarial network using the real interest vector, the generative adversarial network consists of a generator and a discriminator, the generator uses a multi-layer perceptron to generate a short-term interest vector and a long-term interest vector, the discriminator uses a convolutional neural network to output a binary classification probability, and the training uses an adversarial loss. When the discriminator loss no longer decreases, the trained generative adversarial network generator is obtained; Use the trained generative adversarial network generator to infer the quantized spatio-temporal behavior vector to generate a short-term interest vector and a long-term interest vector; Based on the historical click sequence, define M types of interest states, use the Pyro library to construct a Bayesian network, calculate the conditional transition probability, and based on the historical click sequence, calculate the likelihood probability and the initial transition probability to obtain an interest transition probability matrix; Based on the short-term interest vector, the long-term interest vector and the interest transition probability matrix, use the Bayesian-Markov model to predict the user behavior link probability to generate a preliminary click rate prediction probability; Use reinforcement learning to adjust the preliminary click rate prediction probability to generate a final advertisement click rate prediction probability.

2. The user behavior based ad click rate prediction method of claim 1, wherein: The Bayesian-Markov model based on the short-term interest vector, the long-term interest vector and the interest transition probability matrix to predict the user behavior link probability, specifically comprising the following steps, Extract user behavior sequence from distributed file unit; The user behavior sequence is abstracted into a hidden Markov chain, an observation state is defined, N interest levels are defined as hidden states, a state transition matrix is obtained by dimension reduction of an interest transition probability matrix through average pooling, and an initial state distribution is generated based on user portrait data; The conditional transition probability of the hidden state is calculated using a Bayesian network to generate a state transition probability vector; The current hidden state is statistically obtained based on the user behavior sequence through a hidden Markov chain, and the current hidden state and the state transition probability vector are input into a variational autoencoder to infer and generate hidden state parameters, and the user behavior link probability is calculated through sampling.

3. The user behavior based ad click-through rate prediction method of claim 2, wherein: The preliminary click rate prediction probability is generated by inputting the short-term interest vector, the long-term interest vector and the user behavior link probability into a Transformer model, and generating the preliminary click rate prediction probability through an attention mechanism and a Sigmoid function.

4. The user behavior based ad click-through rate prediction method of claim 3, wherein: The preliminary click rate prediction probability is adjusted using reinforcement learning to generate a final advertisement click rate prediction probability, specifically including the following steps, The short-term interest vector, the long-term interest vector and the preliminary click rate prediction probability are spliced to generate a user feature vector, a reinforcement learning framework is defined, the reinforcement learning state is the user feature vector, the action is a prediction adjustment factor, and the reward is prediction accuracy; The Deep-Q-Network uses a multi-layer perceptron, initializes a main network and a target network, verifies the Deep-Q-Network through a mean square error as a loss function, completes training after the loss function meets specified requirements, and obtains a trained Deep-Q-Network; The user feature vector is input into the trained Deep-Q-Network, the optimal prediction adjustment factor is selected, the preliminary click rate prediction probability is adjusted, and the final advertisement click rate prediction probability is generated through a Sigmoid function normalization.

5. The user behavior based ad click-through rate prediction method of claim 4, wherein: The current hidden state is statistically obtained based on the user behavior sequence through a hidden Markov chain, and the current hidden state is generated based on the state transition matrix and the initial state distribution.

6. A system for predicting click-through rate of an advertisement based on user behavior, based on the method for predicting click-through rate of an advertisement based on user behavior according to any one of claims 1 to 5, characterized in that: It includes, The fusion module collects user behavior data and spatio-temporal trajectory data, extracts spatial embedding vectors through a graph neural network after constructing a user movement graph structure, extracts temporal embedding vectors through user behavior data, generates spatio-temporal behavior vectors according to the spatial embedding vectors and the temporal embedding vectors, and constructs a Bayesian network based on historical behavior data to obtain an interest state transition probability matrix. The construction module inputs the spatio-temporal behavior vectors into a generative adversarial network, separates the short-term interest vector and the long-term interest vector through a generator, and constructs a Bayesian network based on historical behavior data to obtain an interest state transition probability matrix; The preliminary prediction module predicts the user behavior link probability using a Bayesian-Markov model based on the short-term interest vector, the long-term interest vector and the interest transition probability matrix to generate a preliminary click rate prediction probability; The optimization module adjusts the preliminary click rate prediction probability using reinforcement learning to generate a final advertisement click rate prediction probability. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: The processor executes the computer program to realize the steps of the user behavior-based advertisement click rate prediction method of any one of claims 1-5.

8. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program, when executed by a processor, implements the steps of the user behavior based advertisement click rate prediction method according to any one of claims 1-5.

Images