An advertisement click rate prediction method and device, an electronic device, and a storage medium

Abstract

The application provides an advertisement click rate prediction method and device, electronic equipment and a computer readable storage medium. The advertisement click rate prediction method provided by the application comprises: obtaining advertisement data, preprocessing the advertisement data to obtain preprocessed advertisement data, encoding the preprocessed advertisement data to obtain a corresponding feature vector; inputting the feature vector into a product optimization-based attention factor decomposition machine deep network model to obtain combined cross features of the advertisement data; and inputting the combined cross features into a convolutional neural network-based advertisement click rate prediction network model to obtain a click rate of the advertisement data. The advertisement click rate prediction method provided by the application can predict the advertisement click rate and improve the accuracy of advertisement click rate prediction.

Description

Technical Field

[0001] This invention relates to the field of Internet data processing technology, and in particular to a method, apparatus, electronic device, and computer-readable storage medium for predicting advertising click-through rates. Background Technology

[0002] With the widespread adoption of the internet and the rapid development of big data technology, advertisers can now utilize internet platforms for targeted advertising. Compared to traditional advertising, online advertising on internet platforms offers advantages such as lower costs, faster speed, and more precise targeting, potentially leading to greater advertising revenue. Predicting ad click-through rates is a key factor in determining the revenue potential of an advertising platform.

[0003] Many existing ad click-through rate (CTR) prediction solutions rely on deep learning models. However, these solutions are unable to effectively extract the high-dimensional sparse features of ads, resulting in low accuracy in predicting CTR. Summary of the Invention

[0004] The purpose of this invention is to provide a method, apparatus, electronic device, and computer-readable storage medium for predicting ad click-through rates, in order to solve the technical problem of low accuracy in predicting ad click-through rates in the prior art.

[0005] The technical solution of the present invention is as follows: a method for predicting ad click-through rates is provided, comprising:

[0006] Acquire advertising data, preprocess the advertising data to obtain preprocessed advertising data, encode the preprocessed advertising data to obtain the corresponding feature vector;

[0007] The feature vector is input into a deep network model of attention factor decomposition machine based on product optimization to obtain the combined cross features of the advertising data. The deep network model of attention factor decomposition machine based on product optimization is trained based on the deep network of attention factor decomposition machine based on product optimization and the historical feature vector obtained from historical advertising data.

[0008] The combined cross features are input into an advertising click-through rate prediction network model based on a convolutional neural network to obtain the click-through rate of the advertising data. The advertising click-through rate prediction network model based on a convolutional neural network is trained based on an advertising click-through rate prediction network based on a convolutional neural network and historical combined cross features obtained from historical advertising data.

[0009] Furthermore, the advertising data is preprocessed to obtain preprocessed advertising data, and the preprocessed advertising data is encoded to obtain a corresponding feature vector. This includes: performing data cleaning and data processing on the advertising data to obtain preprocessed advertising data, and performing one-hot encoding on the preprocessed advertising data to obtain a corresponding feature vector.

[0010] Furthermore, before inputting the feature vector into the attention factorization machine deep network model based on product optimization, the following steps are also included:

[0011] We construct a deep network structure for attention factorization machine based on product optimization, which includes an input layer, an embedding layer, a pair-wise interaction layer, an attention pooling layer, a product layer, and an output layer.

[0012] Furthermore, before inputting the feature vector into the attention factorization machine deep network model based on product optimization, the following steps are also included:

[0013] The attention factorization machine deep network based on product optimization is trained using the historical feature vectors. The input layer receives the historical feature vectors, and the embedding layer reduces the feature dimension of the historical feature vectors to obtain a feature vector with reduced dimension. The pair-wise interaction layer performs element-wise product processing on the feature vectors with reduced dimension to obtain cross features. The attention pooling layer initializes the weights of the cross features to obtain weighted cross features. The product layer performs an inner product on the weighted cross features to obtain combined cross features. After training the attention factorization machine deep network based on product optimization, the attention factorization machine deep network model based on product optimization is obtained.

[0014] Furthermore, the Pair-wise Interaction layer performs element-wise product processing on the reduced-dimensional feature vectors to obtain cross features, including:

[0015] The Pair-wise Interaction layer performs element-wise product processing on the feature vector after dimensionality reduction according to the product formula to obtain cross features, wherein the product formula is: b1 is the first bias, x is the input feature vector after the dimension reduction, and v is the first weight vector. i vj Let R represent the i-th and j-th first weight vectors, respectively. x1 Let P be the set of real numbers i and j. T As the first weight, The output is the cross-feature;

[0016] The AttentionPooling layer initializes the weights of the cross features to obtain the weight-initialized cross features, including:

[0017] The AttentionPooling layer initializes the weights of the cross features according to the weight initialization formula, resulting in weight-initialized cross features. The weight initialization formula is as follows: O represents the initial cross-features of the output weights, a k，g Let y represent the attention factor, y represent the input cross features, and u be the second weight vector. k u g Let R represent the k-th and g-th second weight vectors, respectively. x2 Let g be the set of real numbers for k and g.

[0018] Furthermore, before inputting the combined cross features into the convolutional neural network-based ad click-through rate prediction network model, the method further includes constructing an ad click-through rate prediction network based on a convolutional neural network, comprising an input layer, a first convolutional layer, a second convolutional layer, a pooling layer, a fully connected layer, and an output layer, wherein the pooling layer uses max pooling and the activation function is the Rule function.

[0019] Furthermore, before inputting the combined cross features into the convolutional neural network-based ad click-through rate prediction network model, the method further includes:

[0020] The historical combined cross features are used to train the ad click-through rate prediction network based on the convolutional neural network. The input layer receives the combined cross features, which are then processed by the first convolutional layer, the second convolutional layer, and the pooling layer. The output unit of the fully connected layer outputs a real number F. After the ad click-through rate prediction network based on the convolutional neural network is trained, the ad click-through rate prediction network model based on the convolutional neural network is obtained. Here, F = f(wl + b2), F is the predicted click-through rate, f is the activation function, w is the second weight, b2 is the second bias, and l represents the input of the fully connected layer.

[0021] Another technical solution of the present invention is as follows: an advertising click-through rate prediction device is also provided, including a data processing module, a cross-feature acquisition module, and a click-through rate prediction module;

[0022] The data processing module is used to acquire advertising data, preprocess the advertising data to obtain preprocessed advertising data, and encode the preprocessed advertising data to obtain a corresponding feature vector.

[0023] The cross-feature acquisition module is used to input the feature vector into the product-optimized attention factor decomposition machine deep network model to obtain the combined cross features of the advertising data. The product-optimized attention factor decomposition machine deep network model is trained based on the product-optimized attention factor decomposition machine deep network and the historical feature vectors obtained from historical advertising data.

[0024] The click-through rate prediction module is used to input the combined cross features into an advertising click-through rate prediction network model based on a convolutional neural network to obtain the click-through rate of the advertising data. The advertising click-through rate prediction network model based on a convolutional neural network is trained based on an advertising click-through rate prediction network based on a convolutional neural network and historical combined cross features obtained from historical advertising data.

[0025] Another technical solution of the present invention is as follows: an electronic device is also provided, including a memory and a processor. The memory stores a computer program that can be executed by the processor. When the processor executes the computer program, it implements the advertising click-through rate prediction method as described in any of the above technical solutions.

[0026] Another technical solution of the present invention is as follows: a computer-readable storage medium is also provided, wherein the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the advertising click-through rate prediction method as described in any of the above technical solutions.

[0027] The beneficial effects of this invention are as follows: Advertising data is acquired; the advertising data is preprocessed to obtain preprocessed advertising data; the preprocessed advertising data is encoded to obtain corresponding feature vectors; the feature vectors are input into a deep network model based on a product-optimized attention factor decomposition machine to obtain combined cross features of the advertising data, wherein the deep network model based on a product-optimized attention factor decomposition machine is trained using a deep network based on a product-optimized attention factor decomposition machine and historical feature vectors obtained from historical advertising data; the combined cross features are input into a click-through rate (CTR) prediction network model based on a convolutional neural network to obtain the CTR of the advertising data, wherein the CTR prediction network model based on a convolutional neural network is trained using a CTR prediction network based on a convolutional neural network and historical combined cross features obtained from historical advertising data; through the above technical solution, the prediction of advertising CTR is achieved, which can improve the accuracy of advertising CTR prediction. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating the ad click-through rate prediction method according to an embodiment of the present invention.

[0029] Figure 2 This is a schematic diagram of the structure of the deep network of attention factor decomposition machine based on product optimization and the advertising click-through rate prediction network based on convolutional neural network in an embodiment of the present invention.

[0030] Figure 3 This is a schematic diagram of the advertising click-through rate prediction device according to an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0033] In the description of this application, the terms "first," "second," etc., are used only for distinguishing purposes and should not be construed as indicating or implying relative importance or order. In this specification, the terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0034] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0035] Figure 1 This is a flowchart illustrating the ad click-through rate prediction method according to the first embodiment of the present invention. It should be noted that if substantially the same result is obtained, the ad click-through rate prediction method of the present invention does not necessarily reflect the actual result. Figure 1 The illustrated process sequence is limited. For example... Figure 1 As shown, this method for predicting ad click-through rates mainly includes the following steps:

[0036] S1, acquire advertising data, preprocess the advertising data to obtain preprocessed advertising data, encode the preprocessed advertising data to obtain the corresponding feature vector;

[0037] S2, input the feature vector into the deep network model of attention factor decomposition machine based on product optimization to obtain the combined cross features of the advertising data, wherein the deep network model of attention factor decomposition machine based on product optimization is trained based on the deep network of attention factor decomposition machine based on product optimization and the historical feature vector obtained from the historical advertising data.

[0038] S3, input the combined cross features into the advertising click-through rate prediction network model based on convolutional neural network to obtain the click-through rate of the advertising data, wherein the advertising click-through rate prediction network model based on convolutional neural network is trained based on the advertising click-through rate prediction network based on convolutional neural network and the historical combined cross features obtained from historical advertising data.

[0039] This invention, through its embodiments, acquires advertising data, preprocesses the advertising data to obtain preprocessed advertising data, encodes the preprocessed advertising data to obtain corresponding feature vectors, inputs the feature vectors into a deep network model based on product optimization attention factorization machine to obtain combined cross features of the advertising data, and inputs the combined cross features into a click-through rate (CTR) prediction network model based on convolutional neural networks to obtain the CTR of the advertising data. This achieves CTR prediction and can improve the accuracy of CTR prediction.

[0040] In one optional implementation, the advertising data is preprocessed to obtain preprocessed advertising data, and the preprocessed advertising data is encoded to obtain a corresponding feature vector. This includes: performing data cleaning and data processing on the advertising data to obtain preprocessed advertising data, and performing one-hot encoding on the preprocessed advertising data to obtain a corresponding feature vector.

[0041] It should be noted that the advertising data can be divided into two parts. One part is user behavior data, which is the user behavior data collected when a user enters the page's advertising space, such as advertising space, mobile phone system, device type, region, exposure time, strategy ID, advertising creative ID, advertising bid amount, user information, etc. The other part is advertising business data, namely advertising configuration data, which records configuration type data such as advertising category, advertising query keywords, advertising audience ID, and advertising delivery time. Advertising data can be stored in the ODS layer of the database in the form of messages based on the Kafka platform. Advertising data can contain multiple dimensions, such as several fields of an advertising data, as shown in Table 1.

[0042] Table 1

[0043] User ID 15645334 City Guangzhou system Android Mobile phone type apple Material Classification Insurance Resource slot pop-up window

[0044] In one specific embodiment, the advertising data undergoes data cleaning to remove redundant or missing information. For example, based on a data polling query, if certain data in an advertising dataset is missing, such as key information like user information or mobile device information, then that advertising data is deleted. Data processing of the advertising data may include performing join operations based on data IDs, combining data from business tables and behavioral data tables, processing business data such as creative content, creative categories, and creative keywords into wide table data, and storing the processed advertising data in the ODS layer of the database.

[0045] In another specific embodiment, the preprocessed advertising data is one-hot encoded to convert it into numerical features, i.e., feature vectors. The encoding method used for one-hot encoding is as follows:

[0046]

[0047] In the above formula, based on the n states of the current field, and each state bit has independent features, and only one bit is 1, while the other flag bits are 0; by encoding the preprocessed advertising data using the above formula, a highly sparse feature vector containing high-dimensional bits can be obtained; examples of the feature vector are shown in Table 2.

[0048] Table 2

[0049] 1 1 1 1 1 0 1 1 0 0 1 0 0 0 1

[0050] In an optional implementation, before inputting the feature vector into a product-optimized Attention Factorization Machine (AFM) deep network model, the method further includes:

[0051] We construct a deep network structure for attention factorization machine based on product optimization, which includes an input layer, an embedding layer, a pair-wise interaction layer, an attention pooling layer, a product layer, and an output layer.

[0052] It should be noted that traditional factorization networks, similar to multilayer perceptrons, utilize feature addition. However, for highly sparse advertising features, this is not conducive to the iterative convergence of subsequent convolutional prediction networks. This invention proposes an AFM deep network based on product optimization to extract combined cross-features of advertisements, thereby extracting positively correlated feature data.

[0053] In one specific embodiment, the constructed AFM deep network structure based on product optimization includes six network layers: an input layer, an embedding layer, a pair-wise interaction layer, an attention pooling layer, a product layer, and an output layer.

[0054] In an optional implementation, before inputting the feature vector into the attention factorization machine deep network model based on product optimization, the method further includes:

[0055] The product-optimized attention factorization machine deep network is trained using the historical feature vectors. The input layer receives the historical feature vectors; the embedding layer reduces the feature dimension of the historical feature vectors to obtain a reduced-dimensional feature vector; the pair-wise interaction layer performs element-wise product processing on the reduced-dimensional feature vectors to obtain cross features; the attention pooling layer initializes the weights of the cross features to obtain weighted cross features; the product layer performs an inner product on the weighted cross features to obtain combined cross features. After training the product-optimized attention factorization machine deep network, the product-optimized attention factorization machine deep network model is obtained. The training is considered complete when the product-optimized attention factorization machine deep network reaches its optimal training state.

[0056] In one specific embodiment, the feature vector after one-hot encoding is used as the input to the input layer. The embedding layer is used to reduce the sparse feature dimension of the advertisement and reduce computational complexity. Specifically, the positive feature (with a value of 1) is multiplied by h embeddings to obtain a k*h dimensional feature vector, where k is the length of the corresponding one-hot encoded feature vector. The processing expression of the embedding layer is as follows:

[0057]

[0058] Among them, v h Vx represents the h-th Embedding, and Vx represents the output of the Embedding layer.

[0059] In an optional implementation, the Pair-wise Interaction layer performs element-wise product processing on the reduced-dimensional feature vectors to obtain cross features, including:

[0060] The Pair-wise Interaction layer performs element-wise product processing on the feature vector after dimensionality reduction according to the product formula to obtain cross features, wherein the product formula is: b1 is the first bias, x is the input feature vector after the dimension reduction, and v is the first weight vector. i v j Let R represent the i-th and j-th first weight vectors, respectively. x1 Let P be the set of real numbers i and j. T As the first weight, The output is the cross-feature;

[0061] The AttentionPooling layer initializes the weights of the cross features to obtain the weight-initialized cross features, including:

[0062] The AttentionPooling layer initializes the weights of the cross features according to the weight initialization formula, resulting in weight-initialized cross features. The weight initialization formula is as follows: O represents the initial cross-features of the output weights, a k，g Let y represent the attention factor, y represent the input cross features, and u be the second weight vector. k u g Let R represent the k-th and g-th second weight vectors, respectively. x2 Let P be the set of real numbers for k and g. TThe first weight is the weight of the prediction layer.

[0063] In one specific embodiment, after the embedding layer, a pair-wise interaction layer is constructed to parse the advertising feature data (feature vectors). The output of the embedding layer contains n embedding vectors with dense vector features. The pair-wise interaction layer performs element-wise product processing on the output of the embedding layer to achieve AFM modeling based on product optimization, resulting in combined cross features with feature vector dimension k and a number of feature vectors n(n-1) / 2, where n is the number of embedding vectors. An attention pooling layer is then constructed on top of the pair-wise interaction layer to preserve the position and intensity information of the advertising features. Weights are initialized on the combined features of the product attention score based on an MLP (Multilayer Perceptron Model).

[0064] To better represent the deeper features of advertisements, a multiplicative layer is constructed in the Attention Pooling layer based on the inner product method to obtain better feature representation. The inner product of the multiplicative layer is defined as follows: Here, ⊙ represents the dot product of different ad feature matrices (cross features after weight initialization), and the results are summed. A and B represent different ad feature matrices, respectively. After constructing the entire AFM deep network structure based on product optimization, the network is trained using squared loss.

[0065] In an optional implementation, before inputting the combined cross features into the convolutional neural network-based ad click-through rate prediction network model, the method further includes constructing a convolutional neural network-based ad click-through rate prediction network comprising an input layer, a first convolutional layer, a second convolutional layer, a pooling layer, a fully connected layer, and an output layer, wherein the pooling layer uses max pooling and the activation function is a Rule function.

[0066] In one specific embodiment, the ad click-through rate prediction network based on convolutional neural networks includes a first convolutional layer, a second convolutional layer, a pooling layer, and a fully connected layer; combined cross features are used as input to the input layer; after the input layer is a convolutional layer, which is used for feature extraction, and different convolutional networks extract the combined cross features.

[0067] In an optional implementation, before inputting the combined cross features into the convolutional neural network-based ad click-through rate prediction network model, the method further includes:

[0068] The historical combined cross features are used to train the ad click-through rate prediction network based on a convolutional neural network. The input layer receives the combined cross features, which are then processed through the first convolutional layer, the second convolutional layer, and the pooling layer. Finally, a real number F is output from the output unit of the fully connected layer. After training the ad click-through rate prediction network based on the convolutional neural network, the ad click-through rate prediction network model based on the convolutional neural network is obtained. Here, F = f(wl + b2), F is the predicted click-through rate, l is the activation function, w is the second weight, b2 is the second bias, and l represents the input of the fully connected layer. The training of the ad click-through rate prediction network based on the convolutional neural network reaches its optimal state, i.e., training is complete.

[0069] It should be noted that traditional factorization networks, similar to multilayer perceptrons, utilize feature addition. However, for highly sparse advertising features, this is not conducive to the iterative convergence of subsequent convolutional prediction networks. This invention's embodiment uses product-optimized AFM to extract combined cross-features of advertisements, thereby extracting positively correlated feature data.

[0070] In one specific embodiment, based on the characteristics of advertising data, each dimension of the advertisement represents different aspects of behavior. A one-dimensional convolutional kernel extracts advertising features. Specifically, the convolution operation yields feature s with dimension dx(m+w-1). For the relevance matrix, in its i-th row, a defined one-dimensional convolution can be obtained, defined as follows: Where w is the third weight, s is the feature input, d is one dimension of the filter, o is the filter size, and R i The first convolutional layer is a one-dimensional convolution, j∈[1,m+o+1], where m is the size of the feature input s (the m-th input of the feature parameters). To further explore broadly related features, a second convolutional layer is stacked after the first convolutional layer, with the same stacking structure as the first convolutional layer. Following the convolutional layers is a pooling layer, used to reduce the dimensionality of the advertising features. The pooling layer uses max pooling, with the activation function being the Rule function, expressed as X=down(ze), where down is the downsampling operation, z is the fourth weight, and e is the input to the activation function. Deep-level combined cross-features are obtained after pooling. Following the pooling layer is the output layer, whose output unit is a mapped real number y, representing the predicted click of the advertising click-through rate prediction network based on the convolutional neural network. The predicted click-through rate for a user can be expressed as F=f(wl+b2), where F is the predicted click-through rate, f is the activation function, w is the second weight (weight matrix), b2 is the second bias, and l represents the input to the fully connected layer.

[0071] In another specific embodiment, a schematic diagram of the structure of a deep network based on product optimization attention factorization machine and a network based on convolutional neural network for predicting ad click-through rate is shown below. Figure 2 As shown, Figure 2 From left to right, the image depicts the input layer, embedding layer, pair-wise interaction layer, attention pooling layer, product layer, and output layer of a deep network based on product optimization for attention factorization, as well as the convolutional layer and pooling layer of a network for predicting ad click-through rates based on convolutional neural networks.

[0072] The ad click-through rate (CTR) prediction method provided in this invention involves acquiring ad data, preprocessing the ad data to obtain preprocessed ad data, encoding the preprocessed ad data to obtain corresponding feature vectors, inputting the feature vectors into a deep network model based on product optimization attention factorization machine to obtain combined cross features of the ad data, and inputting the combined cross features into an ad CTR prediction network model based on convolutional neural network to obtain the click-through rate of the ad data. This method achieves ad CTR prediction and can improve the accuracy of ad CTR prediction.

[0073] The ad click-through rate (CTR) prediction method of this invention automatically extracts ad features based on product-optimized attention factor decomposition machine (AFM) and convolutional neural network (CNN) to achieve CTR prediction. It combines AFM to map high-dimensional sparse ad features to a low-dimensional dense feature space. Based on the sparse nature of ad features, a product layer is introduced into the AFM network. Using inner product and dot product, cross-features are combined, and a CNN is constructed to perform secondary feature extraction on the combined cross-features. This more effectively combines the CNN to extract combined cross-features from high-dimensional features, effectively mining deeper ad features. This optimizes the CNN's insensitivity to sparse ad features, thus achieving ad CTR prediction. Compared to existing ad prediction schemes, this method can better extract high-dimensional sparse features of ads, resulting in more accurate predicted CTRs.

[0074] The ad click-through rate prediction method provided in this invention can be built on artificial intelligence (AI). It acquires and processes relevant data using AI technology to achieve unattended AI-powered ad click-through rate prediction. AI is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0075] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0076] Figure 3 This is a schematic diagram of the advertising click-through rate prediction device according to an embodiment of the present invention, as shown below. Figure 3 As shown, the ad click-through rate prediction device 20 includes a data processing module 21, a cross-feature acquisition module 22, and a click-through rate prediction module 23;

[0077] The data processing module 21 is used to acquire advertising data, preprocess the advertising data to obtain preprocessed advertising data, and encode the preprocessed advertising data to obtain a corresponding feature vector.

[0078] The cross-feature acquisition module 22 is used to input the feature vector into the deep network model of attention factor decomposition machine based on product optimization to obtain the combined cross features of the advertising data. The deep network model of attention factor decomposition machine based on product optimization is trained based on the deep network of attention factor decomposition machine based on product optimization and the historical feature vector obtained from historical advertising data.

[0079] The click-through rate prediction module 23 is used to input the combined cross features into the advertising click-through rate prediction network model based on convolutional neural network to obtain the click-through rate of the advertising data. The advertising click-through rate prediction network model based on convolutional neural network is trained based on the advertising click-through rate prediction network based on convolutional neural network and historical combined cross features obtained from historical advertising data.

[0080] Furthermore, the data processing module 21 is also used to perform data cleaning and data processing on the advertising data to obtain preprocessed advertising data, and to perform one-hot encoding on the preprocessed advertising data to obtain the corresponding feature vector.

[0081] Furthermore, the ad click-through rate prediction device 20 also includes a first network construction module, which is used to construct a deep network structure based on product optimization of attention factor decomposition machine, including an input layer, an embedding layer, a pair-wise interaction layer, an attention pooling layer, a product layer, and an output layer, before inputting the feature vector into the deep network model based on product optimization of attention factor decomposition machine.

[0082] Furthermore, the ad click-through rate prediction device 20 also includes a first model training module. This first model training module is used to train the deep network model based on product optimization of the attention factor decomposition machine using the historical feature vector before inputting the feature vector into the deep network model. The input layer receives the historical feature vector; the embedding layer reduces the feature dimension of the historical feature vector to obtain a feature vector with reduced dimension; the pair-wise interaction layer performs element-wise product processing on the feature vector with reduced dimension to obtain cross features; the attention pooling layer initializes the weights of the cross features to obtain weighted cross features; the product layer performs an inner product on the weighted cross features to obtain combined cross features; after training the deep network model based on product optimization of the attention factor decomposition machine, the model is obtained.

[0083] The Pair-wise Interaction layer performs element-wise product processing on the reduced-dimensional feature vectors to obtain cross features, including:

[0084] The Pair-wise Interaction layer performs element-wise product processing on the feature vector after dimensionality reduction according to the product formula to obtain cross features, wherein the product formula is: b1 is the first bias, x is the input feature vector after the dimension reduction, and v is the first weight vector. i v j Let R represent the i-th and j-th first weight vectors, respectively. x1 Let P be the set of real numbers i and j. T As the first weight, The output is the cross-feature;

[0085] The AttentionPooling layer initializes the weights of the cross features to obtain the weight-initialized cross features, including:

[0086] The AttentionPooling layer initializes the weights of the cross features according to the weight initialization formula, resulting in weight-initialized cross features. The weight initialization formula is as follows: O represents the initial cross-features of the output weights, a k，gLet y represent the attention factor, y represent the input cross features, and u be the second weight vector. k u g Let R represent the k-th and g-th second weight vectors, respectively. x2 Let g be the set of real numbers for k and g.

[0087] Furthermore, the ad click-through rate prediction device 20 also includes a second network construction module. The second network construction module is used to construct an ad click-through rate prediction network based on a convolutional neural network, which includes an input layer, a first convolutional layer, a second convolutional layer, a pooling layer, a fully connected layer, and an output layer, before inputting the combined cross features into the ad click-through rate prediction network model based on a convolutional neural network. The pooling layer uses max pooling and the activation function is the Rule function.

[0088] Furthermore, the ad click-through rate prediction device 20 also includes a second model training module. The second model training module is used to train the ad click-through rate prediction network based on the convolutional neural network using the historical combined cross features. The input layer receives the combined cross features, and after passing through the first convolutional layer, the second convolutional layer, and the pooling layer, the input layer outputs a real number F in the output unit of the fully connected layer. After the ad click-through rate prediction network based on the convolutional neural network is trained, the ad click-through rate prediction network model based on the convolutional neural network is obtained, where F = f(wl + b2), F is the predicted click-through rate, f is the activation function, w is the second weight, b2 is the second bias, and l represents the input of the fully connected layer.

[0089] This invention provides an ad click-through rate (CTR) prediction device. The data processing module 21 acquires ad data, preprocesses the ad data to obtain preprocessed ad data, and encodes the preprocessed ad data to obtain corresponding feature vectors. The cross-feature acquisition module 22 inputs the feature vectors into a deep network model based on product optimization attention factorization machine to obtain combined cross-features of the ad data. The CTR prediction module 23 inputs the combined cross-features into an ad CTR prediction network model based on a convolutional neural network to obtain the click-through rate of the ad data. This device achieves ad CTR prediction and can improve the accuracy of ad CTR prediction.

[0090] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Figure 4 As shown, the electronic device 30 includes a processor 31 and a memory 32 communicatively connected to the processor 31.

[0091] The memory 32 stores program instructions for implementing the ad click-through rate prediction method of any of the above embodiments.

[0092] The processor 31 is used to execute program instructions stored in the memory 32 for code testing.

[0093] The processor 31 can also be referred to as a CPU (Central Processing Unit). The processor 31 may be an integrated circuit chip with signal processing capabilities. The processor 31 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor.

[0094] This invention provides a storage medium that stores program instructions capable of implementing all the methods described above. The storage medium can be non-volatile or volatile. These program instructions can be stored in the storage medium as a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0095] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0096] Furthermore, the functional modules in the various embodiments of this invention can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. The above are merely embodiments of this invention and do not limit the patent scope of this invention. Any equivalent structural or procedural transformations made based on the description and drawings of this invention, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this invention.

[0097] The above description is merely an embodiment of the present invention. It should be noted that those skilled in the art can make improvements without departing from the inventive concept of the present invention, but these improvements all fall within the protection scope of the present invention.

Claims

1. A method for predicting ad click-through rate, characterized in that, Includes the following steps: Acquire advertising data, preprocess the advertising data to obtain preprocessed advertising data, encode the preprocessed advertising data to obtain the corresponding feature vector; Construct a deep network structure for attention factorization machine based on product optimization, which includes an input layer, an embedding layer, a pair-wise interaction layer, an attention pooling layer, a product layer, and an output layer; The attention factorization machine deep network based on product optimization is trained using historical feature vectors. The input layer receives the historical feature vectors, and the embedding layer reduces the feature dimension of the historical feature vectors to obtain a feature vector with reduced dimension. The pair-wise interaction layer performs element-wise product processing on the feature vector with reduced dimension to obtain cross features. The attention pooling layer initializes the weights of the cross features to obtain weighted cross features. The product layer performs an inner product on the weighted cross features to obtain combined cross features. After training the deep network of attention factorization machine based on product optimization, the deep network model of attention factorization machine based on product optimization is obtained. The feature vector is input into a deep network model of attention factor decomposition machine based on product optimization to obtain the combined cross features of the advertising data. The deep network model of attention factor decomposition machine based on product optimization is trained based on the deep network of attention factor decomposition machine based on product optimization and the historical feature vector obtained from historical advertising data. The combined cross features are input into an ad click-through rate prediction network model based on a convolutional neural network. In the process, the click-through rate of the advertising data is obtained, wherein the advertising click-through rate prediction network model based on the convolutional neural network is trained based on the advertising click-through rate prediction network based on the convolutional neural network and the historical combination cross features obtained from historical advertising data.

2. The method for predicting ad click-through rates according to claim 1, characterized in that, The advertising data is preprocessed to obtain preprocessed advertising data, and the preprocessed advertising data is encoded to obtain a corresponding feature vector. This includes: performing data cleaning and data processing on the advertising data to obtain preprocessed advertising data, and performing one-hot encoding on the preprocessed advertising data to obtain a corresponding feature vector.

3. The advertising click-through rate prediction method according to claim 1, characterized in that, The Pair-wise Interaction layer performs element-wise product processing on the reduced-dimensional feature vectors to obtain cross features, including: The Pair-wise Interaction layer performs element-wise product processing on the feature vector after dimensionality reduction according to the product formula to obtain cross features, wherein the product formula is: , b 1 represents the first bias, x represents the input feature vector after the dimension reduction, and v represents the first weight vector. i v j Let i and j represent the first weight vectors, respectively. Let i be the set of real numbers i and j. As the first weight, The output is the cross-feature; The Attention Pooling layer initializes the weights of the cross features to obtain the weight-initialized cross features, including: The Attention Pooling layer initializes the weights of the cross features according to the weight initialization formula, resulting in weight-initialized cross features. The weight initialization formula is as follows: , The output is the initial cross-features after weighting. a k，g Indicates attention factor, y The input cross features are represented by u, which is the second weight vector. k u g Let k and g represent the second weight vectors, respectively. Let g be the set of real numbers for k and g.

4. The method for predicting ad click-through rates according to claim 1, characterized in that, Before inputting the combined cross features into the convolutional neural network-based ad click-through rate prediction network model, the method further includes constructing an ad click-through rate prediction network based on a convolutional neural network, comprising an input layer, a first convolutional layer, a second convolutional layer, a pooling layer, a fully connected layer, and an output layer, wherein the pooling layer uses max pooling and the activation function is the ReLU function.

5. The ad click-through rate prediction method according to claim 4, characterized in that, Before inputting the combined cross features into the convolutional neural network-based ad click-through rate prediction network model, the method further includes: The historical combined cross features are used to train the ad click-through rate prediction network based on a convolutional neural network. The input layer receives the combined cross features, which are then processed through the first convolutional layer, the second convolutional layer, and the pooling layer. Finally, a real number F is output from the output unit of the fully connected layer. After training the ad click-through rate prediction network based on the convolutional neural network, the ad click-through rate prediction network model based on the convolutional neural network is obtained. F represents the predicted click-through rate. f For activation function, w As the second weight, b 2 is the second bias. l This represents the input to the fully connected layer.

6. An ad click-through rate prediction device, characterized in that, It includes a data processing module, a first network construction module, a first model training module, a cross-feature acquisition module, and a click-through rate prediction module; The data processing module is used to acquire advertising data, preprocess the advertising data to obtain preprocessed advertising data, and encode the preprocessed advertising data to obtain a corresponding feature vector. The first model building module is used to construct a deep network structure of attention factorization machine based on product optimization, which includes an input layer, an embedding layer, a pair-wise interaction layer, an attention pooling layer, a product layer, and an output layer. The first model training module is used to train the attention factorization machine deep network based on product optimization using historical feature vectors. The input layer receives the historical feature vectors, the embedding layer reduces the feature dimension of the historical feature vectors to obtain a feature vector with reduced dimension, the pair-wise interaction layer performs element-wise product processing on the feature vector with reduced dimension to obtain cross features, the attention pooling layer initializes the weights of the cross features to obtain weighted cross features, and the product layer performs an inner product on the weighted cross features to obtain combined cross features. After training the deep network of attention factorization machine based on product optimization, the deep network model of attention factorization machine based on product optimization is obtained. The cross-feature acquisition module is used to input the feature vector into the product-optimized attention factor decomposition machine deep network model to obtain the combined cross features of the advertising data. The product-optimized attention factor decomposition machine deep network model is trained based on the product-optimized attention factor decomposition machine deep network and the historical feature vectors obtained from historical advertising data. The click-through rate prediction module is used to input the combined cross features into an advertising click-through rate prediction network model based on a convolutional neural network to obtain the click-through rate of the advertising data. The advertising click-through rate prediction network model based on a convolutional neural network is trained based on an advertising click-through rate prediction network based on a convolutional neural network and historical combined cross features obtained from historical advertising data.

7. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program executable by the processor, characterized in that, When the processor executes the computer program, it implements the advertising click-through rate prediction method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the advertising click-through rate prediction method as described in any one of claims 1 to 5.

Images