Advertisement click prediction method based on cross feature extraction model and related device thereof
By employing a cross-feature extraction model and a multi-model fusion method for ad click prediction, the problem of low accuracy in ad click-through rate prediction is solved, achieving efficient ad feature extraction and personalized recommendations while reducing the cost of manual analysis.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PING AN TECH (SHENZHEN) CO LTD
- Filing Date
- 2022-09-20
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies are not very accurate in predicting ad click-through rates and are difficult to extract ad features automatically, resulting in a large demand for manual cross-feature analysis and high costs.
An ad click prediction method based on a cross-feature extraction model is adopted. This method involves data cleaning, sparse feature vector generation, cross-feature extraction, combined feature fusion, and fully connected feature extraction, combined with multiple models such as AFM, GDBT, and LR for ad click prediction.
It improved the accuracy of ad click-through rate prediction, reduced human and material costs, and enhanced the personalized recommendation capabilities of internet advertising platforms.
Smart Images
Figure CN115456680B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to an advertising click prediction method and related equipment based on a cross-feature extraction model. Background Technology
[0002] Since the beginning of the 21st century, the internet has experienced tremendous growth, and online advertising through internet platforms has demonstrated immense value. Compared to traditional advertising, online advertising offers advantages such as richness, diversity, low cost, and personalized recommendations. Therefore, online advertising has become a crucial investment area for various well-known companies and a strategic focus for major internet platforms. Currently, internet advertising primarily takes two forms: online display advertising and online search advertising. A critical aspect of internet advertising lies in predicting clicks. Predicting clicks is the core of performance evaluation for intelligent ad recommendations. Therefore, researching and modeling predictive click-through rates based on user behavior is of great significance.
[0003] Personalized recommendation-based intelligent advertising systems have brought tremendous development opportunities to online advertising. Click-through rate (CTR), as the core of intelligent recommendation, currently has numerous model applications. In academia and industry, researchers have proposed various algorithmic models based on user behavior information to improve ad CTR prediction. Linear regression, as the earliest prediction model, was initially used for predicting ad CTR due to its simple model structure and ease of calculation, and it remains the most widely used advertising model. However, because it requires a large amount of manual feature extraction, it cannot effectively extract ad features automatically. Furthermore, current technologies suffer from low accuracy in predicting ad CTR. Summary of the Invention
[0004] The purpose of this application is to propose an advertising click prediction method and related equipment based on a cross-feature extraction model, thereby improving the accuracy of advertising click prediction.
[0005] To address the aforementioned technical problems, this application provides an ad click prediction method based on a cross-feature extraction model, employing the following technical solution:
[0006] An ad click prediction method based on a cross-feature extraction model includes the following steps:
[0007] Collect user advertising behavior data, and perform data cleaning and data processing operations on the user advertising behavior data in sequence to obtain the first data;
[0008] The cross-feature extraction model is used to iteratively extract the cross features of the first data until the number of iterations is reached, and then the second data is obtained.
[0009] The first data is processed iteratively based on the combined feature extraction model and the leaf feature fusion model until the preset number of iterations is reached to obtain the third data.
[0010] Perform fully connected feature extraction operations on the second data and the third data respectively to obtain the fully connected features of the second data and the fully connected features of the third data;
[0011] By concatenating the fully connected features of the second data and the fully connected features of the third data, the target features are obtained.
[0012] The target features are input into a pre-trained click prediction model to obtain the ad click prediction results.
[0013] Furthermore, the step of sequentially performing data cleaning and data processing operations on the user advertising behavior data to obtain the first data includes:
[0014] Identify key data in the user advertising behavior data and confirm whether any key data is missing;
[0015] If the key data is missing, the corresponding user advertising behavior data is deleted to obtain the cleaned user advertising behavior data.
[0016] The cleaned user advertising behavior data is processed into a sparse feature vector to obtain the first data.
[0017] Furthermore, the user advertising behavior data includes user names, and the step of processing the cleaned user advertising behavior data into a sparse feature vector to obtain the first data includes:
[0018] Based on the user name, a join operation is performed on the user advertising behavior data to obtain a wide table, and the wide table is loaded into the preset big data ADS layer;
[0019] The first data is obtained by performing a dimensional expansion operation on the wide table in the big data ADS layer based on one-hot encoding.
[0020] Furthermore, the cross-feature extraction model includes a cross-input layer, an embedding layer, a feature cross-layer, and an attention pooling layer. The step of iteratively extracting the cross features of the first data based on the cross-feature extraction model until the required number of iterations is reached to obtain the second data includes:
[0021] The first data is input into the cross-feature extraction model, and the embedding layer multiplies the positive features with a value of 1 in the first data with a preset h embedding blocks to obtain the embedding vector;
[0022] The feature cross layer performs element-wise intelligent multiplication on the first data based on the embedding vector to obtain cross features;
[0023] The cross-features are input into the attention pooling layer to obtain the second data output.
[0024] Furthermore, the step of performing element-wise intelligent multiplication on the first data based on the embedding vector to obtain cross features includes:
[0025] The first data is processed using element-wise intelligent multiplication based on the following formula:
[0026] Where b is the bias, x i For the i-th of the first data, x j
[0027] For the j-th first data, v i For the i-th embedding vector, v j Let J be the j-th embedding vector. P is a preset weight matrix. The intersection feature is described above.
[0028] Furthermore, the combined feature extraction model includes multiple decision trees, and the step of iteratively processing the first data based on the combined feature extraction model and the leaf feature fusion model until a preset number of iterations is reached to obtain the third data includes:
[0029] The first data is input into the combined feature extraction model, and the first data is classified by the decision tree;
[0030] Extract the features of the leaf nodes of each decision tree as latent vectors;
[0031] The hidden vectors are input into the leaf feature fusion model for combination until a preset number of iterations is reached to obtain the third data.
[0032] Furthermore, the step of inputting the target features into a pre-trained click prediction model to obtain the ad click prediction result includes:
[0033] The ad click prediction result is calculated based on the following formula:
[0034] Where y is the output, representing the ad click prediction result, Fm is the combined feature, w is the preset initial weight value, b is the bias, and T represents the transpose operator.
[0035] To address the aforementioned technical problems, this application also provides an ad click prediction device based on a cross-feature extraction model, employing the following technical solution:
[0036] An ad click prediction device based on a cross-feature extraction model includes:
[0037] The collection module is used to collect user advertising behavior data, and to perform data cleaning and data processing operations on the user advertising behavior data in sequence to obtain first data.
[0038] The first extraction module is used to iteratively extract the cross features of the first data based on the cross feature extraction model until the number of iterations is reached to obtain the second data.
[0039] The second extraction module is used to process the first data cyclically based on the combined feature extraction model and the leaf feature fusion model until a preset number of iterations is reached to obtain the third data.
[0040] The fully connected feature extraction module is used to perform fully connected feature extraction operations on the second data and the third data respectively to obtain the fully connected features of the second data and the fully connected features of the third data.
[0041] The splicing module is used to splice the fully connected features of the second data and the fully connected features of the third data to obtain the target features;
[0042] The prediction module is used to input the target features into a pre-trained click prediction model to obtain the ad click prediction result.
[0043] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:
[0044] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the above-described advertising click prediction method based on a cross-feature extraction model.
[0045] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:
[0046] A computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the above-described advertising click prediction method based on a cross-feature extraction model.
[0047] Compared with the prior art, the embodiments of this application have the following main advantages:
[0048] This application can effectively improve the prediction of ad click-through rates. It achieves the fusion of cross-features of ad characteristics based on multiple models, effectively mining deep-level features from high-dimensional sparse features to achieve higher prediction accuracy. This can effectively improve the personalized recommendation capabilities of online internet advertising platforms. Furthermore, since this application uses multiple models for prediction, it avoids the problem of relying on large amounts of manpower and resources for manual analysis in traditional ad prediction methods, thus reducing the labor costs of the advertising system. Attached Figure Description
[0049] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0050] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;
[0051] Figure 2 This is a flowchart of an embodiment of the ad click prediction method based on the cross-feature extraction model according to this application;
[0052] Figure 3 This is a schematic diagram of an embodiment of an ad click prediction device based on a cross-feature extraction model according to this application;
[0053] Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application.
[0054] Reference numerals: 200, computer equipment; 201, memory; 202, processor; 203, network interface; 300, advertising click prediction device based on cross-feature extraction model; 301, collection module; 302, first extraction module; 303, second extraction module; 304, fully connected feature extraction module; 305, splicing module; 306, prediction module. Detailed Implementation
[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.
[0056] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0057] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.
[0058] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.
[0059] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.
[0060] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.
[0061] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.
[0062] It should be noted that the ad click prediction method based on the cross-feature extraction model provided in this application embodiment is generally executed by a server / terminal device, and correspondingly, the ad click prediction device based on the cross-feature extraction model is generally set in the server / terminal device.
[0063] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.
[0064] Continue to refer to Figure 2 The diagram illustrates a flowchart of an embodiment of the ad click prediction method based on a cross-feature extraction model according to this application. The ad click prediction method based on a cross-feature extraction model includes the following steps:
[0065] S1: Collect user advertising behavior data, and perform data cleaning and data processing operations on the user advertising behavior data in sequence to obtain the first data.
[0066] In this embodiment, this application collects user data through application software (APP) to obtain multiple pieces of user advertising behavior data. The data collection and processing of this application, the raw data used for model training, is collected through the APP's data tracking and reporting method when the user clicks or browses, and based on the Kafka platform, the collected data is stored in the ODS layer of big data to form raw data (i.e., user advertising behavior data).
[0067] Specifically, in step S1, the steps of sequentially performing data cleaning and data processing operations on the user advertising behavior data to obtain the first data include:
[0068] Identify key data in the user advertising behavior data and confirm whether any key data is missing;
[0069] If the key data is missing, the corresponding user advertising behavior data is deleted to obtain the cleaned user advertising behavior data.
[0070] The cleaned user advertising behavior data is processed into a sparse feature vector to obtain the first data.
[0071] In this embodiment, if the key data is missing, the corresponding user advertising behavior data is deleted to obtain cleaned user advertising behavior data. If the key data is not missing, the user advertising behavior data can be directly considered as cleaned user advertising behavior data. Each piece of user advertising behavior data includes fields such as user name (ID), city, system, mobile phone type, material category, and resource position. For example, as shown in Table 1, Table 1 only shows some fields of this application. Among them, user name and mobile phone type are key data. The specific cleaning steps include: judging the key data such as user name and mobile phone type through data polling query; if the key data is missing in the collected information, the user advertising behavior data corresponding to the key data is deleted; otherwise, the next user advertising behavior data is queried. Finally, all user advertising behavior data that has not been deleted is saved to obtain cleaned user advertising behavior data. The cleaned user advertising behavior data is processed into a sparse feature vector to obtain the first data.
[0072] Table 1 contains user advertising behavior data with multi-dimensional information.
[0073] field name example Username 15645334 City Guangzhou system Android Mobile phone type apple Material Classification Insurance Resource slot pop-up window
[0074] Furthermore, the user advertising behavior data includes user names, and the step of processing the cleaned user advertising behavior data into a sparse feature vector to obtain the first data includes:
[0075] Based on the user name, a join operation is performed on the user advertising behavior data to obtain a wide table, and the wide table is loaded into the preset big data ADS layer;
[0076] The first data is obtained by performing a dimensional expansion operation on the wide table in the big data ADS layer based on one-hot encoding.
[0077] In this embodiment, a join operation is performed based on the user name, including data such as the content, category, and keywords of the user's advertising behavior data, to process it into a wide table. This wide table is then loaded into the big data ADS layer to form the data output. Furthermore, to effectively extract features from the original behavior data, one-hot encoding is used to convert the user's advertising behavior data in the wide table into numerical encoding to expand the dimensions. The categorical variables of the advertising behavior data are used as binary vectors to convert the wide table into numerical features. The main method is as follows: based on the N states existing in the current field of the advertising behavior data (e.g., male / female, with two attributes, and N = 2 states; or, for example, city names, with 663 cities, 663 attributes, and N = 663 states), each state's corresponding state bit has independent features, with only one bit being 1 and the other bits being 0. The formula is as follows:
[0078] Where X is the output (i.e., the output of a certain dimension in the first data), k represents the nth attribute, and n represents the number of state bits.
[0079] The wide table encoded by the above formula can yield a highly sparse feature vector containing high-dimensional bits.
[0080] Examples of converting user advertising behavior data into numerical codes are shown in Table 2:
[0081] Table 2: User Advertising Behavior Data Converted to Numerical Encoding
[0082] Click or not? Android iOS male female 1 0 1 1 0
[0083] S2: Extract the cross features of the first data in a loop based on the cross feature extraction model until the number of iterations is reached to obtain the second data.
[0084] In this embodiment, the cross-feature extraction model is an attentional factorization machine (AFM).
[0085] Specifically, the cross-feature extraction model includes a cross-input layer, an embedding layer, a feature cross-layer, and an attention pooling layer. In step S2, the step of iteratively extracting the cross features of the first data based on the cross-feature extraction model until the required number of iterations is reached to obtain the second data includes:
[0086] The first data is input into the cross-feature extraction model, and the embedding layer multiplies the positive features with a value of 1 in the first data with a preset h embedding blocks to obtain the embedding vector;
[0087] The feature cross layer performs element-wise intelligent multiplication on the first data based on the embedding vector to obtain cross features;
[0088] The cross-features are input into the attention pooling layer to obtain the second data output.
[0089] In this embodiment, the Cross Feature Extraction (AFM) model is an improvement on FM. Its most significant feature is the use of an attention network to learn the importance of different combined features. FM learns an embedding vector, also called a latent vector, for each feature, and the weight of the combined feature is represented by the inner product of two latent vectors. The first data obtained after encoding by AFM is high-dimensional sparse feature data. This application combines AFM to embed the high-dimensional sparse feature data into a low-dimensional vector space to extract cross features from high-dimensional features. Furthermore, different features and weights are combined in subsequent processes to improve the prediction accuracy of linear regression.
[0090] The AFM model comprises five stacked layers: a cross-input layer, an embedding layer, a pair-wise interaction layer, and an attention pooling layer. The first data is input into the AFM model through the input layer. Given the sparse features of the advertisements, the embedding layer multiplies the positive features (value 1) in the first data with h embeddings to obtain a k*h dimensional embedding vector (V). x ), where k is the length of the corresponding feature encoding, that is, k is the length of the matrix (i.e., feature vector) obtained after embedding. The specific operation is as follows:
[0091] Among them, V x The obtained k*h dimensional embedding vector is given, where h is the initial parameter of the AFM model, set according to actual needs. Different data correspond to different values of h, where h represents the h-th embedding block. T represents matrix transpose.
[0092] After the embedding layer, the N embedding vectors of the features are processed by element-wise product. The element-wise product multiplies the first data point of each input by an embedding vector. In other words, it scales each column of the dataset to model the combined features of the advertisements, resulting in a combination of features of number n(n-1) / 2 and dimension k. The operation is as follows:
[0093] Where b is the bias, x i For the i-th of the first data, x j For the j-th first data, v i For the i-th embedding vector, v j Let be the j-th embedding vector. P is a preset weight matrix with random initial values, which will be continuously updated as the model iterates. The obtained cross features are denoted as . Here, i and j are random round-robin combinations, representing pairwise combination features in AFM.
[0094] Furthermore, to reflect the importance of the cross features, the data obtained from the feature cross layer (Pair-wise Interaction layer) undergoes Attention processing. Specifically, different cross features are processed... The attention score (aij) is calculated by multiplying the product of the two values. Further, the attention score is initialized using an MLP (Multilayer Perceptron), which employs a feedforward training method. The chosen mapping function in this MLP is the sigmoid function, used for feedforward training. The final result is:
[0095] Where y is the second data output, w0, w i As the initial weights, For the i-th intersection feature, Let j be the j-th cross feature, n represent the number of the first data entries, p be a preset weight vector, and a be a value. ij The attention score is determined in practice through parameter tuning; v i For the i-th embedding vector, v j Let j be the embedding vector.
[0096] After the entire network is constructed, this application trains the network with features based on the squared loss function until the squared loss function becomes stable, and then extracts the trained features F1, which is the second data.
[0097] S3: Based on the combined feature extraction model and the leaf feature fusion model, process the first data in a loop until the preset number of iterations is reached to obtain the third data.
[0098] In this embodiment, the combined feature extraction model is Gradient Boosting Decision Tree (GDBT), which is used to extract combined features from the first data. The leaf feature fusion model is FatorizationMachine (FM), which is used to fuse the leaf node features of the decision tree of the combined features of the first data.
[0099] In this embodiment, the aforementioned cross-feature extraction model, combined feature extraction model, and leaf feature fusion model are pre-received. The ad click prediction method based on the cross-feature extraction model runs on an electronic device (e.g., Figure 1 The server / terminal device shown can receive preset cross-feature extraction models, combined feature extraction models, and leaf feature fusion models via wired or wireless connections. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultrawideband) connections, and other currently known or future-developed wireless connection methods.
[0100] Specifically, the combined feature extraction model includes multiple decision trees, and the step of iteratively processing the first data based on the combined feature extraction model and the leaf feature fusion model until a preset number of iterations is reached to obtain the third data includes:
[0101] The first data is input into the combined feature extraction model, and the first data is classified by the decision tree;
[0102] Extract the features of the leaf nodes of each decision tree as latent vectors;
[0103] The hidden vectors are input into the leaf feature fusion model for combination until a preset number of iterations is reached to obtain the third data.
[0104] In this embodiment, advertising data, as a type of data containing rich yet sparse features, is used in this patent to construct a hybrid network based on GDBT and FM, enabling parallel training to extract and select combined features. GDBT, as a decision forest, can quickly filter out features with different weights and feature combinations, making it well-suited to the massive and sparse nature of advertising data. The constructed GDBT consists of a strong classifier composed of multiple weak classification decision trees, and gradient descent is implemented based on gradient boosting to achieve the optimal decision forest. Its input is the encoded first data (sparse feature advertising vector).
[0105] This application initializes n-1 decision trees f(x), and the combined decision forest is:
[0106] Where i represents the i-th decision tree, and x represents the first data. The residual function of the constructed GBDT is the derivative of the loss function:
[0107] In the formula, y is the output of the GDBT model, which is called the hidden vector.
[0108] After training with GBDT, features from the leaf nodes of the decision forest are extracted and combined to form a discrete feature vector for the advertisement. Further, an FM model is constructed to further cross-combine features and mine deeper advertising features. The stacked FM network consists of an input layer, a feature combination layer, and an output layer. The input layer uses the latent vectors extracted by GBDT as input. The feature combination layer, based on factorization, performs dot product processing on the latent vectors to cross-feature x. i x j The parameters are not independent, which can effectively achieve quadratic parameter estimation and reduce the sparsity of advertising features. The specific operation of achieving feature cross-combination with different weights is as follows:
[0109] Among them, w0, w i Let v' be the initial weight, b be the bias, <...> denote the implicit vector dot product operation on ij, and v' be the... i Let v' be the latent vector of dimension i. j Let x be the latent vector of dimension j. i For the i-th of the first data, x j Let j be the j-th first data point, and y be the output of the FM network in the formula.
[0110] After the feature combination layer, the feature vector F2 (i.e., the third data) of the hybrid network composed of GDBT and FM is extracted.
[0111] S4: Perform fully connected feature extraction operations on the second data and the third data respectively to obtain the fully connected features of the second data and the fully connected features of the third data.
[0112] In this embodiment, a fully connected feature extraction operation is performed through a fully connected layer to obtain the fully connected features of the second data and the fully connected features of the third data.
[0113] S5: Concatenate the fully connected features of the second data and the fully connected features of the third data to obtain the target features.
[0114] In this embodiment, the fully connected features of the second data and the fully connected features of the third data are concatenated to combine advertising features of different dimensions. The specific operation is as follows:
[0115] Wherein, F1 is the fully connected feature of the second data, F2 is the fully connected feature of the third data, and Fm is the combined feature.
[0116] S6: Input the target features into the pre-trained click prediction model to obtain the ad click prediction result.
[0117] In this embodiment, the click prediction model is Logistic Regression (LR).
[0118] In step S6, the step of inputting the target features into the pre-trained click prediction model to obtain the ad click prediction result includes:
[0119] The ad click prediction result is calculated based on the following formula:
[0120] Where y is the output, representing the ad click prediction result, Fm is the combined feature, w is the preset initial weight value, b is the bias, and T represents the transpose operator.
[0121] In this embodiment, after obtaining the combined features after splicing, a logistic regression model is constructed to obtain the predicted click-through rate of the advertisement.
[0122] This application integrates cross-features of advertising features based on multiple models, effectively mining deep-level features of advertisements from high-dimensional sparse features to achieve higher prediction accuracy. This can effectively improve the personalized recommendation capabilities of online internet advertising platforms. Furthermore, based on a multi-layer network, this application avoids the problem of using a large amount of manpower and resources in traditional advertising prediction methods, thereby reducing the manpower cost of the advertising system.
[0123] In this embodiment, multi-modal fusion is achieved based on Factorization Machine (FM), Attention Factorization Machine (AFM), and Gradient Boosting Decision Tree (GDBT) to improve the predicted click-through rate (CTR) of user ads. First, the original data is processed using one-hot encoding to expand the data dimensionality. Then, high-dimensional sparse feature data is embedded into a low-dimensional vector space based on AFM. Combined with GDBT-FM, the leaf node feature data of GDBT is extracted, and FM performs secondary feature extraction to achieve deep feature extraction. Further, the feature vectors of AFM and GDBT-FM are concatenated using a feature concatenation method. Finally, logistic regression prediction is performed to obtain a high-precision prediction model. This application can effectively improve the prediction of ad CTR.
[0124] It should be emphasized that, in order to further ensure the privacy and security of the aforementioned user advertising behavior data, the aforementioned user advertising behavior data can also be stored in a blockchain node.
[0125] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.
[0126] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.
[0127] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.
[0128] This application can be applied to the field of smart communities, thereby promoting the construction of smart cities.
[0129] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium, and when executed, they can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).
[0130] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0131] Further reference Figure 3 As a response to the above Figure 2 The implementation of the method shown in this application provides an embodiment of an ad click prediction device based on a cross-feature extraction model. This device embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.
[0132] like Figure 3 As shown, the ad click prediction device 300 based on the cross-feature extraction model described in this embodiment includes: a collection module 301, a first extraction module 302, a second extraction module 303, a fully connected feature extraction module 304, a splicing module 305, and a prediction module 306. The system comprises: a collection module 301, used to collect user advertising behavior data and sequentially perform data cleaning and processing operations on the user advertising behavior data to obtain first data; a first extraction module 302, used to iteratively extract cross features of the first data based on a cross feature extraction model until the desired number of iterations is reached to obtain second data; a second extraction module 303, used to iteratively process the first data based on a combined feature extraction model and a leaf feature fusion model until the desired number of iterations is reached to obtain third data; a fully connected feature extraction module 304, used to perform fully connected feature extraction operations on the second data and the third data respectively to obtain fully connected features of the second data and fully connected features of the third data; a splicing module 305, used to splice the fully connected features of the second data and the fully connected features of the third data to obtain target features; and a prediction module 306, used to input the target features into a pre-trained click prediction model to obtain ad click prediction results.
[0133] In this embodiment, the present application can effectively improve the prediction of ad click-through rates. By fusing cross-features of ad features based on a multi-model approach, it effectively mines deep-level features of ads from high-dimensional sparse features to achieve higher prediction accuracy. This can effectively improve the personalized recommendation capabilities of online internet advertising platforms. Furthermore, the multi-model prediction approach avoids the problem of relying on extensive manual analysis in traditional ad prediction methods, thus reducing the labor costs of the advertising system.
[0134] The collection module 301 includes an identification submodule, a deletion submodule, and a processing submodule. The identification submodule identifies key data in the user advertising behavior data and confirms whether the key data is missing. The deletion submodule deletes the corresponding user advertising behavior data when the key data is missing, obtaining cleaned user advertising behavior data. The processing submodule processes the cleaned user advertising behavior data into a sparse feature vector to obtain the first data.
[0135] The user advertising behavior data includes user names. The processing submodule includes a table joining unit and an expansion unit. The table joining unit is used to perform a table joining operation on the user advertising behavior data based on the user names to obtain a wide table, and load the wide table into the preset big data ADS layer. The expansion unit is used to perform a dimension expansion operation on the wide table in the big data ADS layer based on one-hot encoding to obtain the first data.
[0136] The cross-feature extraction model includes a cross-input layer, an embedding layer, a feature cross-layer, and an attention pooling layer. The first extraction module 302 includes an input submodule, a product submodule, and an attention submodule. The input submodule is used to input the first data into the cross-feature extraction model. The embedding layer multiplies the positive features with a value of 1 in the first data with a preset h embedding blocks to obtain an embedding vector. The product submodule is used in the feature cross-layer to perform element-wise intelligent multiplication on the first data based on the embedding vector to obtain cross features. The attention submodule is used to input the cross features into the attention pooling layer to obtain the output second data.
[0137] The attention submodule is also used to perform element-wise intelligent product processing on the first data based on the following formula:
[0138] Where b is the bias, x i For the i-th of the first data, x j For the j-th first data, v i For the i-th embedding vector, v j Let J be the j-th embedding vector. P is a preset weight matrix. The intersection feature is described above.
[0139] The combined feature extraction model includes multiple decision trees. The second extraction module 303 includes a classification submodule, an extraction submodule, and a combination submodule. The classification submodule is used to input the first data into the combined feature extraction model and classify the first data through the decision trees. The extraction submodule is used to extract the features of the leaf nodes of each decision tree as latent vectors. The combination submodule is used to input the latent vectors into the leaf feature fusion model for combination until a preset number of iterations is reached to obtain the third data.
[0140] The prediction module 306 is also used to calculate the ad click prediction result based on the following formula:
[0141] Where y is the ad click prediction result, Fm is the combined feature, w is the preset initial weight value, b is the bias, and T is the transpose operator.
[0142] This application can effectively improve the prediction of ad click-through rates. It achieves the fusion of cross-features of ad characteristics based on multiple models, effectively mining deep-level features from high-dimensional sparse features to achieve higher prediction accuracy. This can effectively improve the personalized recommendation capabilities of online internet advertising platforms. Furthermore, since this application uses multiple models for prediction, it avoids the problem of relying on large amounts of manpower and resources for manual analysis in traditional ad prediction methods, thus reducing the labor costs of the advertising system.
[0143] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.
[0144] The computer device 200 includes a memory 201, a processor 202, and a network interface 203 that are interconnected via a system bus. It should be noted that only the computer device 200 with components 201-203 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.
[0145] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.
[0146] The memory 201 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 201 may be an internal storage unit of the computer device 200, such as the hard disk or memory of the computer device 200. In other embodiments, the memory 201 may also be an external storage device of the computer device 200, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Of course, the memory 201 may also include both internal storage units and external storage devices of the computer device 200. In this embodiment, the memory 201 is typically used to store the operating system and various application software installed on the computer device 200, such as computer-readable instructions for an advertising click prediction method based on a cross-feature extraction model. Furthermore, the memory 201 can also be used to temporarily store various types of data that have been output or will be output.
[0147] In some embodiments, the processor 202 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 202 is typically used to control the overall operation of the computer device 200. In this embodiment, the processor 202 is used to execute computer-readable instructions stored in the memory 201 or to process data, for example, to execute computer-readable instructions for the advertising click prediction method based on the cross-feature extraction model.
[0148] The network interface 203 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 200 and other electronic devices.
[0149] In this embodiment, the cross-feature fusion of advertising features is achieved based on multiple models, effectively mining deep-level features of advertising from high-dimensional sparse features to achieve higher prediction accuracy.
[0150] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the advertising click prediction method based on the cross-feature extraction model described above.
[0151] In this embodiment, the cross-feature fusion of advertising features is achieved based on multiple models, effectively mining deep-level features of advertising from high-dimensional sparse features to achieve higher prediction accuracy.
[0152] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0153] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.
Claims
1. An ad click prediction method based on a cross-feature extraction model, characterized in that, Includes the following steps: Collect user advertising behavior data, and perform data cleaning and data processing operations on the user advertising behavior data in sequence to obtain the first data; The cross-feature extraction model is used to iteratively extract the cross features of the first data until the number of iterations is reached, and then the second data is obtained. The first data is processed iteratively based on the combined feature extraction model and the leaf feature fusion model until the preset number of iterations is reached to obtain the third data. Perform fully connected feature extraction operations on the second data and the third data respectively to obtain the fully connected features of the second data and the fully connected features of the third data; By concatenating the fully connected features of the second data and the fully connected features of the third data, the target features are obtained. The target features are input into a pre-trained click prediction model to obtain the ad click prediction result; The cross-feature extraction model includes a cross-input layer, an embedding layer, a feature cross-layer, and an attention pooling layer. The step of iteratively extracting the cross features of the first data based on the cross-feature extraction model until the required number of iterations is reached to obtain the second data includes: The first data is input into the cross-feature extraction model, and the embedding layer multiplies the positive features with a value of 1 in the first data with a preset h embedding blocks to obtain a k * h dimensional embedding vector. The feature cross layer performs element-wise intelligent multiplication on the first data based on the embedding vector to obtain cross features; The cross features are input into the attention pooling layer to obtain the second data output. The combined feature extraction model includes multiple decision trees. The step of iteratively processing the first data based on the combined feature extraction model and the leaf feature fusion model until a preset number of iterations is reached to obtain the third data includes: The first data is input into the combined feature extraction model, and the first data is classified by the decision tree; Extract the features of the leaf nodes of each decision tree as latent vectors; The hidden vectors are input into the leaf feature fusion model for combination until a preset number of iterations is reached to obtain the third data.
2. The ad click prediction method based on a cross-feature extraction model according to claim 1, characterized in that, The step of sequentially performing data cleaning and data processing operations on the user advertising behavior data to obtain the first data includes: Identify key data in the user advertising behavior data and confirm whether any key data is missing; If the key data is missing, the corresponding user advertising behavior data is deleted to obtain the cleaned user advertising behavior data. The cleaned user advertising behavior data is processed into a sparse feature vector to obtain the first data.
3. The ad click prediction method based on a cross-feature extraction model according to claim 2, characterized in that, The user advertising behavior data includes user names, and the step of processing the cleaned user advertising behavior data into a sparse feature vector to obtain the first data includes: Based on the user name, a join operation is performed on the user advertising behavior data to obtain a wide table, and the wide table is loaded into the preset big data ADS layer; The first data is obtained by performing a dimensional expansion operation on the wide table in the big data ADS layer based on one-hot encoding.
4. The ad click prediction method based on a cross-feature extraction model according to claim 1, characterized in that, The step of performing element-wise intelligent multiplication on the first data based on the embedding vector to obtain cross features includes: The first data is processed using element-wise intelligent multiplication based on the following formula: Where b is the bias. For the i-th of the first data, For the j-th first data, For the i-th embedding vector, Let J be the j-th embedding vector, and P be a preset weight matrix. The intersection feature is described above.
5. The ad click prediction method based on a cross-feature extraction model according to claim 1, characterized in that, The step of inputting the target features into a pre-trained click prediction model to obtain the ad click prediction result includes: The ad click prediction result is calculated based on the following formula: Where y is the ad click prediction result, Fm is the combined feature, w is the preset initial weight value, b is the bias, and T is the transpose operator.
6. An ad click prediction device based on a cross-feature extraction model, characterized in that, include: The collection module is used to collect user advertising behavior data, and to perform data cleaning and data processing operations on the user advertising behavior data in sequence to obtain first data. The first extraction module is used to iteratively extract the cross features of the first data based on the cross feature extraction model until the number of iterations is reached to obtain the second data. The second extraction module is used to process the first data cyclically based on the combined feature extraction model and the leaf feature fusion model until a preset number of iterations is reached to obtain the third data. The fully connected feature extraction module is used to perform fully connected feature extraction operations on the second data and the third data respectively to obtain the fully connected features of the second data and the fully connected features of the third data. The splicing module is used to splice the fully connected features of the second data and the fully connected features of the third data to obtain the target features; The prediction module is used to input the target features into a pre-trained click prediction model to obtain the ad click prediction result; The cross-feature extraction model includes a cross-input layer, an embedding layer, a feature cross-layer, and an attention pooling layer. The first extraction module includes an input submodule, a product submodule, and an attention submodule. The input submodule is used to input the first data into the cross-feature extraction model. The embedding layer multiplies the positive features with a value of 1 in the first data with a preset h embedding blocks to obtain a k * h dimensional embedding vector. The product submodule is used in the feature cross-layer to perform element-wise intelligent multiplication on the first data based on the embedding vector to obtain cross features. The attention submodule is used to input the cross features into the attention pooling layer to obtain the output second data. The combined feature extraction model includes multiple decision trees, and the second extraction module includes a classification submodule, an extraction submodule, and a combination submodule. The classification submodule is used to input the first data into the combined feature extraction model and classify the first data through the decision trees. The extraction submodule is used to extract the features of the leaf nodes of each decision tree as latent vectors. The combination submodule is used to input the latent vectors into the leaf feature fusion model for combination until a preset number of iterations is reached to obtain the third data.
7. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the advertising click prediction method based on the cross-feature extraction model as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the advertising click prediction method based on a cross-feature extraction model as described in any one of claims 1 to 5.