A click-through rate prediction method based on hybrid quantum-classical ultra-deep factorization machine

Abstract

The present invention discloses a click-through rate prediction method based on a hybrid quantum classical deep factorization machine, comprising the following steps: (1) obtaining an open source data set and preprocessing it, and dividing it into a training set and a test set; (2) constructing a new hybrid quantum classical deep factorization machine HQCxDeepFM model, and training it using the preprocessed data set; (3) using a sigmoid activation function to obtain the final prediction result; the present invention improves the computing power of the CTR model and reduces the model complexity, while improving the CTR prediction performance.

Description

Technical Field

[0001] The present invention relates to the field of quantum machine learning technology, and in particular to a click rate prediction method based on a hybrid quantum classical ultra-deep factorization machine. Background Art

[0002] The recommendation system models users by analyzing their historical behaviors, improving the accuracy and personalization of information recommendations to users. Click-Through Rate (CTR) prediction is an important task of the recommendation system. The traditional CTR model has gone through a development process from classical machine learning to deep learning, and then to the introduction of the self-attention mechanism. As an emerging computing paradigm, quantum computing has advantages in optimizing machine learning tasks, especially when dealing with high-dimensional data and complex optimization problems. Quantum computing uses quantum bits for information processing and can provide exponential acceleration in certain tasks. The core features of quantum computing, such as quantum superposition and quantum entanglement, make it naturally parallel and efficient when processing large-scale data. It is possible and feasible to integrate the advantages of quantum computing into the CTR model.

[0003] However, the traditional CTR model has gone through a development process from classical machine learning to deep learning, and then to the introduction of the self-attention mechanism. In this evolving research process, as the amount of data and computing power increases, the traditional CTR model faces performance bottlenecks. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a click-through rate prediction method based on a hybrid quantum classical ultra-deep factorization machine, to solve the computational bottleneck faced by the traditional xDeepFM model in high-dimensional data processing and large-scale data training, and to improve the computational efficiency and accuracy of the click-through rate (CTR) prediction task.

[0005] Technical solution: The click rate prediction method based on a hybrid quantum classical ultra-deep factorization machine described in the present invention comprises the following steps:

[0006] (1) Obtain open source datasets and preprocess them to divide them into training sets and test sets;

[0007] (2) Construct a new hybrid quantum classical deep factorization machine HQCxDeepFM model and train it using the preprocessed dataset;

[0008] (3) Use the sigmoid activation function to obtain the final estimation result.

[0009] Furthermore, step (1) is as follows: using two open benchmark datasets, Criteo and Avazu; setting a frequency threshold for data records; filtering out records below the threshold; for categorical features, using one-hot encoding to convert attributes into digital features; and for continuous features, performing normalization.

[0010] Furthermore, in step (2), the HQCxDeepFM model includes: a quantum compression cross network QCIN, a quantum deep neural network QDNN and a quantum linear model Qlinear; the quantum compression cross network is used to process dense feature vectors to learn explicit high-order feature cross-relationships, and a hybrid angle coding scheme is used to map high-dimensional data to lower-dimensional quantum states to achieve data dimensionality reduction. At the same time, a variational quantum circuit is designed to construct a quantum one-dimensional convolution to perform feature compression on a three-dimensional feature matrix; the quantum deep neural network consists of multiple hidden layers, and is used to process dense feature vectors to learn implicit high-order feature cross-relationships. A hybrid angle coding scheme is used to use multi-layer variational layered variational quantum circuits to quantize traditional deep neural networks; the quantum linear model uses a hybrid angle coding scheme and a single-layer variational quantum circuit to reconstruct the linear model; it is used to use the original features for linear combination, process sparse features, and learn low-order feature linear cross-relationships.

[0011] Furthermore, the specific processing process of the HQCxDeepFM model is as follows: the low-dimensional representation of the data set is obtained through the fully connected neural network of the embedding layer, that is, the sparse features in the data set are mapped to the continuous vector space to form a dense feature vector of fixed length; the obtained fixed-length dense feature vector is input into the quantum compression cross network and the quantum deep neural network for processing, and the original input features are input into the quantum linear model for processing.

[0012] Furthermore, step (3) is as follows: superimpose the outputs of the three modules QCIN, QDNN and QLinear, and obtain the final estimation result through the sigmoid activation function.

[0013] The click rate prediction system based on a hybrid quantum classical ultra-deep factorization machine described in the present invention comprises:

[0014] Data module: used to obtain open source data sets and preprocess them, and divide them into training sets and test sets;

[0015] HQCxDeepFM module: used to build a new hybrid quantum classical deep factorization machine HQCxDeepFM model and train it using the preprocessed dataset;

[0016] Estimation module: used to obtain the final estimation result using the sigmoid activation function.

[0017] Furthermore, in the data module, the details are as follows: two open benchmark datasets, Criteo and Avazu, are used; a frequency threshold for data records is set; records below the threshold are filtered out; for categorical features, one-hot encoding is used to convert attributes into digital features; for continuous features, normalization is performed.

[0018] Furthermore, in the HQCxDeepFM module, the HQCxDeepFM model includes: quantum compression cross network QCIN, quantum deep neural network QDNN and quantum linear model Qlinear; the quantum compression cross network is used to process dense feature vectors to learn explicit high-order feature cross-relationships, and a hybrid angle coding scheme is used to map high-dimensional data to lower-dimensional quantum states to achieve data dimensionality reduction. At the same time, a variational quantum circuit is designed to construct a quantum one-dimensional convolution to perform feature compression on the three-dimensional feature matrix; the quantum deep neural network consists of multiple hidden layers, which is used to process dense feature vectors to learn implicit high-order feature cross-relationships, and a hybrid angle coding scheme is used to use multi-layer variational layered variational quantum circuits to quantize traditional deep neural networks; the quantum linear model adopts a hybrid angle coding scheme and uses a single-layer variational quantum circuit to reconstruct the linear model; it is used to use the original features for linear combination, process sparse features, and learn low-order feature linear cross-relationships.

[0019] Furthermore, in the HQCxDeepFM module, the specific processing process of the HQCxDeepFM model is as follows: the low-dimensional representation of the data set is obtained through the fully connected neural network of the embedding layer, that is, the sparse features in the data set are mapped to the continuous vector space to form a dense feature vector of fixed length; the obtained fixed-length dense feature vector is input into the quantum compression cross network and the quantum deep neural network for processing respectively, and the original input features are input into the quantum linear model for processing.

[0020] Furthermore, in the estimation module, the details are as follows: the outputs of the three modules, QCIN, QDNN and QLinear, are superimposed, and the final estimation result is obtained through the sigmoid activation function.

[0021] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: HQCxDeepFM, which integrates the advantages of quantum computing, uses the hybrid angle encoding scheme (HAE) to map high-dimensional data to lower-dimensional quantum states to achieve data dimensionality reduction, and designs variational quantum circuits to construct QCIN, QDNN and QLinear, using the advantages of quantum computing to improve the click-through rate prediction performance of HQCxDeepFM. Compared with the performance of traditional xDeepFM, the computing power of the CTR model is improved and the model complexity is reduced, while the CTR prediction performance is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of the present invention;

[0023] Figure 2 This is the overall structure diagram of the hybrid quantum classical deep factorization machine HQCxDeepFM of the present invention;

[0024] Figure 3 It is a diagram of the variational quantum circuit structure and parameter optimization process of the quantum one-dimensional convolution of the present invention;

[0025] Figure 4 It is a schematic diagram of the variational quantum circuit structure of the quantum deep neural network QDNN of the present invention;

[0026] Figure 5 It is a schematic diagram of the quantum circuit structure of the quantum linear model QLinear of the present invention. DETAILED DESCRIPTION

[0027] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0028] like Figure 1 As shown, the embodiment of the present invention provides a click rate prediction method based on a hybrid quantum classical ultra-deep factorization machine, comprising the following steps:

[0029] (1) Obtain open source datasets and preprocess them, and divide them into training sets and test sets; specifically: Two open benchmark datasets, Criteo and Avazu, were used for experiments. They are two commonly used datasets for advertising click-through rate prediction. The frequency thresholds of data records were set to 10 and 5, and records below the thresholds were filtered out. For categorical features, one-hot encoding was used to convert attributes into digital features; for continuous features, normalization was performed to reduce the gap between features. Since the experiments were conducted in a quantum simulator and the actual training time was long, 500,000 data samples were selected from each dataset, the positive sample ratio was maintained, and the data were divided into training sets and test sets in a ratio of 8:2.

[0030] (2) Construct a new hybrid quantum classical deep factorization machine HQCxDeepFM model and train it using the preprocessed dataset; Figure 2 As shown in the figure, the overall structure of the hybrid quantum classical deep factorization machine (HQCxDeepFM) is constructed. The present invention will construct a quantum compression intersection network (QCIN), a quantum deep neural network (QDNN) and a quantum linear model (QLinear), and combine them into a hybrid quantum classical algorithm model - HQCxDeepFM, which includes the following steps:

[0031] (21) Figure 2As shown, the fully connected neural network of the embedding layer obtains a low-dimensional representation of the input features, that is, the sparse features in the input features are mapped to the continuous vector space to form a dense feature vector of fixed length for subsequent processing in the model.

[0032] (22) The obtained fixed-length dense feature vectors are input into the quantum compressed intersection network (QCIN) and quantum deep neural network (QDNN), respectively, and the original input features are input into the quantum linear model (QLinear).

[0033] (23) Construct a quantum compressed intersection network (QCIN). QCIN mainly consists of two parts: feature intersection and feature compression. The feature intersection part is a classical calculation, and the D-dimensional original feature matrix composed of m feature domains The output matrix of the hidden layer After performing the Hadamard product, a three-dimensional feature matrix is ​​generated, where the h-th row feature vector of the k-th layer output matrix is The calculation formula is as follows:

[0034] ;

[0035] in, represents the i-th row vector of the output matrix of the k-1th layer, represents the eigenvector of the jth row in the original feature matrix, Denotes the parameter matrix of the h-th eigenvector, denoting the Hadamard product.

[0036] The feature compression part uses quantum computing and variational quantum circuit (VQC) to compress the feature. For the three-dimensional feature matrix obtained by feature crossover, the three-dimensional feature matrix is ​​compressed into a two-dimensional feature matrix through convolution operation. Figure 3 The figure shows the variational quantum circuit structure of quantum one-dimensional convolution constructed by the present invention and its parameter optimization process. It is the coding layer, which uses Hybrid Angle Encoding (HAE) to encode classical data. Hybrid Angle Encoding can encode with n qubits and b qubit blocks. dimensional classical data as quantum states :

[0037] ;

[0038] in, , represents the kth qubit block elements, represents the i-th computational basis state of the k-th quantum bit block.

[0039] First, the 64-dimensional feature vector output by the embedding layer is compressed to 15 dimensions using linear transformation. HAE then uses 1 quantum bit block and 4 quantum bits to encode the quantum state, and uses 16 quantum gate combinations to form the variational layer of the quantum circuit, with a total of 8 training parameters.

[0040] For the variational quantum circuit of quantum one-dimensional convolution, the double-qubit CNOT gate is used to realize the mutual entanglement of qubits to realize quantum parallel computing. The classical optimizer Adam is used to achieve the best expression effect of the variational quantum circuit through multiple iterations. After the quantum circuit variation layer evolves the initial information quantum state, the obtained characteristic information is retained in the quantum state, and the quantum state is measured to make the quantum state collapse to a specific eigenstate, and an observable measurement value is obtained. The measurement value is expanded and output using a linear transformation, and finally the three-dimensional feature matrix is ​​compressed into a two-dimensional feature matrix by convolution, and finally the final output of the QCIN module is obtained through summing and pooling.

[0041] (24) Constructing quantum deep neural networks (QDNNs), mainly using multi-layer variational quantum circuits, such as Figure 4 The figure shows the variational quantum circuit structure of QDNN. It is the encoding layer, and hybrid angle encoding (HAE) is used to encode classical data. First, the 64-dimensional feature vector output by the embedding layer is compressed to 15 dimensions using linear transformation. HAE then uses 1 quantum bit block and 4 quantum bits for quantum state encoding.

[0042] The middle part of QDNN is composed of multiple variation layers, each of which The quantum circuit structures are the same; the specific quantum circuit structures are as follows Figure 4 As shown in the blue dotted box, since the model has the highest ability to learn implicit high-order cross features when the network depth of the deep neural network in xDeepFM is set to 3 layers, the number of variational layers n of the quantum deep neural network is set to 3. Cphase gate parameters Corresponding to the weight parameters in the classic deep neural network, the parameters in the training process They do not interfere with each other, and the classic optimizer Adam is used to achieve the best effect of QDNN learning implicit high-order cross features.

[0043] In the original DNN, each hidden layer neuron is connected to the next hidden layer neuron to form a connection relationship between layers. In order to simulate this connection mode, the output of the previous hidden layer is passed to the next hidden layer through the weight matrix for further feature extraction and representation learning. Therefore, in the QDNN quantum circuit of the present invention, a dual quantum bit gate Cphase gate is used to connect with other circuits to form a fully connected quantum bit topology structure, so as to realize the mutual entanglement between quantum bits.

[0044] In addition, since the Hadamard gate can transform a single quantum pure state into a superposition state, the Hadamard gate is executed at the end of the variation layer to complete the weighted sum addition operation between the hidden layers of the quantum deep neural network. The combination of the Cphase gate and the Hadamard gate realizes the calculation task of linear transformation, and the single quantum bit rotation gate The computational task of rotating the quantum state around the x-axis to achieve nonlinear transformation.

[0045] Finally, each quantum circuit in the QDNN is measured and the expected value is taken for classical post-processing. In the classical post-processing part, a classical fully connected layer is used to perform a linear transformation on the four expected values ​​obtained by measurement, and then the normalization operation BatchNorm1d, the activation function ReLU and the regularization Dropout are performed in sequence. The feature vector after classical post-processing is used as the final output of the QDNN.

[0046] (25) Construct a quantum linear model (QLinear), such as Figure 5 As shown in the figure, it is the quantum circuit structure diagram in QLinear. A single-layer variational quantum circuit similar to the circuit structure in QDNN is used to reconstruct the linear model to enhance the linear expression ability of the model. It is the encoding layer, which also uses hybrid angle encoding (HAE) to encode classical data. The middle variational layer is composed of 12 dual-qubit Cphase gates and 4 single-qubit Hadamard gates to realize the calculation task of linear transformation. Finally, each quantum line is measured and the expected value is obtained. The measurement result is nonlinearly mapped through the activation function ReLU and used as the final output of the QLinear module.

[0047] (3) Use the sigmoid activation function to obtain the final estimation result. Specifically, the outputs of the three modules, QCIN, QDNN, and QLinear, are superimposed, and the final estimation result is obtained through the sigmoid activation function.

[0048] Experimental results:

[0049] For the Criteo and Avazu datasets, the dimension of the embedding vector is set to 16, the batch size of the training dataset is set to 1024, and for the classical computing part of xDeepFM and HQCxDeepFM, the Dropout value is set to 0.5 and the L2 regularization is set to 0.0001. The number of hidden layers of the DNN and the number of layers of the QCIN network are both set to 3, and the number of neurons in each layer is set to 64. In the quantum computing part, the variational quantum circuits in QCIN, QDNN, and QLinear all use 4 qubits, the number of variational layers of the quantum convolution circuit in QCIN is set to 2, and the quantum circuit depths of QDNN and QLinear are set to 3 and 1, respectively. In model training, the Adam optimizer is used to optimize the parameters, the learning rate is set to 0.001, and in order to avoid overfitting in training, the early stopping strategy is used, the number of training epochs is set to 5, and the loss function uses BCEWithLogitsLoss.

[0050] The experimental evaluation indicators are AUC and Logloss. In addition, in order to distinguish the combinations of different modules, the CIN + DNN + QLinear module combination is recorded as HQCxDeepFM1, the QCIN + DNN + Linear module combination is recorded as HQCxDeepFM2, and the CIN + QDNN + Linear module combination is recorded as HQCxDeepFM3.

[0051] Table 1 shows the performance results of different module combinations. Whether a single quantum module is replaced or all quantum modules are replaced, the prediction performance of HQCxDeepFM of the present invention is higher than that of xDeepFM, and the performance advantage is reflected on both real data sets, especially the performance improvement of HQCxDeepFM with the combination of QCIN + QNN + QLinear is the largest. Compared with xDeepFM, HQCxDeepFM improves AUC by 0.775% and 0.725% on Criteo and Avazu data sets, respectively, and Logloss decreases by 0.784% and 0.536%, respectively. Therefore, the HQCxDeepFM model of the present invention can improve the click-through rate prediction performance of the model to a certain extent.

[0052] Table 1 Performance comparison between HQCxDeepFM and xDeepFM

[0053] ;

[0054] The HQCxDeepFM model of the present invention is to improve model performance and model computing power on the one hand, and to reduce the model parameter space complexity on the other hand. During the experiment, the present invention calculates and saves the number of training parameters of the model. Since HAE is used to encode high-dimensional features, the number of quantum bits required is relatively small, and the training parameters in the variational quantum circuit will also be reduced, which has a more obvious effect on reducing the model parameter space complexity. Table 2 shows the number of training parameters of HQCxDeepFM and xDeepFM. Compared with xDeepFM, the parameter space complexity of HQCxDeepFM is reduced by 4.7% to 16.8%. This result proves that the hybrid quantum classical algorithm HQCxDeepFM has improved in reducing the model parameter space complexity.

[0055] Table 2 Comparison of the number of training parameters between HQCxDeepFM and xDeepFM

[0056] .

Claims

1. A click-through rate prediction method based on a hybrid quantum classical ultra-deep factorization machine, characterized in that: The following steps are involved: (1) Obtain open source datasets and preprocess them to divide them into training sets and test sets. Specifically, we use two open benchmark datasets, Criteo and Avazu, for experiments. These are two commonly used datasets for ad click-through rate prediction. (2) Construct a new hybrid quantum classical deep factorization machine HQCxDeepFM model and train it using the preprocessed data set; the HQCxDeepFM model includes: quantum compression cross network QCIN, quantum deep neural network QDNN and quantum linear model Qlinear; the quantum compression cross network is used to process dense feature vectors to learn explicit high-order feature cross relationships, and a hybrid angle coding scheme is used to map high-dimensional data to lower-dimensional quantum states to achieve data dimensionality reduction. At the same time, a variational quantum circuit is designed to construct a quantum one-dimensional convolution and perform feature compression on the three-dimensional feature matrix; the quantum deep neural network consists of multiple hidden layers and is used to process dense feature vectors to learn implicit high-order feature cross relationships. A hybrid angle coding scheme is used to quantize traditional deep neural networks using multi-layer variational quantum circuits; the quantum linear model uses a hybrid angle coding scheme and a single-layer variational quantum circuit to reconstruct the linear model; it is used to use the original features for linear combination, process sparse features, and learn low-order feature linear cross relationships; (3) Use the sigmoid activation function to obtain the final estimation result.

2. The click-through rate prediction method based on a hybrid quantum classical ultra-deep factorization machine according to claim 1, characterized in that: Step (1) is as follows: use two open benchmark datasets, Criteo and Avazu; set a frequency threshold for data records; filter out records below the threshold; for categorical features, use one-hot encoding to convert attributes into digital features; for continuous features, perform normalization.

3. The click-through rate prediction method based on a hybrid quantum classical ultra-deep factorization machine according to claim 1 is characterized in that: The specific processing process of the HQCxDeepFM model is as follows: the low-dimensional representation of the data set is obtained through the fully connected neural network of the embedding layer, that is, the sparse features in the data set are mapped to the continuous vector space to form a dense feature vector of fixed length; the obtained fixed-length dense feature vector is input into the quantum compression cross network and the quantum deep neural network for processing respectively, and the original input features are input into the quantum linear model for processing.

4. The click-through rate prediction method based on a hybrid quantum classical ultra-deep factorization machine according to claim 3 is characterized in that: Step (3) is as follows: superimpose the outputs of the three modules QCIN, QDNN and QLinear, and obtain the final estimation result through the sigmoid activation function.

5. A click-through rate prediction system based on a hybrid quantum classical ultra-deep factorization machine, characterized in that: include: Data module: used to obtain open source data sets and preprocess them, and divide them into training sets and test sets. Specifically, two open benchmark data sets, Criteo and Avazu, are used for experiments. They are two commonly used data sets for ad click-through rate prediction. HQCxDeepFM module: used to build a new hybrid quantum classical deep factorization machine HQCxDeepFM model and train it using the preprocessed dataset; The HQCxDeepFM model includes: quantum compression cross network QCIN, quantum deep neural network QDNN and quantum linear model Qlinear; the quantum compression cross network is used to process dense feature vectors to learn explicit high-order feature cross relationships, and adopts a hybrid angle coding scheme to map high-dimensional data to lower-dimensional quantum states to achieve data dimensionality reduction. At the same time, a variational quantum circuit is designed to construct a quantum one-dimensional convolution and perform feature compression on the three-dimensional feature matrix; the quantum deep neural network consists of multiple hidden layers, which is used to process dense feature vectors to learn implicit high-order feature cross relationships, and adopts a hybrid angle coding scheme to use multi-layer variational layered variational quantum circuits to quantize traditional deep neural networks; the quantum linear model adopts a hybrid angle coding scheme and uses a single-layer variational quantum circuit to reconstruct the linear model; it is used to use the original features for linear combination, process sparse features, and learn low-order feature linear cross relationships; Estimation module: used to obtain the final estimation result using the sigmoid activation function.

6. A click-through rate prediction system based on a hybrid quantum classical ultra-deep factorization machine according to claim 5, characterized in that: In the data module, the details are as follows: use two open benchmark datasets, Criteo and Avazu; set the frequency threshold of data records; filter out records below the threshold; for categorical features, use one-hot encoding to convert attributes into digital features; for continuous features, perform normalization.

7. The click-through rate prediction system based on the hybrid quantum classical ultra-deep factorization machine according to claim 5, characterized in that: In the HQCxDeepFM module, the specific processing process of the HQCxDeepFM model is as follows: the low-dimensional representation of the data set is obtained through the fully connected neural network of the embedding layer, that is, the sparse features in the data set are mapped to the continuous vector space to form a dense feature vector of fixed length; the obtained fixed-length dense feature vector is input into the quantum compression cross network and the quantum deep neural network for processing respectively, and the original input features are input into the quantum linear model for processing.

8. The click-through rate prediction system based on the hybrid quantum classical ultra-deep factorization machine according to claim 7, characterized in that: In the estimation module, the details are as follows: superimpose the outputs of the three modules QCIN, QDNN and QLinear, and obtain the final estimation result through the sigmoid activation function.

Images