A deep learning-driven customer behavior prediction model

Through non-negative matrix decomposition and autoencoder network, combined with Hadamma product fusion hidden vectors, the problems of dynamic changes in user interests and data sparseness in the existing technology are solved, and more accurate customer behavior prediction and reduced computing resource requirements are achieved.

CN118096237BActive Publication Date: 2024-11-29北京嘉华铭品牌策划有限公司
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410263063.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-08
Publication Date
2024-11-29
Estimated Expiration
2044-03-08

AI Technical Summary

Technical Problem

Existing customer behavior prediction methods are difficult to effectively capture the dynamic changes in user interests, and face challenges in dealing with high-dimensional, large-scale, and sparse user-product interaction data, and deep learning models are difficult to play their advantages in the absence of labeled data.

Method used

Non-negative matrix decomposition and autoencoder network are used to construct user and temporal feature matrices through interactive matrix decomposition, learn common hidden vectors of user and timing features, and combine Hadamama product fusion hidden vectors to predict user behavior.

Benefits of technology

Smear data sparsity problem while maintaining data non-negativeness, improve the accuracy of prediction and the richness of feature representation, and reduce the demand for computing resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118096237B_ABST
    Figure CN118096237B_ABST
Patent Text Reader

Abstract

The present invention discloses a deep learning driven customer behavior prediction model, comprising: step 1, constructing an interaction matrix; step 2, performing matrix decomposition on the interaction matrix; step 3, constructing two feature matrices based on the interaction submatrix results of non-negative matrix decomposition; step 4, compressing and learning the latent vector representation of the user behavior sequence based on an autoencoder network; step 5, using the joint latent vector learned by the autoencoder network as input to obtain the behavior prediction of the downstream task. The present invention utilizes time series and user behavior data, combined with a deep learning method, so that the prediction model can more accurately capture user interests and behavior patterns, thereby improving the accuracy of the prediction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular to a deep learning driven customer behavior prediction model. Background Art

[0002] With the rapid development of information technology and big data analysis technology, customer behavior prediction has become increasingly important in e-commerce, financial analysis, marketing and other fields. Traditional customer behavior prediction methods rely on statistical analysis and machine learning techniques, such as logistic regression, decision trees and support vector machines, which are often based on static user characteristics and lack the ability to dynamically capture the evolution of user interests over time.

[0003] In the existing technology, although some studies have tried to track the dynamic changes of user behavior by introducing time series analysis, these methods still face challenges when dealing with high-dimensional, large-scale and sparse user-product interaction data. In addition, existing time series analysis methods usually require complex data preprocessing and feature engineering, which not only increases the complexity of the model, but also limits the generalization ability of the model.

[0004] Deep learning methods can automatically extract high-level features from user behavior, but in practical applications, deep learning models require a large amount of labeled data for training. In many practical scenarios, especially when new products are launched, the available historical interaction data is very limited, which makes it difficult for deep learning models to play their greatest advantages.

[0005] Therefore, how to design a prediction model that can fully utilize user behavior data while adapting to data sparsity and dynamic changes, and can calculate current customer behavior predictions with as few computing resources as possible has become a technical problem that urgently needs to be solved in the current customer behavior prediction field. Summary of the invention

[0006] In view of the defects in the above-mentioned prior art, the present invention provides a deep learning driven customer behavior prediction method, the method comprising:

[0007] Step 1: Obtain historical interaction data such as the number of purchases, views, and ratings of products by users during the data collection phase to construct an interaction matrix;

[0008] Step 2, performing matrix decomposition on the interaction matrix, converting the three-dimensional interaction matrix into a two-dimensional matrix, and applying a non-negative matrix decomposition method to decompose the two-dimensional matrix into two non-negative matrices;

[0009] Step 3, construct two feature matrices based on the interaction submatrix results of non-negative matrix decomposition, and construct the user feature matrix W' and the time feature matrix H' respectively;

[0010] Step 4: compress and learn the latent vector representation of the user behavior sequence based on the autoencoder network, including learning the common latent vector of the user and time series features based on the joint autoencoder of the user feature matrix W' and the time feature matrix H';

[0011] Among them, the latent vector fusion is fused through the Hadamard product;

[0012] Step 5: Use the autoencoder network to learn the joint latent vector as input to obtain behavior predictions for downstream tasks.

[0013] The step 1 specifically includes initializing a three-dimensional matrix M, whose dimensions are determined by the number of products I, the number of features F, and the number of time points T;

[0014] During the matrix initialization phase, all element values ​​are set to zero;

[0015] Afterwards, the matrix is ​​filled in based on the collected data, corresponding to the values ​​of the specific product’s features such as price, number of views, number of purchases, and ratings at different points in time.

[0016] Wherein, the step 2 also includes constructing a decomposed interaction sub-matrix for each user based on the optimization target after the attenuation factor and the confidence adjustment during the interaction matrix decomposition process.

[0017] The characteristic matrix W′ in step 3 is the W matrix directly obtained from non-negative matrix decomposition, and the characteristic matrix H′ is constructed by transposing the H matrix obtained from non-negative matrix decomposition.

[0018] Wherein, the goal of the downstream task in step 5 includes predicting the number of purchases of the user;

[0019] And, use the latent vector Z as the input feature of the linear regression model to predict the number of purchases.

[0020] Here, a three-dimensional matrix M is created, whose dimensions are the number of products I, the number of features F, and the number of time points T, and each element in the three-dimensional matrix is ​​initially set to 0;

[0021] Where i∈I, f∈F and t∈T, i∈{1, 2, ..., I}: product index; f∈{1, 2, 3, 4}: feature index, corresponding to price, number of views, number of purchases, and ratings respectively; t∈{1, 2, ..., T}: time index; M[i][f][t]: element of the three-dimensional matrix, representing the value of the fth feature of the i-th product at time t;

[0022] Fill the three-dimensional matrix M of size I×F×T with m filling elements. ift express;

[0023] For each product i, feature f, and time point t, fill in the corresponding elements in the matrix:

[0024] m i1t : The price of commodity i at time t;

[0025] m i2t : Number of views of product i at time t;

[0026] m i3t : The number of times product i is purchased at time t;

[0027] m i4t : The rating of product i at time t.

[0028] Where, for time point t, the attenuation factor δ(t) can be defined as: δ(t) = e -λ(T-t) ;

[0029] in:

[0030] λ is the decay rate parameter, which determines how fast the interest decays over time;

[0031] T is the most recent time point, used to normalize time decay;

[0032] t is the current time point;

[0033] Among them, for product i, feature f and time point t, the confidence weight ci based on user interaction f t

[0034] Defined as: c if t=1+α·r ift ,

[0035] in:

[0036] r ift is the user's rating of feature f of product i at time t;

[0037] α is a hyperparameter used to adjust the influence of confidence weights.

[0038] Among them, based on the optimization objective after the attenuation factor and confidence adjustment, a decomposed interaction submatrix is ​​constructed for each user, and the optimization objective function is specifically:

[0039]

[0040] in:

[0041] m ift is the value of feature f of product i at time t in the original three-dimensional matrix M;

[0042] [W.H] ift are the corresponding elements in the approximation matrix WH;

[0043] c ift is the confidence weighting;

[0044] δ(t) is the time decay factor.

[0045] Wherein, the step 4 includes defining the autoencoder network structure, determining the hidden layer dimension, determining the loss function, training the autoencoder, and extracting the hidden vector after the training is completed;

[0046] The feature matrix is ​​input, the latent vector is generated by the encoder, and then the output is reconstructed by the decoder, the loss is calculated, and the network weights are updated through back propagation to train the autoencoder.

[0047] The present invention adopts a non-negative matrix decomposition method, which can decompose the original sparse matrix into a low-dimensional feature matrix while maintaining the non-negativity of the data, effectively alleviating the data sparsity problem. By learning the latent vector representation of the user behavior sequence through the autoencoder network, the present invention can learn the common latent vector of the user and time series features, making the feature representation richer and more accurate.

[0048] The present invention utilizes time series and user behavior data, combined with deep learning methods, so that the prediction model can more accurately capture user interests and behavior patterns, thereby improving the accuracy of the prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0050] Figure 1 is a flow chart illustrating a deep learning driven customer behavior prediction model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two.

[0053] It should be understood that although the terms first, second, third, etc. may be used to describe ... in the embodiments of the present invention, these ... should not be limited to these terms. These terms are only used to distinguish .... For example, without departing from the scope of the embodiments of the present invention, the first ... may also be referred to as the second ..., and similarly, the second ... may also be referred to as the first ....

[0054] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0055] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0056] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the commodity or device including the elements.

[0057] Traditional user behavior prediction methods often ignore the dynamics of user interests over time, and user-item interaction data is usually sparse, which brings challenges to the prediction task. In traditional methods, user and item features are often processed independently, ignoring the possible correlation between features.

[0058] like Figure 1 As shown, the present invention proposes a deep learning driven customer behavior prediction method, the method comprising:

[0059] Step 1: Obtain the historical interaction data of users on the number of purchases, views, and ratings of the products through the data collection phase to construct the interaction matrix. Specifically, it includes initializing a three-dimensional matrix M, whose dimensions are determined by the number of products I, the number of features F, and the number of time points T. In the matrix initialization phase, all element values ​​are set to zero. Afterwards, fill the matrix according to the collected data, corresponding to the values ​​of the price, views, purchases, and ratings of the specific products at different time points.

[0060] Step 2, matrix decomposition of the interaction matrix, which takes into account the time factor and user interaction confidence to more accurately reflect the changes in user interests. In the process of interaction matrix decomposition, a decomposed interaction sub-matrix is ​​constructed for each user based on the optimization target adjusted by the attenuation factor and confidence to find the matrix decomposition that best represents the user interaction data.

[0061] Furthermore, after converting the three-dimensional interaction data matrix (user × product feature × time) into a two-dimensional matrix, the non-negative matrix factorization method is applied to decompose the two-dimensional matrix into two smaller non-negative matrices, which respectively capture the implicit relationship between user and product features.

[0062] Step 3: construct a feature matrix based on the result of non-negative matrix decomposition, and construct a user feature matrix W' and a time feature matrix H' respectively.

[0063] The feature matrix W' is the W matrix obtained directly from non-negative matrix factorization.

[0064] The feature matrix H' is constructed by transposing the H matrix obtained by non-negative matrix factorization.

[0065] Step 4: compress and learn the latent vector representation of the user behavior sequence based on the autoencoder network, including learning the common latent vector of user and time series features based on the joint autoencoder of the user feature matrix W' and the time feature matrix H'.

[0066] Among them, the latent vector fusion is carried out through the Hadamard product.

[0067] Step 5, using the autoencoder network to learn the joint latent vector as input to obtain the behavior prediction of the downstream task. Herein, a behavior prediction model for the downstream task is trained. Herein, the goal of the downstream task includes predicting the number of purchases of the user.

[0068] The present invention adopts a non-negative matrix decomposition method, which can decompose the original sparse matrix into a low-dimensional feature matrix while maintaining the non-negativity of the data, effectively alleviating the data sparsity problem. By learning the latent vector representation of the user behavior sequence through the autoencoder network, the present invention can learn the common latent vector of the user and time series features, making the feature representation richer and more accurate.

[0069] In one embodiment, in step 1, the user's historical interaction data needs to be collected first in constructing the interaction matrix, including the number of purchases, views and ratings of each product by a certain user. After the matrix is ​​initialized, the matrix is ​​filled based on the above data.

[0070] Collect data from internal systems using database queries or API calls. If the data is stored in a relational database (such as MySQL, PostgreSQL, SQL Server, etc.), write SQL queries to extract product prices, views, purchases, and ratings at different points in time. If the system provides an API interface, you can get the data through API calls.

[0071] Set up scheduled tasks (such as cron jobs) to execute data extraction scripts regularly. And set up event-driven, including triggering data extraction through event hooks (webhooks) or message queues (such as RabbitMQ, Kafka, etc.) when product information changes, users browse or purchase.

[0072] Store the cleaned data in a suitable storage system, including inserting the data into a relational or non-relational database table and designing the table structure to optimize query performance. Alternatively, use data warehouse technology to store and manage the data.

[0073] In one embodiment, assume that there are I products and T time points. Features F include price, browse count, purchase count, and rating. The size of the three-dimensional matrix M will be I×F×T.

[0074] The matrix construction steps include:

[0075] 1. Initialize the matrix

[0076] Create a three-dimensional matrix M with the dimensions of the number of products I, the number of features F, and the number of time points T. Each element in the matrix is ​​initially set to 0, where i∈I, f∈F, and t∈T.

[0077] in,

[0078] i∈{1, 2, ..., I}: product index;

[0079] f∈{1, 2, 3, 4}: feature index, corresponding to price, number of views, number of purchases, and ratings, respectively;

[0080] t∈{1, 2, ..., T}: time index;

[0081] M[i][f][t]: The elements of the three-dimensional matrix represent the value of the f-th feature of the i-th product at time t.

[0082] 2. Fill the matrix

[0083] Fill the three-dimensional matrix M of size I×F×T with m filling elements. ift express.

[0084] The element A of the filled three-dimensional matrix A′ ift ′ can be defined by the following formula:

[0085]

[0086] in,

[0087] A ift It is the element at position (i, f, t) in the original three-dimensional matrix, and is usually initialized to 0 by default.

[0088] m ift ′ is the element at position (i, f, t) in the filled three-dimensional matrix A′.

[0089] i, f, t represent the depth, row, and column indices respectively, satisfying 1≤i≤I, 1≤f≤F, 1≤t≤T

[0090] m ift is the fill value, where:

[0091] For each element in the matrix M, we have:

[0092] M[i][1][t]: the price of product i at time t; M[i][2][t]: the number of views of product i at time t; M[i][3][t]: the number of purchases of product i at time t; M[i][4][t]: the rating of product i at time t.

[0093] That is, for each product i, feature f, and time point t, fill in the corresponding elements in the matrix:

[0094] m i1t : The price of commodity i at time t;

[0095] m i2t : Number of views of product i at time t;

[0096] m i3t : The number of times product i is purchased at time t;

[0097] m i4t : The rating of product i at time t.

[0098] In one embodiment, step 2, interaction matrix decomposition, includes assigning a decay factor to earlier interaction data to reflect the dynamic changes of user interests, and adjusting the weight of each element according to the confidence of the user interaction (such as the evaluation score). A decomposed interaction sub-matrix is ​​constructed for each user based on the decay factor and the optimization target after the confidence adjustment.

[0099] In one embodiment, first, a time decay factor is defined, which is used to capture the dynamics of user interests changing over time. For time point t, the decay factor δ(t) can be defined as: δ(t) = e -λ(T-t) ,

[0100] in:

[0101] λ is the decay rate parameter, which determines how fast the interest decays over time;

[0102] T is the most recent time point, used to normalize time decay;

[0103] t is the current time point.

[0104] Define confidence weighting. For product i, feature f, and time point t, the confidence weight c based on user interaction ift Defined as: c ift =1+α·r ift ,

[0105] in:

[0106] r ift is the user's rating of feature f of product i at time t;

[0107] α is a hyperparameter used to adjust the influence of confidence weights.

[0108] Based on the optimization objective adjusted by the decay factor and confidence, a decomposed interaction submatrix is ​​constructed for each user. The matrix decomposition problem selects non-negative matrix factorization (NMF) as the basic method, and the goal is to find two non-negative matrices W and H so that their product can approximate the original matrix M. Considering time decay and confidence weighting, the optimization problem can be written as:

[0109]

[0110] in:

[0111] m ift is the value of feature f of product i at time t in the original three-dimensional matrix M;

[0112] [W.H] ift are the corresponding elements in the approximation matrix WH;

[0113] cift is the confidence weighting;

[0114] δ(t) is the time decay factor.

[0115] Approximating a three-dimensional matrix M of size I×F×T by non-negative matrix factorization (NMF) requires converting the three-dimensional matrix M into a two-dimensional matrix, which specifically involves "unfolding" one or more dimensions to form a two-dimensional matrix.

[0116] The time dimension T is regarded as an internal dimension, and M is expanded into a two-dimensional matrix of size I×(F×T), where each row corresponds to a product and each column corresponds to the value of a feature of the product at a specific time point.

[0117] Then, NMF decomposes this two-dimensional matrix M into two smaller non-negative matrices W and H, where W is a matrix of size I×K, H is a matrix of size K×(F×T), and K is the number of latent features selected.

[0118] For example, for a dataset with I products, F features, and T time points, M can be expressed as

[0119]

[0120] In one embodiment, assuming K=2, W and H are as follows:

[0121]

[0122] Among them, each row w of W i Represents the relationship between product i and K latent features. Each column h of H f Represents the value of K latent features corresponding to the feature f of all products at time point t.

[0123] The goal of the decomposition is to choose W and H to minimize the weighted squared error:

[0124]

[0125] Among them, c ift is the confidence weight, δ(t) is the time decay factor, [W·H] ift It corresponds to the matrix resulting from the multiplication of W and H.

[0126] For example, if there are 3 products (I=3), 2 features (F=2), and 4 time points (T=4), then the two-dimensional form of M will be a 3×8 matrix. The unfolded I×(F×T) matrix is ​​decomposed into two matrices W and H using NMF. W is an I×K matrix and H is a K×(F×T) matrix, where K is the number of latent features.

[0127] Assume K = 2, then W and H are as follows:

[0128]

[0129] Among them, w ik represents the relationship between product i and latent feature k, h kf Represents the relationship between latent feature k and feature f at time point t. Each element of the matrix WH [WH] ift It is the sum of the products of the i-th row of W and the f×T+t-th column of H.

[0130] For example, the approximate value of feature 1 of item 1 at time point 1 in the M matrix [WH] 111 , is calculated as follows:

[0131] [WH] 111 =w 11 ·h 11 +w 12 ·h 21 .

[0132] This value is the product of the first row of W and the first column of H. This calculation needs to be done for all i, f, t to fill all the elements of the WH matrix.

[0133] Finally, by adjusting W and H to minimize the element m in the original three-dimensional matrix M ift The corresponding element in the WH matrix [WH] ift The weighted square difference between ift Multiply by the time decay factor δ(t). In this way, the optimal values ​​of W and H can be obtained.

[0134] NMF decomposition may lose some important information, depending on the size of the number of latent features K selected during decomposition. If K is set too small, all important features may not be captured. Therefore, in practical applications, it is necessary to determine the optimal k value through experiments, for example, in practice K may be greater than 2.

[0135] The present invention decomposes the three-dimensional matrix into two two-dimensional matrices using NMF, which can greatly reduce the demand for cloud computing resources in the subsequent training optimization process based on deep learning algorithms and reduce the required computing cost.

[0136] Step 3: Construction of sequence feature matrix

[0137] Determine the sequence length and construct the feature matrix. Each user's behavior sequence is converted into two two-dimensional matrices.

[0138] In one embodiment, for the results W and H of the non-negative matrix decomposition, a feature matrix is ​​constructed, including:

[0139] Construct the feature matrix W′. The W matrix represents the relationship between users and latent features, and W′ can be directly set to W:

[0140]

[0141] Among them, I is the number of products and K is the number of latent features.

[0142] Construct the feature matrix H′H matrix represents the relationship between time steps and features. H′ needs to be constructed to characterize the project features of each time step. First, the sequence length L needs to be determined. H , which is equal to the number of features k in H. H is then transposed to construct H′ so that each row of h j ′ represents the feature vector of time step j:

[0143] After transposition, each row of H′ now represents a feature vector of a time step, and the columns represent different latent features. In this way, the dimension of H′ becomes (F×T)×K.

[0144] Where T is the number of time steps, F is the number of features, and K is the same number of latent features as in W.

[0145] Therefore, the feature matrices W′ and H′ can be viewed as encodings of users’ behaviors over time, and they can be used to train machine learning models to predict users’ future behavior preferences.

[0146] Step 4: Time series behavior matrix self-encoding

[0147] Step 4 involves creating an autoencoder network to learn a compressed representation (latent vector) of the user behavior sequence. An autoencoder is an unsupervised learning algorithm that learns an effective representation of data by minimizing the difference between input and output. The specific implementation of step 4 is as follows: 1. Define the autoencoder network structure. An autoencoder usually contains two main parts, an encoder and a decoder. The encoder is responsible for encoding the input data into a latent vector, while the decoder is responsible for decoding the latent vector back to the original data. When defining the network structure, it is necessary to determine the number of neurons in each layer and the depth of the network. To determine the number of neurons, you can start with a small number of neurons and gradually increase it until the performance of the model is no longer significantly improved. Too many neurons may lead to overfitting, while too few may lead to underfitting. Deeper networks can learn more complex features, but are also more difficult to train and more prone to overfitting. Start with two hidden layers and gradually increase the depth as needed.

[0148] 2. Determine the hidden layer dimension: Choose an appropriate hidden layer dimension d, which is usually a hyperparameter that can be optimized through experiments. The hidden layer dimension determines the size of the hidden vector and the compression capability of the autoencoder.

[0149] 3. Choose the loss function: The training goal of the autoencoder is to minimize the difference between the input and output. The loss function is determined based on the mean squared error (MSE).

[0150] 4. Train the autoencoder: input the feature matrix, generate the latent vector through the encoder, then reconstruct the output through the decoder, calculate the loss and update the network weights through backpropagation.

[0151] 5. After training is completed, extract the latent vector.

[0152] In one embodiment, step 4 is based on a joint autoencoder framework, in which W and H are used as the encoder part of the input to the autoencoder, and the network enables it to simultaneously learn the representation of the user's latent vector and temporal behavior characteristics, specifically including:

[0153] Step 4.1: Define the joint encoder: The encoder will use W and H T As input, the common latent vector Z of user and time series features is learned. The encoder form is:

[0154]

[0155] in,

[0156] is the encoder's weight matrix for the W matrix.

[0157] is the encoder bias vector for the W matrix.

[0158] The encoder is for H T The weight matrix of the matrix.

[0159] The encoder is for H T The bias vector for the matrix.

[0160] is the encoding function of the encoder for W.

[0161] The encoder is for H T The encoding function.

[0162] σ: is the activation function Sigmoid.

[0163] Then Z W and Z HThe fusion obtains the joint latent vector Z: Z = Z W ⊙Z H , where ⊙ is the pair Z W and Z H Perform Hadamard product (element-wise multiplication) to obtain the joint latent vector pair.

[0164] Step 4.2: Define the joint decoder: The goal of the decoder is to reconstruct W and H separately T :

[0165]

[0166] in,

[0167] is the decoder weight matrix for W.

[0168] is the decoder bias vector for W.

[0169] The decoder is for H T The weight matrix of .

[0170] The decoder is for H T The bias vector of .

[0171] is the decoding function of the decoder for W.

[0172] The decoder is for H T The decoding function.

[0173] Step 4.3: Define the loss function: The loss function now takes both W and H into account T The reconstruction error is:

[0174]

[0175] in:

[0176] L: represents the overall loss function.

[0177] α: is the weight coefficient for weighing the reconstruction error of W.

[0178] β: is the trade-off H T Weight coefficient for reconstruction error.

[0179] N W : is the number of samples in the W matrix.

[0180] N H :Yes H T The number of samples in the matrix.

[0181] Step 4.4: Train the autoencoder, which involves updating all the parameters of the encoder and decoder using the gradient descent algorithm:

[0182] During the training of the autoencoder, the following parameters are updated using the gradient descent algorithm:

[0183] The encoder weight matrix is ​​used for the product feature matrix W.

[0184] The encoder bias vector is used for the product feature matrix W.

[0185] The weight matrix of the decoder is used to reconstruct the product feature matrix W.

[0186] The bias vector of the decoder is used to reconstruct the product feature matrix W.

[0187] The encoder weight matrix is ​​used for the feature-time matrix H.

[0188] The encoder bias vector for the feature-time matrix H.

[0189] The decoder weight matrix is ​​used to reconstruct the feature-time matrix H.

[0190] The bias vector of the decoder is used to reconstruct the feature-time matrix H.

[0191] The purpose of updating these parameters is to minimize the loss function L, which is usually achieved by calculating the gradient of the loss function with respect to each parameter and performing parameter updates.

[0192] Step 4.5: After training, the joint latent vector Z can be extracted through the encoder:

[0193]

[0194]

[0195] Z=Z W ⊙Z H ,

[0196] Among them, Z W and Z H are the latent vectors obtained from the product feature matrix W and the feature-time matrix H through the encoder, and ⊙ represents element-by-element multiplication (Hadamard product). W and Z HThe combined joint latent vector combines the information of the two latent vectors.

[0197] Then in step 5, the joint latent vector Z will be used as a feature to train the downstream behavior prediction model.

[0198] Step 5: downstream behavior prediction, using the user latent vector representation learned by the autoencoder network as the feature input of the downstream task.

[0199] In one embodiment, predicting the number of purchases of a user based on the user latent vector learned by the autoencoder network includes:

[0200] The latent vector Z is used as the input feature of the linear regression model to predict the number of purchases. The linear regression model is:

[0201]

[0202] in,

[0203] is the predicted number of purchases.

[0204] Z is determined by the product feature Z W and time feature Z H Combine the obtained joint latent vector.

[0205] θ is the weight parameter vector of the model.

[0206] γ is the bias parameter of the model.

[0207] Based on the mean square error (MSE) as the loss function, it is defined as follows:

[0208] θ and γ are estimated by minimizing the loss function. The gradient descent method is used to optimize the optimization algorithm, and the parameter update rule is as follows:

[0209]

[0210]

[0211] In one embodiment, the linear regression model The training steps include: Initialize parameters: θ can be randomly initialized, and γ is initialized to 0. Set learning rate: Choose a suitable learning rate α. Iterative update: Update parameters θ and γ by gradient descent.

[0212] The present invention adopts a non-negative matrix decomposition method, which can decompose the original sparse matrix into a low-dimensional feature matrix while maintaining the non-negativity of the data, effectively alleviating the data sparsity problem. By learning the latent vector representation of the user behavior sequence through the autoencoder network, the present invention can learn the common latent vector of the user and time series features, making the feature representation richer and more accurate. The present invention uses time series and user behavior data, combined with deep learning methods, so that the prediction model can more accurately capture user interests and behavior patterns, thereby improving the accuracy of the prediction.

[0213] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0214] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0215] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0216] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0217] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0218] The above introduces the preferred embodiments of the present invention, which is intended to make the spirit of the present invention clearer and easier to understand, but is not intended to limit the present invention. All modifications, substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection outlined by the claims attached to the present invention.

Claims

1. A deep learning driven customer behavior prediction method, the method comprising: Step 1: Obtain the historical interaction data of users’ purchase times, browsing times and ratings of products through the data collection phase to construct an interaction matrix; Step 2, performing matrix decomposition on the interaction matrix, converting the three-dimensional interaction matrix into a two-dimensional matrix, and applying a non-negative matrix decomposition method to decompose the two-dimensional matrix into two non-negative matrices; Step 3, construct two feature matrices based on the interaction submatrix results of non-negative matrix decomposition, and construct the user feature matrix W' and the time feature matrix H' respectively; Step 4: compress and learn the latent vector representation of the user behavior sequence based on the autoencoder network, including learning the common latent vector of the user and time series features based on the joint autoencoder of the user feature matrix W' and the time feature matrix H'; Among them, the latent vector fusion is fused through the Hadamard product; Step 5: Use the autoencoder network to learn the joint latent vector as input to obtain the behavior prediction of the downstream task; The step 1 specifically includes initializing a three-dimensional matrix M, whose dimensions are determined by the number of products I, the number of features F, and the number of time points T; During the matrix initialization phase, all element values ​​are set to zero; Afterwards, the matrix is ​​filled based on the collected data, corresponding to the values ​​of the price, number of views, number of purchases, and rating features of specific products at different points in time; Wherein, the characteristic matrix W' in step 3 is the W matrix directly obtained from non-negative matrix decomposition, and the characteristic matrix H' is constructed by transposing the H matrix obtained from non-negative matrix decomposition; Wherein, the step 4 specifically includes: Step 4 is based on the joint autoencoder framework, in which the encoder uses W and H T As input, the network enables it to simultaneously learn the representation of the user's latent vector and temporal behavior characteristics, including: Step 4.1: Define the joint encoder: The encoder will use W and H T As input, the common latent vector Z of user and time series features is learned; the encoder form is: in, is the weight matrix of the encoder for the W matrix; is the encoder's bias vector for the W matrix; The encoder is for H T The weight matrix of the matrix; The encoder is for H T The bias vector of the matrix; is the encoding function of the encoder for W; The encoder is for H T The encoding function of σ: is the activation function Sigmoid; Then Z W and Z H Fusion obtains the joint latent vector Z: Z = Z W ⊙Z H , where ⊙ is the pair Z W and Z H Perform Hadamard product to obtain joint latent vector pairs; Step 4.2: Define the joint decoder: The goal of the decoder is to reconstruct W and H separately T : in, is the weight matrix of the decoder for W; is the decoder bias vector for W; The decoder is for H T The weight matrix of The decoder is for H T The bias vector of is the decoding function of the decoder for W; The decoder is for H T The decoding function of Step 4.3: Define the loss function: The loss function now considers both W and H T The reconstruction error is: in: L: represents the overall loss function; α: is the weight coefficient for weighing the reconstruction error of W; β: is the trade-off H T The weight coefficient of reconstruction error; N W : is the number of samples in the W matrix; N H : Yes H T The number of samples in the matrix; Step 4.4: Train the autoencoder, which involves updating all the parameters of the encoder and decoder using the gradient descent algorithm: During the training of the autoencoder, the following parameters are updated using the gradient descent algorithm: The encoder weight matrix is ​​used for the product feature matrix W; The encoder bias vector is used for the product feature matrix W; The decoder weight matrix is ​​used to reconstruct the product feature matrix W; The decoder’s bias vector is used to reconstruct the product feature matrix W; The encoder weight matrix is ​​used for the feature-time matrix H; The encoder bias vector for the feature-time matrix H; The decoder weight matrix is ​​used to reconstruct the feature-time matrix H; The decoder bias vector is used to reconstruct the feature-time matrix H; The purpose of updating these parameters is to minimize the loss function L, which is achieved by calculating the gradient of the loss function with respect to each parameter and updating the parameters; Step 4.5: After training is completed, the joint latent vector Z is extracted through the encoder: Z=Z W ⊙Z H , Among them, Z W and Z H are the latent vectors obtained from the product feature matrix W and the feature-time matrix H through the encoder, ⊙ represents element-by-element multiplication; thus, Z is Z W and Z H The combined joint latent vector, which fuses the information of the two latent vectors; Then in step 5, the joint latent vector Z will be used as a feature to train the downstream behavior prediction model; Among them, in step 5, the user latent vector representation learned by the autoencoder network is used as the feature input of the downstream task, including predicting the number of purchases of the user based on the user latent vector learned by the autoencoder network, specifically including: The hidden vector Z is used as the input feature of the linear regression model to predict the number of purchases; wherein the linear regression model is: in, is the predicted number of purchases; Z is composed of Z W and Z H Combine the obtained joint latent vector; θ is the weight parameter vector of the model; γ is the bias parameter of the model; Based on the mean square error MSE as the loss function, it is defined as follows: Theta and gamma are estimated by minimizing the loss function; the optimization algorithm is optimized using the gradient descent method, and the parameter update rule is as follows: Among them, α is the learning rate of gradient descent; Among them, the linear regression model The training steps include: Initialization parameters: θ is randomly initialized and γ is initialized to 0; Set the learning rate α and update the parameters θ and γ by gradient descent.

2. The deep learning driven customer behavior prediction method according to claim 1, characterized in that: The step 2 also includes constructing a decomposed interaction sub-matrix for each user based on the optimization target after the attenuation factor and the confidence adjustment during the interaction matrix decomposition process; Where, for time point t, the attenuation factor δ(t) is defined as: δ(t) = e -λ(T-t) ; in: λ is the decay rate parameter, which determines how fast the interest decays over time; T is the most recent time point, used to normalize time decay; t is the current time point; Among them, for product i, feature f and time point t, the confidence weight c based on user interaction ift Defined as: c ift =1+α·r ift , in: r ift is the user's rating of feature f of product i at time t; α is a hyperparameter used to adjust the influence of confidence weight; Among them, based on the optimization objective after the attenuation factor and confidence adjustment, a decomposed interaction submatrix is ​​constructed for each user, and the optimization objective function is specifically: in: m ift is the value of feature f of product i at time t in the original three-dimensional matrix M; W.H] ift are the corresponding elements in the approximation matrix WH; c ift is the confidence weighting; δ(t) is the time decay factor.

3. The deep learning driven customer behavior prediction method according to claim 1, characterized in that: Create a three-dimensional matrix M, whose dimensions are the number of products I, the number of features F, and the number of time points T, and each element in the three-dimensional matrix is ​​initially set to 0; Where i∈I, f∈F and t∈T, i∈{1,2,...,I}: product index; f∈{1,2,3,4}: feature index, corresponding to price, number of views, number of purchases, and ratings respectively; t∈{1,2,...,T}: time index; M[i][f][t]: element of the three-dimensional matrix, representing the value of the fth feature of the i-th product at time t; Fill the three-dimensional matrix M of size I×F×T with m filling elements. ift express; For each product i, feature f, and time point t, fill in the corresponding elements in the matrix: m i1t : the price of commodity i at time t; m i2t : Number of views of product i at time t; m i3t : The number of times product i is purchased at time t; m i4t : The rating of product i at time t.

Citation Information

Patent Citations

  • MOOC forum post recommendation method integrating forum interaction behaviors and user reading preferences

    CN111737427A

  • Personalized recommendation method and system based on collaborative filtering and deep learning

    CN112598462A