User portrait prediction method based on sparse auto-encoder and K-means

Abstract

The invention relates to the technical field of data processing, in particular to a user portrait prediction method based on a sparse auto-encoder and K-means. According to the user portrait prediction method based on the sparse auto-encoder and the K-means, the limitation that a traditional user portrait generation and prediction method mainly depends on artificial feature engineering can be overcome, and the user feature dimension is reduced by using the sparse auto-encoder, so that similar user features and invalid user features are eliminated; the method comprises the following steps: firstly, inputting data into a server, and introducing a sparsity penalty term to better capture important features of the input data so as to automatically extract useful features, finally, dividing the obtained useful features into different categories by using K-means clustering, and constructing a user portrait after analysis.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a user portrait prediction method based on sparse autoencoder and K-means. Background Art

[0002] With the rapid development of Internet technology, user data has shown explosive growth. How to mine valuable information from massive data, accurately profile users, and provide personalized services to users has become a hot topic in current research. Traditional user portrait generation and prediction methods mainly rely on artificial feature engineering, but this method has certain limitations, such as difficulty in feature extraction and weak generalization ability. In recent years, deep learning technology has achieved remarkable results in image recognition, speech recognition and other fields. Applying it to the field of user portrait generation and prediction has great research value. Summary of the invention

[0003] The purpose of this invention is to propose a user profile prediction method based on sparse autoencoder and K-means, aiming to solve the limitations of traditional user profile construction methods. The specific objectives are as follows: 1. Improve the efficiency of user portrait generation: Automatically learn user features through deep learning technology, reduce dependence on manual feature engineering, and thus speed up the process of user portrait generation.

[0004] 2. Improve the prediction accuracy of user portraits: As a deep learning model, sparse autoencoders can automatically learn the intrinsic characteristics of input data, capture the complex relationships of user behavior data, and generate more accurate user portraits, thereby improving the accuracy of user classification and prediction.

[0005] 3. Enhance the generalization ability of the model: By jointly training the user portrait generation model and the prediction model, the model can have good generalization performance in different scenarios and data sets.

[0006] 4. Broaden the application fields: The present invention can be applied to multiple fields such as user personalized recommendation, advertising, user behavior analysis, etc., providing strong data support and services for the Internet industry.

[0007] By achieving the above objectives, the present invention aims to provide users with an efficient, accurate and automated user portrait generation and prediction solution, further optimize the Internet service experience and improve business operation efficiency.

[0008] The following technical solutions are used to solve the existing technical problems: A user portrait prediction method based on sparse autoencoder and K-means includes the following steps: firstly, training the data with the constructed sparse autoencoder, and extracting user feature data based on the sparse autoencoder; secondly, using the K-means algorithm to classify the user feature data after the sparse encoder training, and dividing the user data into a specified number of categories; finally, analyzing the classified data to construct a user portrait.

[0009] This paper proposes a user portrait prediction method based on sparse autoencoder and K-means, which can overcome the limitations of traditional user portrait generation and prediction methods that mainly rely on artificial feature engineering. The sparse autoencoder is used to reduce the dimension of user features, thereby excluding similar user features and invalid user features, and introduces sparsity penalty terms to better capture the important features of the input data, thereby automatically extracting useful features. Finally, the useful features are divided into different categories using K-means clustering, and user portraits are constructed after analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 Schematic diagram of sparse autoencoder; Figure 2 Schematic diagram of sparse autoencoder training; Figure 3 K-means flow chart; Figure 4 Schematic diagram of the process of the present invention; Figure 5 Draw a thermal diagram; Figure 6 Classification scatter plot without sparse autoencoder; Figure 7 Add sparse autoencoder classification scatter plot; Figure 8 Added radar chart for sparse autoencoders. DETAILED DESCRIPTION

[0011] The present invention proposes a user portrait prediction method based on sparse autoencoder and K-means. The method first trains the prepared data with the constructed sparse autoencoder to obtain a sparse autoencoder model, and extracts user feature data based on the sparse autoencoder. Secondly, the K-means algorithm is used to classify the user feature data after the sparse encoder training, so that the user data can be divided into a specified number of categories. Finally, the classified data is analyzed to construct a user portrait. The specific implementation technology and principle are as follows.

[0012] (1) Sparse autoencoder extracts user features User profile construction is an important task in many service areas, such as e-commerce, social media, and online advertising. Traditional user profile construction methods often only classify users directly based on their attributes. This is inefficient when there are many user attributes resulting in high dimensionality, and it also fails to fully utilize the potential structure of user data for classification and prediction. As an unsupervised learning method, autoencoders can learn effective representations of data and reduce the dimensionality of user features, thereby excluding user features that are invalid for classification.

[0013] Since some users have many attributes, and the data dimension of the attributes of string classification will increase after one-hot encoding, which is not conducive to data processing, the sparse autoencoder, as an improved algorithm of the autoencoder, maps the input data to a low-dimensional representation (also called encoding or latent vector) through learning (the technical implementation principle of this part will be given in the principle of sparse encoder below), and then reconstructs the original data from this low-dimensional representation, and introduces a sparsity penalty term to make the activation of hidden layer neurons more sparse. By forcing the sparse activation of hidden layer neurons, the sparse autoencoder can better capture the important features of the input data, thereby automatically extracting useful features.

[0014] The sparse autoencoder consists of two parts: the encoder and the decoder. The encoder is responsible for mapping the input data to a low-dimensional space, while the decoder reconstructs the low-dimensional representation back to the original data space. This method can reduce the redundant features in the user features. Figure 1 shown.

[0015] When encoding, input data X After the autoencoder encoding operation, the output of the hidden layer is used as the feature value through the activation function. H The expression is shown in formula (1).

[0016] (1) In the above formula, input data X for , input data X middle n represents the dimension of the data, and the output of the hidden layer is ,in r Indicates the number of neurons in the hidden layer; the number of neurons in the hidden layer r Usually less than the number of neurons in the input layer n , Represents the weight from the input layer to the hidden layer; Represents the bias from the input layer to the hidden layer; Represents the activation function. This paper adopts the ReLU function, and the function expression is shown in formula (2).

[0017] (2) The decoding process is to extract the feature values ​​from the hidden layer H Reconstructed to approximate X The output vector , the expression is shown in formula (3).

[0018] (3) The output value , is the weight vector from the hidden layer to the output layer, is the bias from the hidden layer to the output layer. The output value should be the same as the input value. X Remain consistent. The activation function used in the reconstruction process is still the ReLU function.

[0019] The sparse autoencoder adds a sparsity constraint to the hidden layer of the autoencoder. When the output of the hidden layer is approximately 1, the neuron is considered to be in an activated state, and when the output of the hidden layer is approximately 0, it is considered to be in an inhibited state. A restriction rule that makes the neurons in the hidden layer inhibited most of the time is called sparsity inhibition. This paper introduces KL discreteness (Kullback-Leibler) to measure the average activation output and sparsity of the hidden layer nodes. The similarity between them is expressed as shown in formula (4).

[0020] (4) (5) It represents the average activity of neuron j in the hidden layer, and the calculation method is shown in formula (5). Indicates sparsity, generally set the sparse coefficient =0.05, m represents the number of samples, and represents the hidden layer node j for the sample i The response output is Represents neurons in the hidden layer j In order to ensure that the hidden layer neurons are at a low level of activity, the smaller the KL discreteness, the better. The smaller the KL discreteness, the better. and The smaller the difference.

[0021] Sparse autoencoders can obtain sparse features of input data when the number of hidden layer neurons is large. These sparse features are reconstructed to obtain output data during the decoding process. , the reconstruction error is calculated through the loss function. In this paper, the cross entropy cost function is used to calculate the loss value, and the expression is shown in formula (6).

[0022] (6) in m is the number of sample data, n Represents the number of tags. Represents a variable 0 or 1. If the label category of the first sample is the same as the label j If the result is consistent, the final result is 1, otherwise it is 0, indicating that the sample data i Belongs to the tag j After adding the penalty factor, the cost function of the algorithm is shown in formula (7).

[0023] (7) in Is the penalty coefficient, which controls the weight of the penalty factor in the cost function.

[0024] After the above introduction, we can derive the training process of the sparse autoencoder. The training diagram of the sparse autoencoder is as follows: Figure 2 shown.

[0025] The pseudo code for implementing feature extraction using the sparse autoencoder is shown in Table 1.

[0026] Table 1 Pseudocode for feature extraction of sparse autoencoder (2) K-means clustering algorithm classifies users according to their characteristics K-means is a clustering algorithm that divides the user feature data set into k clusters so that each data point belongs to the closest cluster. This algorithm is based on iterative optimization, and each iteration step updates the center point of the cluster until the convergence condition is reached. The goal of the K-means algorithm is to n The user features are clustered into K specified clusters according to their similarities, and the distance from each feature to the cluster center is the smallest, so that users are divided into different classes.

[0027] The implementation process of K-means is as follows Figure 3 shown.

[0028] In the K-means algorithm, the number of clusters K is a hyperparameter that needs to be determined by human input. The core task of K-Means is to find the K optimal cluster centers based on the set K, and assign the data closest to these cluster centers to the clusters represented by these cluster centers. The specific process can be summarized as follows: 1) First, randomly select K points in the sample as cluster centers; 2) Calculate the distances between other samples in the sample and the centers of the K clusters, and use these samples as the categories of the cluster center closest to them; 3) For the samples classified above, average the values ​​of each category and find the new cluster centroid; 4) Compare with the K cluster centroids calculated last time. If the cluster centroids have changed, go to process 2), otherwise go to process 5); 5) When the cluster centroid does not change, stop and output the clustering results.

[0029] The distance here uses the Euclidean distance, and the distance formula is shown in formula (8).

[0030] (8) represents the centroid of the cluster, n represents the number of features in each sample point, i Indicates the composition point x Each feature.

[0031] (3) User profile prediction method based on sparse autoencoder and K-means The user portrait prediction method based on sparse autoencoder and K-means uses the data collected by the platform as the original data for user portrait generation and prediction. User data generally contains multiple features and is in various forms. Therefore, after obtaining the user data, the data needs to be preprocessed, cleaned and standardized to ensure data quality. The implementation content of this part is given in the data collection and preprocessing part. Then, each feature is analyzed for correlation, and the features with high correlation are simplified into one feature. After the data is prepared, it is trained with the constructed sparse autoencoder to obtain the sparse autoencoder model, and the user feature data after dimensionality reduction can be obtained. Then, the K-means algorithm is used to classify the user feature data after the sparse encoder training, and the user data can be divided into the specified number of categories. Then, each type of data is analyzed, its potential features are mined, and a user portrait is constructed. Then, the newly obtained user data is preprocessed, and it is predicted and classified in the trained model to obtain the corresponding user portrait.

[0032] The flowchart of the user portrait prediction method based on sparse autoencoder and K-means is as follows Figure 4 As shown: Technical solutions and specific embodiments of the invention In order to make the above-mentioned objects, features and advantages of the present invention more understandable, the present invention is further described in detail through a detailed technical scheme as follows.

[0033] The following are the steps for generating and predicting the user profile prediction method based on sparse autoencoder and K-means: (1) Data collection and preprocessing Collect user data, including basic information and behavioral characteristics of users. Preprocess the data, including deleting redundant rows, data standardization, encoding, etc., to obtain a structured user data set. Since distance is very sensitive to different value ranges between features, if a feature value is very large, it will mask the impact of the distance between features on the total distance. In this way, the distance model cannot distinguish features of different categories well. Therefore, it is very necessary to normalize the data for models based on distance metrics. Such models can improve accuracy after normalization.

[0034] For numerical attributes, this paper adopts Max-Min normalization, which is a linear transformation method. After normalization, the data completely falls into the interval [0,1] and can better maintain the original data structure. The Max-Min normalization mathematical model is shown in formula (9).

[0035] (9) For input x After normalization, max and min are the maximum and minimum values ​​respectively.

[0036] For string categorized attributes, such as gender, type, etc., one-hot encoding is used.

[0037] One-Hot Encoding is a commonly used feature encoding method, which is mainly used to convert discrete features into continuous features. The value of each discrete feature is mapped to a binary vector, in which only one element is 1 and the rest are 0. The position of this element represents the position of the value among all values. For example, for a discrete feature with three different values, one-hot encoding will map each value to a three-element binary vector. The specific implementation is as follows: Assuming that a discrete attribute has three values, namely: A, B, C, then its unique hot encoding is: 1 0 0, 0 10, 0 0 1 respectively.

[0038] Using one-hot encoding, the values ​​of discrete features are extended to Euclidean space, and a certain value of the discrete feature corresponds to a certain point in Euclidean space. Using one-hot encoding for discrete features will make the distance calculation between features more reasonable.

[0039] Then merge all dimensions, that is, merge the normalized data and the one-hot encoded data as the final data set.

[0040] (2) Sparse Autoencoder Construction Construct a sparse autoencoder model, which includes an input layer, an encoder layer, a decoder layer, and an output layer. The encoder layer and the decoder layer are hidden layers. The encoder layer and the decoder layer use a fully connected neural network, and the activation function uses ReLU. The loss function uses the cross entropy loss function after adding sparsity constraints.

[0041] 1) Specific implementation process of sparse autoencoder a. Input layer The input layer of the sparse autoencoder is the data that is merged from the preprocessed, normalized data and the one-hot encoded data, expressed as .

[0042] b. Encoder layer The first step of the encoder layer is to convert the input X and encoder weights W Multiply and add the bias b The second step is to apply the ReLU activation function to get the hidden layer output H , the formula is as follows: Here is the ReLU activation function.

[0043] c. Decoder layer Once the encoding process is complete, the next step is to decode the features to return the final output. To do this, we decode the features from the previous step. H With the corresponding weight Multiply and add bias , and finally a ReLU layer is applied.

[0044] d. Output layer The output layer is the output of the previous step .

[0045] 2) Steps of sparse autoencoder for feature learning: a. Choose a sparse autoencoder architecture: The sparse autoencoder consists of an input layer, a hidden layer, and an output layer. The number of nodes in the input layer is equal to the number of features, and the number of nodes in the output layer is the same as the input layer. The number of nodes in the hidden layer is adjusted according to the task requirements.

[0046] b. Training sparse autoencoder: The sparse autoencoder is trained using an unlabeled dataset. During the training process, the sparse autoencoder learns how to map the input data into a low-dimensional space and reconstruct the data from it. The training goal is to minimize the reconstruction error, which is the difference between the original data and the reconstructed data.

[0047] c. Extract features: Once the sparse autoencoder is trained, the encoder part can be used to extract features for new input data. These features are low-dimensional representations that are efficient representations of the original data and often contain important patterns and structures in the data. These features are then used for clustering without the need for further feature engineering.

[0048] d. Optimization and adjustment: Depending on the requirements of a specific task, the extracted features may need to be further optimized or adjusted. For example, the features may need to be standardized or normalized.

[0049] (3) Model training The preprocessed user data is fed into the sparse autoencoder for training. The back propagation algorithm and Adam optimizer are used to update the parameters. The loss function value is monitored during the training process until the preset threshold or number of iterations is reached.

[0050] 1) Loss Function In this method, the cross-entropy loss function (Cross-Entropy Loss) with sparsity constraints is used. The cross-entropy loss function is a loss function commonly used to measure the difference in class probability distribution, especially in multi-classification problems. The cross-entropy loss function can be regarded as a measure of the amount of information, which measures the difference between the actual output distribution and the ideal output distribution.

[0051] The advantage of the cross entropy loss function is that it is insensitive to outliers and performs well when dealing with class imbalance problems.

[0052] After adding the sparsity constraint to the encoder hidden layer, the loss function is shown in formula (7).

[0053] 2) Optimizer The optimizer is responsible for updating the parameters of the model to minimize the loss function. In this method, the Adam optimizer is used, which is an adaptive learning rate method that combines the advantages of Momentum, RMSprop (Root Mean Square Propagation), and AdaGrad. The Adam optimizer can dynamically adjust the learning rate of each parameter according to the size and direction of the gradient of the parameter. The Adam optimizer defines two exponentially weighted averages: the first exponentially weighted average is the exponentially weighted average of the gradient, and the second exponentially weighted average is the exponentially weighted average of the square of the gradient. These two weighted averages are used to adjust the learning rate of each parameter, thereby achieving the effect of adaptive learning rate. The update rules of the Adam optimizer are as follows: Update the first-order moment vector m , which stores the moving average of the gradient as shown in formula (10).

[0054] (10) We update the second-order moment vector v This vector provides an estimate of the variance (or unpredictability) of the gradient, so it stores the square of the accumulated gradient, calculated as shown in equation (11).

[0055] (11) The deviation of the correction moment is shown in equations (12) and (13).

[0056] (12) (13) Update the model parameters as shown in Equation (14). In this step, the actual optimization process occurs, moving the parameters in the direction of minimizing the loss function. The update process utilizes the adaptive learning rate calculated in the previous step.

[0057] (14) in, is the gradient of the parameter, and is the decay coefficient of the two exponentially weighted averages, and is the bias-corrected moving average of the gradient, is the updated parameter, is the learning rate, is a small constant used to avoid division by zero. The main advantage of the Adam optimizer is that it can adaptively adjust the learning rate of each parameter, thereby improving the convergence speed and generalization ability of the model.

[0058] (4) Obtain encoder output After training the encoder model, it converts the input data (user features) into a lower dimensional representation (usually a feature vector) to simplify the model and eliminate the influence of potentially irrelevant features.

[0059] (5) User classification The extracted low-dimensional features are input into clustering algorithms (such as K-means) for user classification. Clustering algorithms do not require the number of categories to be defined in advance and can automatically discover the distribution of user groups.

[0060] (6) New user prediction For new users, the same data preprocessing and autoencoder feature extraction process is used. The new user features are input into the trained clustering model to obtain the category prediction of the new user.

[0061] 2. Specific examples and analysis: This article uses python programming and pytorch deep learning framework to implement coding.

[0062] (1) Input data This paper uses advertising data as test data, replaces customer attributes with advertising attributes, and constructs a user portrait belonging to advertising similar to customer attributes. The data attribute characteristics are the average daily UV of various advertising channels within 90 days, the average registration rate, average search rate, visit depth, average stay time, order conversion rate, delivery time, material type, advertising type, cooperation method, advertising size and advertising selling points, etc., a total of 890 rows of data, which are used in the user portrait prediction method based on sparse autoencoder and K-means, and their correlation analysis, clustering prediction and result analysis are respectively carried out.

[0063] (2) Correlation analysis Correlation analysis is used to evaluate the relationship between two or more variables, which can help us understand the dependencies between variables and make better predictions and decisions. The Pearson correlation coefficient is used here to analyze the correlation between each feature. The Pearson correlation coefficient ranges from -1 to 1, where 1 indicates a perfect positive correlation, -1 indicates a perfect negative correlation, and 0 indicates no linear correlation. The calculation method of the Pearson correlation coefficient is shown in the formula.

[0064] (15) For input data X No. i Features and j Features The Pearson correlation coefficient between the data, is the covariance, and They are X No. i Features and j Features The standard deviation of the data. and The calculation formulas of are shown in equations (16) and (17).

[0065] (16) (17) for X No. k The first data i Features, kThe value of is 1, 2, 3, ….., n , For the i The feature is the mean of the data.

[0066] After calculating the Pearson correlation coefficient, a heat map is plotted to visualize the correlation between features. By comparing the similarities between features, highly correlated features can be identified, which helps select the most important features. Figure 5 As shown: It can be seen from the above figure that the feature access depth and average residence time are highly correlated, which means that the access depth and average residence time have the same role or effect when building the model. Therefore, any one of the features of access depth and average residence time can be deleted, and then the numerical attributes can be standardized.

[0067] (3) Feature encoding There are 4, 5, 8, 4, and 6 types of string classification features such as material type, advertisement type, cooperation method, advertisement size, and advertisement selling point. The first two lines after one-hot encoding are as follows: [[0 1 0 0 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 1 0 0 0 0 0] [0 1 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0]] Merge the one-hot encoded string classification attributes and the standardized numeric type attributes for training, delete the access depth feature data and merge the first two rows of data as follows: [[0 0.18 0.02 0.01 0.12 0.66 0 1 0 0 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 00 1 0 0 0 0 0] [0.01 0.1 0.03 0.01 0.01 0.62 0 1 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 0 00 0 1 0 0 0 0]] (4) Training sparse autoencoders for feature extraction Set the hidden layer dimension to 10, train the sparse autoencoder according to Algorithm 1 (Sparse Autoencoder for Feature Extraction), and the first two rows of features are as follows: [[-0.2408 -0.4633 -0.5540 0.1800 0.2225 -0.1887 -0.0581 -0.6209 -0.9903 -0.4628], [-0.0679 -1.2058 0.6729 -0.3614 0.8543 -0.2386 -0.7438 -0.3920 -0.6704 -0.2791]] (5) Cluster prediction and result analysis According to the requirements, the number of clusters n_clusters of the K-means clustering algorithm is set to 4. After clustering, the user data is divided into n_clusters classes. After adding the classification of the sparse autoencoder, the proportion of each class is as follows: Cluster1: 0.18, Cluster1: 0.39, Cluster1: 0.2, Cluster1: 0.23.

[0068] Each category has its own characteristics. According to the characteristics of each category, the user's attributes are analyzed to build a user portrait. When predicting a new user, a piece of data is input, the new user data is processed first, and then the category of the new user can be predicted through the trained model.

[0069] In order to see the classification effect more intuitively, draw a classification scatter plot. The classification scatter plot without sparse autoencoder and the classification scatter plot with sparse autoencoder are as follows: Figure 6 and Figure 7 shown.

[0070] It can be seen from the scatter plot that users are divided into four categories according to their features. It can be intuitively seen from the figure that the classification effect of adding sparse autoencoder is obviously better than the classification effect without sparse autoencoder.

[0071] In order to more clearly analyze the characteristics of each category, a radar chart is drawn to analyze the characteristics of each category and its potential value. Figure 8 shown.

[0072] From the radar chart, we can intuitively analyze that the performance of category 1 in each attribute is not much different, and none of them is outstanding. Category 2 has a good performance in average search volume and average daily UV. Category 3 has better performance in visit depth and average registration rate than other attributes. Category 4 has better performance in average registration rate and average daily UV. Based on the above analysis, we can construct a user portrait for each category.

[0073] Therefore, this paper proposes a user portrait prediction method based on sparse autoencoder and K-means, which can overcome the limitations of traditional user portrait generation and prediction methods that mainly rely on artificial feature engineering. The sparse autoencoder is used to reduce the dimension of user features, thereby excluding similar user features and invalid user features, and introduces sparsity penalty terms to better capture the important features of the input data, thereby automatically extracting useful features. Finally, the obtained useful features are divided into different categories using K-means clustering, and user portraits are constructed after analysis.

[0074] Glossary User portrait: A fictional character used to describe a target user group. It summarizes the typical characteristics, behavior patterns, motivations, and needs of users based on market research and user research data. The purpose of user portraits is to help product designers, developers, and marketers better understand users, so as to create products, services, and marketing strategies that better meet user needs and expectations. As an effective tool for outlining target users and linking user demands with design directions, user portraits have been widely used in various fields.

[0075] Sparse Autoencoder: In the field of deep learning, autoencoders are a commonly used unsupervised learning algorithm that can be used for tasks such as data dimensionality reduction, feature extraction, and data reconstruction. Autoencoders usually consist of two parts: an encoder and a decoder. The encoder maps the input data to a low-dimensional feature vector, while the decoder maps the low-dimensional feature vector representation back to the original input data, with the goal of reconstructing the original data as accurately as possible. Sparse Autoencoders, as a variant of autoencoders, can better learn the sparse feature representation of data to a certain extent. Sparse Autoencoders are a neural network-based autoencoder model that aims to reconstruct input data through learned sparse representations. Compared with traditional autoencoders, sparse autoencoders introduce a sparsity penalty term to make the activation of hidden layer neurons more sparse. By forcing the sparse activation of hidden layer neurons, sparse autoencoders can better capture the important features of input data. Therefore, sparse autoencoders can learn the potential characteristics of users and reduce the dimension of user features, which can be used for user classification to construct user profiles.

[0076] Deep Learning: User data is used to learn complex patterns and features through deep learning. Deep learning models consist of multiple layers of nodes, each of which performs some specific calculations and transformations on the input data. These layers are organized into a hierarchy that enables the model to learn higher-level concepts and abstractions from the raw data. Sparse autoencoders, as a deep learning algorithm, can better learn complex patterns and features.

[0077] :K-means clustering algorithm is an unsupervised learning algorithm, which aims to divide data into meaningful or useful groups (or clusters), which does not rely on labeling or classification information when classifying. This division can be done based on business needs or modeling needs, or it can simply help us explore the natural structure and distribution of data. In user portrait construction, clustering can be used to divide users into several groups based on a large amount of current and potential customer information for further analysis and marketing activities.

Claims

1. A user profile prediction method based on sparse autoencoder and K-means, characterized in that The method includes the following steps: first, training the data with the constructed sparse autoencoder, and extracting user feature data based on the sparse autoencoder; second, using the K-means algorithm to classify the user feature data after the sparse encoder training, and dividing the user data into a specified number of categories; finally, analyzing the classified data to construct a user portrait.

2. According to claim 1, a user portrait prediction method based on sparse autoencoder and K-means is characterized in that The sparse autoencoder inputs data during encoding. X After the autoencoder encoding operation, the output of the hidden layer is used as the feature value through the activation function. H The expression is shown in formula (1); (1) In the above formula, input data X for , input data X middle n represents the dimension of the data, and the output of the hidden layer is ,in r Indicates the number of neurons in the hidden layer; the number of neurons in the hidden layer r Less than the number of neurons in the input layer n , Represents the weight from the input layer to the hidden layer; Represents the bias from the input layer to the hidden layer; represents the activation function, using the ReLU function, and the function expression is shown in formula (2); (2) The decoding process is to extract the feature values ​​from the hidden layer H Reconstructed to approximate X The output vector , the expression is shown in formula (3); (3) The output value , is the weight vector from the hidden layer to the output layer, is the bias from the hidden layer to the output layer. The output value should be the same as the input value. X Remain consistent; the activation function used in the reconstruction process is still the ReLU function; The sparse autoencoder adds a sparsity constraint to the hidden layer of the autoencoder; when the output of the hidden layer is approximately 1, the neuron is considered to be in an activated state, and when the output of the hidden layer is approximately 0, it is considered to be in an inhibited state; the KL discreteness Kullback-Leibler is introduced to measure the similarity between the average activation output and sparsity of the hidden layer nodes, and the expression is shown in formula (4); (4) (5) It represents the average activity of neuron j in the hidden layer, and the calculation method is shown in formula (5). Indicates sparsity, sets the sparse coefficient =0.05, m represents the number of samples, Represents the hidden layer node j for the sample i The response output is Represents neurons in the hidden layer j The activation degree; In order to ensure that the hidden layer neurons are at a low activity, the smaller the KL discreteness, the better. The smaller the KL discreteness, the better. and The smaller the difference; Sparse autoencoders can obtain sparse features of input data when the number of hidden layer neurons is large; These sparse features are reconstructed during the decoding process to obtain the output data , the reconstruction error is calculated through the loss function. In this paper, the cross entropy cost function is used to calculate the loss value. The expression is shown in formula (6); (6) in m is the number of sample data, n Represents the number of tags. Represents a variable 0 or 1. If the label category of the first sample is the same as the label j If they are consistent, the final result is 1, otherwise it is 0. Represents sample data i Belongs to the tag j After adding the penalty factor, the cost function of the algorithm is shown in formula (7); (7) in Is the penalty coefficient, which controls the weight of the penalty factor in the cost function.

Images