User classification model training method and apparatus, device, and storage medium

Abstract

The application relates to a user classification model training method and device, equipment and a storage medium. The method can be used in the field of financial technology or other related fields, and comprises the following steps: obtaining historical user behavior data; inputting the historical user behavior data into a user classification model to be trained, obtaining category features including a plurality of sub-features corresponding to the historical user behavior data through the user classification model; filtering the category features through a first module to obtain filtered category features, and performing feature extraction on the filtered category features to obtain first output features; performing similarity extraction on each sub-feature through a second module to obtain second output features; merging the first output features and the second output features to obtain a user prediction category; and training the user classification model to be trained by using the user prediction category to obtain a trained user classification model. The method can obtain a user classification model which can make the user classification result more accurate.

Description

Technical Field

[0001] This application relates to the field of artificial neural network technology, and in particular to a user classification model training method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] With the development of artificial neural network technology, user automatic classification technology has emerged, which automatically classifies users by building user classification models.

[0003] In the above technical solutions, traditional machine learning models are usually used to build user classification models. However, user classification models built based on traditional machine learning models have poor fitting ability for high-dimensional data in user data, and traditional machine learning models suffer from the problem of gradient vanishing during training, which makes the classification results of the trained user classification model inaccurate. Summary of the Invention

[0004] Therefore, it is necessary to provide a user classification model training method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can make user classification results more accurate, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a method for training a user classification model. The method includes:

[0006] Obtain historical user behavior data;

[0007] The historical user behavior data is input into a user classification model to be trained. The user classification model obtains category features corresponding to the historical user behavior data, including multiple sub-features. The first module of the user classification model filters the category features to obtain filtered category features, and extracts features from the filtered category features to obtain the first output feature. The second module of the user classification model extracts similarity from each sub-feature to obtain the second output feature. The first output feature and the second output feature are merged to obtain the predicted user category.

[0008] The user classification model to be trained is trained using the predicted user categories to obtain the trained user classification model.

[0009] In one embodiment, filtering the category features by the first module to obtain filtered category features includes: removing sub-features with a feature value of zero from the category features to obtain filtered category features.

[0010] In one embodiment, the second module extracts the similarity of each sub-feature to obtain the second output feature, including: obtaining the initial similarity between the current sub-feature and the remaining sub-features respectively, and fusing the initial similarity to obtain the similarity corresponding to the current sub-feature; the similarity is used to characterize the weight of the current sub-feature in the category feature, and the remaining sub-features are the sub-features other than the current sub-feature among multiple sub-features; and the second output feature is obtained based on the similarity and the pre-obtained activation function.

[0011] In one embodiment, the user classification model to be trained is trained using the user predicted category to obtain the trained user classification model, including: obtaining the actual user category corresponding to the historical user behavior data based on historical user behavior data; obtaining the loss function value of the user classification model to be trained based on the user predicted category and the actual user category; and obtaining the trained user classification model based on the loss function value.

[0012] In one embodiment, the historical user behavior data includes: first historical user behavior data with continuous text and second historical user behavior data without continuous text; obtaining category features corresponding to the historical user behavior data through a user classification model includes: obtaining a first category feature based on the first historical user behavior data; obtaining a second category feature based on the second historical user behavior data; and fusing the first category feature and the second category feature to obtain the category feature.

[0013] In one embodiment, the user classification model includes a long short-term memory module; based on first historical user behavior data, a first category feature is obtained, including: inputting the first historical user behavior data into the long short-term memory module, filtering the first historical user behavior data through the input gate, forget gate and output gate in the long short-term memory module to obtain output data corresponding to the first historical user behavior data; and transforming the output data through a normalized exponential function to obtain the first category feature.

[0014] In one embodiment, obtaining a second category feature based on the second historical user behavior data includes: obtaining an initial information entropy corresponding to the second historical user behavior data based on the second historical user behavior data; obtaining an information gain value corresponding to the second historical user behavior data based on the initial information entropy; and obtaining the second category feature based on the information gain value.

[0015] Secondly, this application provides a user classification method. The method includes:

[0016] Obtain user behavior data;

[0017] The user behavior data is input into a pre-trained user classification model;

[0018] Based on the user behavior data, the user classification result corresponding to the user behavior data is obtained through the user classification model.

[0019] Thirdly, this application also provides a user classification model training device. The device includes:

[0020] The historical data acquisition module is used to acquire historical user behavior data;

[0021] The historical user behavior data is input into a user classification model to be trained. The user classification model obtains category features corresponding to the historical user behavior data, including multiple sub-features. The first module of the user classification model filters the category features to obtain filtered category features, and extracts features from the filtered category features to obtain the first output feature. The second module of the user classification model extracts similarity from each sub-feature to obtain the second output feature. The first output feature and the second output feature are merged to obtain the predicted user category.

[0022] The classification model training module is used to train the user classification model to be trained using the user prediction category, so as to obtain the trained user classification model.

[0023] Fourthly, this application provides a user classification model. The model includes:

[0024] The user behavior data acquisition module is used to acquire user behavior data.

[0025] The user behavior data input module is used to input the user behavior data into a pre-trained user classification model;

[0026] The user classification result acquisition module is used to obtain the user classification result corresponding to the user behavior data based on the user behavior data and through the user classification model.

[0027] Fifthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0028] Obtain historical user behavior data;

[0029] The historical user behavior data is input into a user classification model to be trained. The user classification model obtains category features corresponding to the historical user behavior data, including multiple sub-features. The first module of the user classification model filters the category features to obtain filtered category features, and extracts features from the filtered category features to obtain the first output feature. The second module of the user classification model extracts similarity from each sub-feature to obtain the second output feature. The first output feature and the second output feature are merged to obtain the predicted user category.

[0030] The user classification model to be trained is trained using the predicted user categories to obtain the trained user classification model.

[0031] Sixthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0032] Obtain historical user behavior data;

[0033] The historical user behavior data is input into a user classification model to be trained. The user classification model obtains category features corresponding to the historical user behavior data, including multiple sub-features. The first module of the user classification model filters the category features to obtain filtered category features, and extracts features from the filtered category features to obtain the first output feature. The second module of the user classification model extracts similarity from each sub-feature to obtain the second output feature. The first output feature and the second output feature are merged to obtain the predicted user category.

[0034] The user classification model to be trained is trained using the predicted user categories to obtain the trained user classification model.

[0035] Seventhly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0036] Obtain historical user behavior data;

[0037] The historical user behavior data is input into a user classification model to be trained. The user classification model obtains category features corresponding to the historical user behavior data, including multiple sub-features. The first module of the user classification model filters the category features to obtain filtered category features, and extracts features from the filtered category features to obtain the first output feature. The second module of the user classification model extracts similarity from each sub-feature to obtain the second output feature. The first output feature and the second output feature are merged to obtain the predicted user category.

[0038] The user classification model to be trained is trained using the predicted user categories to obtain the trained user classification model.

[0039] The aforementioned user classification model training method, apparatus, computer equipment, storage medium, and computer program product acquire historical user behavior data; input the historical user behavior data into the user classification model to be trained; obtain category features corresponding to the historical user behavior data, including multiple sub-features, through the user classification model; filter the category features through the first module of the user classification model to obtain filtered category features; extract features from the filtered category features to obtain a first output feature; extract similarity between each sub-feature through the second module of the user classification model to obtain a second output feature; merge the first output feature and the second output feature to obtain the predicted user category; and train the user classification model to be trained using the predicted user category to obtain a trained user classification model. This application extracts features from the category features through the first module to obtain the first output feature, then merges the second output feature obtained by the second module with the first output feature, weighting the second output feature with the first output feature to obtain the predicted user category. Finally, the user classification model is trained using the predicted user category, resulting in a user classification model with more accurate user classification results. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating a user classification model training method in one embodiment;

[0041] Figure 2 This is a schematic diagram of the process for obtaining the second output feature in one embodiment;

[0042] Figure 3 This is a schematic diagram of the process of training a user classification model in one embodiment;

[0043] Figure 4 This is a flowchart illustrating the process of obtaining category features in one embodiment;

[0044] Figure 5This is a flowchart illustrating a user classification method in one embodiment;

[0045] Figure 6 This is a flowchart illustrating the process of training a user classification model in another embodiment;

[0046] Figure 7 This is a structural block diagram of a long short-term memory model in one embodiment;

[0047] Figure 8 This is a structural block diagram of a user classification model training device in one embodiment;

[0048] Figure 9 This is a structural block diagram of a user classification device in one embodiment;

[0049] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] It should be noted that the terms "first" and "second" used in the embodiments of the present invention are merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permissible. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.

[0052] In one embodiment, such as Figure 1 As shown, a user classification model training method is provided. This embodiment illustrates the method applied to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0053] Step S101: Obtain historical user behavior data.

[0054] Among them, historical user behavior data is pre-processed user business transaction data.

[0055] Specifically, user business processing data is cleaned, including removing discrete values ​​and filling in missing values, to obtain historical user behavior data.

[0056] Step S102: Input historical user behavior data into the user classification model to be trained, and obtain category features corresponding to the historical user behavior data, including multiple sub-features, through the user classification model; filter the category features through the first module of the user classification model to obtain filtered category features, and extract features from the filtered category features to obtain the first output feature; extract the similarity of each sub-feature through the second module of the user classification model to obtain the second output feature; merge the first output feature and the second output feature to obtain the predicted user category.

[0057] The user classification model refers to a model that classifies user behavior data to quickly and accurately categorize the credit scores of user behavior data. The category features are feature vectors of historical user behavior data, influencing the classification of historical user behavior data. For example, historical user behavior data may be a dataset {A, B, C}, where data A has sub-data a, b, and c. Sub-data a can be used to classify user behavior data; therefore, both sub-data a and data A can be data points within the aforementioned category feature vector. The first module, a capsule module within the user classification model, is used to further extract features from the category features. The filtered category features are those obtained by removing sub-features with values ​​of zero. The first output feature is the category feature vector obtained after the category features have been extracted by the first model. The second module, parallel to the first, is an attention module within the user classification model. It extracts the weights of each sub-feature from the category features. For example, if the category features can exist as a matrix vector, then the sub-features are elements within that matrix vector. The second output feature is the feature vector obtained after extracting the weights of each sub-feature from the category features. Similarity represents the degree of similarity between a sub-feature and the remaining sub-features, used to characterize the weight of the sub-feature. Low similarity indicates a larger difference between the sub-feature and the remaining sub-features, hence a higher weight allocation. Finally, the user prediction category mentioned refers to the classification category obtained through the aforementioned user classification model.

[0058] Specifically, historical user behavior data is input into the user classification model to be trained. First, preliminary feature extraction is performed on the historical user behavior data to obtain category features, which are represented in the form of a matrix vector. Then, the category features are input into two parallel modules. When inputting into the first module, the zero elements in the matrix vector are removed, that is, the sub-features with a value of zero in the category features are removed. The remaining sub-features are extracted through the capsule module to obtain the first output feature. At the same time, when the category features are input into the second module, the attention module, the similarity of each sub-feature is extracted through the receptive field of the similarity feature to obtain the second output feature. Finally, the first output feature and the second output feature are merged, and the weights of the first output feature are assigned based on the second output feature to obtain the merged output feature. The merged output feature is then processed for data output to obtain the user predicted category.

[0059] Step S103: Use the user prediction category to train the user classification model to be trained, and obtain the trained user classification model.

[0060] Among them, the trained user classification model is a model that has been trained and can be used for user classification.

[0061] Specifically, the actual category corresponding to the historical user data is calculated. When the difference between the predicted user category and the actual category is less than a preset threshold, the trained user classification model is obtained; otherwise, training continues.

[0062] In the aforementioned user classification model training method, historical user behavior data is acquired; this data is input into the user classification model to be trained, and the model obtains category features corresponding to the historical user behavior data, including multiple sub-features. The first module of the user classification model filters these category features to obtain filtered category features, which are then extracted to obtain a first output feature. A second module of the model extracts similarity between each sub-feature to obtain a second output feature. The first and second output features are then merged to obtain the predicted user category. The predicted user category is then used to train the user classification model, resulting in a trained user classification model. This application extracts category features from the first module to obtain the first output feature, then merges the second output feature obtained from the second module with the first output feature, weighting the second output feature to obtain the predicted user category. Finally, the predicted user category is used to train the user classification model, resulting in a more accurate user classification model.

[0063] In one embodiment, filtering the category features through the first module to obtain the filtered category features includes the following steps: removing sub-features with a feature value of zero from the category features to obtain the filtered category features.

[0064] Specifically, the category feature can be a matrix vector. Removing the elements with a value of zero from this matrix vector yields the filtered matrix vector, which is the filtered category feature.

[0065] In this embodiment, by removing sub-features with a feature value of zero from the category features, we can obtain filtered category features with obvious features.

[0066] In one embodiment, such as Figure 2 As shown, the second module extracts similarity from each sub-feature to obtain the second output feature, including the following steps:

[0067] Step S201: Obtain the initial similarity between the current sub-feature and the remaining sub-features, and fuse the initial similarity to obtain the similarity corresponding to the current sub-feature; the similarity is used to characterize the weight of the current sub-feature in the category features, and the remaining sub-features are the sub-features other than the current sub-feature among multiple sub-features.

[0068] The remaining sub-features are each sub-feature other than the current sub-feature, the initial similarity is the similarity between the current sub-feature and a single sub-feature among the remaining sub-features, and the weight is the proportion of each sub-feature in the category features.

[0069] Specifically, the initial similarity between any one sub-feature and each of the other sub-features is calculated, and then these initial similarities are fused to obtain the similarity corresponding to each sub-feature, as shown in the following expression:

[0070]

[0071]

[0072] in, w represents the similarity between each sub-feature. θ and They are all 1x1 convolutions.

[0073] Step S202: Based on the similarity and the pre-obtained activation function, the second output feature is obtained.

[0074] The activation function obtained beforehand is the sigma activation function.

[0075] Specifically, the similarities corresponding to each sub-feature are merged to obtain the receptive field S of the similarity feature, and then the second output feature is obtained through the sigma activation function, the expression of which is as follows:

[0076] o at (i) = σ(S);

[0077] The output of the attention module is o. at (i), where σ is the sigma activation function.

[0078] In this embodiment, by extracting the similarity corresponding to each sub-feature in the category feature, the weight of each sub-feature in the category feature can be accurately obtained.

[0079] In one embodiment, such as Figure 3 As shown, the user classification model to be trained is trained using the user predicted category to obtain the trained user classification model, including the following steps:

[0080] Step S301: Based on historical user behavior data, obtain the actual user category corresponding to the historical user behavior data.

[0081] The actual user category is the user's true category based on historical user behavior data.

[0082] Specifically, the user's true category is calculated using historical user behavior data.

[0083] Step S302: Based on the predicted user category and the actual user category, obtain the loss function value of the user classification model to be trained.

[0084] The loss function value is the cross-entropy loss function value of the user classification model.

[0085] Specifically, the expression for the cross-entropy loss function is as follows:

[0086]

[0087] Among them, y n and P n Let N be the predicted user category and the actual user category, respectively, and N be the batch size.

[0088] Step S303: Based on the loss function value, the trained user classification model is obtained.

[0089] Specifically, when the loss function value is less than a preset threshold, the trained user classification model is obtained.

[0090] In this embodiment, by calculating the loss function value, the trained user classification model can be accurately obtained.

[0091] In one embodiment, such as Figure 4As shown, historical user behavior data includes: first historical user behavior data with continuous text, and second historical user behavior data without continuous text; the category features corresponding to the historical user behavior data are obtained through a user classification model, including the following steps:

[0092] Step S401: Based on the first historical user behavior data, obtain the first category features.

[0093] Among them, continuous text refers to continuous text in historical user behavior data, while the first historical user behavior data refers to historical user behavior data with continuous text. The first category feature is the category feature obtained by extracting key text based on continuous text.

[0094] Specifically, key text is extracted from continuous text in historical user behavior data to obtain the first category of features.

[0095] Step S402: Based on the second historical user behavior data, obtain the second category features.

[0096] The second historical user behavior data is historical user behavior data without continuous text, that is, historical user behavior data with a large amount of numerical data, while the second category feature is the category feature obtained based on the preliminary feature extraction of the second historical user behavior data.

[0097] Specifically, preliminary feature extraction is performed on the second historical user behavior data to obtain the second category of features.

[0098] Step S403: The first category feature and the second category feature are fused to obtain the category feature.

[0099] Specifically, the first category feature and the second category feature can be matrix vectors. The two matrix vectors are fused to obtain the category feature.

[0100] In this embodiment, by performing preliminary feature extraction on category features with continuous text and category features without continuous text respectively, category features can be accurately obtained.

[0101] In one embodiment, step S401 includes the following steps: inputting first historical user behavior data into a long short-term memory module; filtering the first historical user behavior data through an input gate, a forget gate, and an output gate in the long short-term memory module to obtain output data corresponding to the first historical user behavior data; and transforming the output data through a normalized exponential function to obtain a first category feature.

[0102] Among them, the Long Short-Term Memory module is a module with a Long Short-Term Memory model, which is used to perform preliminary feature extraction on the first historical user behavior data with continuous text. The input gate is a data valve that controls the input of data, the forget gate is a data valve that forgets some useless data, and the input gate is a data valve that controls the output of data.

[0103] Specifically, the first historical user behavior data is input into the Long Short-Term Memory (LSTM) model, which includes an input gate i. t Forgotten Gate t Output gate o t , The expressions used to control the passage of data and filter the data are as follows:

[0104] i t =σ(W i ·[h t-1 ,x t ]+b i ), f t =σ(W f ·[h t-1 ,x t ]+b f ), o t =σ(W o ·[h t-1 ,x t ]+b o ),

[0105]

[0106] Where σ represents the sigma activation function, h t-1 c t-1 h represents the input hidden state of the Long Short-Term Memory (LSTM) model at time t-1. t c t Let x represent the output hidden state of the above LSTM model at time t. t This indicates that the Long Short-Term Memory (LSTM) model receives continuous text data at time t, and W... i W f W o W c b represents the network weights of the Long Short-Term Memory (LSTM) model. i ,b f ,b o ,b c This represents the bias vector of the Long Short-Term Memory (LSTM) model. The LTM model is a loop structure that takes the first historical user behavior data, consisting of continuous text, as input, continuously filters the continuous text data to obtain the output data corresponding to the first historical user behavior data, and finally transforms the output data using a normalized exponential function to obtain the first category feature.

[0107] In this embodiment, by inputting the first historical user behavior data with continuous text into the long short-term memory model, the first category feature can be accurately obtained.

[0108] In one embodiment, step S402 includes the following steps:

[0109] Based on the second historical user behavior data, the initial information entropy corresponding to the second historical user behavior data is obtained; based on the initial information entropy, the information gain value corresponding to the second historical user behavior data is obtained; based on the information gain value, the second category feature is obtained.

[0110] Wherein, the initial information entropy is the average amount of information in the second historical user behavior data after eliminating redundancy, while the information gain value is the difference between the initial information entropy and the second historical user behavior data.

[0111] Specifically, the second set of historical user behavior data can be a dataset F = {f1, f2, ..., f...} m}, where sub-data f m Contains n types of feature values ​​f m1 ,f m2 ,f mn The probability corresponding to each type of feature value is P = {p1, p2, ..., p...} m Assume the user category set L = {l1, l2, ... l...} s The class is divided into s classes, and the probability of each class is... Then f m The initial information entropy calculation formulas for L and L are as follows:

[0112]

[0113] Then f m The information gain of the feature column is:

[0114] IGain(L,f m )=E(L)-E(f m );

[0115] The dataset F = {f1, f2, ..., f2} is the second set of historical user behavior data. m According to information gain IGain(L,f) m Arrange them to obtain the second category features.

[0116] In this embodiment, the information gain value corresponding to the second historical user behavior data can be used to accurately obtain the second category feature.

[0117] In one embodiment, such as Figure 5As shown, a user classification method is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0118] Step S501: Obtain user behavior data.

[0119] This includes user business processing data after preprocessing user behavior data.

[0120] Specifically, user behavior data is retrieved from the database.

[0121] Step S502: Input user behavior data into the pre-trained user classification model.

[0122] Among them, the user classification model is a model that classifies user behavior data and then classifies users, which is used to quickly and accurately classify users' credit scores.

[0123] Specifically, user behavior data is input into a pre-trained user classification model.

[0124] Step S503: Based on user behavior data, obtain the user classification results corresponding to the user behavior data through a user classification model.

[0125] Among them, the user classification result is the user's credit classification result.

[0126] Specifically, the user classification results corresponding to user behavior data are obtained through the above user classification model.

[0127] In the user classification method described above, user behavior data is acquired; this data is then input into a pre-trained user classification model; and based on the user behavior data, the user classification model yields the corresponding user classification result. This user classification method can accurately classify user behavior data, and further, accurately classify users.

[0128] In one embodiment, such as Figure 6 As shown, a user classification model training method is provided. This user classification model includes: an input module, a non-continuous text data feature extraction module, a continuous text data long short-term memory model module, a main capsule module, a high-dimensional capsule module, and a side-channel attention module. The side-channel attention module weights the features output by the high-dimensional capsule module. The specific steps are as follows:

[0129] 1. User historical data preprocessing

[0130] 1.1 Obtain historical business data from customers and perform data preprocessing to obtain the raw dataset.

[0131] 1.2 Data cleaning includes discrete value removal and missing value imputation. The imputation of numerical and categorical missing values ​​is handled as follows:

[0132] V x,y =avg(V x,j ),j=1...n,V x,y =F max (V x,j ),j=1...n;

[0133] Among them, V x,y For the missing values ​​at position (x,y) in the original data, avg(V) x,j F represents the average value of data with the same characteristics. max (V x,j The value that is most frequently displayed for data with the same characteristics is the one that is most frequently displayed.

[0134] 2. Obtain feature vectors for model training

[0135] 2.1 Assume the original dataset has feature columns F = {f1, f2, ..., f...} m}, where f m The feature column contains n types of feature values ​​f m1 ,f m2 ,f mn The probability corresponding to each type of feature value is P = {p1, p2, ..., p...} m Assume the decision column contains s class label values ​​and their corresponding probabilities as L = {l1, l2, ... ln}. s}and Then f m The formulas for calculating the initial information entropy of the feature column and the decision column are as follows:

[0136]

[0137] Then f m The information gain of the feature column is:

[0138] IGain(L,f m )=E(L)-E(f m );

[0139] Feature column F = {f1, f2, ..., f m According to information gain IGain(L,f) m Arrange the eigenvectors to obtain the eigenvector R1;

[0140] 2.2 Input the data with continuous text from the original dataset into the Long Short-Term Memory (LSTM) model, such as... Figure 7 As shown, the Long Short-Term Memory (LSTM) model includes an input gate i t Forgotten Gate t Output gate o t , The expressions used to control the passage of data and filter the data are as follows:

[0141] i t =σ(W i ·[h t-1 ,x t ]+b i ), f t =σ(W f ·[h t-1 ,x t ]+b f ), o t =σ(W o ·[h t-1 ,x t ]+b o )

[0142]

[0143] Where σ represents the sigmoid activation function, h t-1 c t-1 h represents the input hidden state of the LSTM model at time t-1. t c t Let x represent the output hidden state of the above LSTM model at time t. t This indicates that the LSTM model receives continuous text data at time t, W i W f W o W c b represents the network weights of the LSTM model. i ,b f ,b o ,b c This represents the bias vector of the LSTM model described above. This represents the product of two values. This indicates the addition of two values.

[0144] 2.3 The LSTM model is a loop structure. After inputting continuous text data, it continuously filters the continuous text data to extract the feature vector R2. Then, the feature vectors R1 and R2 are fused to obtain the feature vector I = {i1, i2, ..., i...} n}

[0145] 3. User classification model training

[0146] A user classification model is obtained by constructing a model based on capsule networks and attention mechanisms and iteratively updating the model based on the input layer feature vectors.

[0147] 3.1 The input layer of the user classification model consists of I = {i1, i2, ..., i...} n The sequence of n-dimensional vectors corresponding to the basic features formed by}.

[0148] 3.2 In the capsule network model, customer features are sequentially passed through each functional layer using a dynamic routing algorithm. The dynamic routing algorithm calculates the relationships between capsules in different hidden functional layers. The formula for the manifestation coefficient connecting upper capsule i to its adjacent lower capsule j is as follows:

[0149]

[0150] Where C ij Let b be the visibility coefficient from capsule i to capsule j. ij It is the prior probability that capsule i is connected to capsule j, and its initial value is set to 0, according to the coupling coefficient C. ij Calculate the input vector s j The formula is as follows:

[0151]

[0152] Among them, u i For the output of the upper capsule, W is the prediction vector for the upper capsule. ij The weight transformation matrix between two adjacent layers in the network, and the output vector v j The calculation formula is as follows:

[0153]

[0154] The routing vectors between different functional layers are adjusted according to the model iteration process, with a routing bias value b. ij The update formula is as follows:

[0155]

[0156] 3.3 The above feature vectors are first processed by the main capsule module to extract features, resulting in the main capsule layer output features o. pri (i), change o pri (i) Input into the high-dimensional capsule module, first remove o pri (i) takes a sub-feature with a value of zero, and then further extracts features to obtain the output features of the high-dimensional capsule module. hd (i), o hdThe expression for (i) is:

[0157] o hd (i)=σ(w*o pri (i)+b);

[0158] Where x(i) is the input feature of the attention module, w is the module weight, b is the module bias parameter, and σ is the activation function.

[0159] 3.4、o pri (i) contains multiple sub-features. Calculate the initial similarity between any one sub-feature and each of the other sub-features, then fuse these initial similarities to obtain the similarity corresponding to a certain sub-feature. Finally, merge the similarities corresponding to each sub-feature to obtain the receptive field S of the similarity feature, which is expressed as follows:

[0160]

[0161]

[0162] Among them, w θ and All are 1x1 convolutions; the output of the attention module is O. at (i), the expression is as follows:

[0163] o at (i) = σ(S);

[0164] The user classification model fusion layer will use the output of the high-dimensional capsule layer. hd (i) and the output of the attention module o at (i) Merging: The features output by the high-dimensional capsule module are weighted by the bypass attention module, and the merging expression is as follows:

[0165]

[0166] y is the user classification output vector.

[0167] 3.5 Assuming there exists a level k, the expression for the model's cross-entropy loss function L is as follows:

[0168]

[0169] Among them, y n and P n Let N be the predicted class vector and the true class vector of the nth sample, and N be the batch size. When the cross-entropy loss function value L is determined to be greater than or equal to the preset error threshold, the gradient g of the portrait model loss function is calculated. y =▽ y L, then based on the gradient of the loss function g yCalculate the momentum parameter m of the optimizer t ,v t Its expression is as follows:

[0170]

[0171] Among them, g y β1 and β2 represent the gradient of the model loss function, respectively, and the hyperparameters of the optimizer are typically 0.9 and 0.999. t ,v t This represents the first-order and second-order estimates of the profile model by the optimizer, specifically with initial values ​​m0 and v0 set to 0; the optimizer's momentum parameter m t ,v t The update expression is as follows:

[0172]

[0173] in, Let be the hyperparameters of the optimizer model at time t, based on the momentum parameter. The weight matrices of the input layer, hidden layer, and output layer in the image model are adjusted as follows:

[0174]

[0175]

[0176] Then based on momentum parameters The weight matrices of the input layer, hidden layer, and output layer in the image model are adjusted as follows:

[0177]

[0178]

[0179] Where α and ε represent the hyperparameters of the optimizer, with typical values ​​of 10 and 10, respectively. -3 and 10 -8 ;

[0180] Then, based on the obtained model weight matrix and bias matrix at time t, calculate the model output y at time t+1. t+1 The model cross-entropy loss function value L′ is calculated; when the cross-entropy loss function value L′ is less than the preset error threshold, the target profile model is obtained; otherwise, the next model training process continues.

[0181] 3.6 Input the user data to be classified into the obtained target profile model to obtain the current customer's category; assume the threshold for level N is... and φ, if the current customer's value h satisfies Under certain conditions, the customer level is determined to be N.

[0182] In this embodiment, by adding a parallel attention module to the high-dimensional capsule module and weighting the features output by the high-dimensional capsule module, the classification results of the trained user classification model can be made more accurate.

[0183] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0184] Based on the same inventive concept, this application also provides a user classification model training apparatus for implementing the user classification model training method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more user classification model training apparatus embodiments provided below can be found in the limitations of the user classification model training method described above, and will not be repeated here.

[0185] In one embodiment, such as Figure 8 As shown, a user classification model training device is provided, including: a historical data acquisition module 801, a predicted category acquisition module 802, and a classification model training module 803, wherein:

[0186] Historical data acquisition module 801 is used to acquire historical user behavior data;

[0187] The prediction category acquisition module 802 is used to input historical user behavior data into the user classification model to be trained, and obtain category features corresponding to the historical user behavior data, including multiple sub-features, through the user classification model; the first module of the user classification model filters the category features to obtain filtered category features, and extracts features from the filtered category features to obtain the first output feature; the second module of the user classification model extracts the similarity of each sub-feature to obtain the second output feature; the first output feature and the second output feature are merged to obtain the user prediction category;

[0188] The classification model training module 803 is used to train the user classification model to be trained using the user's predicted category, so as to obtain the trained user classification model.

[0189] In one embodiment, the predicted category acquisition module 802 is further used to remove sub-features with a feature value of zero from the category features to obtain filtered category features.

[0190] In one embodiment, the prediction category acquisition module 802 is further used to acquire the initial similarity between the current sub-feature and the remaining sub-features, and to fuse the initial similarity to obtain the similarity corresponding to the current sub-feature; the similarity is used to characterize the weight of the current sub-feature in the category feature, and the remaining sub-features are the sub-features other than the current sub-feature among multiple sub-features; based on the similarity and the pre-obtained activation function, the second output feature is obtained.

[0191] In one embodiment, the classification model training module 803 is further configured to obtain the actual user category corresponding to the historical user behavior data based on the historical user behavior data; obtain the loss function value of the user classification model to be trained based on the predicted user category and the actual user category; and obtain the trained user classification model based on the loss function value.

[0192] In one embodiment, the historical data acquisition module 801 is further configured to obtain a first category feature based on the first historical user behavior data; obtain a second category feature based on the second historical user behavior data; and fuse the first category feature and the second category feature to obtain a category feature.

[0193] In one embodiment, the historical data acquisition module 801 is further configured to input the first historical user behavior data into the long short-term memory module, filter the first historical user behavior data through the input gate, forget gate and output gate in the long short-term memory module to obtain the output data corresponding to the first historical user behavior data, and transform the output data through the normalized exponential function to obtain the first category feature.

[0194] In one embodiment, the historical data acquisition module 801 is further configured to obtain the initial information entropy corresponding to the second historical user behavior data based on the second historical user behavior data; obtain the information gain value corresponding to the second historical user behavior data based on the initial information entropy; and obtain the second category feature based on the information gain value.

[0195] Each module in the aforementioned user classification model training device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0196] In one embodiment, such as Figure 9 As shown, a user classification device is provided, including: a user behavior data acquisition module 901, a user behavior data input module 902, and a user classification result acquisition module 903, wherein:

[0197] User behavior data acquisition module 901 is used to acquire user behavior data;

[0198] User behavior data input module 902 is used to input user behavior data into a pre-trained user classification model;

[0199] The user classification result acquisition module 903 is used to obtain the user classification result corresponding to the user behavior data based on the user behavior data and through the user classification model.

[0200] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a user classification model training method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0201] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0202] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0203] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0204] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0205] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0206] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0207] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0208] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A user classification model training method, characterized in that, The method includes: Obtain historical user behavior data; The historical user behavior data is input into a user classification model to be trained. The user classification model obtains category features corresponding to the historical user behavior data, including multiple sub-features. The first module of the user classification model filters the category features to obtain filtered category features, and extracts features from the filtered category features to obtain a first output feature. The second module of the user classification model extracts similarity between each sub-feature to obtain a second output feature. The first output feature and the second output feature are merged to obtain the predicted user category. The user classification model to be trained is trained using the predicted user categories to obtain the trained user classification model.

2. The method according to claim 1, characterized in that, The step of filtering the category features through the first module to obtain the filtered category features includes: The sub-features with a feature value of zero in the category features are removed to obtain the filtered category features.

3. The method according to claim 2, characterized in that, The second module extracts similarity from each sub-feature to obtain the second output feature, including: The initial similarity between the current sub-feature and the remaining sub-features is obtained, and the initial similarity is fused to obtain the similarity corresponding to the current sub-feature; the similarity is used to characterize the weight of the current sub-feature in the category feature, and the remaining sub-features are the sub-features other than the current sub-feature among the multiple sub-features; Based on the similarity and the pre-obtained activation function, the second output feature is obtained.

4. The method according to claim 1, characterized in that, The step of training the user classification model to be trained using the predicted user category to obtain the trained user classification model includes: Based on the historical user behavior data, the actual user category corresponding to the historical user behavior data is obtained; Based on the predicted user category and the actual user category, the loss function value of the user classification model to be trained is obtained; Based on the loss function value, the trained user classification model is obtained.

5. The method according to claim 1, characterized in that, The historical user behavior data includes: first historical user behavior data with continuous text, and second historical user behavior data without continuous text; obtaining the category features corresponding to the historical user behavior data through the user classification model includes: Based on the first historical user behavior data, the first category features are obtained; Based on the second historical user behavior data, the second category features are obtained; The first category feature and the second category feature are fused together to obtain the category feature.

6. The method according to claim 5, characterized in that, The user classification model includes a long short-term memory module; the first category feature obtained based on the first historical user behavior data includes: The first historical user behavior data is input into the long short-term memory module. The first historical user behavior data is filtered through the input gate, forget gate and output gate in the long short-term memory module to obtain the output data corresponding to the first historical user behavior data. The output data is transformed using a normalized exponential function to obtain the first category feature.

7. The method according to claim 6, characterized in that, The second category of features obtained based on the second historical user behavior data includes: Based on the second historical user behavior data, the initial information entropy corresponding to the second historical user behavior data is obtained; Based on the initial information entropy, the information gain value corresponding to the second historical user behavior data is obtained; Based on the information gain value, the second category feature is obtained.

8. A user classification method, characterized in that, The method includes: Obtain user behavior data; The user behavior data is input into a pre-trained user classification model; Based on the user behavior data, the user classification result corresponding to the user behavior data is obtained through the user classification model; wherein, the pre-trained user classification model is trained by the user classification model training method according to any one of claims 1 to 7.

9. A user classification model training device, characterized in that, The device includes: The historical data acquisition module is used to acquire historical user behavior data; The predicted category acquisition module is used to input the historical user behavior data into a user classification model to be trained, and obtain category features including multiple sub-features corresponding to the historical user behavior data through the user classification model; filter the category features through the first module of the user classification model to obtain filtered category features, and extract features from the filtered category features to obtain a first output feature; extract similarity between each sub-feature through the second module of the user classification model to obtain a second output feature; and merge the first output feature and the second output feature to obtain the predicted user category. The classification model training module is used to train the user classification model to be trained using the user prediction category, so as to obtain the trained user classification model.

10. A user classification device, characterized in that, The device includes: The user behavior data acquisition module is used to acquire user behavior data. The user behavior data input module is used to input the user behavior data into a pre-trained user classification model; The user classification result acquisition module is used to obtain the user classification result corresponding to the user behavior data based on the user behavior data and through the user classification model; wherein, the pre-trained user classification model is trained by the user classification model training method according to any one of claims 1 to 7.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images