Method, device and equipment for classifying users based on user feature data

Abstract

The embodiment of the invention discloses a method, device and equipment for classifying users based on user feature data. The method comprises the following steps: after to-be-classified table data containing various user feature data is obtained, receiving the to-be-classified table data by utilizing a feature coding layer in a classification model, and outputting first feature vector sequence data; wherein Mask Embedding included in the feature coding layer is used for carrying out vector filling on user feature data naturally missing in the table data to be classified; after a Transform Encoder layer in a classification model is used for receiving the first feature vector sequence data, second feature vector sequence data are output; and finally, performing classification processing according to the second feature vector sequence data by utilizing an output layer in a classification model to obtain a classification result for the table data to be classified.

Description

Technical Field

[0001] The present application relates to the technical field of classification models, and in particular to a method, apparatus, and device for classifying users based on user feature data. Background Art

[0002] Missing data is a common problem in the field of tabular data. When counting user feature data, various factors often make it impossible to fully count all feature data. This leads to low accuracy in user classification predictions based on statistics of users with some missing feature data.

[0003] The commonly used processing method at present is to use statistical values ​​(such as mean, median, etc.) or specific values ​​(such as 0 or 1) to fill in the missing data, but this will destroy the distribution pattern of the data. The distribution of the filled data is usually quite different from the actual data distribution, resulting in a low accuracy rate in user classification prediction.

[0004] In view of this, it is necessary to provide a solution for classifying users based on user feature data with high accuracy. Summary of the Invention

[0005] In view of this, embodiments of the present application provide a method, apparatus, and device for classifying users based on user feature data, so as to provide a highly accurate solution for classifying users based on user feature data.

[0006] To solve the above technical problems, an embodiment of this specification provides a method for classifying users based on user feature data. The method uses a classification model, and the classification model includes: a feature coding layer, a Transformer Encoder layer, and an output layer; wherein the output end of the feature coding layer is connected to the input end of the Transformer Encoder layer, and the output end of the Transformer Encoder layer is connected to the input end of the output layer. The method includes:

[0007] Acquire table data to be classified; the table data to be classified includes a variety of user characteristic data; the user characteristic data is data used to reflect user characteristics; the table data to be classified contains naturally missing user characteristic data;

[0008] The feature coding layer receives the table data to be classified and outputs first feature vector sequence data; the Mask Embedding included in the feature coding layer is used to perform vector filling on the naturally missing user feature data;

[0009] After receiving the first feature vector sequence data using the Transformer Encoder layer, output second feature vector sequence data;

[0010] The output layer is used to perform classification processing based on the second feature vector sequence data to obtain a classification result for the table data to be classified.

[0011] An embodiment of this specification provides a classification model training method, wherein the classification model includes: a feature coding layer, a Transformer Encoder layer, and an output layer; wherein the output end of the feature coding layer is connected to the input end of the Transformer Encoder layer, and the output end of the Transformer Encoder layer is connected to the input end of the output layer, and the method includes:

[0012] Obtain a first training sample set; the training samples in the first training sample set are tabular data containing multiple user feature data and corresponding sample classification labels; there is naturally missing user feature data in the first training sample set; the user feature data is data used to reflect user features;

[0013] Performing masking on the first training sample set to obtain a second training sample set; the masking is used to perform masking on at least part of the user feature data in the first training sample set;

[0014] The classification model is trained using the first training sample set and the second training sample set to obtain a trained classification model.

[0015] The embodiments of this specification also provide a device for classifying users based on user feature data. The device uses a classification model, which includes: a feature coding layer, a Transformer Encoder layer, and an output layer; wherein the output end of the feature coding layer is connected to the input end of the Transformer Encoder layer, and the output end of the Transformer Encoder layer is connected to the input end of the output layer. The device includes:

[0016] An acquisition module is used to acquire table data to be classified; the table data to be classified includes a variety of user characteristic data; the user characteristic data is data used to reflect user characteristics; there is naturally missing user characteristic data in the table data to be classified;

[0017] A first output module is configured to receive the table data to be classified using the feature coding layer and output first feature vector sequence data; the Mask Embedding included in the feature coding layer is configured to perform vector filling on the naturally missing user feature data;

[0018] a second output module, configured to output second feature vector sequence data after receiving the first feature vector sequence data using the Transformer Encoder layer;

[0019] The classification module is used to utilize the output layer to perform classification processing according to the second feature vector sequence data to obtain a classification result for the table data to be classified.

[0020] The embodiments of this specification also provide a classification model training device, wherein the classification model includes: a feature coding layer, a Transformer Encoder layer, and an output layer; wherein the output end of the feature coding layer is connected to the input end of the Transformer Encoder layer, and the output end of the Transformer Encoder layer is connected to the input end of the output layer, and the device includes:

[0021] an acquisition module configured to acquire a first training sample set; the training samples in the first training sample set are tabular data containing a variety of user feature data and corresponding sample classification labels; the first training sample set contains naturally missing user feature data; the user feature data is data used to reflect user features;

[0022] a masking module, configured to perform masking processing on the first training sample set to obtain a second training sample set; wherein the masking processing is configured to perform masking processing on at least part of the user feature data in the first training sample set;

[0023] The training module is used to train the classification model using the first training sample set and the second training sample set to obtain a trained classification model.

[0024] The embodiments of this specification also provide a device for classifying users based on user feature data, including:

[0025] at least one processor; and,

[0026] a memory communicatively connected to the at least one processor; wherein,

[0027] The memory stores a classification model, the classification model including: a feature encoding layer, a Transformer Encoder layer, and an output layer; wherein an output end of the feature encoding layer is connected to an input end of the Transformer Encoder layer, and an output end of the Transformer Encoder layer is connected to an input end of the output layer. The memory further stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so as to enable the at least one processor to:

[0028] Acquire table data to be classified; the table data to be classified includes a variety of user characteristic data; the user characteristic data is data used to reflect user characteristics; the table data to be classified contains naturally missing user characteristic data;

[0029] The feature coding layer receives the table data to be classified and outputs first feature vector sequence data; the Mask Embedding included in the feature coding layer is used to perform vector filling on the naturally missing user feature data;

[0030] After receiving the first feature vector sequence data using the Transformer Encoder layer, output second feature vector sequence data;

[0031] The output layer is used to perform classification processing based on the second feature vector sequence data to obtain a classification result for the table data to be classified.

[0032] The present invention also provides a classification model training device, including:

[0033] at least one processor; and,

[0034] a memory communicatively connected to the at least one processor; wherein,

[0035] The memory stores a classification model, the classification model including: a feature encoding layer, a Transformer Encoder layer, and an output layer; wherein an output end of the feature encoding layer is connected to an input end of the Transformer Encoder layer, and an output end of the Transformer Encoder layer is connected to an input end of the output layer. The memory further stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so as to enable the at least one processor to:

[0036] Obtain a first training sample set; the training samples in the first training sample set are tabular data containing multiple user feature data and corresponding sample classification labels; there is naturally missing user feature data in the first training sample set; the user feature data is data used to reflect user features;

[0037] Performing masking on the first training sample set to obtain a second training sample set; the masking is used to perform masking on at least part of the user feature data in the first training sample set;

[0038] The classification model is trained using the first training sample set and the second training sample set to obtain a trained classification model.

[0039] At least one embodiment provided in this specification can achieve the following beneficial effects:

[0040] In an embodiment of the present specification, after obtaining the table data to be classified containing multiple user feature data, the feature coding layer in the classification model is used to receive the table data to be classified and output the first feature vector sequence data; wherein, the Mask Embedding included in the feature coding layer is used to perform vector filling for the user feature data that is naturally missing in the table data to be classified; then, the Transformer Encoder layer in the classification model is used to receive the first feature vector sequence data and output the second feature vector sequence data; finally, the output layer in the classification model is used to perform classification processing based on the second feature vector sequence data to obtain the classification result for the table data to be classified. Since the Mask Embedding and Transformer Encoder layers can be continuously optimized during the model training process of the classification model, they learn the ability to predict the missing user feature data based on other non-missing user feature data. Therefore, by using Mask Embedding to perform vector filling for the user feature data that is naturally missing in the table data to be classified, the filled vector can be closer to the missing true value, which is conducive to improving the accuracy of classifying users based on user feature data. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0042] Figure 1A schematic diagram of an application scenario of a method for classifying users based on user feature data provided in an embodiment of this specification;

[0043] Figure 2 A flowchart of a method for classifying users based on user feature data provided in an embodiment of this specification;

[0044] Figure 3 A schematic diagram of a model architecture of a classification model provided in an embodiment of this specification;

[0045] Figure 4 A flow chart of a classification model training method provided in an embodiment of this specification;

[0046] Figure 5 An overall flow chart of a classification model training method provided in an embodiment of this specification;

[0047] Figure 6 The embodiments of this specification provide corresponding Figure 2 A schematic structural diagram of a device for classifying users based on user feature data;

[0048] Figure 7 The embodiments of this specification provide corresponding Figure 4 A structural diagram of a classification model training device;

[0049] Figure 8 The embodiments of this specification provide corresponding Figure 2 A schematic structural diagram of a device for classifying users based on user feature data;

[0050] Figure 9 The embodiments of this specification provide corresponding Figure 4 A structural diagram of a classification model training device. DETAILED DESCRIPTION

[0051] The following description sets forth many specific details to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present application. Therefore, the present application is not limited to the specific implementations disclosed below.

[0052] The terms used in one or more embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present application. The singular forms "a", "the" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.

[0053] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0054] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0055] In related technologies, there are scenarios where user classification prediction is performed based on tabular data with missing user feature data. This is usually done by calculating statistical values ​​(such as mean, median, etc.) of other data in the data column containing the missing user feature data in the tabular data, or by using specific values ​​(such as 0 or 1) to fill in the missing data. However, this will destroy the data distribution pattern, and the distribution of the filled data is usually significantly different from the actual data distribution. As a result, when using the prediction model to perform user classification prediction based on the filled data, the classification results output by the prediction model are less accurate.

[0056] In order to solve the defects in the related art, this solution provides the following embodiments.

[0057] Figure 1 A schematic diagram of an application scenario of a method for classifying users based on user feature data provided in an embodiment of this specification.

[0058] like Figure 1As shown, after table data 101 to be classified, which includes multiple user feature data, is input into a classification model 102, classification model 102 can determine a classification result 103 for table data 101 based on table data 101. Specifically, classification model 102 includes a feature encoding layer, a Transformer Encoder layer, and an output layer; wherein the output of the feature encoding layer is connected to the input of the Transformer Encoder layer, and the output of the Transformer Encoder layer is connected to the input of the output layer.

[0059] In practical applications, the feature encoding layer in the classification model 102 can output first feature vector sequence data after encoding the table data to be classified 101; wherein, the Mask Embedding included in the feature encoding layer is used to perform vector filling processing on the user feature data that is naturally missing in the table data to be classified 101. The feature encoding layer transmits the output first feature vector sequence data to the Transformer Encoder layer. After receiving the first feature vector sequence data, the Transformer Encoder layer can output second feature vector sequence data. The Transformer Encoder layer can transmit the second feature vector sequence data to the output layer. The output layer can perform classification processing based on the second feature vector sequence data to obtain a classification result 103 for the table data to be classified 101.

[0060] Figure 1In the method, after obtaining the table data to be classified containing multiple user feature data, the feature coding layer in the classification model is used to receive the table data to be classified and output the first feature vector sequence data; wherein, the Mask Embedding included in the feature coding layer is used to fill the vector of the user feature data that is naturally missing in the table data to be classified; then, the Transformer Encoder layer in the classification model is used to receive the first feature vector sequence data and output the second feature vector sequence data; finally, the output layer in the classification model is used to perform classification processing based on the second feature vector sequence data to obtain the classification result for the table data to be classified. Since the Mask Embedding and Transformer Encoder layers can be continuously optimized during the model training process of the classification model, they learn the ability to predict the missing user feature data based on other non-missing user feature data. Therefore, using Mask Embedding to fill the vector of the naturally missing user feature data in the table data to be classified can make the filled vector closer to the missing true value, which is conducive to improving the accuracy of user classification based on user feature data.

[0061] Figure 2 The present invention provides a flowchart of a method for classifying users based on user feature data according to an embodiment of the present invention. The method uses a classification model to classify users based on user feature data.

[0062] Figure 3 A schematic diagram of a model architecture of a classification model provided in an embodiment of this specification, such as Figure 3 As shown, the classification model 300 includes: a feature coding layer 301, a Transformer Encoder layer 302, and an output layer 303; wherein the output end of the feature coding layer 301 is connected to the input end of the Transformer Encoder layer 302, and the output end of the Transformer Encoder layer 302 is connected to the input end of the output layer 303.

[0063] like Figure 2 As shown, the process may include the following steps:

[0064] Step 202: Acquire table data to be classified; the table data to be classified includes a variety of user characteristic data; the user characteristic data is data used to reflect user characteristics; there is naturally missing user characteristic data in the table data to be classified.

[0065] In the embodiments of this specification, the table data to be classified may be user feature data presented in a tabular format and may include multiple rows and columns. The first row may include user feature names corresponding to various user feature data, such as user education level, user ID, user city, user loan amount, etc. The second to last rows of the table data to be classified may include specific user feature data, and the category of the user feature data corresponding to each column is consistent and corresponds to the user feature name in the first row of that column. For example, the user feature data in each row corresponding to the "User City" column may include: Beijing, Shanghai, Guangdong, Henan, etc.

[0066] In practical applications, the table data to be classified can contain a variety of user feature data. This user feature data is used to reflect user characteristics, allowing users to be classified based on this user feature data. For example, based on a user's loan and income data, it can be used to determine whether the user is at high risk of overdue payments. Another example is to recommend products that may be of interest to the user based on their historical purchasing behavior data.

[0067] In actual applications, when counting user feature data, some features may not be fully counted due to various factors, or some user feature data may be missing due to certain reasons. These user feature data that are originally missing in the table data to be classified are naturally missing user feature data.

[0068] Step 204: Utilize the feature coding layer to receive the table data to be classified and output first feature vector sequence data; the Mask Embedding included in the feature coding layer is used to perform vector filling on the naturally missing user feature data.

[0069] In the embodiments of this specification, the feature coding layer can encode the user feature data in the table data to be classified into the form of a feature vector to facilitate model processing. Specifically, the feature coding layer may include a BERT model, a linear layer and a Mask Embedding (mask embedding), wherein the BERT model can convert text-type user feature data or feature name data into feature vector data, the linear layer can convert numerical user feature data into feature vector data, and Mask Embedding can perform vector filling on the user feature data that is naturally missing in the table data to be classified to obtain feature vector data. Since the subsequent embodiments in the embodiments of this specification will explain in detail the process of encoding and processing user feature data by the BERT model, the linear layer and the Mask Embedding (mask embedding), it will not be repeated here.

[0070] In the embodiments of this specification, Mask Embedding can perform vector filling processing on naturally missing user feature data or masked user feature data to obtain a corresponding unified feature vector to represent these naturally missing user feature data or masked user feature data. Mask Embedding has learning capabilities. During the model training process, the parameters of Mask Embedding can be continuously iteratively optimized, thereby improving the accuracy of the model's prediction of masked user feature data or naturally missing user feature data.

[0071] Step 206: After receiving the first feature vector sequence data using the Transformer Encoder layer, output second feature vector sequence data.

[0072] In the embodiments of this specification, the Transformer Encoder is one of the core components of the Transformer model architecture, and is mainly used to extract and encode features of the input sequence. It can efficiently capture long-distance dependencies and contextual information in the sequence through the stacking of the self-attention mechanism and the feed-forward neural network. After training, the Transformer Encoder can have the ability to predict the missing user feature data of a user based on certain user feature data of the user.

[0073] In practical applications, after processing the received first feature vector sequence data using the Transformer Encoder layer, the Transformer Encoder layer may output second feature vector sequence data. The second feature vector sequence data may include prediction information for missing feature data.

[0074] Step 208: Utilize the output layer to perform classification processing based on the second feature vector sequence data to obtain a classification result for the table data to be classified.

[0075] In the embodiment of this specification, the output end of the Transformer Encoder layer is connected to the input end of the output layer. After processing the first feature vector sequence data, the Transformer Encoder layer can output the second feature vector sequence data, and the second feature vector sequence data output by the Transformer Encoder layer serves as the input data of the output layer. After the output layer classifies the second feature vector sequence data, it can obtain the classification result for the table data to be classified.

[0076] In practical applications, the classification results of the table data to be classified can be user classification results for users, classification results for product recommendations for users, or prediction classification results for user behavior prediction, without specific limitations. The classification results of the table data to be classified can be binary classification or multi-classification, without specific limitations.

[0077] In practical applications, the output layer can use the softmax function to classify the received second feature vector sequence data, obtain the probabilities of various classification results, and then use the classification result with the highest probability as the final output. The softmax function, also known as the normalized exponential function, is a generalization of the binary classification function to multi-classification, with the purpose of presenting the multi-classification results in the form of probabilities.

[0078] Figure 2 In the method, after obtaining the table data to be classified containing multiple user feature data, the feature coding layer in the classification model is used to receive the table data to be classified and output the first feature vector sequence data; wherein, the Mask Embedding included in the feature coding layer is used to fill the vector of the user feature data that is naturally missing in the table data to be classified; then, the Transformer Encoder layer in the classification model is used to receive the first feature vector sequence data and output the second feature vector sequence data; finally, the output layer in the classification model is used to perform classification processing based on the second feature vector sequence data to obtain the classification result for the table data to be classified. Since the Mask Embedding and Transformer Encoder layers can be continuously optimized during the model training process of the classification model, they learn the ability to predict the missing user feature data based on other non-missing user feature data. Therefore, using Mask Embedding to fill the vector of the naturally missing user feature data in the table data to be classified can make the filled vector closer to the missing true value, which is conducive to improving the accuracy of user classification based on user feature data.

[0079] based on Figure 2 The method in this specification also provides some specific implementation plans of the method, which are described below.

[0080] Optional, Figure 2 The method in the embodiment can be applied to a credit business scenario; the user feature data may include data used to reflect user features related to the credit business; and the classification result can be used to reflect the user's credit risk.

[0081] In the examples of this specification, credit business may refer to financial activities in which financial institutions (such as banks, consumer finance companies, and microfinance companies) provide financial support to individuals or businesses, with the agreement to repay the principal and interest within a certain period of time in the future. User characteristic data related to credit business may include, but is not limited to: the user's loan amount, the user's city, the user's repaid amount, the user's income, the user's education level, the user's occupation, etc.

[0082] In the embodiments of this specification, the classification results may be used to reflect the credit risk of the user. For example, the classification results may include: the user's credit overdue risk is high, the user's credit overdue risk is low, etc.

[0083] Table 1 is an example table of table data to be classified in a credit business scenario. For example, as shown in Table 1, the first row of this table data includes five feature name data items: "Loan Amount," "City," "Repaid Amount," "Monthly Income," and "Education Level." The second row of this table data includes specific user feature data items, where a "?" next to the repaid amount indicates that the feature is naturally missing or masked. If the data in this table is user feature data for user A, then this table data is input into a classification model, and the classification model outputs a classification result that reflects the level of user A's credit delinquency risk.

[0084] Table 1

[0085] Loan amount City Amount repaid Monthly income Education 30000 Beijing ？ 10000 undergraduate

[0086] Optional, Figure 2 The method can also be applied to a target content recommendation scenario; the user feature data includes data used to reflect user features related to the target content; and the classification result is used to reflect the value of recommending the target content to the user.

[0087] In the embodiments of this specification, the target content may be a product, an advertisement, or other promotional information or coupons, without specific limitation. User characteristic data related to the target content may include, but is not limited to: historical user purchase behavior data for a certain type of product, historical user browsing history for a certain type of advertisement, user purchase preference analysis data, user income, user age, user occupation, etc.

[0088] In actual applications, taking product A as the target content, before pushing product A's advertising information to user A, the value of pushing product A's advertising information to user A can be analyzed based on user characteristic data such as user A's age, occupation, and historical product purchase records. If the classification result indicates that the value of pushing product A's advertising information to user a is low, the advertising information of product A may not be pushed to user A, or advertising information of other products may be pushed to user A.

[0089] In the embodiments of the present specification, by determining the classification results of the value of recommending the target content to the user based on data reflecting user characteristics related to the target content, the target content can be pushed to the user in a targeted manner, which is beneficial to improving the accuracy and push value of the target content push, and is also beneficial to avoiding the disturbance to the user caused by pushing a large amount of target content that the user is not interested in.

[0090] Optional, Figure 2 In the method, the table data to be classified includes feature name data corresponding to various user feature data; the user feature data includes numerical user feature data and / or textual user feature data; the feature encoding layer also includes a BERT model and a linear layer; the BERT model is used to convert textual user feature data or feature name data into feature vector data; the linear layer is used to convert numerical user feature data into feature vector data.

[0091] In the embodiments of this specification, the BERT (Bidirectional Encoder Representations from Transformers) model is a pre-trained language model (PLM). The BERT model can process text data input into the BERT model and convert the text data into feature vector data. The BERT model can use token embedding to convert each word into a fixed-dimensional feature vector.

[0092] In the embodiments of this specification, a linear layer (also known as a fully connected layer or dense layer) is a basic neural network layer. A linear layer performs a linear transformation on the input data, transforming the input vector through a weight matrix and a bias vector, and mapping the input data into a high-dimensional or low-dimensional space through the linear transformation to obtain an output vector.

[0093] In the embodiments of this specification, numerical user feature data may refer to user feature data in numerical format; textual user feature data may refer to user feature data in text format. Still taking Table 1 as an example for explanation, the "loan amount", "city", "repaid amount", "monthly income" and "education level" in the first row of the table data to be classified shown in Table 1 are feature name data; the "30000" and "10000" in the second row of the table data to be classified shown in Table 1 are numerical user feature data, and "Beijing" and "undergraduate" are textual user feature data. The BERT model in the feature encoding layer of the classification model is used to convert textual user feature data or feature name data into feature vector data; the linear layer in the feature encoding layer of the classification model is used to convert numerical user feature data into feature vector data.

[0094] In practical applications, numbers may be used to replace categories. In this case, the numbers can be formatted as text. This data, unlike numeric user feature data, can also be classified as text-based user feature data. For example, for city classifications, there are "Beijing," "Shanghai," and "Guangzhou." When using numbers like 0, 1, and 2 to represent different cities, these numbers are text-based user feature data. The BERT model can be used to convert these user feature data into feature vectors.

[0095] Based on this, Figure 2 The method, step 204: using the feature coding layer to receive the table data to be classified and outputting first feature vector sequence data, may specifically include:

[0096] Using the BERT model to encode the feature name data to obtain a first set of feature vector data;

[0097] Using the linear layer to encode the numerical user feature data to obtain a second set of feature vector data;

[0098] Using the BERT model to encode the text-based user feature data to obtain a third set of feature vector data;

[0099] Performing vector filling on the naturally missing user feature data using the Mask Embedding method to obtain a fourth set of feature vector data;

[0100] Performing vector fusion processing on each feature vector in the second set of feature vector data, the third set of feature vector data, and the fourth set of feature vector data with the feature vector corresponding to the feature name in the first set of feature vector data to obtain a fifth set of feature vector data;

[0101] Each eigenvector in the fifth set of eigenvector data is concatenated to obtain the first eigenvector sequence data.

[0102] In the embodiments of this specification, the BERT model is used to encode the feature name data in the classification table data to obtain a first set of feature vector data. Taking Table 1 above as an example, after using the BERT model to encode the five feature name data items in the first row of the classification table data ("Loan Amount," "City," "Repaid Amount," "Monthly Income," and "Education Level"), five feature vectors can be obtained. These five feature vectors can constitute the first set of feature vector data.

[0103] In the embodiments of this specification, a linear layer is used to encode the numerical user feature data in the classification table data to obtain a second set of feature vector data. Taking Table 1 above as an example, after encoding the two numerical user feature data "30000" and "10000" in the second row of the classification table data using a linear layer, two feature vectors can be obtained. These two feature vectors can constitute the second set of feature vector data.

[0104] In the embodiments of this specification, the BERT model is used to encode the textual user feature data in the classification table data to obtain a third set of feature vector data. Taking Table 1 above as an example, after encoding the two textual user feature data "Beijing" and "Undergraduate" in the second row of the classification table data using the BERT model, two feature vectors can be obtained. These two feature vectors can form the third set of feature vector data.

[0105] In the embodiment of the present specification, Mask Embedding is used to perform vector filling on the naturally missing user feature data in the table data to be classified, and a fourth set of feature vector data can be obtained. Taking the above Table 1 as an example, after using Mask Embedding to perform vector filling on the missing "?" in the second row of the table data to be classified, one feature vector can be obtained, and this feature vector can constitute the fourth set of feature vector data. It should be noted that if there are multiple naturally missing user feature data in the table data to be classified, multiple feature vectors can be obtained after using Mask Embedding to perform vector filling on these naturally missing user feature data, but these feature vectors are the same.

[0106] In practical applications, each feature vector in the second set of feature vector data, the third set of feature vector data, and the fourth set of feature vector data is vector-fused with the feature vector corresponding to the feature name in the first set of feature vector data to obtain a fifth set of feature vector data. Taking Table 1 above as an example, after encoding the feature name data in the first row of the table using the BERT model, 5 feature vectors can be obtained. After processing the user feature data in the second row of the table using the BERT model, the linear layer, and Mask Embedding, 5 feature vectors are also obtained. For the feature vectors obtained in the second row, vector fusion processing can be performed with the feature vector corresponding to the feature name, that is, the feature vectors determined by the data in the same column can be vector-fused. For example: after encoding "city" using the BERT model, feature vector A is obtained, and after encoding "Beijing" using the BERT model, feature vector B is obtained. Feature vector A and feature vector B can be vector-fused to obtain feature vector AB. Specifically, vector fusion processing can be performed by vector addition, or by other vector fusion processing methods, which are not specifically limited.

[0107] Taking Table 1 above as an example, after performing vector fusion processing on the feature vectors in the second group of feature vector data, the third group of feature vector data and the fourth group of feature vector data and the feature vector corresponding to the feature name in the first group of feature vector data, 5 feature vectors can be obtained, and these 5 feature vectors can form the fifth group of feature vector data.

[0108] In practical applications, after obtaining the fifth set of feature vector data, the feature vectors in the fifth set of feature vector data can be concatenated to obtain first feature vector sequence data. Specifically, the feature vectors in the fifth set of feature vector data can be concatenated sequentially according to the sorting order of the feature data in the table data to be classified to obtain the first feature vector sequence data.

[0109] Figure 4 A flow chart of a classification model training method provided in the embodiment of this specification is shown in FIG. Figure 3 As shown, since the above content has been Figure 3 The model architecture shown in is described in detail in

[15] , so it will not be repeated here.

[0110] like Figure 4 As shown, the process may include the following steps:

[0111] Step 402: Obtain a first training sample set; the training samples in the first training sample set are tabular data containing multiple user feature data and corresponding sample classification labels; there is naturally missing user feature data in the first training sample set; the user feature data is data used to reflect user features.

[0112] Step 404: performing mask processing on the first training sample set to obtain a second training sample set; the mask processing is used to perform mask processing on at least part of the user feature data in the first training sample set.

[0113] Step 406: Use the first training sample set and the second training sample set to train the classification model to obtain a trained classification model.

[0114] In the embodiment of this specification, the user characteristic data and the naturally missing user characteristic data can be Figure 2 The meanings of the terms mentioned in the embodiments of the method are consistent. Since these contents have been explained in detail above, they will not be repeated here.

[0115] It should be noted that the first training sample set may include multiple training samples, and the presence of naturally missing user feature data in the first training sample set may refer to training samples with naturally missing user feature data among the multiple training samples. Therefore, the training samples in the first training sample set may also include training samples that do not contain naturally missing user feature data.

[0116] In the embodiments of this specification, the training samples in the first training sample set are tabular data containing multiple user feature data and corresponding sample classification labels. Taking Table 1 as an example, after adding sample classification labels to the data in Table 1, it can be used as a training sample in the first training sample set. The sample classification label can be "user has a high risk of overdue payment" or "user has a low risk of overdue payment." In actual applications, the first training sample set can include multiple training samples, each of which can correspond to the user feature data of a single user.

[0117] In the embodiments of this specification, masking is performed on a first training sample set to obtain a second training sample set; the masking is used to mask at least a portion of the user feature data in the first training sample set. Feature masking is a technique for selectively shielding or modifying input features through masking during data processing or model training.

[0118] In the embodiments of this specification, the core purpose of mask processing is to continuously optimize Mask Embedding through self-supervision and supervised training, so that Mask Embedding can learn the optimal representation of missing features in the training phase and fill in the positions of missing features in the prediction phase. The preset mask probability and dynamic mask ratio help to improve the robustness of the model, so that the model can learn and process samples with different missing rates. In practical applications, the mask processing method can adopt Boolean masking, floating point masking, or dynamic masking, without specific limitation.

[0119] In practical applications, the classification tasks that can be performed on the classification model using the first training sample set and the second training sample set include supervised training, self-supervised training for mask feature restoration, and self-supervised training for classification of missing mask features. Among them, self-supervised training for mask feature restoration and self-supervised training for classification of missing mask features can be used as two auxiliary training tasks, and these two types of training can improve the classification accuracy of the final classification task. Since the subsequent embodiments in this specification will explain in detail the process of training the classification model using the first training sample set and the second training sample set, they will not be repeated here.

[0120] In practical applications, the classification model is trained using the first training sample set and the second training sample set to obtain a trained classification model. By training the classification model using the first training sample set and the second training sample set, the classification model can learn the ability to predict missing user feature data based on other non-missing user feature data, and the ability to accurately classify users based on user feature data with partially missing feature data. The trained classification model can be used as Figure 2 The classification model used in the step.

[0121] Figure 4In the method, after obtaining the first training sample set, the first training sample set is masked to obtain the second training sample set, and the first training sample set and the second training sample set are simultaneously used to train the classification model to obtain the trained classification model. Since the Transformer Encoder layer can be continuously optimized during the model training process of the classification model, it learns the ability to predict missing user feature data based on other non-missing user feature data and the ability to accurately classify users based on user feature data with partially missing feature data. This is conducive to improving the accuracy of the trained classification model in classifying users based on user feature data. In addition, by masking the first training sample set to obtain the second training sample set, and using the two sample sets to train the model, it is beneficial to enhance the robustness of the model and the accuracy of the training results when the number of samples is small.

[0122] based on Figure 4 The method in this specification also provides some specific implementation plans of the method, which are described below.

[0123] Optional, Figure 4 In the method, step 404: performing mask processing on the first training sample set to obtain a second training sample set may specifically include:

[0124] For any training sample in the first training sample set, determine whether masking is required for the training sample based on a preset mask probability, and obtain a first determination result;

[0125] If the first judgment result indicates that masking processing needs to be performed on the training sample, determining the amount of user feature data that needs to be masked in the training sample based on the mask ratio;

[0126] Determining the user feature data that needs to be masked in the training sample by using a random function according to the amount of user feature data that needs to be masked in the training sample;

[0127] Masking is performed on user feature data that needs to be masked in each training sample in the first training sample set to obtain the second training sample set.

[0128] In an embodiment of the present specification, the preset mask probability can be a mask probability value preset based on expert experience. For example, if the preset mask probability is 60%, then for any training sample in the first training sample set, the probability of the sample being masked is 60%, and the probability of not being masked is 40%.

[0129] In the embodiments of this specification, if it is determined that masking is required for a training sample, the amount of user feature data in the training sample that needs to be masked can be further determined based on the masking ratio. The masking ratio can be a preset fixed value or a dynamic value randomly selected within a preset range, and is not specifically limited to this. For example, if a training sample contains five user feature data and the masking ratio is 40%, then two of the five user feature data need to be masked.

[0130] In practical applications, after determining the number of user feature data that need to be masked in the training sample, a random function can be used to randomly determine the user feature data that need to be masked from multiple user feature data of the training sample. The random function is not specifically limited in this application.

[0131] In practical applications, if there are naturally missing user feature data in the training sample, a random function is used to randomly determine the user feature data that needs to be masked from multiple user feature data other than the naturally missing user feature data in the training sample.

[0132] Optional, Figure 4 In the method, the mask ratio can be a dynamic ratio; the mask ratio changes dynamically within a preset range; the dynamic change means that the mask ratio randomly takes values ​​within the preset range when processing different sample data.

[0133] In the embodiments of this specification, the preset range may be a value range pre-set based on expert experience. For example, if the preset range is 0.1-0.3, the mask ratio may be randomly selected within the range of 0.1-0.3. When processing different sample data based on the mask ratio, the mask ratio may be revalued each time a new sample data is processed, or the mask ratio may be revalued each time a few new sample data are processed.

[0134] In the embodiment of this specification, by setting the mask ratio to a dynamic ratio, the randomness of masking the features can be improved, which is beneficial to improving the robustness of the trained classification model.

[0135] Optional, Figure 4In the method, the Transformer Encoder layer may include a first Transformer encoder and a second Transformer encoder; parameters are shared between the first Transformer encoder and the second Transformer encoder. Accordingly, step 406: training the classification model using the first training sample set and the second training sample set to obtain a trained classification model may specifically include:

[0136] The first training sample set and the second training sample set are input into the classification model, and the parameters of the first Transformer encoder, the parameters of the second Transformer encoder, and the parameters of the Mask Embedding are adjusted according to the output of the Transformer Encoder layer in the classification model to obtain a trained classification model.

[0137] In practical applications, by setting up parameter-sharing Transformer first encoder and Transformer second encoder in the Transformer Encoder layer, the first and second Transformer encoders can be trained simultaneously using training samples during the model training process, and supervised training and self-supervised training can be performed separately in a targeted manner, which is conducive to improving training efficiency.

[0138] In practical applications, the process of training the classification model using the first training sample set and the second training sample set is mainly a process of iteratively tuning the parameters of the first Transformer encoder, the parameters of the second Transformer encoder, and the parameters of the Mask Embedding in the classification model. Specifically, the parameters of the first Transformer encoder, the parameters of the second Transformer encoder, and the parameters of the Mask Embedding can be adjusted with the minimum loss value during the model training process as the adjustment goal to obtain the trained classification model.

[0139] Based on this, Figure 4 The method, wherein the parameters of the first Transformer encoder, the parameters of the second Transformer encoder, and the parameters of the MaskEmbedding are adjusted according to the output of the Transformer Encoder layer in the classification model to obtain a trained classification model, specifically comprising:

[0140] Determining a first loss value for supervised training of the first Transformer encoder for a classification task using the first training sample set; the supervised training for the classification task is used to train the Transformer Encoder layer to classify users based on user feature data; the first loss value is used to reflect the difference between a result output by the first Transformer encoder and a sample classification label;

[0141] Determining a second loss value for supervised training of the second Transformer encoder for a classification task using the second training sample set; the second loss value is used to reflect the difference between a result output by the second Transformer encoder and a sample classification label;

[0142] Determining a third loss value for self-supervised training of masked feature restoration on the second Transformer encoder using the second training sample set; the self-supervised training of masked feature restoration is used to train the second Transformer encoder to predict the masked user feature data; the third loss value is used to reflect the difference between the user feature data predicted by the second Transformer encoder and the actual user feature data;

[0143] Determining a fourth loss value for self-supervised training of missing mask feature classification for the Transformer second encoder using the second training sample set; the missing mask feature classification self-supervised training is used to train the Transformer second encoder to predict the missing classification results of missing user feature data; the missing classification results include natural missing and masked missing; the fourth loss value is used to reflect the difference between the missing classification results of the missing user feature data predicted by the Transformer second encoder and the missing classification results of the actual user feature data;

[0144] Performing a weighted summation on the first loss value, the second loss value, the third loss value, and the fourth loss value to obtain a comprehensive loss value;

[0145] Taking the minimum comprehensive loss value as the adjustment goal, the parameters of the Transformer first encoder, the parameters of the Transformer second encoder, and the parameters of the Mask Embedding are adjusted to obtain a trained classification model.

[0146] In the embodiments of this specification, supervised training is a training method in machine learning, which refers to training a model with data with clear labels so that the model can learn the mapping relationship between input features and output labels. The training samples in the first training sample set carry the sample classification labels corresponding to the training samples, so that the first training sample set can be used to perform supervised training on the first encoder of the Transformer for classification tasks, and train the Transformer Encoder layer to classify users based on user feature data. During the training process, the loss value of the loss function of the model can be used to measure the error between the model prediction result and the true label. Therefore, the first loss value of the first training sample set for supervised training of the first encoder of the Transformer for classification tasks can be used to reflect the difference between the result output by the first encoder of the Transformer and the sample classification label.

[0147] In practical applications, the second loss value of the second training sample set for supervised training of the Transformer second encoder for classification tasks can be used to reflect the difference between the output result of the Transformer second encoder and the sample classification label.

[0148] In practical applications, since the parameters between the Transformer first encoder and the Transformer second encoder are shared, using the first training sample set to perform supervised training on the Transformer first encoder for classification tasks and using the second training sample set to perform supervised training on the Transformer second encoder for classification tasks is equivalent to using the first training sample set and the second training sample set to perform supervised training on the Transformer Encoder layer for classification tasks.

[0149] In the examples of this specification, self-supervised training is a model training method that does not require manually labeled data. By designing predefined tasks, supervisory signals are generated from the data itself, allowing the model to learn the data's inherent structure and feature representations. The core goal of self-supervised training is to enable the model to automatically mine supervisory information from unlabeled data, thereby learning features that are meaningful for downstream tasks (such as downstream classification and detection).

[0150] In practical applications, the second training sample set can be used to perform self-supervised training on the Transformer second encoder using masked feature restoration. This self-supervised training is used to train the Transformer second encoder to predict masked user feature data. During training, the third loss value of the model's loss function reflects the difference between the user feature data predicted by the Transformer second encoder and the actual user feature data.

[0151] In practical applications, the second training sample set can be used to perform self-supervised training on the Transformer second encoder for missing mask feature classification. This self-supervised training is used to train the Transformer second encoder to predict missing classification results for missing user feature data (including both natural and masked missing). During training, the fourth loss value of the model's loss function reflects the difference between the missing classification results for the user feature data predicted by the Transformer second encoder and the actual missing classification results for the user feature data.

[0152] In practical applications, after determining the first, second, third, and fourth loss values, these four loss values ​​can be weighted and summed to obtain a comprehensive loss value. The weight coefficients corresponding to the respective loss values ​​can be set and adjusted based on experience and are not specifically limited thereto.

[0153] After determining the comprehensive loss value, the parameters of the Transformer's first encoder, the Transformer's second encoder, and the Mask Embedding can be adjusted with the goal of minimizing the comprehensive loss value to obtain a trained classification model. This allows for joint training of the classification model using supervised training for the classification task, self-supervised training for mask feature restoration, and self-supervised training for classification of missing mask features. This improves both model training efficiency and the classification accuracy of the trained classification model.

[0154] In the embodiments of this specification, on the one hand, the self-supervised training of mask feature restoration and the self-supervised training of missing mask feature classification enable the Transformer Encoder layer and Mask Embedding to restore the representation of missing features through other existing features. On the other hand, the supervised training of the classification task enables the masked samples to achieve the same classification prediction effect as the samples before masking after being filled by Mask Embedding. After Mask Embedding learns the effective representation of the missing features, Mask Embedding can be used to fill in the real missing values ​​in the actual reasoning process, further improving the classification prediction effect.

[0155] Figure 5 This is an overall flow chart of a classification model training method provided in the embodiment of this specification. Figure 5 As shown, the classification model training method may specifically include the following steps:

[0156] Step 502: Obtain a first training sample set.

[0157] Step 504: For any training sample in the first training sample set, determine whether the training sample requires masking. If masking is required for the training sample, proceed to step 506. If masking is not required for the training sample, directly add the training sample to the second training sample set. Specifically, the determination of whether masking is required for the training sample can be made based on a preset masking probability.

[0158] Step 506: Determine the amount of user feature data that needs to be masked in the training sample based on the mask ratio.

[0159] Step 508: using a random function, the user feature data that needs to be masked is determined according to the number of user feature data that needs to be masked in the training sample.

[0160] Step 510: Masking is performed on the user feature data that needs to be masked in each training sample in the first training sample set.

[0161] Step 512: Obtain a second training sample set.

[0162] Step 514: Input the first training sample set and the second training sample set into the classification model.

[0163] Step 516: Adjust the parameters of the first Transformer encoder, the second Transformer encoder, and the Mask Embedding parameters according to the output of the Transformer Encoder layer in the classification model.

[0164] Step 518: Obtain the trained classification model.

[0165] Figure 6 The embodiments of this specification provide corresponding Figure 2 A schematic diagram of the structure of a device for classifying users based on user feature data. The device uses a classification model, which includes: a feature encoding layer, a Transformer Encoder layer, and an output layer; wherein the output end of the feature encoding layer is connected to the input end of the Transformer Encoder layer, and the output end of the Transformer Encoder layer is connected to the input end of the output layer, such as Figure 6 As shown, the device may include:

[0166] The acquisition module 602 is used to acquire table data to be classified; the table data to be classified includes a variety of user characteristic data; the user characteristic data is data used to reflect user characteristics; there is naturally missing user characteristic data in the table data to be classified.

[0167] The first output module 604 is configured to receive the table data to be classified using the feature coding layer and output first feature vector sequence data; the Mask Embedding included in the feature coding layer is configured to perform vector filling on the naturally missing user feature data.

[0168] The second output module 606 is configured to output second feature vector sequence data after receiving the first feature vector sequence data using the Transformer Encoder layer.

[0169] The classification module 608 is configured to utilize the output layer to perform classification processing based on the second feature vector sequence data to obtain a classification result for the table data to be classified.

[0170] based on Figure 6 The present specification also provides some specific implementation plans of the device, which are described below.

[0171] Optionally, the device can be applied to a credit business scenario; the user characteristic data includes data used to reflect user characteristics related to the credit business; and the classification result is used to reflect the user's credit risk.

[0172] Optionally, the table data to be classified includes feature name data corresponding to various user feature data; the user feature data includes numerical user feature data and / or textual user feature data; the feature encoding layer also includes a BERT model and a linear layer; the BERT model is used to convert textual user feature data or feature name data into feature vector data; the linear layer is used to convert numerical user feature data into feature vector data.

[0173] Optionally, the first output module 604 may specifically include:

[0174] The first encoding unit is used to encode the feature name data using the BERT model to obtain a first set of feature vector data.

[0175] The second encoding unit is configured to use the linear layer to encode the numerical user feature data to obtain a second set of feature vector data.

[0176] The third encoding unit is used to encode the text-based user feature data using the BERT model to obtain a third set of feature vector data.

[0177] A vector filling unit is used to perform vector filling on the naturally missing user feature data using the Mask Embedding to obtain a fourth set of feature vector data.

[0178] A vector fusion processing unit is used to perform vector fusion processing on each feature vector in the second group of feature vector data, the third group of feature vector data and the fourth group of feature vector data with the feature vector corresponding to the feature name in the first group of feature vector data to obtain a fifth group of feature vector data.

[0179] The vector concatenation unit is configured to concatenate the eigenvectors in the fifth set of eigenvector data to obtain the first eigenvector sequence data.

[0180] Figure 7 The embodiments of this specification provide corresponding Figure 4 A structural diagram of a classification model training device. The classification model includes: a feature coding layer, a Transformer Encoder layer, and an output layer; wherein the output end of the feature coding layer is connected to the input end of the Transformer Encoder layer, and the output end of the Transformer Encoder layer is connected to the input end of the output layer. The device includes:

[0181] Acquisition module 702 is used to acquire a first training sample set; the training samples in the first training sample set are tabular data containing multiple user feature data and corresponding sample classification labels; there is naturally missing user feature data in the first training sample set; the user feature data is data used to reflect user characteristics.

[0182] The masking module 704 is configured to perform masking processing on the first training sample set to obtain a second training sample set; the masking processing is configured to perform masking processing on at least part of the user feature data in the first training sample set.

[0183] The training module 706 is configured to train the classification model using the first training sample set and the second training sample set to obtain a trained classification model.

[0184] based on Figure 7 The present specification also provides some specific implementation plans of the device, which are described below.

[0185] Optionally, the mask module 704 may specifically include:

[0186] The judgment unit is configured to judge, for any training sample in the first training sample set, whether masking is required for the training sample based on a preset masking probability, and obtain a first judgment result.

[0187] The first determining unit is configured to determine the amount of user feature data that needs to be masked in the training sample based on the mask ratio if the first judgment result indicates that the training sample needs to be masked.

[0188] The second determining unit is configured to determine the user feature data that needs to be masked in the training sample according to the amount of user feature data that needs to be masked in the training sample by using a random function.

[0189] The masking unit is configured to perform masking on the user feature data that needs to be masked in each training sample in the first training sample set to obtain the second training sample set.

[0190] Optionally, the mask ratio is a dynamic ratio; the mask ratio changes dynamically within a preset range; the dynamic change means that the mask ratio randomly takes a value within the preset range when processing different sample data.

[0191] Optionally, the Transformer Encoder layer includes a first Transformer encoder and a second Transformer encoder; parameters are shared between the first Transformer encoder and the second Transformer encoder.

[0192] Correspondingly, the training module 706 may specifically include:

[0193] A training unit is used to input the first training sample set and the second training sample set into the classification model, and adjust the parameters of the first Transformer encoder, the parameters of the second Transformer encoder, and the parameters of the Mask Embedding according to the output of the Transformer Encoder layer in the classification model to obtain a trained classification model.

[0194] Optionally, the training unit may specifically include:

[0195] The first determination subunit is used to determine the first loss value of the first training sample set for supervised training of the Transformer first encoder for a classification task; the supervised training of the classification task is used to train the Transformer Encoder layer to classify users based on user feature data; the first loss value is used to reflect the difference between the result output by the Transformer first encoder and the sample classification label.

[0196] The second determination subunit is used to determine the second loss value of the second training sample set for supervised training of the Transformer second encoder for the classification task; the second loss value is used to reflect the difference between the result output by the Transformer second encoder and the sample classification label.

[0197] The third determination subunit is used to determine the third loss value of the second training sample set for the masked feature restoration self-supervised training of the Transformer second encoder; the masked feature restoration self-supervised training is used to train the ability of the Transformer second encoder to predict the masked user feature data; the third loss value is used to reflect the difference between the user feature data predicted by the Transformer second encoder and the actual user feature data.

[0198] The fourth determination subunit is used to determine the fourth loss value of the second training sample set for the missing mask feature classification self-supervised training of the Transformer second encoder; the missing mask feature classification self-supervised training is used to train the ability of the Transformer second encoder to predict the missing classification results of missing user feature data; the missing classification results include natural missing and masked missing; the fourth loss value is used to reflect the difference between the missing classification results of the missing user feature data predicted by the Transformer second encoder and the missing classification results of the actual user feature data.

[0199] The weighted summation subunit is used to perform weighted summation on the first loss value, the second loss value, the third loss value and the fourth loss value to obtain a comprehensive loss value.

[0200] The parameter adjustment subunit is used to adjust the parameters of the first Transformer encoder, the second Transformer encoder, and the Mask Embedding parameters with the minimum comprehensive loss value as the adjustment target to obtain a trained classification model.

[0201] It is understood that the above modules refer to computer programs or program segments for performing one or more specific functions. In addition, the distinction between the above modules does not mean that the actual program codes must also be separated.

[0202] Based on the same idea, the embodiments of this specification also provide devices corresponding to the above methods.

[0203] Figure 8 The embodiments of this specification provide corresponding Figure 2 A schematic diagram of the structure of a device for classifying users based on user feature data. Figure 8 As shown, the device 800 may include:

[0204] at least one processor 810; and,

[0205] A memory 830 in communication with the at least one processor; wherein,

[0206] The memory 830 stores a classification model, which includes: a feature encoding layer, a Transformer Encoder layer, and an output layer; wherein the output end of the feature encoding layer is connected to the input end of the Transformer Encoder layer, and the output end of the Transformer Encoder layer is connected to the input end of the output layer. The memory 830 also stores instructions 820 executable by the at least one processor 810, and the instructions are executed by the at least one processor 810 to enable the at least one processor 810 to:

[0207] Acquire table data to be classified; the table data to be classified includes a variety of user characteristic data; the user characteristic data is data used to reflect user characteristics; the table data to be classified contains naturally missing user characteristic data;

[0208] The feature coding layer receives the table data to be classified and outputs first feature vector sequence data; the Mask Embedding included in the feature coding layer is used to perform vector filling on the naturally missing user feature data;

[0209] After receiving the first feature vector sequence data using the Transformer Encoder layer, output second feature vector sequence data;

[0210] The output layer is used to perform classification processing based on the second feature vector sequence data to obtain a classification result for the table data to be classified.

[0211] Figure 9 The embodiments of this specification provide corresponding Figure 4A structural diagram of a classification model training device. Figure 9 As shown, the device 900 may include:

[0212] at least one processor 910; and,

[0213] A memory 930 in communication with the at least one processor; wherein,

[0214] The memory 930 stores a classification model, which includes: a feature encoding layer, a Transformer Encoder layer, and an output layer; wherein the output end of the feature encoding layer is connected to the input end of the Transformer Encoder layer, and the output end of the Transformer Encoder layer is connected to the input end of the output layer. The memory 930 also stores instructions 920 executable by the at least one processor 910, and the instructions are executed by the at least one processor 910 to enable the at least one processor 910 to:

[0215] Obtain a first training sample set; the training samples in the first training sample set are tabular data containing multiple user feature data and corresponding sample classification labels; there is naturally missing user feature data in the first training sample set; the user feature data is data used to reflect user features;

[0216] Performing masking on the first training sample set to obtain a second training sample set; the masking is used to perform masking on at least part of the user feature data in the first training sample set;

[0217] The classification model is trained using the first training sample set and the second training sample set to obtain a trained classification model.

[0218] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. Figure 8 The equipment shown and Figure 9 As for the device shown, since it is basically similar to the corresponding method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0219] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0220] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures such as diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to hire a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0221] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0222] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0223] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0224] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0225] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0226] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0227] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0228] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0229] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0230] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0231] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0232] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0233] The foregoing is merely an example of the present invention and is not intended to limit the present invention. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A method for classifying users based on user feature data, the method using a classification model, the classification model comprising: A feature coding layer, a Transformer Encoder layer, and an output layer; wherein the output end of the feature coding layer is connected to the input end of the Transformer Encoder layer, and the output end of the Transformer Encoder layer is connected to the input end of the output layer, and the method includes: Acquire table data to be classified; the table data to be classified includes a variety of user characteristic data; the user characteristic data is data used to reflect user characteristics; the table data to be classified contains naturally missing user characteristic data; The feature coding layer receives the table data to be classified and outputs first feature vector sequence data; the Mask Embedding included in the feature coding layer is used to perform vector filling on the naturally missing user feature data; After receiving the first feature vector sequence data using the Transformer Encoder layer, output second feature vector sequence data; The output layer is used to perform classification processing based on the second feature vector sequence data to obtain a classification result for the table data to be classified.

2. The method according to claim 1, wherein the method is applied to a credit business scenario; the user characteristic data includes data used to reflect user characteristics related to the credit business; and the classification result is used to reflect the user's credit risk.

3. The method as claimed in claim 1 is applied to a target content recommendation scenario; the user feature data includes data used to reflect user features related to the target content; and the classification result is used to reflect the value of recommending the target content to the user.

4. The method as claimed in claim 1, wherein the table data to be classified includes feature name data corresponding to various user feature data; the user feature data includes numerical user feature data and / or textual user feature data; the feature encoding layer further includes a BERT model and a linear layer; the BERT model is used to convert textual user feature data or feature name data into feature vector data; and the linear layer is used to convert numerical user feature data into feature vector data.

5. The method according to claim 4, wherein the step of receiving the table data to be classified using a feature encoding layer and outputting first feature vector sequence data specifically comprises: Using the BERT model to encode the feature name data to obtain a first set of feature vector data; Using the linear layer to encode the numerical user feature data to obtain a second set of feature vector data; Using the BERT model to encode the text-based user feature data to obtain a third set of feature vector data; Performing vector filling on the naturally missing user feature data using the Mask Embedding to obtain a fourth set of feature vector data; Performing vector fusion processing on each feature vector in the second set of feature vector data, the third set of feature vector data, and the fourth set of feature vector data with the feature vector corresponding to the feature name in the first set of feature vector data to obtain a fifth set of feature vector data; Each eigenvector in the fifth set of eigenvector data is concatenated to obtain the first eigenvector sequence data.

6. A classification model training method, the classification model comprising: Feature Encoding layer, a Transformer Encoder layer, and an output layer; wherein the output end of the feature encoding layer is connected to the input end of the Transformer Encoder layer, and the output end of the Transformer Encoder layer is connected to the input end of the output layer, and the method includes: Obtain a first training sample set; the training samples in the first training sample set are tabular data containing multiple user feature data and corresponding sample classification labels; there is naturally missing user feature data in the first training sample set; the user feature data is data used to reflect user features; Performing masking on the first training sample set to obtain a second training sample set; the masking is used to perform masking on at least part of the user feature data in the first training sample set; The classification model is trained using the first training sample set and the second training sample set to obtain a trained classification model.

7. The method according to claim 6, wherein the masking process is performed on the first training sample set to obtain the second training sample set, specifically comprising: For any training sample in the first training sample set, determine whether masking is required for the training sample based on a preset mask probability, and obtain a first determination result; If the first judgment result indicates that masking processing needs to be performed on the training sample, determining the amount of user feature data that needs to be masked in the training sample based on the mask ratio; Determining the user feature data that needs to be masked in the training sample by using a random function according to the amount of user feature data that needs to be masked in the training sample; Masking is performed on the user feature data that needs to be masked in each training sample in the first training sample set to obtain the second training sample set.

8. The method as claimed in claim 7, wherein the mask ratio is a dynamic ratio; the mask ratio changes dynamically within a preset range; the dynamic change means that the mask ratio randomly takes values ​​within the preset range when processing different sample data.

9. The method of claim 6, wherein the Transformer Encoder layer includes a first Transformer encoder and a second Transformer encoder; parameters are shared between the first Transformer encoder and the second Transformer encoder; and the training of the classification model using the first training sample set and the second training sample set to obtain the trained classification model specifically comprises: The first training sample set and the second training sample set are input into the classification model, and the parameters of the first Transformer encoder, the parameters of the second Transformer encoder, and the parameters of the Mask Embedding are adjusted according to the output of the Transformer Encoder layer in the classification model to obtain a trained classification model.

10. The method according to claim 9, wherein the adjusting parameters of the first Transformer encoder, the second Transformer encoder, and the Mask Embedding according to the output of the Transformer Encoder layer in the classification model to obtain the trained classification model specifically comprises: Determining a first loss value for supervised training of the first Transformer encoder for a classification task using the first training sample set; the supervised training for the classification task is used to train the Transformer Encoder layer to classify users based on user feature data; the first loss value is used to reflect the difference between a result output by the first Transformer encoder and a sample classification label; Determining a second loss value for supervised training of the second Transformer encoder for a classification task using the second training sample set; the second loss value is used to reflect the difference between a result output by the second Transformer encoder and a sample classification label; Determining a third loss value for self-supervised training of masked feature restoration on the second Transformer encoder using the second training sample set; the self-supervised training of masked feature restoration is used to train the second Transformer encoder to predict the masked user feature data; the third loss value is used to reflect the difference between the user feature data predicted by the second Transformer encoder and the actual user feature data; Determining a fourth loss value for self-supervised training of missing mask feature classification for the Transformer second encoder using the second training sample set; the missing mask feature classification self-supervised training is used to train the Transformer second encoder to predict the missing classification results of missing user feature data; the missing classification results include natural missing and masked missing; the fourth loss value is used to reflect the difference between the missing classification results of the missing user feature data predicted by the Transformer second encoder and the missing classification results of the actual user feature data; Performing a weighted summation on the first loss value, the second loss value, the third loss value, and the fourth loss value to obtain a comprehensive loss value; Taking the minimum comprehensive loss value as the adjustment goal, the parameters of the Transformer first encoder, the parameters of the Transformer second encoder, and the parameters of the Mask Embedding are adjusted to obtain a trained classification model.

11. A device for classifying users based on user feature data, the device using a classification model, the classification model comprising: A feature coding layer, a Transformer Encoder layer, and an output layer; wherein the output end of the feature coding layer is connected to the input end of the Transformer Encoder layer, and the output end of the Transformer Encoder layer is connected to the input end of the output layer, and the device includes: An acquisition module is used to acquire table data to be classified; the table data to be classified includes a variety of user characteristic data; the user characteristic data is data used to reflect user characteristics; there is naturally missing user characteristic data in the table data to be classified; A first output module is configured to receive the table data to be classified using the feature coding layer and output first feature vector sequence data; the Mask Embedding included in the feature coding layer is configured to perform vector filling on the naturally missing user feature data; a second output module, configured to output second feature vector sequence data after receiving the first feature vector sequence data using the Transformer Encoder layer; The classification module is used to utilize the output layer to perform classification processing according to the second feature vector sequence data to obtain a classification result for the table data to be classified.

12. A classification model training device, the classification model comprising: A feature coding layer, a Transformer Encoder layer, and an output layer; wherein the output end of the feature coding layer is connected to the input end of the Transformer Encoder layer, and the output end of the Transformer Encoder layer is connected to the input end of the output layer, and the device includes: an acquisition module configured to acquire a first training sample set; the training samples in the first training sample set are tabular data containing a variety of user feature data and corresponding sample classification labels; the first training sample set contains naturally missing user feature data; the user feature data is data used to reflect user features; a masking module, configured to perform masking processing on the first training sample set to obtain a second training sample set; wherein the masking processing is configured to perform masking processing on at least part of the user feature data in the first training sample set; The training module is used to train the classification model using the first training sample set and the second training sample set to obtain a trained classification model.

13. A device for classifying users based on user feature data, comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores a classification model, the classification model including: a feature encoding layer, a Transformer Encoder layer, and an output layer; wherein an output end of the feature encoding layer is connected to an input end of the Transformer Encoder layer, and an output end of the Transformer Encoder layer is connected to an input end of the output layer. The memory further stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so as to enable the at least one processor to: Acquire table data to be classified; the table data to be classified includes a variety of user characteristic data; the user characteristic data is data used to reflect user characteristics; the table data to be classified contains naturally missing user characteristic data; The feature coding layer receives the table data to be classified and outputs first feature vector sequence data; the Mask Embedding included in the feature coding layer is used to perform vector filling on the naturally missing user feature data; After receiving the first feature vector sequence data using the Transformer Encoder layer, output second feature vector sequence data; The output layer is used to perform classification processing based on the second feature vector sequence data to obtain a classification result for the table data to be classified.

14. A classification model training device comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores a classification model, the classification model including: a feature encoding layer, a Transformer Encoder layer, and an output layer; wherein an output end of the feature encoding layer is connected to an input end of the Transformer Encoder layer, and an output end of the Transformer Encoder layer is connected to an input end of the output layer. The memory further stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so as to enable the at least one processor to: Obtain a first training sample set; the training samples in the first training sample set are tabular data containing multiple user feature data and corresponding sample classification labels; there is naturally missing user feature data in the first training sample set; the user feature data is data used to reflect user features; Performing masking on the first training sample set to obtain a second training sample set; the masking is used to perform masking on at least part of the user feature data in the first training sample set; The classification model is trained using the first training sample set and the second training sample set to obtain a trained classification model.

Images