Representation model training method and feature representation method

Abstract

Provided in the present application are a representation model training method and a feature representation method. During the training of a representation model, a gray sample is first acquired, wherein the gray sample comprises user features, a preset first label of a classification task, and information, and the information is determined on the basis of a user feature of the gray sample, which user feature corresponds to a preset second label of the classification task. Next, the user features are inputted into a feature representation model, so as to obtain feature representation data; the first label is inputted into a label representation model, so as to obtain main label representation data; and the information is inputted into the label representation model, so as to obtain sub-label representation data. Label representation data is determined on the basis of the main label representation data and the sub-label representation data. The similarity between the feature representation data and the label representation data is calculated, and parameters of the representation model are adjusted with the objective of increasing the similarity. By means of main label representation data and sub-label representation data, labels of different characteristics of a gray sample are separately represented, thereby making full use of the gray sample, and improving the utilization rate of training samples.

Description

A representation model training method and a feature representation method

[0001] This application claims priority to Chinese Patent Application No. 2025102440076, filed on February 28, 2025, entitled “A Representation Model Training Method and a Feature Representation Method”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This specification relates to the field of computer application technology, and in particular to a representation model training method and a feature representation method. Background Technology

[0003] With the continuous development of computer technology and machine learning, artificial intelligence (AI) is being applied to various scenarios. For example, in many situations, AI technology is being used to segment users into groups, essentially training user classification models to categorize users. User segmentation facilitates the provision of targeted services based on user groups, such as customizing user-specific content and pushing different information to users. Segmentation based on user fraud risk can also be used to determine whether to allow specific user actions, such as rejecting large transfer requests or frequent login requests from users with a high fraud risk. Therefore, segmenting users into groups not only allows users to access more needed services or content, improving user experience, but also reduces unnecessary data and information dissemination, optimizes the network environment, and enhances network security through risk prevention and control.

[0004] However, traditional classification models may use a large number of gray samples (i.e., samples that do not clearly belong to a certain classification category) as training samples. Since it is difficult to determine the corresponding label of gray samples, they are often unusable for training classification models, resulting in low data utilization during training. Summary of the Invention

[0005] In view of this, one or more embodiments of this specification provide a method for training a representation model and a method for representing features.

[0006] According to a first aspect of one or more embodiments of this specification, a representation model training method is proposed, comprising:

[0007] Obtain a first user sample, which includes a first user feature, a first label, and first information. The first label is one of a plurality of preset labels for a first classification task. The first information is obtained based on the feature in the first user feature that corresponds to the second label among the plurality of labels.

[0008] The first user features are processed using a feature representation model to obtain first feature representation data;

[0009] The first label and the first information are processed using a label representation model to obtain main label representation data and sub-label representation data, respectively.

[0010] First label representation data is calculated based on the main label representation data and the sub-label representation data, and a first similarity is calculated between the first label representation data and the feature representation data.

[0011] With the goal of increasing the first similarity, the parameters of the feature representation model are adjusted.

[0012] According to a second aspect of one or more embodiments of this specification, a feature characterization method is proposed, comprising:

[0013] Obtain user characteristics of the target users;

[0014] A feature representation model is obtained, which is trained by the representation model training method as described in the first aspect;

[0015] The user features are processed using the feature representation model to obtain target feature representation data for representing the user features.

[0016] According to a third aspect of the embodiments of this specification, a representation model training apparatus is provided, comprising:

[0017] The sample acquisition module is used to acquire a first user sample, which includes a first user feature, a first label, and first information. The first label is one of a plurality of preset labels for a first classification task, and the first information is acquired based on a feature in the first user feature that corresponds to a second label among the plurality of labels.

[0018] The first embedding module is used to process the first user features using a feature representation model to obtain first feature representation data;

[0019] The second embedding module is used to process the first label and the first information using a label representation model to obtain main label representation data and sub-label representation data, respectively.

[0020] The similarity calculation module is used to calculate the first label representation data based on the main label representation data and the sub-label representation data, and to calculate the first similarity between the first label representation data and the feature representation data;

[0021] The parameter adjustment module is used to adjust the parameters of the feature representation model with the goal of increasing the first similarity.

[0022] According to a fourth aspect of the embodiments of this specification, a feature characterization apparatus is provided, comprising:

[0023] The feature acquisition module is used to acquire the user features of the target user;

[0024] The model acquisition module is used to acquire a feature representation model, which is trained by the representation model training method as described in the first aspect.

[0025] The feature processing module is used to process the user features using the feature representation model to obtain target feature representation data for representing the user features.

[0026] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the method described in the first or second aspect of the embodiments of this specification.

[0027] According to a sixth aspect of the embodiments of this specification, a computer device is provided, the computer device comprising:

[0028] processor;

[0029] Memory used to store processor-executable instructions;

[0030] The processor executes the executable instructions to implement the method as described in the first or second aspect of the embodiments of this specification.

[0031] According to a seventh aspect of the embodiments of this specification, a computer program product is provided that, when executed by a processor, implements the method described in the first or second aspect of the embodiments of this specification.

[0032] This specification provides a representation model training method and a feature representation method. During representation model training, gray samples are first acquired. These gray samples include user features, a first label preset for the classification task, and information. The information is determined based on the user features corresponding to the second label preset for the classification task. Then, the user features are input into the feature representation model to obtain feature representation data; the first label is input into the label representation model to obtain main label representation data; and the information is input into the label representation model to obtain sub-label representation data. Label representation data is determined based on the main label representation data and the sub-label representation data. The similarity between the feature representation data and the representation data is calculated, and the parameters of the representation model are adjusted with the goal of increasing this similarity.

[0033] The above method uses both main label representation data and sub-label representation data to characterize the different characteristics of gray samples. This fully utilizes gray samples and improves the utilization rate of training samples.

[0034] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0035] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this specification and, together with the description, serve to explain the principles of this specification.

[0036] Figure 1 is an architecture diagram of the classification model in the related art shown in this specification.

[0037] Figure 2 is an architectural diagram of a classification model illustrated in this specification according to an exemplary embodiment.

[0038] Figure 3 is a flowchart illustrating a representation model training method according to an exemplary embodiment of this specification.

[0039] Figure 4 is a schematic diagram of a gray sample embedding shown in this specification.

[0040] Figure 5 is a flowchart illustrating a feature characterization method according to an exemplary embodiment of this specification.

[0041] Figure 6 is a block diagram of a representation model training apparatus according to an exemplary embodiment of this specification.

[0042] Figure 7 is a block diagram of a feature characterization device according to an exemplary embodiment of this specification.

[0043] Figure 8 is a hardware structure diagram of a computer device according to an exemplary embodiment of this specification. Detailed Implementation

[0044] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0045] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0046] To classify users, it's often necessary to label them into categories, obtaining a label for each user sample. However, in some cases, user samples exhibit characteristics of both categories; or these samples may be inaccurately classified due to missing or insufficient information, noise, or other reasons. These user samples, often unclassifiable, are called gray samples.

[0047] Gray samples generally have the following characteristics: First, high uncertainty: They contain ambiguous or insufficient information, making their category difficult for classification models or humans to determine. Second, overlapping attributes: They may simultaneously possess some features of both positive and negative samples, making classification ambiguous. Third, missing information: The lack of some key information may lead to the inability to accurately identify their type.

[0048] For example, when classifying users' fraud risk, the preset categories are no fraud risk and fraud risk. However, a user sample may have characteristics of both no fraud risk (e.g., no overdue payments in the past 3 years) and fraud risk (e.g., transactions with users with fraud risk in the past 3 years). Such a user sample often cannot be classified into a specific category and is called a gray sample.

[0049] As shown in Figure 1, in traditional classification models, user features are typically embedded into feature vectors through a feature representation model, and a classifier is implemented using activation functions such as sigmoid or softmax in the output layer. The loss between the classifier and the labels is then calculated to classify the user features. This classification method generally treats labels as discrete categories, preventing gray samples that cannot be classified into a specific category from participating in training.

[0050] During classification model training, it was found that in some scenarios, the proportion of gray samples is relatively high. For example, in a scenario of classifying user fraud risk, out of a total of 70 million user samples, there may be 20 million samples with clear labels. This means that if only samples with clear labels are used for training, the remaining 50 million valuable samples will be lost, resulting in low data utilization and significant losses in data integrity and model robustness.

[0051] To address these issues, related technologies typically employ methods such as pseudo-label learning. However, these solutions often require the incorporation of substantial expert experience, and even with extensive expert input, the final training results may not meet expectations in some cases.

[0052] The improvement methods in several related technologies will be explained in the following sections:

[0053] 1. Pseudo Labeling:

[0054] A semi-supervised learning technique assigns predicted labels to unlabeled data (i.e., gray samples) and uses these pseudo-labels for the model's training process. This method can expand the training set, but the accuracy of label assignment is highly dependent on the model's current performance.

[0055] 2. Active Learning:

[0056] In this approach, the model proactively selects and labels the samples it deems most difficult to classify, and then uses this newly labeled data for training. This can effectively utilize grey samples, but requires an efficient sample selection strategy and incurs additional labeling costs.

[0057] 3. Data Augmentation:

[0058] Data augmentation can expand datasets and improve model generalization by transforming existing labeled data to generate new samples. However, it does not directly address the use of grey samples; it only indirectly improves model robustness through other means.

[0059] 4. Ensemble Learning:

[0060] Integrating predictions from multiple models, perhaps through voting or averaging, can improve overall predictive performance. This approach typically enhances the model's robustness to noise and outliers, but it can present challenges when dealing with large numbers of grey samples.

[0061] 5. Transfer Learning:

[0062] By using a model pre-trained on a relevant task as a starting point, transfer learning can accelerate the learning process for new tasks and, in some cases, better understand gray samples. Nevertheless, transfer learning may require a sufficiently relevant source task to guarantee effectiveness.

[0063] 6. Weakly Supervised Learning:

[0064] A range of methods use incompletely labeled, inaccurately labeled, or other forms of weakly labeled data for model training. These methods attempt to understand gray samples through weak labels, but accuracy and reliability can vary depending on label quality.

[0065] 7. Cost-sensitive Learning:

[0066] By modifying the learning algorithm or loss function, the model can be made more sensitive to different types of errors. When dealing with gray samples, cost-sensitive learning can assign different weights to these difficult-to-classify samples to optimize model performance.

[0067] Each method has its advantages and applicable scenarios, but also its limitations. For example, pseudo-labeling and active learning rely on the model's current predictive ability, data augmentation and ensemble learning increase the computational burden, transfer learning and weakly supervised learning may not be suitable for all tasks, and cost-sensitive learning requires careful design of error costs. It is evident that methods in related techniques rely on the incorporation of expert experience or do not directly process gray samples.

[0068] To address the aforementioned issues, this specification proposes a representation model training method, as shown in Figure 2. First, the sigmoid or softmax activation function in the output layer of the traditional classification model is removed. Instead, labels are converted into label representation data (e.g., label vectors) using a label representation model. User features are still converted into feature representation data using a feature representation model. The output layer calculates the loss by comparing the similarity between the label representation data and the feature representation data, classifying the features. By training the feature representation model to make the label representation data and feature representation data of the same user sample more similar, the model can better classify user samples.

[0069] Secondly, regarding gray samples, since the labels are converted into representational data (e.g., embedded in a vector space), the representational data is continuous compared to discrete categories. Therefore, the representational data can also be used to represent the category of gray samples (e.g., using vectors to represent the category of gray samples in a vector space). Furthermore, according to the definition of gray samples above, a gray sample may include: a large number of features belonging to the category corresponding to the first label, and a small number of features belonging to the category corresponding to the second label.

[0070] Therefore, a label representation model can be used to process the first label of the gray sample to obtain the main label representation data (such as the main label vector), and to process the information obtained based on the features corresponding to the second label category to obtain the sub-label representation data. The label representation data corresponding to the user sample can then be determined based on the main label representation data and the sub-label representation data. Thus, the gray sample can be used for model training.

[0071] The above method uses main label representation data and sub-label representation data to represent the labels of different characteristics of gray samples separately. This makes full use of gray samples and improves the utilization rate of training samples. Furthermore, by introducing gray samples, the model learns the differences and relationships between gray samples and other samples with definite labels, enabling the model to better represent user features and labels and thus better complete the classification task.

[0072] The training method for the representation model will be explained next.

[0073] This section will explain the process of obtaining a single sample and adjusting parameters based on that sample. It is easy to understand that if there are multiple samples, the process described in this article can be used to process each sample separately, which will not be elaborated here.

[0074] As shown in Figure 3, Figure 3 is a flowchart of a representation model training method illustrated in this specification, including the following steps:

[0075] Step 301: Obtain the first user sample.

[0076] The first user sample includes a first user feature, a first label, and first information. The first label is one of a plurality of preset labels for a first classification task. The first information is obtained based on the feature in the first user feature that corresponds to the second label among the plurality of labels.

[0077] The first user sample here is also known as a gray sample. Figure 3 illustrates the processing of the gray sample. As mentioned earlier, the gray sample differs from other user samples with definite categories in that it does not have a definite category. It has features corresponding to one label and features corresponding to another label. Therefore, the first label here refers to the label that matches some features of the sample, or it could be a label that is closer to the sample. The first information corresponds to the features of the user sample that correspond to another sample.

[0078] As mentioned earlier, user samples are divided into samples with defined labels and gray samples. Similar to the first user sample in step 301, each gray sample includes: user features, a corresponding label, and information. The corresponding label is the label most closely related to the sample, which can be the first label, the second label, or other labels corresponding to the first classification task. The information refers to the information obtained based on the features of the sample corresponding to other labels.

[0079] Correspondingly, samples with definite labels include: user characteristics and corresponding labels. In some cases, samples with definite labels may also have corresponding information that characterizes their behavior as particularly consistent with a certain label.

[0080] The user samples will be explained in detail below.

[0081] User characteristics, such as primary user characteristics, refer to the features of a user sample corresponding to the primary classification task domain. For example, if the primary classification task is to classify a user's fraud risk or financial credit risk, then user characteristics may include: user profile, transaction history, historical risk behavior, whether they belong to a certain group, scene activity, etc.

[0082] The first classification task could be classifying a user's fraud risk, such as determining whether a user has engaged in fraudulent behavior. The first classification task could also be classifying a user's credit risk in the financial sector. For example, it could classify a user's rental fraud risk, determining whether a user might rent an item without a deposit and not return it. Another example is classifying a user's insurance fraud risk, determining whether a user might commit insurance fraud. Yet another example is classifying a user's loan fraud risk, determining whether a user might default on a loan. The above examples are merely possible illustrations of the first classification task and do not constitute a limitation of this specification.

[0083] A label, for example, is one of several labels pre-defined for the first classification task. In a scenario where a user's financial credit risk is classified, the first label could be either "financial credit risk exists" or "financial credit risk does not exist." In more detailed classification scenarios, the first label could be one of "primary financial credit risk exists," "secondary financial credit risk exists," or "financial credit risk does not exist."

[0084] For gray samples such as the first user sample, the first label can come from manual annotation or other methods.

[0085] For gray samples such as the first user sample, the first information refers to the reason why it cannot be classified into the category corresponding to the first label. The first information is determined based on the first user feature corresponding to the second label, that is, based on the feature that does not conform to the first label. For example, in the scenario of classifying users' fraud credit risk, although most of the features of a user indicate that they do not have fraud risk (i.e., corresponding to the first label), if they had a transfer with a user with fraud risk last year, this feature corresponds to the presence of fraud risk (second label), and the first information can be determined through this feature.

[0086] Regarding the specific form of the first information, the first information can be a specific feature or a sub-label derived from the generalization of that feature. For example, in the above example, the first information can be the transfer behavior or a sub-label such as "transfer behavior with users at risk of fraud within the past three years".

[0087] Regarding the source of the primary information, it can be obtained through manual annotation. For example, in addition to labeled tags, user samples can also include notes indicating that gray samples cannot be classified into a specific category; these notes constitute the primary information.

[0088] In contrast to gray samples, for user samples with definite labels, the labels can originate from established past facts. For example, in classifying fraud risk, if a user has a history of fraudulent behavior, their label could be "fraud risk present." Secondly, labels can also originate from past judgments. For instance, using a traditional fraud risk classification model, if a fraud risk assessment was performed on the recipient of a transfer before the transaction was blocked, and that recipient's sample would then be labeled "fraud risk present."

[0089] As mentioned earlier, in some implementations, user samples with defined labels may also have corresponding information. This information can be obtained based on features in the user characteristics that exceed preset conditions corresponding to their labels, that is, features that characterize the user sample as particularly consistent with its corresponding label. For example, in the presence of black and white samples, if a user sample is a black sample, then this information could be features characterizing that the user is particularly black.

[0090] For example, if a user is tagged as having a fraud risk, then generally, having transferred money more than 10 times with a user of this risk would raise a potential fraud risk. If a user has transferred money 150 times with a user of this risk, then information corresponding to that sample is determined based on these 150 transfer transactions. Specifically, this information could be the specific transactions themselves, or it could be a predefined condition such as "having transferred money more than 100 times with a user of this risk within the past 3 years."

[0091] The specific form and acquisition method of this information are similar to those of the gray samples mentioned earlier, and will not be repeated here.

[0092] Step 303: Process the first user features using a feature representation model to obtain the first feature representation data.

[0093] After obtaining the samples, the first user features need to be embedded using a feature representation model to obtain the first feature representation data.

[0094] In one optional implementation, the first feature representation data may specifically be in vector form. Step 303 will now be explained using the representation data in vector form as an example.

[0095] The aforementioned feature representation model can be implemented as a Transformer-based encoder or a BERT model, or as a deep neural network (DNN), recurrent neural network (RNN), or convolutional neural network (CNN) or other neural networks.

[0096] Taking an encoder implemented as a Transformer or a BERT model as an example, the features of each first user sample can be directly input into the feature representation model. Then, in the embedding layer of this feature representation model, word embedding processing is performed on each first user feature to obtain the corresponding word representation data, such as word vectors. Subsequently, based on the obtained word representation data, the overall representation data of each first user feature can be obtained, such as the first feature vector.

[0097] After obtaining the feature vectors of the aforementioned user characteristics, the feature vectors can be normalized, for example, by performing L2 norm normalization, to ensure that the magnitude of the feature vector is 1.

[0098] In related technologies, classifiers are generally used to classify different user samples. The representation data obtained in steps 303 and 305 differs from the output of classifiers in these technologies, which output probability distributions for discrete categories. However, the representation data in this specification is a continuous result representation, such as existing as vectors in a vector space. From another perspective, a classifier is equivalent to a decoder in a feature space, while the label embedding method is equivalent to an encoder. Through label embedding, information can interact at higher dimensions and richer levels, thereby improving classification accuracy.

[0099] Step 305: Process the first label and the first information using the label representation model to obtain main label representation data and sub-label representation data, respectively.

[0100] By embedding the labels and information of user samples using a label representation model, we can obtain main label representation data and sub-label representation data, which respectively represent different characteristics of the gray samples. By embedding the labels, for example, converting them into points in vector space, and through learning in subsequent steps, the feature representation data and label representation data are made to be similar, thus solving the classification problem based on the similarity between the label representation data and the user feature representation data. This improves the flexibility of the classification task and helps to capture more subtle relationships between different categories.

[0101] The label representation model here can be implemented as any word embedding model, such as Word2Vec, GloVe, FastText, etc.

[0102] Step 305 includes two processes: label embedding and sub-label embedding. The two processes will be explained separately below.

[0103] The process of label embedding involves inputting a label (such as the first label) into a label representation model to obtain the embedding result, which is the main label representation data. The main label representation data can be the main label representation vector.

[0104] The sub-label embedding process is similar to the label embedding process. It involves inputting the information of each user sample (e.g., the first information of the first user sample) into the label representation model to obtain the sub-label representation result. This label representation model can be the same as the label representation model described above. In an optional implementation, the sub-label representation result can be a sub-label vector.

[0105] For gray samples, as shown in Figure 4, the gray sample is deconstructed, comprising features corresponding to a certain label (e.g., the first label) and features corresponding to other labels. For example, the first user sample consists of some features corresponding to the first label and some features corresponding to the second label, with the features corresponding to the first label having a larger proportion. Therefore, in step 305 above, for the first user sample, the first label closest to it is embedded to obtain the main label representation data. At the same time, for the features in the first user sample corresponding to the second label, the first information is determined from these features, and the sub-label vector is obtained by embedding these features through the label representation model.

[0106] By using a sub-label embedding mechanism, the characteristics of gray samples are deconstructed and these substandard characteristics are quantified, allowing gray samples to participate in model training. This effectively utilizes a large amount of data that might otherwise be discarded, improving data utilization and model performance without compromising the stability of already learned categories. Moreover, this allows the model to learn representations of fuzzy information, making it more capable of handling complex and ambiguous scenarios.

[0107] Furthermore, in emerging fields that require handling large amounts of uncertain data, the methods provided in this specification can make better use of existing data and improve model performance.

[0108] Furthermore, the size of the sub-label representation data can be limited. Specifically, the first user sample is closer to the first label, and the first label is more important than the first information in classifying the first user sample. Therefore, for the sub-label embedding part, if the overall influence of the sub-label representation data on the label representation data is greater than that of the main label representation data, it may affect the stability of the label representation data and the training process. Therefore, the value range of the sub-label representation data can be limited to a preset range to reduce the influence of the sub-label representation data.

[0109] For specific methods to limit the value range of sublabel representation data, one approach is to add a hyperparameter that indicates the degree of restriction on the sublabel representation data.

[0110] Specifically, after obtaining the sub-label representation data through the label representation model, the sub-label representation data can be multiplied by the hyperparameter to limit its size. For example, if the hyperparameter is 0.1, the output of the label representation model is usually within the range of (-1,1) or (0,1) (it should be noted that the output of the label representation model may sometimes exceed these ranges). By multiplying the output of the label representation model by 0.1, the output of the label representation model for the sub-label can be reduced. The above example is not intended to limit this explanation.

[0111] By controlling the size of the sub-label representation data, the influence of the main label representation data and sub-label representation data on the main label representation data is adjusted, which fine-tunes the model's response to the main label representation data and sub-label representation data, achieving more precise model tuning. At the same time, it also ensures the model's stability when absorbing new information. Furthermore, controlling the value range of the sub-label representation data allows the model to be suitable not only for clearly defined classification tasks but also to efficiently handle scenarios with ambiguous and incomplete data.

[0112] Step 307: Calculate first label representation data based on the main label representation data and the sub-label representation data, and calculate the first similarity between the first label representation data and the feature representation data.

[0113] The first label representation data is the label representation data corresponding to the first user sample. The first label representation data can be obtained by calculating the sum of the main label representation data and the sub-label representation data. Specifically, the sum of the main label representation data and the sub-label representation data can be directly used as the first label representation data, or the sum of the main label representation data and the sub-label representation data can be multiplied by a preset multiple, or added to or subtracted from a preset value, and the result can be used as the first label representation data.

[0114] In an alternative implementation, the first label representation data can also be determined based on a weighted sum of the main label representation data and the sub-label representation data. In this case, the weight corresponding to the sub-label representation data can be lower than the weight corresponding to the main label representation data, so as to limit the influence of the sub-label representation data on the first label representation data.

[0115] The first similarity score represents the degree of similarity between the model's output labels and features. Specifically, the first similarity score can be calculated using methods such as cosine similarity, Euclidean distance, Manhattan distance, and Pearson correlation coefficient.

[0116] Step 309: Adjust the parameters of the feature representation model with the goal of increasing the first similarity.

[0117] The ultimate training objective of the model is to make the label representation data and feature representation data corresponding to the same user sample closer to each other. Therefore, in actual training, the model parameters can be adjusted by aiming to increase the first similarity corresponding to the first user sample, so that the label representation data and feature representation data are closer to each other.

[0118] In one alternative implementation, the specific adjustment can be to the parameters of the feature representation model. In this way, given that the representation data corresponding to each label is predefined, the embedding results of user features are adjusted so that the feature representation model can obtain representation data that better meets the requirements.

[0119] In an optional implementation, the adjustment may specifically involve the feature representation model and the label representation model. Specifically, step 309 may include adjusting the parameters of the feature representation model and the label representation model with the goal of increasing the first similarity. This allows for more flexible adjustments by adjusting the parameters of the feature representation model and the label representation model to make the first feature representation data and the first label representation data more similar.

[0120] In an alternative implementation, the requirement can also be achieved by adjusting the label model while keeping the feature representation model unchanged; this specification does not limit this to the latter.

[0121] Furthermore, in order to achieve better classification, it is not only necessary to make the feature representation data and label representation data of each user sample close to each other, but also to reduce the similarity between the label representation data corresponding to different types, so that the label representation data corresponding to different types are far apart from each other, such as far apart from each other in the vector space, to achieve better classification results.

[0122] Specifically, the similarity between the label representation data corresponding to each label in the first classification task can also be calculated. Specifically, the label representation data corresponding to each label in the first classification task can be obtained through the label representation model; for each label representation data, the similarity is calculated pairwise to obtain several second similarities.

[0123] Correspondingly, in step 309, the parameters of the feature representation model and the label representation model can be adjusted with the goal of increasing the first similarity and decreasing the plurality of second similarities.

[0124] The feature representation model and label representation model adjusted in this way can have significantly different embedding results for different types of labels, thus enabling the model to better distinguish different categories in practical applications.

[0125] The above mainly described the processing method for gray samples. For samples with definite labels, the processing method can be similar. User features and labels of samples with definite labels can be obtained. The user features are input into a feature representation model to obtain the feature representation data of the sample, and the label is input into a label representation model to obtain the main label representation data. Then, the main label representation data can be directly used as the label representation data of the sample, and the similarity between it and the feature representation data can be calculated. The parameters of the representation model are adjusted with the goal of increasing this similarity.

[0126] In an optional implementation, samples with defined labels may also have corresponding information. The specific meaning of this information is detailed above and will not be repeated here. This information can also be input into the label representation model to obtain sub-label representation data for the sample. This enriches the meaning of the label representation data, allowing the model to learn more information and thus more easily handle subsequent classification tasks.

[0127] In other words, it is also possible to: obtain a second user sample, which includes a second user feature, a first label, and second information, wherein the second information is obtained based on features in the second user feature that correspond to the first label and exceed preset conditions; process the second user feature using a feature representation model to obtain second feature representation data; process the second information and the first label using a label representation model to obtain second label representation data; calculate a third similarity between the second feature representation data and the second label representation data; and adjust the parameters of the feature representation model with the goal of increasing the third similarity.

[0128] Specifically, processing the second information and the first label using a label representation model to obtain the second label representation data can be achieved by processing the second information using the label representation model to obtain the sub-label representation data corresponding to the second user sample, processing the first label using the label representation model to obtain the main label representation data corresponding to the second user sample, and obtaining the second label representation data corresponding to the second user sample based on the main label representation data and the sub-label representation data corresponding to the second user sample.

[0129] The other steps are implemented in a similar manner to those in Figure 3, and will not be described in detail here.

[0130] This allows the model to learn which user features are strongly correlated with a certain label, thus learning richer knowledge and performing classification tasks better when faced with a data distribution that is richer than the training samples.

[0131] The preceding text primarily describes the method presented in this specification from the perspective of the first classification task. Furthermore, this specification can also train feature representation models and label representation models capable of performing multiple classification tasks; that is, it can simultaneously execute multiple user classification tasks, such as simultaneously executing the first and second classification tasks belonging to the financial credit risk domain. In an optional embodiment, the first classification task is to classify the user's lease credit risk, and the second classification task is to classify the user's insurance credit risk.

[0132] Correspondingly, user samples can also have labels corresponding to different classification tasks. For example, the first user sample may also include a third label, which is one of a set of preset labels for the second classification task. If other user samples are obtained, these other user samples may also have labels corresponding to the second classification task, provided they already have labels corresponding to the first classification task.

[0133] It should also be noted that the concept of gray samples mentioned above applies to a single classification task. In one classification task, the first user sample may be a gray sample; in other classification tasks, the first user sample may have a definite label and is not a gray sample.

[0134] In the above scenario, since the label representations for different classification tasks may overlap—for example, high-risk users in both the first and second classification tasks might be represented by 0—different task identifiers can be set to enable the label representation model to distinguish between different classification tasks. For instance, the first label and the first information might correspond to a first task identifier. Correspondingly, the third label for the second classification task could correspond to the second task identifier.

[0135] In step 305, the first task identifier, first label, and first information can be processed using a label representation model to obtain main label representation data and sub-label representation data. Specifically, the first task identifier and first label can be input together into the label representation model to obtain main label representation data, and the first information and first task identifier can be input together into the label representation model to obtain sub-label representation data.

[0136] Correspondingly, for the second classification task, the third label and the second task identifier can be input into the label representation model to obtain the third label representation data corresponding to the second classification task.

[0137] When adjusting the model, the first similarity between the first feature representation data and the first label representation data, and the fourth similarity between the first feature representation data and the third label representation data can be calculated separately. The model parameters are then adjusted with the goal of increasing the first and fourth similarities.

[0138] In one optional implementation, for the label representation data of different classification tasks, the label representation data corresponding to different classification tasks can be in different spaces (such as vector space). That is, there is no correlation between the label representation data corresponding to different classification tasks, and the two can be repeated. In the case of repetition, it will not affect the final result.

[0139] In another alternative implementation, the label representation data for different classification tasks can reside in the same space (e.g., a vector space). This allows the feature representation model and the label representation model to learn the differences and relationships between different classification tasks, ensuring the robustness of the trained model.

[0140] Correspondingly, in order for the model to perform classification tasks better and distinguish different categories better, the similarity between labels corresponding to different classification tasks can be reduced.

[0141] Specifically, the following can be achieved: obtaining label representation data corresponding to each label under the first classification task and the second classification task through the label representation model; calculating the similarity between each pair of the label representation data to obtain several third similarities; and adjusting the parameters of the feature representation model and the label representation model with the goal of reducing the number of third similarities.

[0142] The third similarity can be the similarity between any two labels in the first and second classification tasks. For example, it can be the similarity between the first and second labels of the first classification task, or the similarity between the first label of the first classification task and the third label of the second classification task.

[0143] This allows the model to better distinguish categories under different classification tasks and perform classification tasks better.

[0144] Furthermore, compared to classifiers in related technologies, the method in this specification is more suitable for incremental learning.

[0145] Specifically, in related technologies, when it is necessary to add new categories or change the classification method of existing categories after the classifier has been trained, it is often necessary to retrain the classifier.

[0146] The method described in this specification flexibly represents labels by using labeled data, thus enabling the learning of new categories without forgetting previously learned categories when adding new ones. This facilitates continuous learning and rapid model updates, improving the model's applicability and scalability in dynamic real-world environments.

[0147] Specifically, when adding a new category, we can refer to the method described above and adjust the model parameters based on the training samples corresponding to the new category to better adapt to it. This also improves the robustness of the model.

[0148] Furthermore, when existing categories change, the method described in this specification can easily modify the label representation data corresponding to each category by using sub-label representation data. For example, the originally defined sub-label "had made one transfer to a user with fraud risk within the past three years" originally referred to users who were considered high-risk. However, after adding a new category or changing the category, high-risk users no longer include users who "had made one transfer to a user with fraud risk within the past three years." Therefore, the label representation data corresponding to the sub-label can be subtracted from the original label representation data corresponding to high-risk users to obtain the new label representation data for high-risk users.

[0149] The above methods first improve flexibility and scalability while reducing resource consumption. Specifically, traditional classifiers often require retraining or modifying the model structure when adding new categories. However, label embedding and sub-label embedding allow for incremental training, reducing the resource consumption caused by frequent retraining.

[0150] Secondly, this method, through a sub-label embedding mechanism, allows gray samples to be added as supplementary information to the training process, enabling a more comprehensive learning of the data distribution. It provides a better understanding and handling of data points that do not perfectly conform to the predetermined categories, thus improving model robustness. Furthermore, it optimizes data utilization and may improve the model's predictive ability for unseen data.

[0151] Furthermore, the aforementioned method, through label embedding, enables the model to learn subtle relationships between different categories in a high-dimensional space, avoiding information loss and improving classification accuracy. Moreover, when facing different classification tasks, by representing the labels corresponding to different classification tasks in the same vector space, the model can leverage the subtle differences in data between similar classification tasks to learn the characteristics of different classification categories, providing more refined classification judgments.

[0152] Finally, compared with the active learning or pseudo-labeling methods described in related technologies, the method provided in this specification does not require additional labeling work or relies on unstable predictions at the beginning of the model, thereby reducing labor costs and potential error propagation.

[0153] This specification also provides a feature representation method based on a model trained using the above method. As shown in Figure 5, Figure 5 illustrates a flowchart of a feature representation method, including:

[0154] Step 501: Obtain the user characteristics of the target user.

[0155] The user characteristics of the target users are similar to those of the first user sample mentioned earlier, and will not be repeated here.

[0156] Step 503: Obtain the feature representation model.

[0157] The feature representation model is obtained through the aforementioned representation model training method. A detailed explanation of the feature representation model can be found above and will not be repeated here.

[0158] Step 505: Process the user features using the feature representation model to obtain target feature representation data for representing the user features.

[0159] By inputting user features into the feature representation model, target feature representation data that meets the requirements (such as the requirements of a classification task) can be obtained, thereby enabling the completion of the required classification task.

[0160] Specifically, after step 505, target label representation data corresponding to the target task can be obtained; target similarity between the target label representation data and the target feature representation data can be calculated respectively, and the classification result of the target user can be determined based on the target similarity.

[0161] The label representation model mentioned earlier can output label representation data corresponding to each category type. Therefore, the label representation data corresponding to each label can be obtained through the label representation model. When it is necessary to classify users, the user's label representation data can be compared with the label representation data corresponding to the target task to determine the classification result of the target user.

[0162] The number of target labels representing data can be one or more, and the specific number can be determined according to the target task.

[0163] For example, if the objective is to identify high-risk users in the rental market, the target feature representation data and the target label representation data of the high-risk user can be used to calculate the target similarity. If the target similarity is greater than a preset threshold, the target user is determined to be a high-risk user in the rental market.

[0164] For example, if the objective is to classify the rental credit risk of users, which includes three categories: high risk, medium risk, and low risk, then the target feature representation data can be compared with the target label representation data corresponding to the three categories to calculate the target similarity. If the highest similarity among the three targets is greater than a preset threshold, the target user can be classified into the category with the highest target similarity.

[0165] For example, if the objective is to identify high-risk users in the rental and lending credit sectors, then the target feature representation data can be compared with the target representation data of the two categories to calculate target similarity. For each target similarity score, if the similarity score is greater than a preset threshold, the target user can be assigned to the corresponding category.

[0166] This allows for more flexible and accurate user classification. Furthermore, the same model can perform multiple classification tasks simultaneously, improving the ease of classification.

[0167] Corresponding to the embodiments of the foregoing methods, this specification also provides embodiments of the apparatus and the computer equipment to which it is applied.

[0168] As shown in Figure 6, Figure 6 is a block diagram of a representation model training apparatus according to an exemplary embodiment of this specification, the apparatus comprising:

[0169] The sample acquisition module 610 is used to acquire a first user sample, the first user sample including a first user feature, a first label and first information, the first label being one of a plurality of preset labels for a first classification task, and the first information being acquired based on a feature in the first user feature that corresponds to a second label in the plurality of labels;

[0170] The first embedding module 620 is used to process the first user features using a feature representation model to obtain first feature representation data;

[0171] The second embedding module 630 is used to process the first label and the first information using a label representation model to obtain main label representation data and sub-label representation data, respectively.

[0172] The similarity calculation module 640 is used to calculate a first label representation data based on the main label representation data and the sub-label representation data, and to calculate a first similarity between the first label representation data and the feature representation data;

[0173] The parameter adjustment module 650 is used to adjust the parameters of the feature representation model with the goal of increasing the first similarity.

[0174] In an optional implementation, the parameter adjustment module 650 is specifically used to adjust the parameters of the feature representation model and the label representation model with the goal of increasing the first similarity.

[0175] In an optional implementation, the similarity calculation module 640 is further configured to obtain label representation data corresponding to each label under the first classification task through the label representation model; and to calculate the similarity between each pair of the label representation data to obtain several second similarities. The parameter adjustment module 650 is specifically configured to adjust the parameters of the feature representation model and the label representation model with the goal of increasing the first similarity and decreasing the several second similarities.

[0176] In an optional embodiment, the device further includes a second user sample processing module 660 (not shown in the figure), used to acquire a second user sample, the second user sample including a second user feature, a first label, and second information, wherein the second information is acquired based on features in the second user feature that correspond to the first label and exceed a preset condition; the second user feature is processed using a feature representation model to obtain second feature representation data; the second information and the first label are processed using a label representation model to obtain second label representation data; a third similarity is calculated between the second feature representation data and the second label representation data; and the parameters of the feature representation model are adjusted with the goal of increasing the third similarity.

[0177] In one optional implementation, the sub-label represents the value range of the data within a preset value range.

[0178] In an optional implementation, the first user sample further includes a third label, which is one of a plurality of preset labels for the second classification task. The first label and the first information correspond to a first task identifier. The second embedding module 630 is specifically used to process the first task identifier, the first label, and the first information using a label representation model to obtain main label representation data and sub-label representation data.

[0179] In an optional implementation, the first user sample further includes a third label, which is one of a plurality of preset labels for the second classification task. The similarity calculation module 640 is further configured to obtain label representation data corresponding to each label under the first classification task and the second classification task through the label representation model; and to calculate pairwise similarity for each label representation data to obtain several third similarities. The parameter adjustment module 650 is further configured to adjust the parameters of the feature representation model and the label representation model with the goal of reducing several of the third similarities.

[0180] In one optional implementation, the first classification task is to classify the user's lease credit risk, and the second classification task is to classify the user's insurance credit risk.

[0181] As shown in Figure 7, Figure 7 is a block diagram illustrating a feature characterization device according to an exemplary embodiment of this specification. The device includes:

[0182] Feature acquisition module 710 is used to acquire user features of the target user;

[0183] The model acquisition module 720 is used to acquire a feature representation model, which is trained by the aforementioned representation model training method.

[0184] The feature processing module 730 is used to process the user features using the feature representation model to obtain target feature representation data for representing the user features.

[0185] In an optional embodiment, the device includes a classification module 740, which is used to acquire multiple target label representation data corresponding to the target task; calculate the target similarity between each of the target label representation data and the target feature representation data; and determine the classification result for the target user based on each of the target similarities.

[0186] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0187] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the solution in this specification according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0188] Figure 8 shows a hardware structure diagram of a computer device, which may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0189] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification. The processor implements the above-described methods by running executable instructions.

[0190] The memory 1020 for storing processor-executable instructions can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020.

[0191] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0192] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0193] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0194] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0195] This specification also provides a computer program product that, when executed by a processor, implements the above-described representation model training method or feature representation method.

[0196] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned representation model training method or feature representation method.

[0197] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0198] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0199] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0200] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

Claims

1. A method for training a representation model, comprising: Obtain a first user sample, which includes a first user feature, a first label, and first information. The first label is one of a plurality of preset labels for a first classification task. The first information is obtained based on the feature in the first user feature that corresponds to the second label among the plurality of labels. The first user features are processed using a feature representation model to obtain first feature representation data; The first label and the first information are processed using a label representation model to obtain main label representation data and sub-label representation data, respectively. First label representation data is calculated based on the main label representation data and the sub-label representation data, and a first similarity is calculated between the first label representation data and the feature representation data. With the goal of increasing the first similarity, the parameters of the feature representation model are adjusted.

2. The method according to claim 1, wherein adjusting the parameters of the feature representation model with the goal of increasing the first similarity includes: With the goal of increasing the first similarity, the parameters of the feature representation model and the label representation model are adjusted.

3. The method according to claim 2, further comprising: The label representation model is used to obtain the label representation data corresponding to each label in the first classification task. For each of the label representation data, the similarity is calculated pairwise to obtain several second similarities; The adjustment of the parameters of the feature representation model includes: With the goal of increasing the first similarity and decreasing the plurality of second similarities, the parameters of the feature representation model and the label representation model are adjusted.

4. The method according to claim 1, further comprising: A second user sample is obtained, which includes a second user feature, a first label, and second information. The second information is obtained based on the features in the second user feature that correspond to the first label and exceed a preset condition. The second user features are processed using a feature representation model to obtain second feature representation data; The second information and the first label are processed using a label representation model to obtain second label representation data; Calculate the third similarity between the second feature representation data and the second label representation data; The parameters of the feature representation model are adjusted with the goal of increasing the third similarity.

5. The method according to claim 1, wherein the value range of the sub-label characterizing data is within a preset value range.

6. The method according to claim 1, wherein the first user sample further includes a third tag, the third tag being one of a plurality of preset tags for the second classification task, and the first tag and the first information correspond to a first task identifier; The process of using a label representation model to process the first label and the first information of the sample to obtain the main label representation data and sub-label representation data of the sample, includes: The first task identifier, the first label, and the first information are processed using a label representation model to obtain main label representation data and sub-label representation data.

7. The method according to claim 1, wherein the first user sample further includes a third label, the third label being one of a plurality of preset labels for the second classification task; the method further includes: The label representation model is used to obtain the label representation data corresponding to each label under the first classification task and the second classification task. For each of the label representation data, the similarity is calculated pairwise to obtain several third similarities; With the goal of reducing several of the third similarities, the parameters of the feature representation model and the label representation model are adjusted.

8. The method according to claim 6 or 7, wherein the first classification task is to classify the user's lease credit risk, and the second classification task is to classify the user's insurance credit risk.

9. A feature characterization method, comprising: Obtain user characteristics of the target user; A feature representation model is obtained, wherein the feature representation model is trained by the method described in any one of claims 1-9; The user features are processed using the feature representation model to obtain target feature representation data for representing the user features.

10. The method of claim 9, further comprising: Obtain the target label representation data corresponding to the target task; Calculate the target similarity between the target label representation data and the target feature representation data, and determine the classification result for the target user based on the target similarity.

11. A representation model training device, comprising: The sample acquisition module is used to acquire a first user sample, which includes a first user feature, a first label, and first information. The first label is one of a plurality of preset labels for a first classification task, and the first information is acquired based on a feature in the first user feature that corresponds to a second label among the plurality of labels. The first embedding module is used to process the first user features using a feature representation model to obtain first feature representation data; The second embedding module is used to process the first label and the first information using a label representation model to obtain main label representation data and sub-label representation data, respectively. The similarity calculation module is used to calculate the first label representation data based on the main label representation data and the sub-label representation data, and to calculate the first similarity between the first label representation data and the feature representation data; The parameter adjustment module is used to adjust the parameters of the feature representation model with the goal of increasing the first similarity.

12. A feature characterization device, comprising: The feature acquisition module is used to acquire the user features of the target user; A model acquisition module is used to acquire a feature representation model, wherein the feature representation model is trained by the method described in any one of claims 1-9; The feature processing module is used to process the user features using the feature representation model to obtain target feature representation data for representing the user features.

13. A computer program product that, when executed by a processor, implements the method as described in any one of claims 1-10.

14. A computer device, comprising: processor; Memory used to store processor-executable instructions; The processor implements the method as described in any one of claims 1-10 by executing the executable instructions.

Images