Classification method, apparatus, device, storage medium and product

Abstract

This application discloses a classification method, apparatus, device, storage medium, and product, belonging to the field of artificial intelligence technology. The method includes: acquiring at least two classification models, each corresponding to a business scenario related to a target model; acquiring unlabeled user account-related training data within the business scenarios; performing relevance-based knowledge distillation on the knowledge of each of the at least two classification models based on the training data to train the target model, where the relevance characterizes the degree of correlation between each classification model and the target model; acquiring a target account within the business scenarios; inputting the target account into the trained target model to obtain a classification result corresponding to the target account. This application embodiment can improve the cold-start effect of the model used for user account classification by relying on multiple related non-cold-start state models in unlabeled scenarios through knowledge distillation.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a classification method, apparatus, device, storage medium, and product. Background Technology

[0002] In an era of rapid development in internet technology, with the continuous expansion of business scenarios and the increasing demands for user account classification, new models for user account classification need to be constantly trained based on actual needs. For each new user account classification model, the cold start problem is inevitable. Cold start solutions for related technologies usually rely heavily on expert knowledge, knowledge transfer, or knowledge distillation of a single model. However, expert knowledge is often difficult to obtain, and the applicable scenarios for knowledge transfer or knowledge distillation of a single model are also very limited. This makes it difficult to solve the problem of poor cold start performance in many scenarios. Summary of the Invention

[0003] This application provides a classification method, apparatus, device, storage medium, and product that can improve the cold start effect of the model used for user account classification by relying on multiple related non-cold start state models in unlabeled scenarios through knowledge distillation, and is especially suitable for rapid classification in cold start scenarios.

[0004] According to one aspect of the embodiments of this application, a classification method is provided, the method comprising:

[0005] Obtain at least two classification models, wherein the business scenarios corresponding to each of the at least two classification models are related to the business scenarios corresponding to the target model;

[0006] Obtain unlabeled training data related to user accounts in the aforementioned business scenario;

[0007] Based on the training data, knowledge distillation based on relevance is performed on the knowledge of each of the at least two classification models to train the target model, wherein the relevance characterizes the degree of correlation between each classification model and the target model;

[0008] Obtain the target account in the business scenario, input the target account into the trained target model, and obtain the classification result corresponding to the target account.

[0009] According to one aspect of the embodiments of this application, a sorting apparatus is provided, the apparatus comprising:

[0010] The association model acquisition module is used to acquire at least two classification models, wherein the business scenarios corresponding to each of the at least two classification models are related to the business scenarios corresponding to the target model.

[0011] The training data acquisition module is used to acquire unlabeled training data related to user accounts in the business scenario.

[0012] A knowledge distillation model is used to perform relevance-based knowledge distillation on the knowledge of each of the at least two classification models based on the training data, and to train the target model, wherein the relevance characterizes the degree of correlation between each classification model and the target model;

[0013] A classification model is used to obtain the target account in the business scenario. The target account is input into the trained target model to obtain the classification result corresponding to the target account.

[0014] According to one aspect of the embodiments of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the above classification method.

[0015] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored in the storage medium, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the above-described classification method.

[0016] According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform to implement the above-described classification method.

[0017] The technical solution provided in this application can bring the following beneficial effects:

[0018] This application provides a classification method that can perform knowledge distillation on other classification models related to the target model to be cold-started to different degrees, and transfer the knowledge obtained from the distillation to the target model, thereby enabling the target model to be quickly and accurately classified during the cold-start stage and improving the cold-start effect. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of an application runtime environment provided in one embodiment of this application;

[0021] Figure 2 This is a flowchart of a classification method provided in one embodiment of this application;

[0022] Figure 3 This is a flowchart of a second classification result determination method provided in one embodiment of this application;

[0023] Figure 4 This is a schematic diagram illustrating an embodiment of the classification method provided in this application;

[0024] Figure 5 This is a schematic diagram of a phased implementation method provided in one embodiment of this application;

[0025] Figure 6 This is a schematic diagram of the phased implementation steps provided in one embodiment of this application;

[0026] Figure 7 This is a block diagram of a sorting device provided in one embodiment of this application;

[0027] Figure 8 This is a structural block diagram of a computer device provided in one embodiment of this application. Detailed Implementation

[0028] Before introducing the method embodiments provided in this application, a brief introduction will be given to the relevant terms or nouns that may be involved in the method embodiments of this application, so as to facilitate the understanding of those skilled in the art.

[0029] BERT (Bidirectional Encoder Representation from Transformers) is a large-scale text pre-trained model. BERT uses a 12-layer transformer encoder to significantly improve the benchmark performance of natural language processing tasks. Compared to word2vec (word vectors), BERT, pre-trained on massive amounts of text, can incorporate more transfer knowledge into classification algorithms, providing more accurate text features.

[0030] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0031] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0032] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0033] Deep learning: The concept of deep learning originated from research on artificial neural networks. A multilayer perceptron with multiple hidden layers is a type of deep learning architecture. Deep learning discovers distributed feature representations of data by combining low-level features to form more abstract high-level representations of attribute categories or features.

[0034] Computer vision (CV) is a science that studies how to make machines "see," or more specifically, how to use cameras and computers to do so.

[0035] Machine vision, which involves the human eye recognizing and measuring targets, further processes images to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0036] Key technologies in speech technology include Automatic Speech Recognition (ASR), Text-to-Speech (TTS), and voiceprint recognition. Enabling computers to hear, see, speak, and feel is the future direction of human-computer interaction, with speech emerging as one of the most promising methods.

[0037] Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close relationship with linguistic research. NLP technologies typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0038] Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to achieve data computing, storage, processing, and sharing.

[0039] Cloud technology is a general term encompassing network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will all require robust system support, which can only be achieved through cloud computing.

[0040] CNN (Convolutional Neural Networks) is a type of feedforward neural network that includes convolutional computation and has a deep structure. It is one of the representative algorithms of deep learning, has representation learning capabilities, and can perform translation-invariant classification of input information according to its hierarchical structure. It can be used as the model framework in the embodiments of this application.

[0041] The KS test, short for Kolmogorov–Smirnov test, is based on the cumulative distribution function and is used to test whether two empirical distributions are different or whether one empirical distribution is different from another ideal distribution. For example, the KS value can be used as an evaluation metric for how well an artificial intelligence model can separate positive and negative samples.

[0042] The OOT set is a cross-time validation set that is temporally isolated from the training data. It is divided from the entire labeled dataset based on a specific time point. Before this time point is the training and testing set, and after this time point is the OOT set. Samples with the most recent application date are usually reserved as OOT to measure the model's stability over time.

[0043] Wasserstain Distance (WD distance): Wasserstain distance (Kantorovich–Rubinstein metric), also known as Earth-Mover distance (EM distance), is used to measure the distance between two distributions.

[0044] Zero samples / unlabeled samples: A sample set with X features but no true Y labels.

[0045] In the fields of machine learning and artificial intelligence, the cold start problem is a common challenge. Cold start refers to the process of building a model from scratch with little or no data. Taking financial business as an example, this involves the cold start problem of risk control models. In the early stages of financial operations, the long delinquency period (from a user's application to the financial institution to normal repayment or the emergence of delinquency risk can take 1 to 6 months or more) makes it difficult to collect enough labeled samples for effective supervised risk control model building in a short period. This leads to the cold start problem. Therefore, how to build an effective risk control model using unlabeled samples of target customers during the cold start phase of financial operations is crucial for identifying risky targets, improving risk control capabilities, and controlling business risk gaps early. In fact, a risk control model is essentially a classification model, aiming to identify targets belonging to the dangerous category in a timely manner. This means that many classification models suffer from the cold start problem in their initial construction, and related technologies offer relatively limited solutions to this problem.

[0046] For classification problems based on unlabeled samples, relevant techniques can be used to solve them, such as expert policy rules, adversarial transfer learning, and knowledge distillation. Specifically:

[0047] Expert strategy rules: This method primarily relies on the expertise of business or technical personnel. Through their understanding of the business, they specify a classification engine suitable for the target business, or select and reuse existing classification models or strategies from similar existing businesses based on expert experience as the classification model or strategy for the target scenario. This approach, based on expert strategy rules, heavily depends on the experience and knowledge of professionals, making it difficult to achieve the maximum classification effect.

[0048] Adversarial transfer learning: Transfer learning methods centered on adversarial training primarily utilize Domain Adversarial Neural Networks (DANNs) to mix labeled samples from the source domain and unlabeled samples from the target domain in the same feature space, training the model using the labeled samples from the source domain. A standard DANN typically includes a feature extractor, a domain classifier, and a domain classifier. The adversarial idea is embedded in the feature extractor and the domain classifier. During the forward propagation of the network, the information extracted by the feature extractor is fed into the domain classifier. The domain classifier determines whether the input information comes from the source or target domain and calculates the domain classification loss, where the goal of the domain classifier is to distinguish as accurately as possible whether the input features come from the source or target domain. During the backpropagation, the gradient reversal layer between the domain classifier and the feature extractor makes the training objective of the feature extractor opposite to that of the domain classifier; that is, the feature extractor aims to output features that prevent the domain classifier from correctly determining which domain the information comes from. This adversarial relationship ultimately prevents the domain classifier from correctly distinguishing the received information, and the feature extractor successfully mixes the source and target domain samples in a common feature space. Since the source domain samples are labeled, the classification model trained on the source domain samples in the common feature space can be applied to the classification task of the target scenario, thus achieving cold-start training of the target domain classification model. Adversarial transfer learning methods can build a cold-start classification model for the target scenario based on a large number of unlabeled samples from the target domain and labeled samples from the source domain. However, it requires the source domain to have a large number of labeled samples and to train the classification model on the transformed common feature space using the source domain samples. However, due to the special properties of certain scenarios, business personnel usually only have access to the pre-trained risk control model in the source domain (the business scenario related to the cold-start scenario), which is a "black box" to the target business. Traditional adversarial transfer learning is not suitable for transfer learning based solely on "black box" models and unlabeled samples.

[0049] Knowledge distillation, initially used to address model compression, is now widely applied in transfer learning. Its core principle is to transfer knowledge from a complex neural network to a simpler one. The complex network being transferred is called the teacher network, and the compressed network is called the student network. The process involves the teacher network outputting the feature representations of its intermediate layers or the final classification prediction as "soft labels" from a large number of labeled or unlabeled samples. The student network learns from these "soft labels," mimicking the teacher network's predictive behavior by learning and replicating its feature representations and classification results. Compared to adversarial transfer learning methods, knowledge distillation can transfer knowledge from a "black box" teacher model to a student model in a target scenario based solely on unlabeled samples. However, it primarily targets transfer learning from a single teacher network to a single student network. In many classification scenarios, business users often face the problem of transferring knowledge from multiple teacher models to the same student model, and the relevance of the teacher model to the target scenario's business context determines the quality of knowledge learned by the student model. Current technologies are unable to address this issue effectively.

[0050] Obviously, in solving the cold start problem of classification models, related technologies struggle to automatically improve the cold start effect by using knowledge distillation based on knowledge from other models with varying degrees of relevance to the target model. Therefore, this application provides a classification method that can perform knowledge distillation on other classification models related to the target model to be cold-started to different degrees, and transfer the distilled knowledge to the target model, thereby enabling the target model to be quickly and accurately classified during the cold start phase, thus improving the cold start effect.

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0052] Please refer to Figure 1 This diagram illustrates an application runtime environment provided in one embodiment of this application. The application runtime environment may include: terminal 10 and server 20.

[0053] Terminal 10 includes, but is not limited to, electronic devices such as mobile phones, computers, smart voice interaction devices, smart home appliances, in-vehicle terminals, game consoles, e-book readers, multimedia playback devices, and wearable devices. Application clients can be installed on terminal 10.

[0054] In this embodiment, the application can be any application capable of providing user account-based services. Typically, this application is a media application, financial application, social application, news application, interactive entertainment application, browser application, shopping application, content sharing application, virtual reality (VR) application, augmented reality (AR) application, etc., and this embodiment does not limit this. Furthermore, different applications may use different criteria to classify user accounts, and their corresponding functions may also differ. These can be pre-configured according to actual needs, and this embodiment does not limit this. Optionally, the terminal 10 runs a client of the aforementioned application.

[0055] Server 20 provides background services to clients of applications in terminal 10. For example, server 20 can be a background server for the aforementioned applications. Server 20 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, server 20 can simultaneously provide background services to applications in multiple terminals 10.

[0056] Optionally, terminal 10 and server 20 can communicate with each other via network 30. Terminal 10 and server 20 can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0057] Please refer to Figure 2 The diagram illustrates a flowchart of a classification method provided in one embodiment of this application. This method can be applied to computer devices, which refer to electronic devices capable of data computation and processing. For example, the entity executing each step may be... Figure 1 The application runtime environment shown is server 20. This method may include the following steps:

[0058] S101, Obtain at least two classification models, where the business scenarios corresponding to each of the at least two classification models are related to the business scenarios corresponding to the target model.

[0059] Specifically, at least two of the aforementioned classification models are not in a cold start state, while the target model before training is in a cold start state. That is, the two classification models can be considered teacher models, containing relevant knowledge specific to the business scenario, while the target model is a student model. By distilling the knowledge from the teacher models, a better output can be quickly obtained in the cold start state. Both the teacher and student models reside in the same business scenario. For example, in the financial field, both the teacher and student models reside in a risk control scenario, outputting classification results for user accounts, indicating whether a user account is a risky account. In the media recommendation field, both the teacher and student models reside in a media content recommendation scenario, outputting classification results for user accounts, indicating whether a user account is a media content sensitive account.

[0060] Of course, the embodiments of this application do not limit the structure of any of the mentioned models. For example, it can be a model constructed by any machine learning method such as Bayesian, decision tree, random forest, neural network, etc.

[0061] S102. Obtain unlabeled training data related to user accounts in the above business scenarios.

[0062] The cold start phase typically lacks labeled data. To train the target model, unlabeled training data can be directly obtained. For example, in a user account classification scenario, unclassified user accounts and their corresponding features can be directly obtained as training data. This training data might include at least one of the following: user account behavioral characteristics, user static attribute characteristics, user social attribute characteristics, and user social behavior history. Based on this training data, user accounts can be classified, such as whether a user account poses a business risk.

[0063] S103. Based on the above training data, perform knowledge distillation based on relevance on the knowledge of each of the above at least two classification models, and train the above target model. The above relevance represents the degree of relevance between each of the above classification models and the above target model.

[0064] Specifically, the above-mentioned relevance-based knowledge distillation of the knowledge of each of the at least two classification models based on the above-mentioned training data, and training the above-mentioned target model, includes: obtaining first training data for each of the above-mentioned classification models and second training data for the above-mentioned target model based on the above-mentioned training data; inputting the second training data into each of the above-mentioned classification models to obtain a first classification result; fusing the first classification results of each of the above-mentioned classification models according to the relevance corresponding to each of the above-mentioned classification models to obtain a second classification result, wherein the relevance is calculated by the difference between the first training data and the second training data corresponding to the above-mentioned classification models; inputting the second training data into the above-mentioned target model to obtain a classification prediction result; and adjusting the parameters of the above-mentioned target model based on the difference between the classification prediction result and the second classification result.

[0065] To obtain the aforementioned relevance, a relevance calculator can be trained. This calculator is used to calculate the correlation between the first model and the second model. The first model can be any model within the aforementioned business scenario, and the second model can be any model different from the first model within the same business scenario. Based on the relevance calculator, the relevance of the classification models is determined. This application does not limit the measurement criteria for the relevance. For example, W distance (also known as WD distance) from relevant technologies can be used, or any distance calculation method such as KL divergence or Euclidean distance can be used. The concept of KL divergence originates from probability theory and information theory. KL divergence is also known as relative entropy, cross-entropy, discriminative information, Kullback entropy, and Kullback-Leible divergence (short for KL divergence). Either the first model or the second model can be any of the aforementioned classification models or a target model.

[0066] The aforementioned relevance calculator is trained using the following method: acquiring sample data from different source domains within the same business scenario; inputting any two sets of sample data into the distance calculator to obtain the relevance between the two sets of sample data, whereby the relevance represents the probability that the two sets of sample data belong to the same source domain; training the distance calculator based on the relevance between the two sets of sample data and the true source domain of the two sets of sample data, and determining the training result as the aforementioned relevance calculator.

[0067] A specific implementation describes the training method for the relevance calculator proposed in this application. The sample data includes all samples S = {S...} from multiple source domains. iEach source domain can be viewed as a domain formed by the input data of an independent model, where samples may be labeled or unlabeled. The source domain can be highly correlated with the domain formed by the input data of the aforementioned classification model or target model; for example, both could be domains formed by user account-related features. This application does not limit the number of samples to two sets. For example, each set of samples may include several samples; that is, the two sets of samples can be understood as two batches of samples, with two batches of data randomly selected each time. and By judging the degree of correlation, this degree of correlation represents and Whether they come from the same domain is used to iteratively update the model (distance calculator). If they come from the same domain, the distance is 0; otherwise, the distance is 1. The trained distance calculator can be used to calculate the correlation between the target model and each classification model. The greater the correlation, the smaller the corresponding distance.

[0068] The specific training process can be expressed by the following pseudocode:

[0069]

[0070] The formula for updating a parameter indicates how the parameter iterates. During training, two batches of data are randomly selected each time. and By judgment and Whether the model is updated from the same domain or not, the distance is 0 if they are from the same domain and 1 otherwise. The distance calculator after training can be used to calculate the correlation between the target model and the classification model. In, θ w This means that the right side represents the parameters from the previous iteration, and the left side represents the parameters from the next iteration. This indicates the calculation of cross-entropy. This indicates that the derivative of the loss function is taken, and the parameters move in the direction of gradient descent. α1 represents the learning rate.

[0071] In one example, the distance calculator is a W-distance calculation device, which can be used to obtain the W-distance matrix between the input data of two models. The smaller the W-distance, the greater the correlation between the two models. Through this step, the W-distance between the training samples of each classification model and the target model can be calculated, that is, the distance between the first training data and the second training data, which serves as the basis for knowledge distillation.

[0072] In one embodiment, such as Figure 3 As shown, based on the relevance of each of the above classification models, the first classification results of each of the above classification models are fused to obtain the second classification result, including:

[0073] S1. For each classification model, input the first training data and the second training data corresponding to each classification model into the correlation model to obtain the correlation between each classification model and the target model.

[0074] S2. Based on the correlation between each of the above classification models and the above target model, obtain the weight associated with the first classification result corresponding to each of the above classification models.

[0075] The relevance of each classification model is normalized to obtain the weight associated with the first classification result of each classification model.

[0076] S3. Perform a weighted summation on the results of the first classification above to obtain the results of the second classification above.

[0077] In fact, the second classification result can be considered as the pseudo-label corresponding to the second training data. The pseudo-label generation process is the same as steps S1-S3 described above. First, the correlation between the classification model and the target model is calculated using a correlation calculator based on the first and second training data. The correlation calculation process has been described above and will not be repeated here. Finally, each correlation is normalized to obtain the weight corresponding to each classification model, and then pseudo-labels are generated by weighted summation. In this process, the introduction of correlation mechanism ensures that the more relevant the classification model is to the target model, the greater its contribution to the pseudo-label, thereby reducing the noise impact of classification models with low correlation.

[0078] Specifically, in one example, the preceding text can be expressed using the following pseudocode:

[0079]

[0080] The first and second training data come from the same business scenario. For example, in a risk control scenario, both data originate from user account data corresponding to borrowers. Therefore, both the first and second training data belong to a subspace of the large borrower group, sharing significant commonalities. Of course, differences also exist. The calculated W-distance between the first and second training data quantifies their similarity, using this similarity as weight to jointly create the learning objective. When multiple classification models exist, the outputs of the second training data on each model are weighted using the W-distance, allowing for the transfer of commonalities. Ultimately, the learning objective of the target model is a weighted combination of the outputs of multiple classification models, thus complementing each other and effectively transferring knowledge to the target model.

[0081] S104. Obtain the target account in the above business scenario, input the target account into the trained target model, and obtain the classification result corresponding to the target account.

[0082] Based on the classification results of the target accounts, various subsequent processing can be carried out. For example, in a financial scenario, the classification result can characterize the risk level of the target account, or a fraud risk score. The higher the score, the higher the probability of fraud. In this case, it is possible to choose to provide services to the target account with caution, or to directly refuse to provide services to the target account.

[0083] This invention relates to the fields of machine learning and artificial intelligence. It aims to achieve rapid user account classification during the cold start phase by using multi-model distillation learning technology to train a target model based on unlabeled samples. This classification can improve classification capabilities in many scenarios. Taking the financial sector as an example, in the early stages of lending, the long delinquency period (typically 1 to 6 months or more from when a user applies for a loan to when they repay or show signs of delinquency) makes it difficult for financial institutions to collect sufficient labeled samples for effective supervised machine learning model construction. This results in difficulty in identifying risky user accounts in the early stages. The method described in this application allows for the construction of unlabeled samples using relevant user account data during the cold start phase, enabling the training of the target model and effectively identifying risky user accounts, thereby improving risk control capabilities.

[0084] Taking a financial scenario as an example, the solution in this application embodiment can be achieved through... Figure 4 In the first stage, a W-distance calculator can be trained, which is the relevance calculator mentioned earlier. Existing classification models in the financial scenario are used as teacher models to train a new target model (student model). For each teacher model and target model, corresponding unlabeled samples are obtained, which are the first and second training data mentioned above. The W-distance calculator can be trained based on these unlabeled samples. The relevance between each teacher model and target model can be obtained based on the training results. Based on this relevance and the classification results of each teacher model for the second training data, pseudo-labels are generated. Distillation training based on the pseudo-labels yields the trained target model. This target model can classify user accounts, thereby enabling risk control in the financial scenario. Therefore, after deploying the trained target model applied to this financial scenario as a risk control model, a deployment service (Model Service) can be provided. Inputting the relevant information of the target account to be classified into this deployment service will yield the output of the target model (fraud risk score).

[0085] Please refer to Figure 5The diagram illustrates an implementation of an embodiment of this application. The method mainly consists of three stages: W-distance calculation, pseudo-label generation, and multi-model distillation. Specifically: 1) W-distance calculation primarily calculates the business relevance between the unlabeled samples of the target scene and the teacher model. The more similar a teacher model is to the target scene, the smaller its W-distance to the unlabeled samples of the target scene. The unlabeled samples of the target scene are the second training data mentioned earlier, and the teacher model is the classification model mentioned earlier. Specific details can be found in the preceding text. 2) Pseudo-label generation is responsible for generating "pseudo-labels" for the unlabeled samples of the target scene based on the outputs of multiple teacher models. The smaller the distance between a teacher model and the target sample, the greater its contribution to the generation of "pseudo-labels." 3) Multi-model distillation uses the "pseudo-labels" generated by the teacher models as training targets to train the student network (target model), enabling the student model to have the same predictive ability as the teacher model, thereby achieving multi-teacher model guidance for training the target scene student model under zero-labeled sample conditions.

[0086] Please refer to Figure 6 The diagram illustrates the implementation steps of an embodiment of this application.

[0087] Step 1: Training Preparation. For the unlabeled samples, a high-performance model (target model) that closely matches the new scenario is custom-trained. Before modeling, unlabeled samples (secondary training data), unlabeled training samples for multiple teacher models (classification models) (primary training data), and multiple teacher models are required.

[0088] Step 2: Training the relevance calculator. Specifically, this relevance calculator can be a self-supervised W-distance calculation network. The training data includes all training samples from multiple source domains. Each source domain can be viewed as an independent specific scenario, but all correspond to the same business scenario. It can serve as the first training data for the teacher model or the second training data for the target scenario. Two batches of data are randomly selected each time, and the relevance calculator is iteratively updated by determining whether they come from the same domain. For a well-trained relevance calculator, the more relevant the inputs of two models are, the higher the relevance between the two models, and the smaller the W-distance.

[0089] Step 3: W-distance calculation. By inputting the first training data and the second training data into the W-distance calculation network, the correlation between the corresponding classification model and the target model can be obtained, which serves as the basis for generating pseudo-labels.

[0090] Step 4: Calculation of the first classification result. Input the unlabeled samples (second training data) into each teacher model to obtain the classification result (first classification result) of each teacher model.

[0091] Step 5: Pseudo-label generation. The first classification results are weighted based on relevance to obtain the second classification results (pseudo-labels).

[0092] Step 6: Student Model Training. Unlabeled samples and "pseudo-labels" are input into the multi-model distillation device. The multi-model distillation device uses unlabeled samples from the target scenario as training data and "pseudo-labels" generated by the pseudo-label generator as the training objective. It trains the student model (objective) using regression, ensuring that the student model has the same predictive ability as the teacher model. Finally, given only unlabeled samples from the target scenario, a risk control model for that scenario is obtained. The student model can be trained using any machine learning method, such as Bayesian, decision tree, random forest, or neural network.

[0093] Step 7: Deploy the student model. Deploy the distilled student model as a general-purpose model to serve customers.

[0094] It is evident that the entire modeling process does not require the use of labeled samples from new customers. Based on distillation learning, and considering the differences between the target scenarios of new customers and the teacher models, the predictive power of multiple teacher models is transferred to the student model with different weights. The student model trained by distillation can then be used as a formal model for user account prediction in actual production. Specifically, the embodiments of this application mainly solve the following three problems:

[0095] First, a relevance calculator was developed during training. This calculator measures the correlation between different models. Taking a financial scenario as an example, there may be multiple classification models for user accounts, each with varying degrees of correlation to the target model. The relevance calculator can be used to calculate this correlation. Generally, the more relevant the classification model is to the specific business scenario corresponding to the target model, the greater the probability that the predicted classification result will match the expected result of the target model.

[0096] Second, based on the generation of "pseudo-labels" from multi-classification models, multiple classification models are used as teacher models. By combining the prediction results of multiple teacher models with the corresponding relevance of multiple teacher models, "pseudo-labels" can be obtained for training the target model.

[0097] Third, based on the target model distillation of the multi-teacher model, the "pseudo-labels" generated by the teacher network are used as pseudo-real labels for the unlabeled training data of the target model to train the target network. Ultimately, the goal is to transfer the optimal combination ability of multiple teacher models to the same student model (target model) without providing labeled samples.

[0098] The method proposed in this application can effectively improve the user account classification ability during the cold start phase by training a cold-start target model based on multi-teacher model capability transfer learning under conditions of only unlabeled samples. The table below shows the KS validation results on 12 datasets. In each experiment, one dataset was extracted as the student network and only its unlabeled samples were used, while the risk prediction models corresponding to the other 11 datasets were used as teacher networks. As a benchmark scheme, the table below also compares:

[0099] Mean_Teacher: Generates pseudo-labels using the average prediction results of the other 11 teacher models (classification models), which are then used to distill and train the student model (target model);

[0100] WD_Teacher: The proposed scheme in this application involves weighting the prediction results of the other 11 teacher models based on their relevance to generate pseudo-labels for unlabeled samples, and then distilling them to train the student model.

[0101] Self_Train: Train the risk model using real labels on each dataset.

[0102]

[0103]

[0104] Table 1 shows that the multi-model distillation scheme proposed in this application outperforms the modeling scheme based on average distillation learning overall, proving that selecting the relevance between the target unlabeled samples and the teacher model can effectively ensure the high relevance of the teacher model while suppressing the guiding influence of the low relevance of the teacher model on the student model. Furthermore, the comparison results between Self_Train and WD_Teacher show that the scheme proposed in this application can achieve prediction results close to those based on real label training even in unlabeled target scenarios. On some datasets (such as D3, D9, and D12), the scheme even surpasses the actual training results. This demonstrates that the scheme can effectively improve user account classification capabilities during the business cold start phase.

[0105] Specifically, the labels for Self_Train come from the students. This employs a cross-validation approach, aiming to split the dataset multiple times and average the results of each evaluation to eliminate the negative impact of imbalanced data splits in a single partition. Currently, the dataset with y-labels is divided into 12 parts (D1-D12). Each time, one part is selected as the student dataset, and the other 11 parts are used to train 11 teacher models using their respective y-labels. For example, dataset D1 is used as the student sample, and D2-D12 are used to train 11 teacher models using their own y-labels. First, the W distance is calculated between the outputs of D1 and the remaining D2-D12 models to quantify the similarity between D1 and D2-D12. Next, the outputs of D1 on the teacher models of D2-D12 are weighted and combined to form the learning objective WD_teacher, which is then used to train the student model. After obtaining the training results, the results need to be evaluated to determine their quality. The Self_Train comparison here uses student sample D1 and the student sample's own y-label to train the Self_Train result. This comparison shows that even without using the student sample's y-label, the distillation result is extremely close to the effect of the self-modeling using the student sample with the y-label. The experiment was conducted 12 times, with each dataset used as a student sample, resulting in 12 sets of experimental results.

[0106] The following are embodiments of the apparatus of this application, which can be used to execute embodiments of the method of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method of this application.

[0107] Please refer to Figure 7 This diagram illustrates a block diagram of a classification apparatus according to an embodiment of this application. The apparatus is applied to a first recommendation system and has the function of implementing the above-described classification method. This function can be implemented in hardware or by hardware executing corresponding software. The apparatus can be a computer device or can be installed within a computer device. The apparatus may include:

[0108] The association model acquisition module 101 is used to acquire at least two classification models, and the business scenarios corresponding to each of the at least two classification models are related to the business scenarios corresponding to the target model.

[0109] The training data acquisition module 102 is used to acquire unlabeled training data related to user accounts in the above business scenarios.

[0110] Knowledge distillation model 103 is used to perform relevance-based knowledge distillation on the knowledge of each of the at least two classification models based on the above training data, and to train the above target model, wherein the above relevance characterizes the degree of relevance between each of the above classification models and the above target model.

[0111] Classification model 104 is used to obtain the target account in the above business scenario. The target account is input into the trained target model to obtain the classification result corresponding to the target account.

[0112] In one embodiment, the above knowledge distillation model 103 is used for:

[0113] Based on the above training data, we obtain the first training data for each of the above classification models and the second training data for the above target model.

[0114] For each of the above classification models, the second training data is input into each of the above classification models to obtain the first classification result;

[0115] Based on the relevance of each of the above classification models, the first classification results of each of the above classification models are fused to obtain the second classification result. The relevance is calculated by the difference between the first training data and the second training data corresponding to the above classification models.

[0116] Input the second training data into the target model to obtain the classification prediction results;

[0117] Based on the difference between the above classification prediction results and the above second classification results, the parameters of the above target model are adjusted.

[0118] In one embodiment, the above knowledge distillation model 103 is used for:

[0119] Training relevance calculator, the relevance calculator is used to calculate the degree of relevance between the first model and the second model, the first model is any model under the above business scenario, and the second model is any model different from the first model under the above business scenario;

[0120] Based on the aforementioned relevance calculator, the relevance of the classification model is determined.

[0121] In one embodiment, the above knowledge distillation model 103 is used for:

[0122] Acquire sample data from different source domains within the same business scenario;

[0123] Input any two sets of sample data into the distance calculator to obtain the correlation between the two sets of sample data. The correlation represents the probability that the two sets of sample data belong to the same source domain.

[0124] Based on the correlation between the two sets of sample data and the true source domain of the two sets of sample data, the distance calculator is trained, and the training result is determined as the correlation calculator.

[0125] In one embodiment, the above knowledge distillation model 103 is used for:

[0126] For each classification model, the first training data and the second training data corresponding to each classification model are input into the correlation model to obtain the correlation between each classification model and the target model.

[0127] Based on the correlation between each classification model and the target model, the weight associated with the first classification result corresponding to each classification model is obtained.

[0128] The weighted sum of the results of the first classification above is used to obtain the results of the second classification above.

[0129] In one embodiment, the above knowledge distillation model 103 is used for:

[0130] The relevance of each classification model is normalized to obtain the weight associated with the first classification result of each classification model.

[0131] In one embodiment, in a user account classification scenario, the training data includes at least one of the following:

[0132] User account behavioral characteristics, user static attribute characteristics, user social attribute characteristics, and user social behavior history.

[0133] In one embodiment, at least two of the above classification models are not in a cold start state, while the target model before training is in a cold start state.

[0134] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0135] Please refer to Figure 8 This diagram illustrates a structural block diagram of a computer device according to an embodiment of this application. The computer device may be a server for executing the above-described classification method. Specifically:

[0136] Computer device 1600 includes a central processing unit (CPU) 1601, a system memory 1604 including random access memory (RAM) 1602 and read-only memory (ROM) 1603, and a system bus 1605 connecting the system memory 1604 and the CPU 1601. Computer device 1600 also includes a basic input / output system (I / O system) 1606 that facilitates information transfer between various devices within the computer, and a mass storage device 1607 for storing the operating system 1613, application programs 1614, and other program modules 1615.

[0137] The basic input / output system 1606 includes a display 1608 for displaying information and an input device 1609, such as a mouse or keyboard, for inputting information from content consumers. Both the display 1608 and the input device 1609 are connected to the central processing unit 1601 via an input / output controller 1610 connected to the system bus 1605. The basic input / output system 1606 may also include the input / output controller 1610 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1610 also provides output to a display screen, printer, or other types of output devices.

[0138] Mass storage device 1607 is connected to central processing unit 1601 via a mass storage controller (not shown) connected to system bus 1605. Mass storage device 1607 and its associated computer-readable media provide non-volatile storage for computer device 1600. That is, mass storage device 1607 may include computer-readable media (not shown) such as hard disk or CD-ROM (Compact Disc Read-Only Memory) drive.

[0139] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 1604 and mass storage device 1607 described above can be collectively referred to as memory.

[0140] According to various embodiments of this application, the computer device 1600 can also be connected to a remote computer on a network, such as the Internet. That is, the computer device 1600 can be connected to the network 1612 via the network interface unit 1611 connected to the system bus 1605, or the network interface unit 1611 can be used to connect to other types of networks or remote computer systems (not shown).

[0141] The aforementioned memory also includes a computer program stored in the memory and configured to be executed by one or more processors to implement the aforementioned classification method.

[0142] In an exemplary embodiment, a computer-readable storage medium is also provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, at least one program, code set, or instruction set is executed by a processor to implement the classification method described above.

[0143] Optionally, the computer-readable storage medium may include: ROM (Read Only Memory), RAM (Random Access Memory), SSD (Solid State Drives), or optical disc, etc. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0144] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the classification method described above.

[0145] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0146] Furthermore, in the specific embodiments of this application, data related to content consumption object information is involved. When the above embodiments of this application are applied to specific products or technologies, it is necessary to obtain the permission or consent of the content consumption object, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0147] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application shall be included within the protection scope of this application.

Claims

1. A classification method characterized by, The method comprises: obtaining at least two classification models, each corresponding to a business scenario related to a business scenario corresponding to a target model; obtaining unannotated user account-related training data under the business scenario; based on the training data, performing knowledge distillation of the knowledge of each classification model in the at least two classification models based on the relevance, training the target model, the relevance representing the degree of relevance between the each classification model and the target model; obtaining a target account under the business scenario, inputting the target account into the trained target model, and obtaining a classification result corresponding to the target account.

2. The method of claim 1, wherein, The method comprises: obtaining first training data for each classification model and second training data for the target model according to the training data; for each classification model, inputting the second training data into the each classification model to obtain a first classification result; based on the relevance corresponding to each classification model, fusing the first classification results of each classification model to obtain a second classification result, the relevance being calculated based on the difference between the first training data corresponding to the classification model and the second training data; inputting the second training data into the target model to obtain a classification prediction result; based on the difference between the classification prediction result and the second classification result, adjusting the parameters of the target model.

3. The method of claim 2, wherein, The method further comprises: training a relevance calculator, the relevance calculator being used to calculate the degree of relevance between a first model and a second model, the first model being any model under the business scenario, and the second model being any model different from the first model and under the business scenario; based on the relevance calculator, determining the relevance corresponding to the classification model.

4. The method according to claim 2 or 3, characterized in that, The relevance calculator is trained by the following method: obtaining sample data from different source domains in the same business scenario; inputting any two groups of sample data into a distance calculator to obtain the association degree of the two groups of sample data, the association degree representing the probability that the two groups of sample data belong to the same source domain; based on the association degree of the two groups of sample data and the real source domain of the two groups of sample data, training the distance calculator, and determining the training result as the relevance calculator.

5. The method of claim 3, wherein, The method comprises: for each classification model, inputting the first training data corresponding to the each classification model and the second training data into the relevance calculator to obtain the relevance between the each classification model and the target model; based on the relevance between the each classification model and the target model, obtaining the weight associated with the first classification result corresponding to the each classification model; weighting and summing each first classification result to obtain the second classification result.

6. The method of claim 5, wherein, The weight associated with the first classification result corresponding to each classification model is obtained according to a correlation degree of each classification model and the target model, and the weight associated with the first classification result corresponding to each classification model comprises: The correlation degrees corresponding to the classification models are normalized to obtain the weight associated with the first classification result corresponding to each classification model.

7. The method of claim 1, wherein, In the user account classification scenario, the training data comprises at least one of: behavior characteristics of a user account, static attribute characteristics of a user, social attribute characteristics of a user, and social behavior history records of a user.

8. The method of claim 1, wherein, The at least two classification models are not in a cold start state, and the target model before training is in a cold start state.

9. A sorting device, characterized in that The device comprises: a correlation model acquisition module configured to acquire at least two classification models, each of the at least two classification models corresponding to a business scenario related to a target model; a training data acquisition module configured to acquire unannotated training data related to a user account in the business scenario; a knowledge distillation model configured to perform knowledge distillation based on a correlation degree on knowledge of each of the at least two classification models according to the training data, and train the target model, the correlation degree representing a correlation degree of each of the at least two classification models and the target model; a classification model configured to acquire a target account in the business scenario, input the target account into the trained target model, and obtain a classification result corresponding to the target account.

10. The apparatus of claim 9, wherein, The knowledge distillation model is configured to: obtain first training data for each of the at least two classification models and second training data for the target model according to the training data; input the second training data into each of the at least two classification models to obtain a first classification result for each of the at least two classification models; fuse the first classification results of the at least two classification models to obtain a second classification result according to correlation degrees corresponding to the at least two classification models, the correlation degrees being calculated based on differences between the first training data and the second training data corresponding to the at least two classification models; input the second training data into the target model to obtain a classification prediction result; adjust parameters of the target model based on a difference between the classification prediction result and the second classification result.

11. The apparatus of claim 10, wherein, The knowledge distillation model is configured to: a training correlation degree calculator configured to calculate a correlation degree of a first model and a second model, the first model being any model in the business scenario, and the second model being any model different from the first model and in the business scenario; determine the correlation degrees corresponding to the at least two classification models based on the training correlation degree calculator.

12. The apparatus of claim 10 or 11, wherein, The knowledge distillation model is configured to: acquire sample data from different source domains in the same business scenario; input any two groups of sample data into a distance calculator to obtain a correlation degree of the two groups of sample data, the correlation degree representing a probability that the two groups of sample data belong to the same source domain; train the distance calculator based on the correlation degree of the two groups of sample data and real source domains of the two groups of sample data, and determine a training result as the correlation degree calculator.

13. The apparatus of claim 11, wherein, The knowledge distillation model is configured to: For each classification model, the first training data corresponding to the classification model and the second training data are input into the correlation model, to obtain a correlation between the classification model and the target model; According to the correlation between the classification model and the target model, a weight associated with a first classification result corresponding to the classification model is obtained; The first classification results are weighted and summed to obtain the second classification result.

14. The apparatus of claim 13, wherein, The knowledge distillation model is configured to: normalize the correlation corresponding to each classification model to obtain a weight associated with a first classification result corresponding to each classification model.

15. The apparatus of claim 9, wherein, In the user account classification scenario, the training data includes at least one of: behavioral characteristics of a user account, static attribute characteristics of a user, social attribute characteristics of a user, and social behavior history records of a user.

16. The apparatus of claim 9, wherein, The at least two classification models are not in a cold start state, and the target model before training is in a cold start state.

17. A computer device, comprising: The computer device includes a processor and a memory, and the memory stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by the processor to implement the classification method of any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by the processor to implement the classification method of any one of claims 1 to 8.

19. A computer program product, characterised in that, The computer program product includes computer instructions stored in a computer readable storage medium, and the processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes to implement the classification method of any one of claims 1 to 8.

Images