Domain adaptive method and device based on utility driven sample fusion and dynamic knowledge distillation, and electronic equipment
By constructing a utility-driven sample fusion and dynamic knowledge distillation method, high-quality source domain samples are selected and dynamic knowledge transfer is performed, which solves the problem of performance degradation of the model in cross-domain transfer and achieves better generalization performance and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI HUARUI BANK CO LTD
- Filing Date
- 2026-01-13
- Publication Date
- 2026-04-24
AI Technical Summary
In machine learning, when a model is transferred from a source domain to a target domain, the performance degrades due to domain shift, especially when labeled data in the target domain is scarce. Existing methods suffer from negative transfer, overfitting, and insufficient robustness.
By constructing a utility function that combines fusion domain similarity with model prediction reliability, high-quality source domain samples with similar distributions to the target domain and high prediction accuracy of the source model are selected. These samples are then fused with the target domain samples through dynamic knowledge distillation. A composite loss function is constructed for training to dynamically correct distribution shifts.
It effectively suppresses the negative transfer problem, improves the generalization performance and robustness of the target model in the target domain, enhances the model's adaptability to the characteristics of the target domain, and accelerates model convergence and training stability.
Smart Images
Figure CN121920455A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of risk control and artificial intelligence technology, and in particular to a domain adaptation method, apparatus and electronic device based on utility-driven sample fusion and dynamic knowledge distillation. Background Technology
[0002] In machine learning practice, a model that performs well when trained on source domain data often experiences a significant performance drop when deployed to a target domain with a different feature distribution. This phenomenon is called domain shift. This problem is particularly pronounced when labeled data in the target domain is extremely scarce (small sample size).
[0003] Existing solutions have the following limitations: 1. Direct transfer of source domain model: Due to domain offset, the model performs poorly in the target domain, which may result in negative transfer.
[0004] 2. Training with only a small number of samples from the target domain: This can easily lead to overfitting of the model and poor generalization ability.
[0005] 3. Traditional domain adaptation methods: (1) Most methods treat all source domain samples equally and do not distinguish their transfer value. This may introduce noisy samples with low similarity to the target domain or unreliable predictions of the source model itself. (2) In the knowledge transfer process, fixed transfer strategies are usually adopted and cannot be dynamically adjusted according to the characteristics of the samples, resulting in insufficient robustness to the source model's erroneous knowledge. (3) Sample selection, distribution alignment and model training are often optimized in stages, making it difficult to obtain the global optimal solution.
[0006] Therefore, a domain adaptation method, device, and electronic device based on utility-driven sample fusion and dynamic knowledge distillation are proposed. Summary of the Invention
[0007] This specification provides a domain adaptation method, apparatus, and electronic device based on utility-driven sample fusion and dynamic knowledge distillation. By constructing a utility function that combines the similarity of the fusion domain with the reliability of model prediction, it can accurately select high-quality source domain samples that are similar in distribution to the target domain and have high accuracy and confidence in the prediction of the source model.
[0008] This specification provides a domain adaptation method based on utility-driven sample fusion and dynamic knowledge distillation, including: Obtain the domain similarity score output by the domain discriminator and the prediction reliability score of the source domain risk scoring model for the source domain samples; A utility function is constructed based on the domain similarity score and the prediction reliability score, and the top N% of source domain samples in terms of utility value are selected based on the utility function and fused with the target domain samples to form an enhanced training set. Based on the domain similarity score, the importance weights of the top N% source domain samples in the utility value of each sample in the enhanced training set are determined, and the distribution offset between the top N% source domain samples and the target domain samples is corrected based on the importance weights of the top N% source domain samples in the utility value. A composite loss function is constructed, and the composite loss function is used to perform dynamic knowledge distillation on the corrected augmented training set to obtain the target model.
[0009] Optionally, obtaining the prediction reliability score of the source domain risk scoring model for source domain samples includes: in, To predict the reliability score, The log loss of the source domain risk model on the samples, The Shannon entropy is the prediction result of the source domain risk model. The default rate predicted by the source domain risk model for the sample. The sample contains genuine default labels.
[0010] Optionally, constructing the utility function based on the domain similarity score and the prediction reliability score includes: in, For utility function, For domain similarity scores, This is a hyperparameter.
[0011] Optionally, determining the importance weights of the top N% of source domain samples in the augmented training set based on the domain similarity score includes: in, To enhance the importance weights of the top N% of source domain samples in the training set for each utility value, The normalization constant is To enhance the overall importance weight of the target domain samples in the training set, To enhance the overall importance weight of the top N% of source domain samples in the training set, For the target domain sample size, To enhance the training set.
[0012] Optionally, the step of correcting the distribution shift between the top N% of source domain samples and the target domain samples based on the importance weights of the top N% of source domain samples in terms of utility value further includes: The importance weights are truncated, and weight values exceeding a preset quantile are set to that quantile value.
[0013] Optionally, constructing the composite loss function includes: A composite loss function is constructed based on the hard label loss of the real labels and the KL divergence loss of the soft labels in the source domain risk model. Specifically: in, For composite loss function, This is a hyperparameter for distillation intensity. For real labels, The hard label loss was measured. The default rate output by the target model currently being trained. The soft label loss is measured by KL divergence. Soft labels for the source domain risk model after temperature smoothing; This is the distillation temperature coefficient; This represents the logits value corresponding to the default rate predicted by the source domain risk model for the sample.
[0014] Optional, also includes: Using the performance metrics of the target domain validation set as the optimization objective, the hyperparameters in the utility function are jointly optimized. The target model is optimized using the sample selection ratio N%, knowledge distillation parameters, and parameters of the target model; wherein the knowledge distillation parameters include at least the distillation temperature coefficient T and the distillation intensity hyperparameter. .
[0015] This specification provides a domain adaptation device based on utility-driven sample fusion and dynamic knowledge distillation, comprising: The acquisition module is used to acquire the domain similarity score output by the domain discriminator and the prediction reliability score of the source domain risk scoring model for the source domain samples. The fusion module is used to construct a utility function based on the domain similarity score and the prediction reliability score, and to select the top N% of source domain samples in terms of utility value based on the utility function, and fuse them with the target domain samples to form an enhanced training set. The correction module is used to determine the importance weight of each source domain sample in the top N% of the utility value in the enhanced training set based on the domain similarity score, and to correct the distribution offset between the source domain samples in the top N% of the utility value and the target domain samples based on the importance weight of the source domain samples in the top N% of the utility value. An adaptive module is used to construct a composite loss function and use the composite loss function to perform dynamic knowledge distillation on the corrected augmented training set to obtain the target model.
[0016] Optionally, the acquisition module includes: in, To predict the reliability score, The log loss of the source domain risk model on the samples, The Shannon entropy is the prediction result of the source domain risk model. The default rate predicted by the source domain risk model for the sample. The sample contains genuine default labels.
[0017] Optionally, the fusion module includes: in, For utility function, For domain similarity scores, This is a hyperparameter.
[0018] Optionally, the correction module includes: in, To enhance the importance weights of the top N% of source domain samples in the training set for each utility value, The normalization constant is To enhance the overall importance weight of the target domain samples in the training set, To enhance the overall importance weight of the top N% of source domain samples in the training set, For the target domain sample size, To enhance the training set.
[0019] Optionally, the correction module further includes: The importance weights are truncated, and weight values exceeding a preset quantile are set to that quantile value.
[0020] Optionally, the adaptive module includes: A composite loss function is constructed based on the hard label loss of the real labels and the KL divergence loss of the soft labels in the source domain risk model. Specifically: in, For composite loss function, This is a hyperparameter for distillation intensity. For real labels, The hard label loss was measured. The default rate output by the target model currently being trained. The soft label loss is measured by KL divergence. Soft labels for the source domain risk model after temperature smoothing; This is the distillation temperature coefficient; This represents the logits value corresponding to the default rate predicted by the source domain risk model for the sample.
[0021] Optional, also includes: Using the performance metrics of the target domain validation set as the optimization objective, the hyperparameters in the utility function are jointly optimized. The target model is optimized using the sample selection ratio N%, knowledge distillation parameters, and parameters of the target model; wherein the knowledge distillation parameters include at least the distillation temperature coefficient T and the distillation intensity hyperparameter. .
[0022] This specification also provides an electronic device, wherein the electronic device includes: A processor; and a memory storing computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.
[0023] This specification also provides a computer-readable storage medium that stores one or more programs that, when executed by a processor, implement any of the methods described above.
[0024] In this invention, by constructing a utility function that integrates domain similarity and model prediction reliability, high-quality source domain samples with similar distributions to the target domain and high prediction accuracy and confidence of the source model can be accurately selected. This effectively suppresses the "negative transfer" problem caused by directly mixing all source domain data in traditional methods, enabling the target model to achieve better and more robust generalization performance in the target domain. Importance weights are assigned to the selected source domain samples based on domain similarity, and these weights are truncated to dynamically correct the conditional distribution differences between the source and target domains at the sample level. This mechanism mitigates the negative impact of distribution shift on model training and enhances the model's adaptability to the characteristics of the target domain. A dynamic knowledge distillation framework is employed, using temperature-smoothed soft labels and adjustable distillation intensity to safely and controllably transfer complex knowledge (such as inter-category relationships) from the source domain prediction model to the target model. Joint training is performed using a weighted, augmented training set, which utilizes valuable knowledge from the source domain and strengthens the model's learning of true labels, accelerating model convergence and improving training stability. Guided by the performance of the target domain validation set, the hyperparameters of key processes such as sample selection, weight correction, and knowledge distillation are jointly optimized, enabling the entire adaptive process to automatically adapt to the data characteristics of different cross-domain scenarios, thus enhancing the practicality and robustness of the method. Attached Figure Description
[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0026] Figure 1 A schematic diagram illustrating the principle of a domain adaptation method based on utility-driven sample fusion and dynamic knowledge distillation, provided for embodiments of this specification; Figure 2 A schematic diagram of the structure of a domain adaptive device based on utility-driven sample fusion and dynamic knowledge distillation, provided for an embodiment of this specification; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification; Figure 4 This is a schematic diagram of a computer-readable medium provided for embodiments of this specification. Detailed Implementation
[0027] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.
[0028] The following is in conjunction with the appendix Figure 1-4 Exemplary embodiments of the invention will be described more fully here. However, exemplary embodiments can be implemented in many forms and should not be construed as limiting the invention to the embodiments set forth herein. Rather, these exemplary embodiments are provided to make the invention more comprehensive and complete, and to facilitate a full communication of the inventive concept to those skilled in the art. The same reference numerals in the figures denote the same or similar elements, components, or parts, and therefore repeated descriptions of them are omitted.
[0029] Subject to the technical concept of this invention, the features, structures, characteristics or other details described in a particular embodiment may be combined in one or more other embodiments in a suitable manner.
[0030] In the description of specific embodiments, the features, structures, characteristics, or other details described in this invention are intended to enable those skilled in the art to fully understand the embodiments. However, it is not excluded that those skilled in the art can practice the technical solutions of this invention without one or more of the specific features, structures, characteristics, or other details.
[0031] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.
[0032] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0033] The terms “and / or” or “and / or” include all combinations of any one or more of the listed items.
[0034] Figure 1 This specification provides a schematic diagram illustrating the principle of a domain adaptation method based on utility-driven sample fusion and dynamic knowledge distillation, which may include: S110: Obtain the domain similarity score output by the domain discriminator and the prediction reliability score of the source domain risk scoring model for the source domain samples; In the specific implementation of this specification, the prediction model trained on historical business data of the source domain (such as existing sample data) takes sample features (such as basic attributes, historical behavior records, etc.) as input and outputs the target prediction probability of the sample, which is used to provide the knowledge base of the source domain.
[0035] Domain Discriminator: A binary classification model used to distinguish the source of samples. The input is the same as the source model (sample features), and the output is the probability that the sample belongs to the target domain (quantifying the distribution similarity between the source samples and the target domain). During training, the source domain sample label is set to 0, and the target domain sample label is set to 1. It is only used as an auxiliary tool (and does not participate in the final risk control decision).
[0036] Optionally, obtaining the prediction reliability score of the source domain risk scoring model for source domain samples includes: in, To predict the reliability score, The log loss of the source domain risk model on the samples, The Shannon entropy is the prediction result of the source domain risk model. The default rate predicted by the source domain risk model for the sample. The sample contains genuine default labels.
[0037] In the specific implementation of this specification The smaller the value, the more accurate the prediction. The value is mapped to the [0,1] interval through exponential transformation. The smaller the value, the higher the prediction confidence.
[0038] S120: Construct a utility function based on the domain similarity score and the prediction reliability score, and select the top N% of source domain samples in terms of utility value based on the utility function, and fuse them with the target domain samples to form an enhanced training set; In the specific implementation of this specification, the top N% (e.g., 30%) of source domain samples (i.e., high transfer value samples) in terms of utility value are selected and fused with all target domain samples to form an enhanced training set (also known as an enhanced source domain), providing a high-quality data foundation for subsequent distribution correction.
[0039] Optionally, constructing the utility function based on the domain similarity score and the prediction reliability score includes: in, For utility function, For domain similarity scores, This is a hyperparameter.
[0040] In the specific implementation of this specification Used for domain similarity scoring With predicted reliability score Weights, for example When the value is 0.6, it places more emphasis on domain similarity matching. It combines the accuracy and confidence of the source model's predictions; only when the source model's predictions are accurate and confident... Take the higher value.
[0041] Optionally, determining the importance weights of the top N% of source domain samples in the augmented training set based on the domain similarity score includes: in, To enhance the importance weights of the top N% of source domain samples in the training set for each utility value, The normalization constant is To enhance the overall importance weight of the target domain samples in the training set, To enhance the overall importance weight of the top N% of source domain samples in the training set, For the target domain sample size, To enhance the training set.
[0042] In the specific implementation of this specification The weights of high-transfer-value samples are scaled to ensure that the training set is enhanced after weighting. Total weight of samples in the middle source domain Total weight of samples in the target domain Equal contributions ensure that both types of samples have a balanced contribution to the loss function. Do not multiply by a coefficient Its value may be far greater than the former, causing the model to overfit the feature distribution of high-value samples in the source domain while ignoring the real risk characteristics of samples in the target domain. Ultimately, this may result in the model performing well on relevant data in the source domain but failing to adapt to the actual risk control needs of the target domain, thus violating the core intention of adaptive optimization of the target domain performance.
[0043] Optionally, the step of correcting the distribution shift between the top N% of source domain samples and the target domain samples based on the importance weights of the top N% of source domain samples in terms of utility value further includes: The importance weights are truncated, and weight values exceeding a preset quantile are set to that quantile value.
[0044] In the specific implementation of this specification, the calculated importance weights are truncated to the 90th percentile; that is, weights exceeding the 90th percentile are uniformly set to the value of that quantile to avoid extreme weights (such as those for individual samples). Approaching 1 leads to Excessive variance can interfere with model training; therefore, the weight variance should be controlled.
[0045] S130: Based on the domain similarity score, determine the importance weight of each source domain sample in the top N% of the utility value in the enhanced training set, and correct the distribution offset between the source domain samples in the top N% of the utility value and the target domain samples based on the importance weight of the source domain samples in the top N% of the utility value. In the specific implementation of this specification, the enhanced training set is composed of high transfer value samples and target domain samples. Since the feature distributions of these two parts may still differ, it is necessary to use importance weights to make the weighted source domain sample distribution approximate the target domain sample distribution, so as to ensure that the two types of samples contribute equally to the loss of subsequent model training (avoiding that too many source domain samples mask the target domain signal).
[0046] S140: Construct a composite loss function and use the composite loss function to perform dynamic knowledge distillation on the corrected augmented training set to obtain the target model.
[0047] Optionally, constructing the composite loss function includes: A composite loss function is constructed based on the hard label loss of the real labels and the KL divergence loss of the soft labels in the source domain risk model. Specifically: in, For composite loss function, This is a hyperparameter for distillation intensity. For real labels, The hard label loss was measured. The default rate output by the target model currently being trained. The soft label loss is measured by KL divergence. Soft labels for the source domain risk model after temperature smoothing; This is the distillation temperature coefficient; This represents the logits value corresponding to the default rate predicted by the source domain risk model for the sample.
[0048] In the specific implementation of this specification Let logits be the default rate predicted by the source domain risk model for the sample. Established, among which This is the sigmoid function. The larger the temperature coefficient T, the flatter the soft-label probability distribution, and the better it reflects the relative confidence of the source model in the samples.
[0049] Optional, also includes: Using the performance metrics of the target domain validation set as the optimization objective, the hyperparameters in the utility function are jointly optimized. The target model is optimized using the sample selection ratio N%, knowledge distillation parameters, and parameters of the target model; wherein the knowledge distillation parameters include at least the distillation temperature coefficient T and the distillation intensity hyperparameter. .
[0050] In the specific implementation of this specification, the AUC or KS on the target domain validation set is used as the optimization objective, and the Bayesian optimization method is employed to optimize the hyperparameters in the utility function. Sample screening ratio N%, distillation temperature coefficient T, and distillation intensity hyperparameters An end-to-end joint search is performed on the hyperparameters of the target model to avoid suboptimal solutions caused by staged optimization. Target model hyperparameters, if using LightGBM (suitable for financial industry deployment), include tree depth (max_depth), learning rate (learning_rate), and number of iterations (n_estimators). The parameters are iteratively searched on independent time-series validation sets, and the parameter combination with the highest AUC on the target domain validation set is selected to obtain the optimal target domain risk scoring model. .
[0051] In this invention, by constructing a utility function that integrates domain similarity and model prediction reliability, high-quality source domain samples with similar distributions to the target domain and high prediction accuracy and confidence of the source model can be accurately selected. This effectively suppresses the "negative transfer" problem caused by directly mixing all source domain data in traditional methods, enabling the target model to achieve better and more robust generalization performance in the target domain. Importance weights are assigned to the selected source domain samples based on domain similarity, and these weights are truncated to dynamically correct the conditional distribution differences between the source and target domains at the sample level. This mechanism mitigates the negative impact of distribution shift on model training and enhances the model's adaptability to the characteristics of the target domain. A dynamic knowledge distillation framework is employed, using temperature-smoothed soft labels and adjustable distillation intensity to safely and controllably transfer complex knowledge (such as inter-category relationships) from the source domain prediction model to the target model. Joint training is performed using a weighted, augmented training set, which utilizes valuable knowledge from the source domain and strengthens the model's learning of true labels, accelerating model convergence and improving training stability. Guided by the performance of the target domain validation set, the hyperparameters of key processes such as sample selection, weight correction, and knowledge distillation are jointly optimized, enabling the entire adaptive process to automatically adapt to the data characteristics of different cross-domain scenarios, thus enhancing the practicality and robustness of the method.
[0052] Figure 2 This specification provides a schematic diagram of a domain adaptation device based on utility-driven sample fusion and dynamic knowledge distillation, which may include: The acquisition module 10 is used to acquire the domain similarity score output by the domain discriminator and the prediction reliability score of the source domain risk scoring model for the source domain samples. The fusion module 20 is used to construct a utility function based on the domain similarity score and the prediction reliability score, and to select the top N% of source domain samples in terms of utility value based on the utility function, and fuse them with the target domain samples to form an enhanced training set. The correction module 30 is used to determine the importance weight of each source domain sample in the top N% of the utility value in the enhanced training set based on the domain similarity score, and to correct the distribution offset between the source domain samples in the top N% of the utility value and the target domain samples based on the importance weight of the source domain samples in the top N% of the utility value. The adaptive module 40 is used to construct a composite loss function and use the composite loss function to perform dynamic knowledge distillation on the corrected augmented training set to obtain the target model.
[0053] Optionally, the acquisition module 10 includes: in, To predict the reliability score, The log loss of the source domain risk model on the samples, The Shannon entropy is the prediction result of the source domain risk model. The default rate predicted by the source domain risk model for the sample. The sample contains genuine default labels.
[0054] Optionally, the fusion module 20 includes: in, For utility function, For domain similarity scores, This is a hyperparameter.
[0055] Optionally, the correction module 30 includes: in, To enhance the importance weights of the top N% of source domain samples in the training set for each utility value, The normalization constant is To enhance the overall importance weight of the target domain samples in the training set, To enhance the overall importance weight of the top N% of source domain samples in the training set, For the target domain sample size, To enhance the training set.
[0056] Optionally, the correction module 30 further includes: The importance weights are truncated, and weight values exceeding a preset quantile are set to that quantile value.
[0057] Optionally, the adaptive module 40 includes: A composite loss function is constructed based on the hard label loss of the real labels and the KL divergence loss of the soft labels in the source domain risk model. Specifically: in, For composite loss function, This is a hyperparameter for distillation intensity. For real labels, The hard label loss was measured. The default rate output by the target model currently being trained. The soft label loss is measured by KL divergence. Soft labels for the source domain risk model after temperature smoothing; This is the distillation temperature coefficient; This represents the logits value corresponding to the default rate predicted by the source domain risk model for the sample.
[0058] Optional, also includes: Using the performance metrics of the target domain validation set as the optimization objective, the hyperparameters in the utility function are jointly optimized. The target model is optimized using the sample selection ratio N%, knowledge distillation parameters, and parameters of the target model; wherein the knowledge distillation parameters include at least the distillation temperature coefficient T and the distillation intensity hyperparameter. .
[0059] The functions of the apparatus in this embodiment have been described in the above method embodiments. Therefore, for any parts not detailed in this embodiment, please refer to the relevant descriptions in the foregoing embodiments, which will not be repeated here.
[0060] Based on the same inventive concept, embodiments of this specification also provide an electronic device.
[0061] The following describes embodiments of the electronic device of the present invention, which can be considered as specific implementations of the methods and apparatus embodiments of the present invention described above. Details described in the embodiments of the electronic device of the present invention should be considered as supplements to the methods or apparatus embodiments described above; details not disclosed in the embodiments of the electronic device of the present invention can be implemented with reference to the methods or apparatus embodiments described above.
[0062] Figure 3 This is a schematic diagram of an electronic device provided as an embodiment of this specification. Refer to the following... Figure 3 The electronic device 300 according to this embodiment of the present invention will be described. Figure 3 The electronic device 300 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.
[0063] like Figure 3 As shown, the electronic device 300 is presented in the form of a general-purpose computing device. The components of the electronic device 300 may include, but are not limited to: at least one processing unit 310, at least one storage unit 320, a bus 330 connecting different system components (including storage unit 320 and processing unit 310), a display unit 340, etc.
[0064] The storage unit stores program code that can be executed by the processing unit 310, causing the processing unit 310 to perform the steps described in the processing method section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 310 can perform, for example... Figure 1 The steps are shown.
[0065] The storage unit 320 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 3201 and / or a cache storage unit 3202, and may further include a read-only memory unit (ROM) 3203.
[0066] The storage unit 320 may also include a program / utility 3204 having a set (at least one) program module 3205, such program module 3205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.
[0067] Bus 330 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.
[0068] Electronic device 300 can also communicate with one or more external devices 400 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable viewers to interact with electronic device 300, and / or with any device that enables electronic device 300 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 350. Furthermore, electronic device 300 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 360. Network adapter 360 can communicate with other modules of electronic device 300 via bus 330. It should be understood that, although... Figure 3 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0069] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described in this invention can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the method described above according to this invention. When the computer program is executed by a data processing device, it enables the computer-readable medium to implement the method described above, i.e.: as... Figure 1 The method shown.
[0070] Figure 4 This is a schematic diagram of a computer-readable medium provided for embodiments of this specification.
[0071] accomplish Figure 1The computer program of the method shown can be stored on one or more computer-readable media. A computer-readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.
[0072] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.
[0073] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java and C++, and conventional procedural programming languages such as C or similar languages. The program code can execute entirely on the audience's computing device, partially on the audience's device, as a standalone software package, partially on the audience's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the audience's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).
[0074] In summary, the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that in practice, general-purpose data processing devices such as microprocessors or digital signal processors (DSPs) can be used to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
[0075] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the present invention is not inherently related to any specific computer, virtual device, or electronic device, and various general-purpose devices can also implement the present invention. The above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
[0076] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.
[0077] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A domain adaptation method based on utility-driven sample fusion and dynamic knowledge distillation, characterized in that, include: Obtain the domain similarity score output by the domain discriminator and the prediction reliability score of the source domain risk scoring model for the source domain samples; A utility function is constructed based on the domain similarity score and the prediction reliability score, and the top N% of source domain samples in terms of utility value are selected based on the utility function and fused with the target domain samples to form an enhanced training set. Based on the domain similarity score, the importance weights of the top N% source domain samples in the utility value of each sample in the enhanced training set are determined, and the distribution offset between the top N% source domain samples and the target domain samples is corrected based on the importance weights of the top N% source domain samples in the utility value. A composite loss function is constructed, and the composite loss function is used to perform dynamic knowledge distillation on the corrected augmented training set to obtain the target model.
2. The domain adaptation method based on utility-driven sample fusion and dynamic knowledge distillation as described in claim 1, characterized in that, The method of obtaining the prediction reliability score of the source domain risk scoring model for source domain samples includes: in, To predict the reliability score, The log loss of the source domain risk model on the samples, The Shannon entropy is the prediction result of the source domain risk model. The default rate predicted by the source domain risk model for the sample. The sample contains genuine default labels.
3. The domain adaptation method based on utility-driven sample fusion and dynamic knowledge distillation as described in claim 2, characterized in that, The construction of the utility function based on the domain similarity score and the prediction reliability score includes: in, For utility function, For domain similarity scores, This is a hyperparameter.
4. The domain adaptation method based on utility-driven sample fusion and dynamic knowledge distillation as described in claim 3, characterized in that, The step of determining the importance weights of the top N% of source domain samples in the enhanced training set based on the domain similarity score includes: in, To enhance the importance weights of the top N% of source domain samples in the training set for each utility value, The normalization constant is To enhance the overall importance weight of the target domain samples in the training set, To enhance the overall importance weight of the top N% of source domain samples in the training set, For the target domain sample size, To enhance the training set.
5. The domain adaptation method based on utility-driven sample fusion and dynamic knowledge distillation as described in claim 3, characterized in that, The method of correcting the distribution shift between the top N% of source domain samples and the target domain samples based on the importance weights of the top N% of source domain samples in terms of utility value further includes: The importance weights are truncated, and weight values exceeding a preset quantile are set to that quantile value.
6. The domain adaptation method based on utility-driven sample fusion and dynamic knowledge distillation as described in claim 4, characterized in that, The construction of the composite loss function includes: A composite loss function is constructed based on the hard label loss of the real labels and the KL divergence loss of the soft labels in the source domain risk model. Specifically: in, For composite loss function, This is a hyperparameter for distillation intensity. For real labels, The hard label loss was measured. The default rate output by the target model currently being trained. The soft label loss is measured by KL divergence. Soft labels for the source domain risk model after temperature smoothing; This is the distillation temperature coefficient; This represents the logits value corresponding to the default rate predicted by the source domain risk model for the sample.
7. The domain adaptation method based on utility-driven sample fusion and dynamic knowledge distillation as described in claim 1, characterized in that, Also includes: Using the performance metrics of the target domain validation set as the optimization objective, the hyperparameters in the utility function are jointly optimized. The target model is optimized using the sample selection ratio N%, knowledge distillation parameters, and parameters of the target model; wherein the knowledge distillation parameters include at least the distillation temperature coefficient T and the distillation intensity hyperparameter. .
8. A domain adaptive device based on utility-driven sample fusion and dynamic knowledge distillation, characterized in that, include: The acquisition module is used to acquire the domain similarity score output by the domain discriminator and the prediction reliability score of the source domain risk scoring model for the source domain samples. The fusion module is used to construct a utility function based on the domain similarity score and the prediction reliability score, and to select the top N% of source domain samples in terms of utility value based on the utility function, and fuse them with the target domain samples to form an enhanced training set. The correction module is used to determine the importance weight of each source domain sample in the top N% of the utility value in the enhanced training set based on the domain similarity score, and to correct the distribution offset between the source domain samples in the top N% of the utility value and the target domain samples based on the importance weight of the source domain samples in the top N% of the utility value. An adaptive module is used to construct a composite loss function and use the composite loss function to perform dynamic knowledge distillation on the corrected augmented training set to obtain the target model.
9. An electronic device, wherein, The electronic device includes: A processor; and a memory storing computer-executable instructions, which, when executed, cause the processor to perform the method according to any one of claims 1-7.
10. A computer-readable storage medium, wherein, The computer-readable storage medium stores one or more programs that, when executed by a processor, implement the method of any one of claims 1-7.