Training method and device of pseudo label model, storage medium and electronic equipment
By using a pseudo-label model training method, pseudo-labels are added to unlabeled data using difference analysis and a preset loss function. This solves the problem that existing models cannot utilize unlabeled data, and achieves effective model training and performance improvement.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA TELECOM CORP LTD
- Filing Date
- 2022-12-09
- Publication Date
- 2026-06-09
AI Technical Summary
Existing machine learning-based fraud user identification models cannot effectively utilize unlabeled data in real-world scenarios, leading to model overfitting and an inability to leverage important semantic information from massive amounts of unlabeled data.
The pseudo-label model training method uses the first and second classification models to generate differential analysis, and uses a preset loss function model to add pseudo-labels to unlabeled data and noisy labeled data to generate pseudo-label data. The model is then trained using clean labeled data and pseudo-label data.
This approach enables the effective use of unlabeled data for model training, improves the model's recognition performance, and solves the problem of not being able to use unlabeled data for model training.
Smart Images
Figure CN115859107B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the Internet field, and more specifically, to a training method, apparatus, storage medium, and electronic device for a pseudo-label model. Background Technology
[0002] With the development of mobile internet, mobile phones have become an indispensable part of people's daily lives. However, telephone fraud has emerged frequently in recent years. Fraudulent activities harm users' interests, damage operators' reputation, and cause customer churn. Therefore, detecting and dealing with fraudulent users can effectively prevent losses. With the widespread application of machine learning, some machine learning-based fraud user identification models have emerged. Their basic method is to construct a network of call relationships between users and then use machine learning to identify fraudulent users. However, most existing machine learning-based fraud classification models are based on supervised learning methods, requiring the acquisition of a large amount of labeled data, i.e., data explicitly marked as fraudulent users and normal users, and then performing supervised learning in an ideal environment. In real-world scenarios, the number of explicitly marked fraudulent users is limited; the vast majority of data obtained is unlabeled, which restricts the application of supervised learning models.
[0003] In the design of fraud detection systems, the real-time sample data acquired is enormous and rich in semantic information, but most of it is unlabeled. Currently, researchers mainly use supervised learning-based methods to identify fraudulent users. This method ignores unlabeled data during model training, which leads to two problems: 1) it easily causes model overfitting, and 2) it fails to utilize the important semantic information contained in the massive amount of unlabeled data.
[0004] There is currently no effective solution to the problem of not being able to train models using unlabeled data. Summary of the Invention
[0005] This invention provides a training method, apparatus, storage medium, and electronic device for a pseudo-labeled model, to at least solve the technical problem of being unable to train a model using unlabeled data.
[0006] According to one aspect of the present invention, a method for training a pseudo-label model is provided, comprising: acquiring a plurality of sample communication relationship data, wherein each sample communication relationship data is used to represent a communication relationship between a group of communication objects that have been labeled with sample tags; analyzing the sample communication relationship data using a first classification model to generate a first classification label, wherein the first classification model is trained using clean label data through machine learning, and the clean label data is preset communication relationship data with preset labels; and analyzing the sample communication relationship data using a second classification model to generate a second classification label, wherein the second classification model is trained using mixed data through machine learning, and the mixed data includes: the clean label data, noisy label data, and unlabeled data. The system identifies several data sets: 1) Noise-labeled data, which is preset relationship data with added noise labels (where noise labels are interference from the preset labels); 2) Unlabeled data, which is preset communication relationship data without added labels; 3) The system determines the difference between the sample label and the second classification label as a first difference, and the difference between the first classification label and the second classification label as a second difference; 4) A preset loss function model is used to analyze the first difference and the second difference to determine a pseudo-label model, wherein the preset loss function model is used to assign preset trust weights to the first difference and the second difference, and the pseudo-label model is defined based on the first difference, the second difference, and the preset trust weights. The pseudo-label model is used to add the preset labels to the unlabeled data and the noise-labeled data.
[0007] Optionally, obtaining multiple sample communication relationship data includes: obtaining a preset communication relationship network, wherein the preset communication relationship network records multiple preset communication relationship data, each of the preset communication relationship data representing a communication relationship between a group of communication objects; in the preset communication relationship network, determining the preset communication relationship data with the preset label as the clean label data; and selecting the sample communication relationship data from the clean label data.
[0008] Optionally, after selecting the sample communication relationship data from the clean label data, the method further includes: determining the clean label data other than the sample communication relationship data as the clean label data for training the first classification model.
[0009] Optionally, determining the difference between the sample label and the second classification label as a first difference, and determining the difference between the first classification label and the second classification label as a second difference, includes: using a preset cross-entropy loss function to determine the loss of the sample label relative to the second classification label as a first difference; and using the preset cross-entropy loss function to determine the loss of the first classification label relative to the second classification label as a second difference.
[0010] Optionally, the preset trust weights include: a first trust weight and a second trust weight. Analyzing the first difference and the second difference using a preset loss function model to determine the pseudo-label model includes: analyzing the difference between the sample label and the second classification label to determine the degree of trust in the first classification model; assigning the first trust weight to the first difference based on the degree of trust; assigning the second trust weight to the second difference based on the degree of trust; and determining the pseudo-label model based on the product of the first difference and the first trust weight, and the product of the second difference and the second trust weight.
[0011] Optionally, analyzing the difference between the sample label and the second classification label to determine the degree of trust in the first classification model includes: analyzing the difference between the sample label and the second classification label to determine the label matching probability of the first classification label matching the sample label; and determining the degree of trust in the first classification model based on the label matching probability.
[0012] Optionally, after analyzing the first difference and the second difference using a preset loss function model to determine the pseudo-label model, the method further includes: using the pseudo-label model to add pseudo-labels to the unlabeled data and the noisy labeled data to generate pseudo-label data; using the pseudo-label data and the clean labeled data as training data to train a preset classification model, wherein the preset classification model is used to analyze the target communication relationship data and determine the target label of the target communication relationship data.
[0013] According to another aspect of the present invention, a training apparatus for a pseudo-label model is also provided, comprising: an acquisition module, configured to acquire multiple sample communication relationship data, wherein each sample communication relationship data represents a communication relationship between a group of communication objects for which sample labels have been added; a first analysis module, configured to analyze the sample communication relationship data using a first classification model to generate a first classification label, wherein the first classification model is trained using clean label data through machine learning, and the clean label data is preset communication relationship data for which preset labels have been added; and a second analysis module, configured to analyze the sample communication relationship data using a second classification model to generate a second classification label, wherein the second classification model is trained using mixed data through machine learning, and the mixed data includes: the clean label data and noise labels. The system comprises: data and unlabeled data, wherein the noise-labeled data is preset relationship data with added noise labels, the noise labels being interference from the preset labels, and the unlabeled data is preset communication relationship data without added labels; a determination module, used to determine the difference between the sample label and the second classification label as a first difference, and the difference between the first classification label and the second classification label as a second difference; and a third analysis module, used to analyze the first difference and the second difference using a preset loss function model to determine a pseudo-label model, wherein the preset loss function model is used to assign preset trust weights to the first difference and the second difference, and to define the pseudo-label model based on the first difference, the second difference, and the preset trust weights, and the pseudo-label model is used to add the preset labels to the unlabeled data and the noise-labeled data.
[0014] According to another aspect of the present invention, a non-volatile storage medium is also provided, characterized in that the non-volatile storage medium stores a program, wherein the program controls the device where the non-volatile storage medium is located to execute the above-described training method for the pseudo-label model when the program is running.
[0015] According to another aspect of the present invention, an electronic device is also provided, characterized in that it includes: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when running, executes the training method of the pseudo-label model described above.
[0016] In this embodiment of the invention, multiple sample communication relationship data are acquired, wherein each sample communication relationship data represents the communication relationship between a group of communication objects that have been labeled with sample tags; the sample communication relationship data are analyzed using a first classification model to generate a first classification label, wherein the first classification model is trained using clean-label data through machine learning, and the clean-label data is preset communication relationship data with preset labels; the sample communication relationship data are analyzed using a second classification model to generate a second classification label, wherein the second classification model is trained using mixed data through machine learning, and the mixed data includes: clean-label data, noisy-label data, and unlabeled data, the noisy-label data is preset relationship data with added noise labels, the noise labels are interference of the preset labels, and the unlabeled data is preset communication data without added labels. The data is related to the following: The difference between the sample label and the second classification label is defined as the first difference; the difference between the first classification label and the second classification label is defined as the second difference; a pre-defined loss function model is used to analyze the first and second differences to determine a pseudo-label model. The pre-defined loss function model is used to assign pre-defined trust weights to the first and second differences. Based on the first and second differences and the pre-defined trust weights, a pseudo-label model is defined. This pseudo-label model is used to add pre-defined labels to unlabeled data and noisy labeled data. By adding pseudo-labels to unlabeled data and noisy labeled data using the pseudo-label model, pseudo-labeled data is generated. Then, clean labeled data and pseudo-labeled data are used together for model training, thus achieving the technical effect of training a model using unlabeled data and solving the technical problem of not being able to train a model using unlabeled data. Attached Figure Description
[0017] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:
[0018] Figure 1 This is a flowchart of a training method for a pseudo-label model according to an embodiment of the present invention;
[0019] Figure 2 This is a schematic diagram of training a preset classification model based on pseudo-label technology according to an embodiment of the present invention;
[0020] Figure 3 This is a schematic diagram of constructing a spatiotemporal heterogeneous communication network according to an embodiment of the present invention;
[0021] Figure 4 This is a schematic diagram illustrating the generation of pseudo-labels using conditional trust training according to an embodiment of the present invention;
[0022] Figure 5This is a schematic diagram of a training device for a pseudo-label model according to an embodiment of the present invention;
[0023] Figure 6 This is a structural block diagram of a computer terminal according to an embodiment of the present invention. Detailed Implementation
[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0026] According to an embodiment of the present invention, a training method for a pseudo-label model is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0027] Figure 1 This is a flowchart of a training method for a pseudo-label model according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:
[0028] Step S102: Obtain multiple sample communication relationship data, wherein each sample communication relationship data is used to represent the communication relationship between a group of communication objects that have been labeled with sample tags;
[0029] Step S104: Analyze the sample communication relationship data using the first classification model to generate the first classification label. The first classification model is trained by machine learning using clean label data, and the clean label data is the preset communication relationship data with preset labels added.
[0030] Step S106: Analyze the sample communication relationship data using the second classification model to generate a second classification label. The second classification model is trained by machine learning using mixed data. The mixed data includes: clean labeled data, noisy labeled data, and unlabeled data. The noisy labeled data is the preset relationship data with added noise labels. The noise labels are the interference of the preset labels. The unlabeled data is the preset communication relationship data without added labels.
[0031] Step S108: Determine the difference between the sample label and the second classification label as the first difference, and determine the difference between the first classification label and the second classification label as the second difference;
[0032] Step S110: Analyze the first difference and the second difference using a preset loss function model to determine the pseudo-label model. The preset loss function model is used to assign preset trust weights to the first difference and the second difference, and the pseudo-label model is defined based on the first difference, the second difference, and the preset trust weights. The pseudo-label model is used to add preset labels to unlabeled data and noisy labeled data.
[0033] In this embodiment of the invention, multiple sample communication relationship data are acquired, wherein each sample communication relationship data represents the communication relationship between a group of communication objects that have been labeled with sample tags; the sample communication relationship data are analyzed using a first classification model to generate a first classification label, wherein the first classification model is trained using clean-label data through machine learning, and the clean-label data is preset communication relationship data with preset labels; the sample communication relationship data are analyzed using a second classification model to generate a second classification label, wherein the second classification model is trained using mixed data through machine learning, and the mixed data includes: clean-label data, noisy-label data, and unlabeled data, the noisy-label data is preset relationship data with added noise labels, the noise labels are interference of the preset labels, and the unlabeled data is preset communication data without added labels. The data is related to the following: The difference between the sample label and the second classification label is defined as the first difference; the difference between the first classification label and the second classification label is defined as the second difference; a pre-defined loss function model is used to analyze the first and second differences to determine a pseudo-label model. The pre-defined loss function model is used to assign pre-defined trust weights to the first and second differences. Based on the first and second differences and the pre-defined trust weights, a pseudo-label model is defined. This pseudo-label model is used to add pre-defined labels to unlabeled data and noisy labeled data. By adding pseudo-labels to unlabeled data and noisy labeled data using the pseudo-label model, pseudo-labeled data is generated. Then, clean labeled data and pseudo-labeled data are used together for model training, thus achieving the technical effect of training a model using unlabeled data and solving the technical problem of not being able to train a model using unlabeled data.
[0034] In step S102 above, the sample tag is used to indicate the nature of the sample communication data. For example, the sample tag is used to indicate whether the sample communication data is fraudulent communication data or non-fraudulent communication data.
[0035] In step S102 above, the communication object is a user or a device, and the sample communication relationship data is used to represent the communication relationship or call relationship between users as samples, and the communication relationship or call relationship between users and devices as samples.
[0036] As an optional embodiment, obtaining multiple sample communication relationship data includes: obtaining a preset communication relationship network, wherein the preset communication relationship network records multiple preset communication relationship data, each preset communication relationship data being used to represent the communication relationship between a group of communication objects; in the preset communication relationship network, determining the preset communication relationship data with preset labels as clean label data; and selecting sample communication relationship data from the clean label data.
[0037] Optionally, the preset communication relationship network records preset communication relationship data, which includes preset communication relationship data with added tags and preset communication relationship data without added tags, and the preset communication relationship data without added tags is determined to be untagged data.
[0038] Optionally, the preset communication objects of the two communicating parties in the preset communication relationship data that has been tagged are identified, and it is determined whether the preset communication objects exist in the preset whitelist or preset blacklist. If the preset communication objects of the two communicating parties exist in the preset whitelist or preset blacklist, the preset communication relationship data is determined to be clean tagged data; if the preset communication objects of the two communicating parties exist in the preset whitelist or preset blacklist, the preset communication relationship data is determined to be noisy tagged data.
[0039] The above embodiments of the present invention can select sample communication relationship data from clean label data.
[0040] As an optional embodiment, after selecting sample communication relationship data from the clean label data, the method further includes: determining clean label data other than the sample communication relationship data as clean label data for training the first classification model.
[0041] In the above embodiments of the present invention, a portion of clean label data can be selected from the clean label data as sample communication relationship data; the remaining clean label data in the clean label data can be used as clean label data for training the first classification model, and then the first classification model can be trained based on the portion of clean label data. The first classification model can be tested based on the sample communication relationship data, and then a pseudo-label model can be trained.
[0042] As an optional embodiment, determining the difference between the sample label and the second classification label as a first difference, and determining the difference between the first classification label and the second classification label as a second difference, includes: using a preset cross-entropy loss function to determine the loss of the sample label relative to the second classification label as the first difference; and using a preset cross-entropy loss function to determine the loss of the first classification label relative to the second classification label as the second difference.
[0043] In the above embodiments of the present invention, a preset cross-entropy loss function is used to determine the sample label y. i Relative to the second category label f(x) i The loss l(y) i ,f(x i The first difference is used to determine the first classification label s using a preset cross-entropy loss function. i Relative to the second category label f(x) i The loss l(s) i ,f(x i )) is the second difference.
[0044] As an optional embodiment, the preset trust weights include: a first trust weight and a second trust weight. The first and second differences are analyzed using a preset loss function model. Determining the pseudo-label model includes: analyzing the difference between the sample label and the second classification label to determine the degree of trust in the first classification model; assigning a first trust weight to the first difference based on the degree of trust; assigning a second trust weight to the second difference based on the degree of trust; and determining the pseudo-label model based on the product of the first difference and the first trust weight, and the product of the second difference and the second trust weight.
[0045] In the above embodiments of the present invention, the trust level of the first classification model is determined based on the difference between the sample label and the second classification label. Then, a first trust weight is assigned to the first difference and a second trust weight is assigned to the second difference based on the trust level. The product of the first difference and the first trust weight and the product of the second difference and the second trust weight are determined as a preset loss function model. Then, a pseudo-label model can be defined based on the preset loss function model.
[0046] As an optional embodiment, analyzing the difference between the sample label and the second classification label to determine the degree of trust in the first classification model includes: analyzing the difference between the sample label and the second classification label to determine the label matching probability of the first classification label matching the sample label; and determining the degree of trust in the first classification model based on the label matching probability.
[0047] In the above embodiments of the present invention, the label matching probability is used to represent the probability that the first classification label obtained by the second classification model from analyzing the target communication relationship data is the same as the target label pre-added to the target communication relationship. Based on this label matching probability, the degree of trust in the first classification model can be determined, that is, S can be determined. i .
[0048] Optionally, the preset loss function model is: L D (y i ,f(x i ))=S i γ l(y i ,f(x i ))+(1-S i γ)l(s i ,f(x i ), where y i These are sample labels, s i f(x) is the first category label. i ) represents the second classification label, l is the cross-entropy loss function, and l(y) is the cross-entropy loss function. i ,f(x i )) represents the first difference, l(s) i ,f(xi The second difference is S. i S represents the probability that the label matches. i γ As the first trust weight, (1-S i ) γ The second trust weight, γ∈(0,1), is S i The larger the value, the more trust is placed in the auxiliary judgment model (i.e., the second classification model); conversely, the smaller the value, the lower the weight of the auxiliary model (i.e., the second classification model).
[0049] As an optional embodiment, after analyzing the first and second differences using a preset loss function model to determine the pseudo-label model, the method further includes: using the pseudo-label model to add pseudo-labels to the unlabeled data and noisy labeled data to generate pseudo-label data; using the pseudo-label data and clean labeled data as training data to train a preset classification model, wherein the preset classification model is used to analyze the target communication relationship data and determine the target label of the target communication relationship data.
[0050] The above embodiments of the present invention utilize a pseudo-label model to add pseudo-labels to unlabeled data and noisy labeled data, and use the pseudo-labeled data and clean labeled data as training data for a preset classification model. Then, the target communication relationship data is analyzed based on the trained preset classification model to determine the target label of the target communication relationship data. This achieves the technical effect of using unlabeled data for model training, thereby solving the technical problem of not being able to use unlabeled data for model training.
[0051] The present invention also provides a preferred embodiment, which provides a method for identifying fraudulent users based on pseudo-tag technology.
[0052] To fully utilize the semantic information of both labeled and unlabeled data, this application proposes a fraud user identification method based on pseudo-labeling technology. The method uses a model trained on labeled data to label the unlabeled data, filters the samples based on the labeling results, and then retrains to improve the model's performance. Robust pseudo-labeling generation technology can be used to alleviate the problem of insufficient data labels in fraud user identification.
[0053] The technical solution provided by this invention constructs a call user relationship network using collected raw data, then filters out a label-clean subgraph (i.e., clean label data) from the network, and trains a classification model Fc (i.e., the first classification model) using the clean subgraph; then, it uses the proposed conditional trust learning framework (i.e., the pseudo-label model) to generate robust pseudo-labels, and finally trains a classification model Fa (i.e., the preset classification model). By generating robust pseudo-labels, the model is further trained to improve detection performance; the conditional trust learning framework (i.e., the preset loss function model) utilizes the semantic information of labeled data (i.e., clean label data) and unlabeled data to generate more robust pseudo-labels.
[0054] Figure 2 This is a schematic diagram illustrating the training of a preset classification model based on pseudo-label technology according to an embodiment of the present invention, as shown below. Figure 2 As shown, the steps are as follows:
[0055] S21, Data Preprocessing.
[0056] S22, constructing a spatiotemporal heterogeneous communication network.
[0057] S23, select a clean label subgraph (i.e. determine clean label data).
[0058] S24, train the classification model Fc (i.e., train the first classification model).
[0059] S25, Conditional Trust Learning, generates pseudo-labels.
[0060] S26, train the classification model Fa (i.e., train the preset classification model).
[0061] Figure 3 This is a schematic diagram of constructing a spatiotemporal heterogeneous communication network according to an embodiment of the present invention, such as... Figure 3 As shown, a heterogeneous call network G = <V,E,h,w> is constructed by acquiring relevant information about user calls. Here, V is the vertex set, including users or devices (i.e., communication relationships), E is the edge set, h represents node attributes such as user type, and w represents edge attributes such as call time. This is represented by triples, which include two types: (user, call, user) and (user, call, device), meaning G has two node types and two edge types. To avoid excessive network discretization, time information is added. Call relationships within a week are statistically analyzed, highly isolated branches are removed, and only the denser and more important statistical information remains. At this point, the call network contains a large amount of unlabeled data.
[0062] As an optional embodiment, constructing a spatiotemporally heterogeneous call network includes: filtering clean label subgraphs (i.e., determining clean label data).
[0063] Optionally, users on the whitelist (a list of confirmed normal users) and the blacklist (a list of confirmed malicious users) are marked as clean users, from G a We retain only nodes and edges associated with nodes whose labels are pure (whitelist and blacklist), thus constructing a spatiotemporal heterogeneous communication network with pure labels, denoted as G. c This serves as clean-labeled data. Due to the limited number of clean labels, G... c The scale will be much smaller than G a Define G n =G a -G c Then G n It contains a large amount of unlabeled data and noisy labeled data.
[0064] As an optional embodiment, training the classification model Fc includes: training the classification model Fc (i.e., training the first classification model) using a pure subgraph (i.e., clean labeled data).
[0065] Alternatively, GraphSAGE is a classic inductive graph embedding algorithm that learns the embedding representation of network nodes by sampling neighbor nodes and using aggregation functions. However, traditional GraphSAGE cannot aggregate all neighbor information (due to sampling a fixed number of neighbors) and cannot handle weighted heterogeneous networks. Therefore, based on the GraphSAGE framework, we propose an inductive graph embedding learning method adapted to weighted heterogeneous graphs, abbreviated as RW-GraphSAGE, which can generalize to graph embedding algorithms with different network structures.
[0066] Alternatively, in the case of preprocessing, in order to mitigate the potential loss of useful information due to GraphSAGE's inability to aggregate all neighbor information, all node features (i.e., the user features and device features mentioned above) are first initialized as the average of the first-order neighbor node features concatenated with the node's own features.
[0067] Alternatively, in the case of weighted feature aggregation, in RW-GraphSAGE, the neighboring node with the larger the edge weight has a greater influence on the target node.
[0068] Optionally, after converting high-dimensional graph data into low-dimensional vector representation using RW-GraphSAGE, a classification model Fc (i.e., the first classification model) can be trained as follows: F c =argmin f E Dct {l[y * ,f(x c )]}, where Dct is G c Test data, y * The true label of sample x (i.e., the preset label of the preset communication relationship data), f(x)c ) is the model output (i.e., the first classification label), l is the cross-entropy loss function, and E represents the expectation.
[0069] It should be noted that if directly through the real-time spatiotemporal heterogeneous communication network G... a The training classification model Fa (i.e., the second classification model) is: F a =argmin f E Dct {L[y * ,f(x a )]}, where Dct is G a Test data, y * The true label of sample x (i.e., the preset label of the preset communication relationship data), f(x) a ) represents the model output. L represents the loss function. E represents the expectation. However, due to G a The model contains a large number of noisy labeled samples (i.e., noisy labeled data) and unlabeled samples (i.e., unlabeled data), resulting in limited model performance.
[0070] Figure 4 This is a schematic diagram illustrating the generation of pseudo-labels using conditional trust training according to an embodiment of the present invention, as shown below. Figure 4 As shown, in order to fully learn G a The semantic information in the model is used to generate pseudo-labels through conditional trust training, and then Fa is further trained. Specifically, this includes: training the model F... c (i.e., the first classification model) serves as the guiding model, due to G c The labels of the samples are all clean, therefore F c The output results have a certain degree of credibility; a new loss function (i.e., a pre-defined loss function model) is used as the LD, which is called Conditional Trust Loss (CTLoss).
[0071] It should be noted that, unlike some existing works that choose unconditional trust auxiliary models, the degree of trust is determined based on the output of the auxiliary model.
[0072] Optionally, the default loss function model is: Among them, y i These are pre-defined, preset labels (i.e., sample labels), s i f(x) is the first category label. i ) represents the second classification label, l is the cross-entropy loss function, and l(y) is the cross-entropy loss function. i ,f(x i The primary loss (i.e., the first difference) is l(s). i ,f(x i To aid in determining the loss (i.e., the second difference), Si S represents the probability that the label matches. i γ As the first trust weight, (1-S i ) γ The second trust weight, γ∈(0,1), is S i The larger the value, the more trust is placed in the auxiliary judgment model (i.e., the second classification model); conversely, the smaller the value, the lower the weight of the auxiliary model (i.e., the second classification model).
[0073] It should be noted that the second classification model can output the first classification label based on the sample communication relationship data, and can also carry the label matching probability of the first classification label and the sample label in the first classification label.
[0074] Optionally, the label matching probability is used to represent the probability that the first classification label obtained by the second classification model from analyzing the target communication relationship data is the same as the target label pre-added to the target communication relationship.
[0075] Alternatively, since l is the cross-entropy loss, the loss function for label y i It is linear, therefore, it can be rewritten as: Therefore, the pseudo-label model can be defined as: Robust pseudo-labels can be obtained through conditional trust training.
[0076] Alternatively, the generated pseudo-labels can be used to retrain the model (i.e., the pre-defined classification model) to obtain a more robust system for identifying fraudulent users.
[0077] The technical solution provided by this invention can make full use of labeled, noisy labeled and unlabeled data for learning; the conditional trust loss CTLoss (i.e., the preset loss function model) can flexibly use auxiliary models for conditional trust training.
[0078] According to an embodiment of the present invention, a training device for a pseudo-label model is also provided. It should be noted that the training device for the pseudo-label model can be used to execute the training method for the pseudo-label model in the embodiment of the present invention, and the training method for the pseudo-label model in the embodiment of the present invention can be executed in the training device for the pseudo-label model.
[0079] Figure 5 This is a schematic diagram of a training device for a pseudo-label model according to an embodiment of the present invention, as shown below. Figure 5As shown, the device may include: an acquisition module 51, used to acquire multiple sample communication relationship data, wherein each sample communication relationship data represents the communication relationship between a group of communication objects that have been labeled with sample tags; a first analysis module 53, used to analyze the sample communication relationship data using a first classification model to generate a first classification label, wherein the first classification model is trained using clean label data through machine learning, and the clean label data is preset communication relationship data with preset labels; and a second analysis module 55, used to analyze the sample communication relationship data using a second classification model to generate a second classification label, wherein the second classification model is trained using mixed data through machine learning, and the mixed data includes: clean label data, noise labels, and other data. The system includes labeled data and unlabeled data. Noise-labeled data consists of preset relationship data with added noise labels, where noise labels are interference from preset labels. Unlabeled data consists of preset communication relationship data without added labels. A determination module 57 is used to determine the difference between the sample label and the second classification label as the first difference, and the difference between the first classification label and the second classification label as the second difference. A third analysis module 59 is used to analyze the first and second differences using a preset loss function model to determine a pseudo-label model. The preset loss function model is used to assign preset trust weights to the first and second differences, and to define a pseudo-label model based on the first and second differences and the preset trust weights. The pseudo-label model is used to add preset labels to the unlabeled data and the noise-labeled data.
[0080] It should be noted that the acquisition module 51 in this embodiment can be used to execute step S102 in this application embodiment, the first analysis module 53 in this embodiment can be used to execute step S104 in this application embodiment, the second analysis module 55 in this embodiment can be used to execute step S106 in this application embodiment, the determination module 57 in this embodiment can be used to execute step S108 in this application embodiment, and the third analysis module 59 in this embodiment can be used to execute step S110 in this application embodiment. The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments.
[0081] In this embodiment of the invention, multiple sample communication relationship data are acquired, wherein each sample communication relationship data represents the communication relationship between a group of communication objects that have been labeled with sample tags; the sample communication relationship data are analyzed using a first classification model to generate a first classification label, wherein the first classification model is trained using clean-label data through machine learning, and the clean-label data is preset communication relationship data with preset labels; the sample communication relationship data are analyzed using a second classification model to generate a second classification label, wherein the second classification model is trained using mixed data through machine learning, and the mixed data includes: clean-label data, noisy-label data, and unlabeled data, the noisy-label data is preset relationship data with added noise labels, the noise labels are interference of the preset labels, and the unlabeled data is preset communication data without added labels. The data is related to the following: The difference between the sample label and the second classification label is defined as the first difference; the difference between the first classification label and the second classification label is defined as the second difference; a pre-defined loss function model is used to analyze the first and second differences to determine a pseudo-label model. The pre-defined loss function model is used to assign pre-defined trust weights to the first and second differences. Based on the first and second differences and the pre-defined trust weights, a pseudo-label model is defined. This pseudo-label model is used to add pre-defined labels to unlabeled data and noisy labeled data. By adding pseudo-labels to unlabeled data and noisy labeled data using the pseudo-label model, pseudo-labeled data is generated. Then, clean labeled data and pseudo-labeled data are used together for model training, thus achieving the technical effect of training a model using unlabeled data and solving the technical problem of not being able to train a model using unlabeled data.
[0082] As an optional embodiment, the acquisition module includes: an acquisition module for acquiring a preset communication relationship network, wherein the preset communication relationship network records multiple preset communication relationship data, each preset communication relationship data representing a communication relationship between a group of communication objects; a first determining unit for determining, in the preset communication relationship network, preset communication relationship data with preset labels as clean label data; and a selection unit for selecting sample communication relationship data from the clean label data.
[0083] As an optional embodiment, the apparatus further includes: a second determining unit, configured to determine clean label data other than sample communication relationship data as clean label data for training the first classification model after selecting sample communication relationship data from the clean label data.
[0084] As an optional embodiment, the determining module includes: a third determining unit, configured to determine the loss of the sample label relative to the second classification label as a first difference using a preset cross-entropy loss function; and a fourth determining unit, configured to determine the loss of the first classification label relative to the second classification label as a second difference using a preset cross-entropy loss function.
[0085] As an optional embodiment, the preset trust weights include: a first trust weight and a second trust weight. The third analysis module includes: an analysis unit for analyzing the difference between the sample label and the second classification label to determine the degree of trust in the first classification model; a first allocation unit for allocating a first trust weight to the first difference according to the degree of trust; a second allocation unit for allocating a second trust weight to the second difference according to the degree of trust; and a fifth determination unit for determining the pseudo-label model based on the product of the first difference and the first trust weight, and the product of the second difference and the second trust weight.
[0086] As an optional embodiment, the analysis unit includes: an analysis subunit for analyzing the difference between the sample label and the second classification label, and determining the label matching probability that the first classification label matches the sample label; and a determination subunit for determining the degree of trust in the first classification model based on the label matching probability.
[0087] As an optional embodiment, the apparatus further includes: a generation module, used to analyze the first difference and the second difference using a preset loss function model, and after determining the pseudo-label model, use the pseudo-label model to add pseudo-labels to the unlabeled data and noisy labeled data to generate pseudo-label data; and a training module, used to train a preset classification model using the pseudo-label data and clean labeled data as training data, wherein the preset classification model is used to analyze the target communication relationship data and determine the target label of the target communication relationship data.
[0088] Embodiments of the present invention can provide a computer terminal, which can be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the computer terminal can also be replaced by a mobile terminal or other terminal device.
[0089] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.
[0090] In this embodiment, the computer terminal described above can execute the program code for the following steps in the training method of the pseudo-label model: acquiring multiple sample communication relationship data, wherein each sample communication relationship data is used to represent the communication relationship between a group of communication objects that have been labeled with sample tags; analyzing the sample communication relationship data using a first classification model to generate a first classification label, wherein the first classification model is trained using clean label data through machine learning, and the clean label data is preset communication relationship data with preset labels added; analyzing the sample communication relationship data using a second classification model to generate a second classification label, wherein the second classification model is trained using mixed data through machine learning, and the mixed data includes: dry The dataset consists of net-labeled data, noisy-labeled data, and unlabeled data. Noisy-labeled data refers to preset relationship data with added noise labels, where the noise labels represent interference with the preset labels. Unlabeled data refers to preset communication relationship data without added labels. The difference between the sample label and the second classification label is defined as the first difference, and the difference between the first classification label and the second classification label is defined as the second difference. A preset loss function model is used to analyze the first and second differences to determine a pseudo-label model. The preset loss function model is used to assign preset trust weights to the first and second differences, and the pseudo-label model is defined based on the first and second differences and the preset trust weights. The pseudo-label model is used to add preset labels to the unlabeled data and the noisy-labeled data.
[0091] Optionally, Figure 6 This is a structural block diagram of a computer terminal according to an embodiment of the present invention. Figure 6 As shown, the computer terminal 60 may include one or more (only one is shown in the figure) processors 62 and memory 64.
[0092] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the training method and apparatus for the pseudo-label model in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned training method for the pseudo-label model. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to terminal A via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0093] The processor can access information and applications stored in memory via a transmission device to perform the following steps: acquiring multiple sample communication relationship data, wherein each sample communication relationship data represents the communication relationship between a group of communication objects that have been labeled; analyzing the sample communication relationship data using a first classification model to generate a first classification label, wherein the first classification model is trained using clean label data through machine learning, and the clean label data is preset communication relationship data with preset labels; analyzing the sample communication relationship data using a second classification model to generate a second classification label, wherein the second classification model is trained using mixed data through machine learning, and the mixed data includes: clean labels... The system comprises labeled data, noisy-labeled data, and unlabeled data. Noisy-labeled data consists of preset relationship data with added noise labels, where the noise labels represent interference from the preset labels. Unlabeled data consists of preset communication relationship data without added labels. The system identifies the difference between the sample label and the second classification label as the first difference and the difference between the first classification label and the second classification label as the second difference. A preset loss function model is used to analyze the first and second differences to determine a pseudo-label model. The preset loss function model is used to assign preset trust weights to the first and second differences, and the pseudo-label model is defined based on the first and second differences and the preset trust weights. The pseudo-label model is used to add preset labels to the unlabeled data and the noisy-labeled data.
[0094] Optionally, the processor may also execute program code for the following steps: obtaining a preset communication relationship network, wherein the preset communication relationship network records multiple preset communication relationship data, each preset communication relationship data being used to represent a communication relationship between a group of communication objects; in the preset communication relationship network, determining the preset communication relationship data with preset labels as clean label data; and selecting sample communication relationship data from the clean label data.
[0095] Optionally, the processor may also execute program code that performs the following steps: after selecting sample communication relationship data from the clean label data, determines the clean label data other than the sample communication relationship data as the clean label data for training the first classification model.
[0096] Optionally, the processor may also execute program code that performs the following steps: using a preset cross-entropy loss function to determine the loss of the sample label relative to the second classification label as the first difference; using a preset cross-entropy loss function to determine the loss of the first classification label relative to the second classification label as the second difference.
[0097] Optionally, the preset trust weights include: a first trust weight and a second trust weight. The processor can also execute program code that performs the following steps: analyzes the difference between the sample label and the second classification label to determine the degree of trust in the first classification model; assigns a first trust weight to the first difference based on the degree of trust; assigns a second trust weight to the second difference based on the degree of trust; and determines the pseudo-label model based on the product of the first difference and the first trust weight, and the product of the second difference and the second trust weight.
[0098] Optionally, the processor may also execute program code that performs the following steps: analyzes the difference between the sample label and the second classification label, determines the label matching probability of the first classification label matching the sample label, and determines the degree of trust in the first classification model based on the label matching probability.
[0099] Optionally, the processor may also execute program code for the following steps: after analyzing the first and second differences using a preset loss function model and determining the pseudo-label model, the pseudo-label model is used to add pseudo-labels to the unlabeled data and noisy labeled data to generate pseudo-label data; the pseudo-label data and clean labeled data are used as training data to train a preset classification model, wherein the preset classification model is used to analyze the target communication relationship data and determine the target label of the target communication relationship data.
[0100] This invention provides a training scheme for a pseudo-label model. Multiple sample communication relationship data are acquired, where each sample communication relationship data represents the communication relationship between a group of communication objects that have been labeled. A first classification model is used to analyze the sample communication relationship data to generate a first classification label. The first classification model is trained using clean-label data via machine learning, and the clean-label data consists of preset communication relationship data with pre-defined labels. A second classification model is used to analyze the sample communication relationship data to generate a second classification label. The second classification model is trained using mixed data via machine learning, where the mixed data includes clean-label data, noisy-label data, and unlabeled data. The noisy-label data consists of preset relationship data with added noise labels, where the noise labels are interference from the preset labels. The unlabeled data consists of preset communication relationship data without added labels. The difference between the sample label and the second classification label is defined as the first difference, and the difference between the first classification label and the second classification label is defined as the second difference. A pre-defined loss function model is used to analyze the first and second differences to determine a pseudo-label model. The pre-defined loss function model is used to assign pre-defined trust weights to the first and second differences. Based on the first and second differences and the pre-defined trust weights, a pseudo-label model is defined. This pseudo-label model is used to add pre-defined labels to unlabeled data and noisy labeled data. By adding pseudo-labels to unlabeled data and noisy labeled data using the pseudo-label model, pseudo-labeled data is generated. Then, clean labeled data and pseudo-labeled data are used together for model training, thus achieving the technical effect of training a model using unlabeled data and solving the technical problem of not being able to train a model using unlabeled data.
[0101] Those skilled in the art will understand that Figure 6 The structure shown is for illustrative purposes only. The computer terminal can also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a mobile internet device (MID), a PAD, and other terminal devices. Figure 6 This does not limit the structure of the aforementioned electronic device. For example, computer terminal 60 may also include components that are more... Figure 6 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 6 The different configurations shown.
[0102] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0103] Embodiments of the present invention also provide a non-volatile storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the training method of the pseudo-label model provided in the above embodiments.
[0104] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.
[0105] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: acquiring multiple sample communication relationship data, wherein each sample communication relationship data represents a communication relationship between a group of communication objects that have been labeled with sample tags; analyzing the sample communication relationship data using a first classification model to generate a first classification label, wherein the first classification model is trained using clean label data through machine learning, and the clean label data is preset communication relationship data with preset labels added; analyzing the sample communication relationship data using a second classification model to generate a second classification label, wherein the second classification model is trained using mixed data through machine learning, and the mixed data includes: dry The dataset consists of net-labeled data, noisy-labeled data, and unlabeled data. Noisy-labeled data refers to preset relationship data with added noise labels, where the noise labels represent interference with the preset labels. Unlabeled data refers to preset communication relationship data without added labels. The difference between the sample label and the second classification label is defined as the first difference, and the difference between the first classification label and the second classification label is defined as the second difference. A preset loss function model is used to analyze the first and second differences to determine a pseudo-label model. The preset loss function model is used to assign preset trust weights to the first and second differences, and the pseudo-label model is defined based on the first and second differences and the preset trust weights. The pseudo-label model is used to add preset labels to the unlabeled data and the noisy-labeled data.
[0106] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: obtaining a preset communication relationship network, wherein the preset communication relationship network records multiple preset communication relationship data, each preset communication relationship data being used to represent a communication relationship between a group of communication objects; in the preset communication relationship network, determining the preset communication relationship data with preset tags as clean tag data; and selecting sample communication relationship data from the clean tag data.
[0107] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: after selecting sample communication relationship data from the clean label data, determining the clean label data other than the sample communication relationship data as the clean label data for training the first classification model.
[0108] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: determining the loss of the sample label relative to the second classification label as a first difference using a preset cross-entropy loss function; and determining the loss of the first classification label relative to the second classification label as a second difference using a preset cross-entropy loss function.
[0109] Optionally, in this embodiment, the preset trust weights include: a first trust weight and a second trust weight. The non-volatile storage medium is configured to store program code for performing the following steps: analyzing the difference between the sample label and the second classification label to determine the degree of trust in the first classification model; assigning a first trust weight to the first difference based on the degree of trust; assigning a second trust weight to the second difference based on the degree of trust; and determining the pseudo-label model based on the product of the first difference and the first trust weight, and the product of the second difference and the second trust weight.
[0110] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: analyzing the difference between the sample label and the second classification label, determining the label matching probability of the first classification label matching the sample label; and determining the degree of trust in the first classification model based on the label matching probability.
[0111] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: after analyzing the first difference and the second difference using a preset loss function model to determine the pseudo-label model, the pseudo-label model is used to add pseudo-labels to the unlabeled data and the noisy labeled data to generate pseudo-label data; the pseudo-label data and the clean labeled data are used as training data to train a preset classification model, wherein the preset classification model is used to analyze the target communication relationship data and determine the target label of the target communication relationship data.
[0112] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0113] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0114] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.
[0115] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0116] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0117] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.
[0118] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A training method for a pseudo-label model, characterized in that, include: Acquire multiple sample communication relationship data, wherein each sample communication relationship data is used to represent the communication relationship between a group of communication objects that have been labeled with sample tags; The sample communication relationship data is analyzed using a first classification model to generate a first classification label. The first classification model is trained using clean label data through machine learning. The clean label data is preset communication relationship data with preset labels added. The sample communication relationship data is analyzed using a second classification model to generate a second classification label. The second classification model is trained by machine learning using mixed data, which includes: clean labeled data, noisy labeled data, and unlabeled data. The noisy labeled data is preset relationship data with added noise labels, and the noise labels are interferences of the preset labels. The unlabeled data is preset communication relationship data without added labels. The difference between the sample label and the second classification label is determined as the first difference, and the difference between the first classification label and the second classification label is determined as the second difference; The first difference and the second difference are analyzed using a preset loss function model to determine a pseudo-label model. The preset loss function model is used to assign preset trust weights to the first difference and the second difference, and the pseudo-label model is defined based on the first difference, the second difference, and the preset trust weights. The pseudo-label model is used to add the preset labels to the unlabeled data and the noisy labeled data. The preset trust weights include: a first trust weight and a second trust weight. A preset loss function model is used to analyze the first difference and the second difference to determine the pseudo-label model, which includes: Analyze the differences between the sample labels and the second classification labels to determine the level of trust in the first classification model; Assign a first trust weight to the first difference based on the level of trust; Assign a second trust weight to the second difference based on the level of trust; The pseudo-label model is determined based on the product of the first difference and the first trust weight, and the product of the second difference and the second trust weight.
2. The method according to claim 1, characterized in that, Obtaining communication relationship data from multiple samples includes: Obtain a preset communication relationship network, wherein the preset communication relationship network records multiple preset communication relationship data, and each preset communication relationship data is used to represent the communication relationship between a group of communication objects; In the preset communication relationship network, the preset communication relationship data with the preset tag is determined as the clean tag data; Select the sample communication relationship data from the clean label data.
3. The method according to claim 2, characterized in that, After selecting the sample communication relationship data from the clean label data, the method further includes: Clean label data other than the sample communication relationship data is determined as clean label data for training the first classification model.
4. The method according to claim 1, characterized in that, Determining the difference between the sample label and the second classification label as the first difference, and determining the difference between the first classification label and the second classification label as the second difference, includes: The loss of the sample label relative to the second classification label is determined using a preset cross-entropy loss function as the first difference; The loss of the first classification label relative to the second classification label is determined using the preset cross-entropy loss function as the second difference.
5. The method according to claim 1, characterized in that, Analyzing the differences between the sample labels and the second classification labels to determine the level of trust in the first classification model includes: Analyze the differences between the sample label and the second classification label to determine the label matching probability of the first classification label matching the sample label; The level of trust in the first classification model is determined based on the probability of matching the labels.
6. The method according to claim 1, characterized in that, After analyzing the first difference and the second difference using a preset loss function model to determine the pseudo-label model, the method further includes: The pseudo-label model is used to add pseudo-labels to the unlabeled data and the noisy labeled data to generate pseudo-labeled data. The pseudo-label data and the clean label data are used as training data to train a preset classification model, wherein the preset classification model is used to analyze the target communication relationship data and determine the target label of the target communication relationship data.
7. A training device for a pseudo-labeled model, characterized in that, include: The acquisition module is used to acquire multiple sample communication relationship data, wherein each sample communication relationship data is used to represent the communication relationship between a group of communication objects that have been labeled with sample tags; The first analysis module is used to analyze the sample communication relationship data using a first classification model and generate a first classification label, wherein the first classification model is trained by machine learning using clean label data, and the clean label data is preset communication relationship data with preset labels added. The second analysis module is used to analyze the sample communication relationship data using a second classification model and generate a second classification label. The second classification model is trained by machine learning using mixed data. The mixed data includes: clean labeled data, noisy labeled data, and unlabeled data. The noisy labeled data is preset relationship data with added noise labels, and the noise labels are interferences of the preset labels. The unlabeled data is preset communication relationship data without added labels. The determination module is used to determine the difference between the sample label and the second classification label as a first difference, and to determine the difference between the first classification label and the second classification label as a second difference; The third analysis module is used to analyze the first difference and the second difference using a preset loss function model to determine a pseudo-label model. The preset loss function model is used to assign preset trust weights to the first difference and the second difference, and to define the pseudo-label model based on the first difference, the second difference, and the preset trust weights. The pseudo-label model is used to add the preset labels to the unlabeled data and the noisy labeled data. The preset trust weights include: a first trust weight and a second trust weight; the third analysis module includes: An analysis unit is used to analyze the differences between the sample labels and the second classification labels to determine the degree of trust in the first classification model; The first allocation unit is configured to allocate the first trust weight to the first difference based on the degree of trust. The second allocation unit is used to allocate the second trust weight to the second difference according to the degree of trust. The fifth determining unit is used to determine the pseudo-label model based on the product of the first difference and the first trust weight, and the product of the second difference and the second trust weight.
8. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores a program, wherein when the program is executed, it controls the device where the non-volatile storage medium is located to execute the training method of the pseudo-label model according to any one of claims 1 to 6.
9. An electronic device, characterized in that, include: A memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when running, executes the training method for the pseudo-label model according to any one of claims 1 to 6.