Partial mark learning method based on PLTSK

Abstract

The invention discloses a partial mark learning method based on PLTSK, and belongs to the technical field of partial mark learning. According to the method, consequent output of a traditional TSK fuzzy system is improved, and a confidence coefficient matrix is introduced to enhance processing of candidate tags; a PLOCTSK framework is constructed, a conventional PLTSK fuzzy system mines effective information in a feature space through a fuzzy rule, and preliminary disambiguation is carried out according to a category prototype; the correction of the PLTSK fuzzy system focuses on learning of accurate non-candidate labels; the two fuzzy systems interact with each other through a confrontation mechanism, and the distinction degree between disambiguated labels and non-real labels is effectively maximized. According to the method, the noise problem can be effectively solved, the accuracy of tag disambiguation is improved, and the method is suitable for various data types.

Description

Technical Field

[0001] The present invention relates to a partial label learning method based on PLTSK, which belongs to the technical field of partial label learning. Background Art

[0002] Machine learning has achieved great success in supervised learning tasks such as classification and regression. However, the high cost of manually labeled data makes it more challenging to obtain reliable information. The need for large amounts of data when training models inevitably leads to some instances with ambiguous labels. Therefore, weakly supervised learning has become an important research direction.

[0003] Partially labeled learning (PLL), as a weakly supervised learning framework, aims to identify unique true labels from data with ambiguous labels in order to summarize multi-class classifiers. This learning framework has a wide range of applications in the real world. For example, in automatic face naming, each face detected in the image can be used as an instance, and the name extracted from the relevant title can be used as a candidate label, but the true correspondence between each face and the specific name is unknown; in crowdsourcing-based classification tasks, when the label space is very large (may contain hundreds to thousands of labels), it is difficult for the annotator to accurately select the true label, and thus tends to provide inaccurate label information for the query; in part-of-speech tagging (POS) tasks, a word may correspond to multiple part-of-speech tags, only one of which is correct, and a model that can accurately perform part-of-speech tagging is learned.

[0004] The TSK fuzzy system is a classic intelligent model based on fuzzy reasoning. Due to its powerful learning ability, it has been widely used in many fields. The core of the TSK fuzzy system is the fuzzy reasoning rules. Unlike the fuzzy sets used in the Mamdani fuzzy system, the rule consequences of the TSK fuzzy system are composed of precise input values ​​or functions. This feature makes it more accurate and flexible when dealing with specific applications.

[0005] The main challenge of partial label learning (PLL) is that the true labels of training instances are not directly available to the learning algorithm. The core strategy for dealing with the PLL problem is to disambiguate candidate labels, specifically, to recover the true label information by distinguishing the confidence of each candidate label. Current disambiguation methods can be roughly divided into two categories: one is the average-based method, which treats each candidate label equally and attempts to learn by balancing the distribution of candidate labels with non-candidate labels. For example, some studies try to distinguish the average distribution of candidate labels from the distribution of non-candidate labels, while others classify new instances by analyzing the candidate labels of neighbors. Although this strategy is simple and easy to implement, there is a significant problem that the output of the true label may be overwhelmed by the output of false positive labels, resulting in a significant decrease in model performance. Another disambiguation strategy is recognition-based disambiguation. This strategy treats the true label as a latent variable and determines it through iterative optimization during model training. Generally speaking, an iterative optimization process (such as EM) is used to predict the latent variable, which can be achieved by the maximum likelihood or maximum margin method. Although such strategies are usually more effective than averaging-based methods, they rely on identifying a single true label from the candidate label set. If an iteration produces an inaccurate prediction, it may seriously affect the effect of subsequent model training. In addition to the above strategies, some works try to mine the neglected information in the feature space and make full use of it through feature-aware disambiguation. For example, some methods use the graph structure in the training data to generate label confidence for the candidate label set.

[0006] However, these methods usually fail to fully utilize the non-candidate label information in the label space and do not consider the false disambiguation problem caused by feature space noise, which makes the model sensitive to instances and prone to misjudgment. Summary of the invention

[0007] In order to solve the problems existing in the prior art, the present invention provides a partial label learning method based on PLTSK. This method introduces non-candidate label information to correct the wrong disambiguation label, constructs a PLTSK fuzzy system suitable for partial label learning, and applies fuzzy theory to solve the partial label learning problem for the first time. It has important technical value and application prospects. Using dual feature input and two-stage disambiguation strategy, the PLOCTSK partial label fuzzy system is designed, and the integration of two PLTSK fuzzy systems is realized.

[0008] The first step is to modify the post-processing part of the TSK fuzzy system, introduce the confidence matrix, and propose a PLTSK fuzzy system suitable for partial label learning (PLL) to enhance the processing ability of candidate labels.

[0009] Step 2: Build a PLOCTSK framework that integrates two PLTSK fuzzy systems. The conventional PLTSK fuzzy system performs preliminary disambiguation based on prototypes, and the PLTSK correction fuzzy system focuses on learning accurate non-candidate labels to correct the incorrect disambiguated labels in conventional PLTSK. Through alternating iterations, the two PLTSK systems promote each other, effectively improve the disambiguation effect, and maximize the distinction between disambiguated labels and non-true labels.

[0010] Step 3: Extensive experiments are conducted on datasets (including four synthetic datasets and five real-world datasets), and the proposed method outperforms six existing partial label learning algorithms in performance.

[0011] Furthermore, a partial labeling learning method based on PLTSK comprises the following steps:

[0012] S1. Modify the post-processing output of the TSK fuzzy system and introduce the confidence matrix; the output of the TSK fuzzy system is:

[0013]

[0014] in, represents the activation level of the rth rule, is a fuzzy set The associated membership function, is the normalized activation level of the rth rule;

[0015] The selected HTSK-LN-ReLU is used as the classification model. The output part and loss function of the consequent of HTSK-LN-ReLU are modified, and the conventional PLTSK fuzzy system is proposed. In the PLTSK fuzzy system, the fuzzy set in the antecedent of the rule is used to describe The membership function of is as follows:

[0016]

[0017] in, , are the center and standard deviation of the fuzzy set respectively;

[0018] After obtaining the membership of the sample in each rule, the softmax function is used and the membership function is changed to an average operation to calculate the activation level of the sample in each rule:

[0019]

[0020] The normalized activation level is expressed as follows:

[0021]

[0022] In order to more intuitively represent the defuzzification process, the input of the rule consequent is rewritten as:

[0023]

[0024] in,

[0025]

[0026] Use layer normalization scaling , a LayerNorm layer is added after the input of the rewriting part, and a ReLU function is introduced after the LayerNorm layer;

[0027]

[0028] The input of the rule postcondition is finally rewritten as:

[0029]

[0030]

[0031] The new post-processing output is expressed as:

[0032]

[0033] The final output of the PLTSK correction fuzzy system is:

[0034] Defining the label confidence matrix , and initialize it using a uniform distribution, where Indicates The label is the confidence of the true label of the i-th instance;

[0035] Define the non-candidate label confidence matrix As the training target of the calibration model; Indicates The confidence that the label is not the true label of the i-th instance;

[0036] S2. Construct a PLOCTSK framework that integrates two PLTSK fuzzy systems. The conventional PLTSK fuzzy system performs preliminary disambiguation based on prototypes, and the PLTSK correction fuzzy system focuses on learning accurate non-candidate labels to correct incorrect disambiguated labels in conventional PLTSK.

[0037] Through the adversarial relationship between the conventional PLTSK fuzzy system and the PLTSK corrected fuzzy system, the disambiguated label confidence matrix is ​​dynamically adjusted during the entire PLOCTSK training process; then, the conventional PLTSK model is trained in a new round of iterations based on the corrected label confidence.

[0038] The PLTSK correction fuzzy system is:

[0039]

[0040] The PLTSK correction fuzzy system is a multi-label problem that predicts the non-true label of the instance; ideally, the output results of the two classifiers are complementary; the complementary state between the two is expressed as ; Therefore, the optimization goal of the PLTSK correction classifier is:

[0041]

[0042]

[0043] in: is a vector of all 1s of length L; by using the AdamW optimizer, the gradient calculated by backpropagation is used to minimize the formula, passing valuable information to the label confidence matrix , optimize the label confidence matrix of the sample;

[0044] In the PLTSK correction model, the confidence of the disambiguated label is used as a parameter of the loss function to make adjustments during the disambiguation correction process, and the corrected label confidence is used to guide the update of the next round of disambiguation.

[0045] A prototype-based disambiguation approach is adopted to iteratively drive performance improvement through repeated interaction of the correction model with the disambiguation process until convergence.

[0046] Compared with the prior art, the present invention has the following significant advantages: (1) The dual feature input and two-stage disambiguation strategy are adopted to enhance the disambiguation ability and accuracy of the model. (2) By introducing non-candidate label information, the PLOCTSK method can effectively correct the erroneous disambiguation labels in the conventional PLOCTSK fuzzy system, thereby improving the robustness and accuracy of the model. (3) The disambiguation process is optimized by alternating iterations to maximize the distance between the disambiguation label and the non-candidate label, ensuring more accurate label recognition. (4) PLOCTSK does not rely on a specific feature space structure and can flexibly adapt to various types of data sets, especially in data sets with high-dimensional features or labels. (5) Experimental results on real-world data sets show that the PLOCTSK method performs well in partial label learning tasks and has higher classification performance and disambiguation ability than other existing algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is an example diagram of the PLOCTSK framework.

[0048] Figure 2 It is a schematic diagram of the PLTSK fuzzy system model. DETAILED DESCRIPTION

[0049] A PLTSK-based partial labeling learning disambiguation correction method comprises the following steps:

[0050] The first step is to modify the output part of the TSK fuzzy system and introduce the confidence matrix. The output of the TSK fuzzy system can be obtained through a simple defuzzification process, namely:

[0051] in, represents the activation level of the rth rule, is a fuzzy set The associated membership function, is the normalized activation level of the rth rule.

[0052] Firstly, HTSK-LN-ReLU, which is more suitable for processing high-dimensional data, is selected as the classification model. The output part and loss function of the consequent of HTSK-LN-ReLU are modified, and a conventional PLTSK fuzzy system is proposed. The PLTSK fuzzy system directly performs feature enhancement operations in its rule antecedent, avoiding the separation of the feature enhancement process into two independent steps: constructing a weighted graph and applying a multi-output regressor.

[0053] In the PLTSK fuzzy system, it is necessary to calculate the membership of each rule for the input sample, so the fuzzy set used to describe the rule antecedent is defined as The membership function of is defined as follows:

[0054]

[0055] in , are the center and standard deviation of the fuzzy set respectively. After obtaining the membership of the sample on each rule, the softmax function is used to change the membership function to an average operation to calculate the activation level of the sample in each rule:

[0056]

[0057] The normalized activation level is expressed as follows:

[0058]

[0059] In order to more intuitively represent the defuzzification process, the rule consequent is rewritten as:

[0060]

[0061] in,

[0062]

[0063] As the number of rules increases, the expected mean activation level decreases, and layer normalization is used to scale After rewriting the input of the subsequent part, a LayerNorm layer is added to deal with the gradient vanishing problem that may occur in the subsequent parameters. In order to ensure the scalability and interpretability of the model, the ReLU function is introduced after the LayerNorm layer.

[0064]

[0065] The input of the rule postcondition is finally rewritten as:

[0066]

[0067]

[0068] In order to adapt to the PLL multi-label learning scenario, the rule consequence is improved and expanded from a single label to multiple category vectors to adapt to multi-class classification. The new consequence output can be expressed as:

[0069]

[0070] The final output of the system is:

[0071] Secondly, the introduction of confidence helps the model to more accurately identify and disambiguate labels when dealing with partial label learning tasks. Define the label confidence matrix , and initialize it using a uniform distribution, where Indicates The labels are the confidence (probability) of the true label of the i-th instance.

[0072] Define the non-candidate label confidence matrix As the training target of the calibration model. Indicates The confidence (probability) that the label is not the true label of the i-th instance.

[0073] The second step is to build a PLOCTSK framework that integrates two PLTSK fuzzy systems. The conventional PLTSK fuzzy system performs preliminary disambiguation based on prototypes, and the PLTSK correction fuzzy system focuses on learning accurate non-candidate labels to correct incorrect disambiguated labels in conventional PLTSK.

[0074] By exploring the adversarial relationship between the conventional PLOCTSK fuzzy system and the PLOCTSK corrected fuzzy system, the disambiguated label confidence matrix is ​​dynamically adjusted during the entire PLOCTSK training process.

[0075] Then, based on the corrected label confidence, the regular PLTSK model is trained in a new round of iterations.

[0076] The constructed PLTSK correction fuzzy system is as follows:

[0077]

[0078] The PLTSK correction fuzzy system is a multi-label problem, predicting the non-true label of the instance. Ideally, the output results of the two classifiers are complementary. In PLL, in order to restore this state, The higher the confidence level of the label, the higher the The lower the confidence level. and The values ​​in are non-negative and not greater than 1, so the complementary state between the two can be expressed as In view of this, the optimization goal of the PLTSK correction classifier is:

[0079]

[0080]

[0081] in is a vector of all 1s of length L. The non-candidate label set provides directly accessible and more accurate supplementary information than the candidate label set. By using the AdamW optimizer, the gradient calculated by backpropagation is used to minimize the formula, passing valuable information to the label confidence matrix , thereby further optimizing the label confidence matrix of the sample and correcting the label noise problem in the disambiguation process where false positive labels cause non-candidate labels to show high confidence. In the PLTSK correction model, the disambiguated label confidence is used as a parameter of the loss function for continuous adjustment during the disambiguation correction process. Afterwards, the corrected label confidence is used to guide the update of the next round of disambiguation. The prototype-based disambiguation method is adopted, and the performance is continuously improved in an iterative manner through repeated interactions between the correction model and the disambiguation process until convergence.

[0082] The third step is to evaluate the performance of PLOCTSK and other methods on real datasets. On each dataset, ten-fold cross validation is used, and the average accuracy and its standard deviation are reported. The experimental results are shown in Table 1.

[0083] Table 1: Comparative experimental results of PLOCTSK and other methods on real-world datasets

[0084]

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the present invention. Although the above embodiments describe the present invention in detail, it should be understood that a person skilled in the art can still modify the technical solutions or replace some of the technical features with equivalents. These modifications or replacements do not change the essence of the corresponding technical solutions, and still fall within the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A partial labeling learning method based on PLTSK, characterized in that: The following steps are involved: S1. Modify the post-processing output of the TSK fuzzy system and introduce the confidence matrix; the output of the TSK fuzzy system is: ； in, represents the activation level of the rth rule, is a fuzzy set The associated membership function, is the normalized activation level of the rth rule; The selected HTSK-LN-ReLU is used as the classification model. The output part and loss function of the consequent of HTSK-LN-ReLU are modified, and the conventional PLTSK fuzzy system is proposed. In the PLTSK fuzzy system, the fuzzy set in the antecedent of the rule is used to describe The membership function of is as follows: ； in, , are the center and standard deviation of the fuzzy set respectively; After obtaining the membership of the sample in each rule, the softmax function is used and the membership function is changed to an average operation to calculate the activation level of the sample in each rule: ； The normalized activation level is expressed as follows: ； In order to more intuitively represent the defuzzification process, the input of the rule consequent is rewritten as: ； in, ； Use layer normalization scaling , a LayerNorm layer is added after the input of the rewriting part, and a ReLU function is introduced after the LayerNorm layer; ； The input of the rule postcondition is finally rewritten as: ； ； The new post-processing output is expressed as: ； The final output of the PLTSK correction fuzzy system is: ； Defining the label confidence matrix , and initialize it using a uniform distribution, where Indicates The label is the confidence of the true label of the i-th instance; Define the non-candidate label confidence matrix As the training target of the calibration model; Indicates The confidence that the label is not the true label of the i-th instance; S2. Construct a PLOCTSK framework that integrates two PLTSK fuzzy systems. The conventional PLTSK fuzzy system performs preliminary disambiguation based on prototypes, and the PLTSK correction fuzzy system focuses on learning accurate non-candidate labels to correct incorrect disambiguated labels in conventional PLTSK. Through the adversarial relationship between the conventional PLTSK fuzzy system and the PLTSK corrected fuzzy system, the disambiguated label confidence matrix is ​​dynamically adjusted during the entire PLOCTSK training process; then, the conventional PLTSK model is trained in a new round of iterations based on the corrected label confidence. The PLTSK correction fuzzy system is: ； The PLTSK correction fuzzy system is a multi-label problem that predicts the non-true label of the instance; ideally, the output results of the two classifiers are complementary; the complementary state between the two is expressed as ; Therefore, the optimization goal of the PLTSK correction classifier is: ； ； in: is a vector of all 1s of length L; by using the AdamW optimizer, the gradient calculated by backpropagation is used to minimize the formula, passing valuable information to the label confidence matrix , optimize the label confidence matrix of the sample; In the PLTSK correction model, the confidence of the disambiguated label is used as a parameter of the loss function to make adjustments during the disambiguation correction process, and the corrected label confidence is used to guide the update of the next round of disambiguation. A prototype-based disambiguation approach is adopted to iteratively drive performance improvement through repeated interaction of the correction model with the disambiguation process until convergence.

Images