Two-stage label noise learning classification method and system

Abstract

The invention discloses a two-stage label noise learning classification method and system, and the method comprises the following steps: constructing a relative similarity filter through the comprehensive measurement of relative similarity and Euclidean distance, inputting an original data set into the relative similarity filter, screening out noise samples through the calculation of the relative similarity of the samples, and carrying out the classification of the noise samples. Outputting the filtered data set; inputting the filtered data set into a robust sorting support vector machine model based on a non-convex slope loss function and a local label weighting strategy for training, and converting the non-convex loss function into a convex optimization problem through a convex-concave process for solving to obtain an optimal model parameter; and updating a model threshold by using the optimal model parameter, and classifying a to-be-classified sample by using the updated model. According to the method, data-level screening and model-level noise suppression are both considered, the problem of unstable performance under a single processing path in a traditional method is solved, and the method is particularly suitable for stable learning tasks under high-dimensional and high-noise scenes.

Description

Technical Field

[0001] The present invention belongs to the technical field of noise label processing in machine learning, and more specifically, relates to a two-stage label noise learning classification method and system. Background Art

[0002] Label noise is a common problem in machine learning, significantly impacting model performance. Label noise can arise from various sources, including manual annotation errors and data transmission errors, and is particularly prevalent when constructing large-scale datasets. Research has shown that label noise has a greater negative impact on model training than feature noise, potentially causing the model to learn incorrect patterns and thus reducing generalization capabilities. Therefore, developing effective methods to combat label noise is crucial for the application of machine learning in key areas such as medical diagnosis and financial risk management.

[0003] Existing methods for addressing label noise typically focus solely on a single dimension: data filtering or robust model construction. These methods lack effective coordination mechanisms and struggle to maintain stable performance in high-noise environments. While absolute distance metrics, such as relative density (RD) or weighted probability density (kRPD), possess a certain degree of noise recognition capability in low-dimensional spaces, they are susceptible to the "curse of dimensionality" in high-dimensional spaces, leading to failures in similarity calculations and significantly reduced filtering accuracy. Furthermore, traditional support vector machine (SVM) algorithms generally employ convex loss functions, lacking mechanisms to suppress anomalous samples. This makes it easy for residual noise labels that have not been effectively filtered to overly influence classification boundaries, thereby reducing the model's generalization capabilities.

[0004] The invention patent with prior art publication number CN110543898 proposes a supervised learning method, a data classification processing method and a device for noise labels. The method includes determining a first loss function for a classification model and constructing a second loss function based on the loss function; replacing the first loss function with the second loss function, and training the classification model based on the second loss function. This method lacks explicit data filtering and only relies on weighted loss to suppress noise. It is susceptible to the "dimensionality curse" in high-dimensional space, resulting in failure of similarity calculation, thereby significantly reducing the filtering accuracy. Summary of the Invention

[0005] In order to overcome the problems in the prior art that the filtering accuracy of the noise learning method drops sharply in high-dimensional space and the traditional support vector machine algorithm is insufficiently sensitive to noise, the present invention provides a two-stage label noise learning classification method and system.

[0006] The primary purpose of the present invention is to solve the above technical problems, and the technical solutions of the present invention are as follows: A first aspect of the present invention provides a two-stage label noise learning classification method, comprising the following steps: A relative similarity filter is constructed using a comprehensive measure of relative similarity and Euclidean distance, the original data set is input into the relative similarity filter, noise samples are filtered out by calculating the relative similarity of the samples, and the filtered data set is output; Inputting the filtered data set into a robust sorting support vector machine model based on a non-convex slope loss function and a local label weighting strategy for training, converting the non-convex loss function into a convex optimization problem through a convex-concave process for solving, and obtaining the optimal model parameters; The model threshold is updated using the optimal model parameters, and the updated model is used to classify the samples to be classified.

[0007] Furthermore, the original data set is input into the relative similarity filter, and the filtered data set is output, comprising the following steps: The observation labels of each sample in the original data set are used to construct the similar sample set and the heterogeneous sample set, and the similar neighbor set and heterogeneous neighbor set of each sample are determined based on the cosine similarity metric; The similarity between the target sample and the target sample is calculated by calculating the similarity between the target sample and the target sample, and the absolute similarity between the target sample and the target sample is obtained. The expression for degree is as follows:

[0008] Heterogeneous similarity The expression is as follows:

[0009] in, represents the i-th sample, and Represent the same sample data set and the heterogeneous sample data set respectively, represents the kth nearest neighbor sample, K represents the number of nearest neighbor samples, j is the sample feature number, d represents the number of sample features, Represents a sample The value on the jth feature dimension, Representation sample The value on the jth feature dimension, Indicates the inner product of two samples; Calculate the relative similarity of samples based on the absolute similarity of the same type and the absolute similarity of different types , the expression is as follows:

[0010] Sort the relative similarities of all samples in ascending order, and calculate the average relative similarity of the sample set as the filtering threshold; The samples whose relative similarity is lower than the filtering threshold are determined as suspected label noise samples and are removed, and the filtered data set is output.

[0011] Furthermore, the optimal model parameters are solved using a robust ranking support vector machine model based on a non-convex slope loss function and a local label weighting strategy, including the following steps: Use the filtered dataset as input to generate positive and negative sample pairs and calculate the predicted labels of the positive and negative sample pairs ; Use positive and negative sample pairs and predicted labels to construct the objective function of the model; Decompose the slope loss function in the objective function into convex function and concave function; The convex-concave process algorithm is combined with the Lagrange multiplier method to transform the optimization problem of concave functions into a dual problem, and the dual problem is solved using the Karush-Kuhn-Tucker condition to obtain the dual solution. Use the dual solution to update the model weight coefficients; Repeat the above steps until the model weight coefficient changes below the set threshold, and output the optimal model parameters.

[0012] Furthermore, the method for calculating the predicted labels of positive and negative sample pairs is Nearest neighbor method.

[0013] Furthermore, the expression of the objective function of the model constructed using positive and negative sample pairs and predicted labels is as follows:

[0014] Among them, i represents the sequence number of the positive sample pair, j represents the sequence number of the negative sample pair, represents the feature vector weight of the classification model, represents the regularization parameter, which is used to control the trade-off between the classification boundary interval and the classification error. represents the number of positive sample pairs, n represents the number of negative sample pairs, represents the slope loss function, Represents a sample pair ( , )’s noise labels, represents the predicted label, Represents a hyperparameter responsible for balancing noisy labels and predicted labels , Represents a positive sample and negative samples The positive sample pairs are composed of Represents the model weight vector Compared with the positive sample The result of the inner product operation.

[0015] Furthermore, the slope loss function is a non-convex function, and the slope loss function is decomposed into a convex function With concave function The objective function expression is as follows:

[0016]

[0017] in, is the convex part, The non-convex part.

[0018] Furthermore, the convex-concave process (CCCP) algorithm is combined with the Lagrange multiplier method to transform the optimization problem of the convex function into a dual problem and obtain the dual solution, which includes the following steps: According to expression (7), each iteration of the convex-concave process is used to approximate the concave part with a tangent line, and the resulting convex function is minimized as follows:

[0019] in, Represent the parameters obtained by the kth and k+1th iterations respectively, Represents the feature vector weight of the classification model, The derivative of is defined as , the expression is as follows:

[0020] Among them, i represents the sequence number of the positive sample pair, j represents the sequence number of the negative sample pair, represents the number of positive sample pairs, n represents the number of negative sample pairs, Represents a sample pair ( , )’s noise labels, represents the predicted label, represents the hyperparameter, Represents a positive sample and negative samples The positive sample pairs, weight coefficient The expression is as follows:

[0021] Among them, s represents the upper limit threshold of the slope loss function, represents the regularization parameter, Represents the model weight vector Compared with the positive sample The result of the inner product operation of ; Rewrite the k+1th iterative optimization problem as follows with slack variables The convex optimization form of , using expressions (8) and (9) to analyze and simplify expression (7), we get the following expression:

[0022] Constructing the corresponding Lagrangian function for expression (10) yields the following expression:

[0023] in, and is the Lagrange multiplier, for the variables in Eq. (11) and Taking partial derivatives, we can derive the following Karush-Kuhn-Tucker conditions:

[0024] Substituting expression (12) into expression (11) and simplifying it, we get the dual problem expression as follows:

[0025] in, represents the sequence number of the positive sample pair, Both represent the negative sample pair numbers, represents a positive sample pair, represents the weight coefficient; make , , ,in, represent 、 , represent 、 represent , the expression is as follows:

[0026] Substituting (14) into (13) yields the dual equation, which is expressed as follows:

[0027] Solve expression (15) to obtain the dual optimal solution .

[0028] Furthermore, the method of updating the model weight coefficient using the dual solution includes the following steps: Using the dual optimal solution Calculate the parameters of the model , the expression is as follows:

[0029] Among them, i represents the sequence number of the positive sample pair, j represents the sequence number of the negative sample pair, represents the dual optimal solution, Represents a sample pair ( , )’s noise labels, represents the predicted label, represents the hyperparameter, Represents a positive sample and negative samples The positive sample pairs constituted by The parameters Expression (9) is used to obtain the updated weight coefficient.

[0030] Furthermore, the optimal model parameters are used to update the model threshold, and the updated model is used to classify the samples to be classified, including the following steps: Constructing a scoring function using optimal model parameters ,in, Represents a sample, and calculates the minimum score value of all positive samples for all samples in the dataset and the maximum score of negative samples ; Based on the five-fold cross-validation strategy, the intermediate discriminant value is calculated in each fold, and the obtained , the expression is as follows:

[0031] Among them, i represents the fold number of i, represents the highest score of the negative sample in the ith fold, Represents the lowest score value of the positive sample in the i-th fold, averages the intermediate discriminant values ​​obtained, and determines the final discrimination threshold , the expression is as follows:

[0032] The score value obtained by the scoring function and the discrimination threshold For comparison, if , it is judged as positive; otherwise it is judged as negative. The expression of the prediction function is as follows:

[0033] Use the prediction function to classify the samples to be classified and output the classification results.

[0034] The second aspect of the present invention provides a two-stage label noise learning classification system, including a memory and a processor, wherein the memory includes a two-stage label noise learning classification method program, and when the two-stage label noise learning classification method program is executed by the processor, it implements the steps of a two-stage label noise learning classification method.

[0035] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: The present invention realizes efficient suppression of label noise by constructing a two-stage collaborative mechanism of relative similarity filter and robust sorting support vector machine model. The relative similarity filter integrates cosine similarity and Euclidean distance information, taking into account both feature direction and spatial distance, and can effectively identify and eliminate low-credibility samples in high-dimensional data, overcoming the problem of sudden drop in filtering accuracy of traditional absolute distance methods in high-dimensional space. The robust sorting support vector machine introduces a non-convex slope loss function and a local label weighting strategy to further weaken the interference of residual noise samples on the classification boundary at the model level, significantly improving the classification robustness. The overall solution takes into account both data-level screening and model-level noise suppression, improving the problem of unstable performance under a single processing path in traditional methods, and is particularly suitable for stable learning tasks in high-dimensional and high-noise scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to make the purpose and technical solution of the present invention clearer, the present invention provides the following drawings and descriptions: Figure 1 A flow chart of a two-stage label noise learning classification method provided by an embodiment of the present invention; Figure 2 A schematic diagram of a two-stage label noise learning classification model training provided by an embodiment of the present invention; Figure 3 A comparison chart of experimental results of the label noise filtering algorithm provided by an embodiment of the present invention; Figure 4 A comparison chart of the experimental results of the sorting support vector machine provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0038] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0039] Example 1: The present invention provides a two-stage label noise learning classification method, such as Figure 1 The figure shows a flow chart of a two-stage label noise learning classification method provided by the present invention. Figure 2 The following is a diagram of a two-stage label noise learning classification model training process. S1: A relative similarity filter (RSF) is constructed using a comprehensive measure of relative similarity and Euclidean distance. The original dataset is input into the relative similarity filter, noise samples are filtered out by calculating the relative similarity of the samples, and the filtered dataset is output.

[0040] The specific process is: The observation labels of each sample in the original data set are used to construct the similar sample set and the heterogeneous sample set, and the similar neighbor set and heterogeneous neighbor set of each sample are determined based on the cosine similarity metric; The similarity between the target sample and the target sample is calculated by calculating the similarity between the target sample and the target sample, and the absolute similarity between the target sample and the target sample is obtained. The expression for degree is as follows:

[0041] Heterogeneous similarity The expression is as follows:

[0042] in, represents the i-th sample, and Represent the same sample data set and the heterogeneous sample data set respectively, represents the kth nearest neighbor sample, K represents the number of nearest neighbor samples, j is the sample feature number, d represents the number of sample features, Representation sample The value on the jth feature dimension, Representation sample The value on the jth feature dimension, Indicates the inner product of two samples; Calculate the relative similarity of samples based on the absolute similarity of the same type and the absolute similarity of different types , the expression is as follows:

[0043] Sort the relative similarities of all samples in ascending order, and calculate the average relative similarity of the sample set as the filtering threshold; The samples whose relative similarity is lower than the filtering threshold are determined as suspected label noise samples and are removed, and the filtered data set is output.

[0044] S2: The filtered dataset is input into a robust ranking support vector machine model based on a non-convex ramp loss function and a local label weighting strategy (LWR-RSVM) for training. The non-convex loss function is converted into a convex optimization problem through a convex-concave process (CCCP) to obtain the optimal model parameters.

[0045] The specific process is: Use the filtered dataset as input to generate positive and negative sample pairs, and use The nearest neighbor method calculates the positive and negative sample pairs ( ) ; The objective function of the model is constructed using positive and negative sample pairs and predicted labels. The expression is as follows:

[0046] Among them, i represents the sequence number of the positive sample pair, j represents the sequence number of the negative sample pair, represents the feature vector weight of the classification model, represents the regularization parameter, which is used to control the trade-off between the classification boundary interval and the classification error. represents the number of positive sample pairs, n represents the number of negative sample pairs, represents the slope loss function, Represents a sample pair ( , )’s noise labels, represents the predicted label, Represents a hyperparameter responsible for balancing noisy labels and predicted labels , by combining the noise labels with the predicted labels to reduce the impact of the noise labels, The larger the value, the higher the weight given to the original label of the sample. Represents a positive sample and negative samples The positive sample pairs are composed of Represents the model weight vector Compared with the positive sample The result of the inner product operation.

[0047] Since the ramp function is a non-convex function, decompose the ramp function into a convex function With concave function The objective function expression is as follows:

[0048]

[0049] in, is the convex part, The non-convex part.

[0050] The convex-concave process (CCCP) algorithm is combined with the Lagrange multiplier method to transform the optimization problem of the concave function into a dual problem. The dual problem is solved using the Karush-Kuhn-Tucker (KKT) condition to obtain the dual solution. The specific process is as follows: According to expression (7), each iteration of the convex-concave process is used to approximate the concave part with a tangent line, and the resulting convex function is minimized as follows:

[0051] in, Represent the parameters obtained by the kth and k+1th iterations respectively, Represents the feature vector weight of the classification model, The derivative of is defined as , the expression is as follows:

[0052] Among them, i represents the sequence number of the positive sample pair, j represents the sequence number of the negative sample pair, represents the number of positive sample pairs, n represents the number of negative sample pairs, Represents a sample pair ( , )’s noise labels, represents the predicted label, represents the hyperparameter, Represents a positive sample and negative samples The positive sample pairs, weight coefficient The expression is as follows:

[0053] Among them, s represents the upper limit threshold of the slope loss function, represents the regularization parameter, Represents the model weight vector Compared with the positive sample The result of the inner product operation of ; Rewrite the k+1th iterative optimization problem as follows with slack variables The convex optimization form of , using expressions (8) and (9) to analyze and simplify expression (7), we get the following expression:

[0054] Constructing the corresponding Lagrangian function for expression (10) yields the following expression:

[0055] in, and is the Lagrange multiplier, for the variables in Eq. (11) and Taking partial derivatives, we can derive the following Karush-Kuhn-Tucker (KKT) conditions:

[0056] Substituting expression (12) into expression (11) and simplifying it, we get the dual problem expression as follows:

[0057] in, Both represent the sequence number of positive sample pairs, Represents the negative sample pair number, Both represent positive sample pairs, Both represent weight coefficients, In the process of simplifying expression (12) into expression (11), since expression (11) contains , so the expression (12) When substituting into an expression, a distinction needs to be made, so new symbols appear, and the meaning of the symbols is the same as the original symbols; make , , ,in, represent 、 , represent 、 represent , the expression is as follows:

[0058] Substituting (14) into (13) yields the dual equation, which is expressed as follows:

[0059] Solve expression (15) to obtain the dual optimal solution ; Use the dual optimal solution to update the model weight coefficient. The specific process is: Using the dual optimal solution Calculate the parameters of the model , the expression is as follows:

[0060] Among them, i represents the sequence number of the positive sample pair, j represents the sequence number of the negative sample pair, represents the dual optimal solution, Represents a sample pair ( , )’s noise labels, represents the predicted label, represents the hyperparameter, Represents a positive sample and negative samples The positive sample pairs constituted by The parameters Expression (9) is used to obtain the updated weight coefficient; Repeat the above steps until the model weight coefficient changes below the set threshold, and output the optimal model parameters. In this embodiment, the threshold value is [0.001, 0.01].

[0061] S3: Using the optimal model parameters to update the model threshold, and using the updated model to classify the samples to be classified.

[0062] The specific process is: Constructing a scoring function using optimal model parameters ,in, Represents a sample, and calculates the minimum score value of all positive samples for all samples in the dataset and the maximum score of negative samples ; Based on the five-fold cross-validation strategy, the intermediate discriminant value is calculated in each fold, and the obtained , the expression is as follows:

[0063] Among them, i represents the fold number of i, represents the highest score of the negative sample in the ith fold, Represents the lowest score value of the positive sample in the i-th fold, averages the intermediate discriminant values ​​obtained, and determines the final discrimination threshold , the expression is as follows:

[0064] The score value obtained by the scoring function and the discrimination threshold For comparison, if , it is judged as positive; otherwise it is judged as negative. The expression of the prediction function is as follows:

[0065] Use the prediction function to classify the samples to be classified and output the classification results.

[0066] The present invention realizes efficient suppression of label noise by constructing a two-stage collaborative mechanism of relative similarity filter and robust sorting support vector machine model. The relative similarity filter integrates cosine similarity and Euclidean distance information, taking into account both feature direction and spatial distance, and can effectively identify and eliminate low-credibility samples in high-dimensional data, overcoming the problem of sudden drop in filtering accuracy of traditional absolute distance methods in high-dimensional space. The robust sorting support vector machine introduces a non-convex slope loss function and a local label weighting strategy to further weaken the interference of residual noise samples on the classification boundary at the model level, significantly improving the classification robustness. The overall solution takes into account both data-level screening and model-level noise suppression, improving the problem of unstable performance under a single processing path in traditional methods, and is particularly suitable for stable learning tasks in high-dimensional and high-noise scenarios.

[0067] To quantify the performance improvement of the relative similarity label noise filtering algorithm (RSF) proposed in this paper, four representative existing label noise filtering methods were selected: relative density (RD), multi-class completely random forest (mCRF), k-nearest neighbor graph neural network based on the EIRS module (EIRS-kGNN), and k-relative probability density (kRPD). The performance of these five filtering algorithms was systematically evaluated on four representative datasets, with noise rates ranging from 0.1 to 0.4. The relevant datasets are listed in Table 1.

[0068] Table 1

[0069] The recall rate (the higher the better) and the false positive rate (the lower the better) are used as evaluation indicators. Figure 3 As shown in the table, the two best-performing algorithms are highlighted in bold. Experimental results demonstrate the significant advantages of the RSF algorithm in the task of label noise filtering. In terms of recall, RSF performs well across multiple datasets and noise rates, maintaining particularly strong recall capabilities at high noise levels of 0.3-0.4. Taking Breast, Sonar, and Votes as examples: on the Breast dataset, RSF achieves recall exceeding 0.8 at all four noise rates, reaching a peak of 0.9956 at a noise rate of 0.1. On the Sonar dataset, RSF achieves a recall of 1.0000 at a noise rate of 0.1, nearly perfectly filtering out all noisy samples. On the Votes dataset, RSF also achieves a perfect recall of 1.0000 at a noise rate of 0.1. While the mCRF algorithm performs competitively on the Votes dataset, its stability in high-noise scenarios is significantly weaker than RSF. This is attributed to the random forest-based mCRF's susceptibility to overfitting to noisy data.

[0070] RSF also demonstrated excellent performance in the false positive rate metric, which measures the false positives of normal samples. Taking the Breast dataset as an example, RSF's false positive rate at a noise rate of 0.4 was only 0.3973, significantly outperforming other algorithms. On datasets such as Sonar and Votes, RSF's false positive rate remained consistently below 0.2, demonstrating its precise filtering capabilities. Although some algorithms achieve lower false positive rates in specific scenarios (such as mCRF's 0.3208 at a noise rate of 0.3 on the Breast dataset), their cross-dataset stability is significantly insufficient. In contrast, RSF exhibits more stable low false positive rates across different noise environments and datasets. Comprehensive analysis shows that RSF strikes a good balance between two key metrics: recall and false positive rate. The algorithm effectively filters out noisy samples while maximizing the retention of normal samples. Experiments also found that when the noise ratio exceeds 0.35, the recall of traditional algorithms such as kRPD and EIRS-kGNN generally decreases by 15%-20%, while that of RSF only decreases by 8%-12%, validating the effectiveness of its robust design. These characteristics make RSF an ideal choice for handling high-noise scenarios, particularly in practical applications where a balance between filtering efficiency and accuracy is crucial.

[0071] To quantify the performance improvement of our proposed robust ranking support vector machine (LWR-RSVM) based on a nonconvex slope loss function and local label weighting, we compared the classification performance of four existing high-performance models: the classical ranking support vector machine (Ranking SVM), the robust least squares ranking support vector machine (RLSSVM), the relabeled support vector machine (RESVM), and the Pinball-based graph-structured bilateral support vector machine (Pin-GBTSVM). To mitigate the effects of randomness, experimental results are the average of five runs. The "raw" and "filter" subscripts indicate whether RSF filtering was used, respectively. The two best algorithms are highlighted in bold.

[0072] The experimental results are as follows Figure 4As shown in the table, the LWR-RSVM algorithm performs well across all datasets and noise levels. LWR-RSVM achieves high classification accuracy in most cases, both on raw data and on RSF-filtered data. In particular, on the Sonar and Heart datasets, after RSF filtering, LWR-RSVM achieves optimal or near-optimal accuracy, demonstrating its strong generalization across different datasets. Analyzing the noise level dimension, LWR-RSVM exhibits remarkable stability, maintaining high classification accuracy even under high noise levels of 0.3-0.4. In contrast, algorithms such as RankingSVM and RLSSVM exhibit significant performance degradation in high-noise environments, with RLSSVM exhibiting the largest fluctuations, indicating its weaker stability against noisy data.

[0073] The RSF filtering algorithm significantly improves the performance of all classifiers. After RSF filtering, the performance of all five classifiers increased. With the exception of Pin-GBTSVM, which only saw a 1.21% improvement, the accuracy of the other classifiers increased by nearly 3%. This demonstrates that RSF effectively reduces the negative impact of noise, and the improved algorithm, LWR-RSVM, further improves classifier accuracy when processing RSF-filtered data. As a baseline model, Ranking SVM performs well at low noise levels of 0.1-0.2, but its accuracy decreases significantly as the noise level increases. Although RSF filtering improves its performance, it still lags behind advanced algorithms such as LWR-RSVM on multiple datasets, indicating that Ranking SVM lacks the robustness of LWR-RSVM when dealing with complex noise.

[0074] RLSSVM and RESVM exhibit relatively balanced performance characteristics. RLSSVM performs well at low noise levels, but its performance deteriorates significantly at high noise levels, such as 0.4. Although RSF filtering improves its accuracy, it still cannot surpass LWR-RSVM. RESVM performs well on some datasets, but overall its stability at high noise levels is inferior to LWR-RSVM, and the performance gap is particularly significant at a noise level of 0.4. Pin-GBTSVM exhibits instability at high noise levels, with large fluctuations in accuracy, which may be related to its sensitivity to noise. Although RSF filtering improves its accuracy, it is still insufficient compared to other algorithms.

[0075] It's important to note that RSF has limitations in certain scenarios. For example, in the Votes dataset, due to overly simple data features, RSF's false negative rate remains consistently high. This suggests that RSF filters out too many normal samples, causing the classifier to lose a large number of representative samples during training, which in turn leads to reduced classification performance. This phenomenon suggests that when applying RSF, adaptive threshold adjustment should be performed based on the dataset's feature dimensions.

[0076] In summary, LWR-RSVM demonstrates superior classification performance under both low-noise and high-noise conditions. In particular, its classification accuracy is further improved on the RSF-filtered dataset, fully demonstrating its noise resistance and stability. This demonstrates that LWR-RSVM has significant advantages in handling label noise, and the introduction of the RSF filtering algorithm effectively improves classifier accuracy, providing strong support for LWR-RSVM.

[0077] Example 2: This embodiment provides a two-stage label noise learning and classification system, including a memory and a processor, wherein the memory includes a two-stage label noise learning and classification method program, and when the two-stage label noise learning and classification method program is executed by the processor, the steps of a two-stage label noise learning and classification method as described in Example 1 are implemented.

[0078] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A two-stage label noise learning classification method, characterized by: The following steps are involved: A relative similarity filter is constructed using a comprehensive measure of relative similarity and Euclidean distance, the original data set is input into the relative similarity filter, noise samples are filtered out by calculating the relative similarity of the samples, and the filtered data set is output; Inputting the filtered data set into a robust sorting support vector machine model based on a non-convex slope loss function and a local label weighting strategy for training, converting the non-convex loss function into a convex optimization problem through a convex-concave process for solving, and obtaining the optimal model parameters; The model threshold is updated using the optimal model parameters, and the updated model is used to classify the samples to be classified.

2. A two-stage label noise learning classification method according to claim 1, characterized in that: Inputting the original data set into the relative similarity filter and outputting the filtered data set comprises the following steps: The observation labels of each sample in the original data set are used to construct the similar sample set and the heterogeneous sample set, and the similar neighbor set and heterogeneous neighbor set of each sample are determined based on the cosine similarity metric; The similarity between the target sample and the target sample is calculated by calculating the similarity between the target sample and the target sample, and the absolute similarity between the target sample and the target sample is obtained. The expression of the absolute similarity between the target sample and the target sample is as follows: Heterogeneous similarity The expression is as follows: in, represents the i-th sample, and Represent the same sample data set and the heterogeneous sample data set respectively, represents the kth nearest neighbor sample, K represents the number of nearest neighbor samples, j is the sample feature number, d represents the number of sample features, Representation sample The value on the jth feature dimension, Representation sample The value on the jth feature dimension, Indicates the inner product of two samples; Calculate the relative similarity of samples based on the absolute similarity of the same type and the absolute similarity of different types , the expression is as follows: Sort the relative similarities of all samples in ascending order, and calculate the average relative similarity of the sample set as the filtering threshold; The samples whose relative similarity is lower than the filtering threshold are determined as suspected label noise samples and are removed, and the filtered data set is output.

3. A two-stage label noise learning classification method according to claim 1, characterized in that: The optimal model parameters are solved using a robust ranking support vector machine model based on a non-convex slope loss function and a local label weighting strategy, which includes the following steps: Use the filtered dataset as input to generate positive and negative sample pairs and calculate the predicted labels of the positive and negative sample pairs ; Use positive and negative sample pairs and predicted labels to construct the objective function of the model; Decompose the slope loss function in the objective function into convex function and concave function; The convex-concave process algorithm is combined with the Lagrange multiplier method to transform the optimization problem of convex functions into a dual problem, and the dual problem is solved using the Karush-Kuhn-Tucker condition to obtain the dual solution. Use the dual solution to update the model weight coefficients; Repeat the above steps until the model weight coefficient changes below the set threshold, and output the optimal model parameters.

4. A two-stage label noise learning classification method according to claim 3, characterized in that: The method for calculating the predicted labels of positive and negative sample pairs is Nearest neighbor method.

5. A two-stage label noise learning classification method according to claim 3, characterized in that: The expression of the objective function of the model constructed using positive and negative sample pairs and predicted labels is as follows: Among them, i represents the sequence number of the positive sample pair, j represents the sequence number of the negative sample pair, represents the feature vector weight of the classification model, represents the regularization parameter, which is used to control the trade-off between the classification boundary interval and the classification error. represents the number of positive sample pairs, n represents the number of negative sample pairs, represents the slope loss function, Represents a sample pair ( , )’s noise labels, represents the predicted label, Represents a hyperparameter responsible for balancing noisy labels and predicted labels , Represents a positive sample and negative samples The positive sample pairs are composed of Represents the model weight vector Compared with the positive sample The result of the inner product operation.

6. A two-stage label noise learning classification method according to claim 5, characterized in that: The slope loss function is a non-convex function, and the slope loss function is decomposed into a convex function With concave function The objective function expression is as follows: in, is the convex part, The non-convex part.

7. A two-stage label noise learning classification method according to claim 6, characterized in that: The convex-concave process (CCCP) algorithm is combined with the Lagrange multiplier method to transform the optimization problem of the convex function into a dual problem and obtain the dual solution, which includes the following steps: According to expression (7), each iteration of the convex-concave process is used to approximate the concave part with a tangent line, and the resulting convex function is minimized as follows: in, Represent the parameters obtained by the kth and k+1th iterations respectively, Represents the feature vector weight of the classification model, The derivative of is defined as , the expression is as follows: Among them, i represents the sequence number of the positive sample pair, j represents the sequence number of the negative sample pair, represents the number of positive sample pairs, n represents the number of negative sample pairs, Represents a sample pair ( , )’s noise labels, represents the predicted label, represents the hyperparameter, Represents a positive sample and negative samples The positive sample pairs, weight coefficient The expression is as follows: Among them, s represents the upper limit threshold of the slope loss function, represents the regularization parameter, Represents the model weight vector Compared with the positive sample The result of the inner product operation of ; Rewrite the k+1th iterative optimization problem as follows with slack variables The convex optimization form of , using expressions (8) and (9) to analyze and simplify expression (7), we get the following expression: Constructing the corresponding Lagrangian function for expression (10) yields the following expression: in, and is the Lagrange multiplier, for the variables in Eq. (11) and Taking partial derivatives, we can derive the following Karush-Kuhn-Tucker conditions: Substituting expression (12) into expression (11) and simplifying it, we get the dual problem expression as follows: in, Both represent the sequence number of positive sample pairs, Both represent the negative sample pair numbers, Both represent positive sample pairs, Both represent weight coefficients; make , , ,in, represent 、 , represent 、 represent , the expression is as follows: Substituting (14) into (13) yields the dual equation, which is expressed as follows: Solve expression (15) to obtain the dual optimal solution .

8. A two-stage label noise learning classification method according to claim 7, characterized in that: The method for updating the model weight coefficient using the dual solution includes the following steps: Using the dual optimal solution Calculate the parameters of the model , the expression is as follows: Among them, i represents the sequence number of the positive sample pair, j represents the sequence number of the negative sample pair, represents the dual optimal solution, Represents a sample pair ( , )’s noise labels, represents the predicted label, represents the hyperparameter, Represents a positive sample and negative samples The positive sample pairs constituted by The parameters Expression (9) is used to obtain the updated weight coefficient.

9. A two-stage label noise learning classification method according to claim 1, characterized in that: Updating the model threshold using the optimal model parameters and classifying the samples to be classified using the updated model includes the following steps: Constructing a scoring function using optimal model parameters ,in, Represents a sample, and calculates the minimum score value of all positive samples for all samples in the dataset and the maximum score of negative samples ; Based on the five-fold cross-validation strategy, the intermediate discriminant value is calculated in each fold, and the obtained , the expression is as follows: Among them, i represents the fold number of the i-th fold, represents the highest score of the negative sample in the ith fold, Represents the lowest score value of the positive sample in the i-th fold, averages the intermediate discriminant values ​​obtained, and determines the final discrimination threshold , the expression is as follows: The score value obtained by the scoring function and the discrimination threshold For comparison, if , it is judged as positive; otherwise it is judged as negative. The expression of the prediction function is as follows: Use the prediction function to classify the samples to be classified and output the classification results.

10. A two-stage label noise learning classification system, characterized in that The system includes: a memory and a processor, wherein the memory includes a two-stage label noise learning classification method program, and when the two-stage label noise learning classification method program is executed by the processor, the steps of a two-stage label noise learning classification method as described in any one of claims 1 to 9 are implemented.

Images