Sample weight real-time adaptive statistical adjustment method for classification model

Abstract

The invention discloses a sample weight real-time adaptive statistical adjustment method for a classification model, and the method comprises the following steps: obtaining the sample data of a current training batch, inputting the sample data into a main classification model, processing the sample data through the main classification model, and generating prediction result data; the real-time statistical tracking module receives the prediction result data and the sample data, and updates a sample level statistical magnitude and a batch level statistical magnitude based on the prediction result data; the weight decision network receives the sample level statistics and the batch level statistics, and calculates a real-time weight value of each sample; updating the parameters of the main classification model by using a weighted loss function, and updating the parameters of the main classification model through a back propagation algorithm; updating parameters of the weight decision network according to updating feedback of the main classification model; therefore, the feature learning of minority class samples and difficult samples can be continuously optimized, and the classification balance among the classes is remarkably improved while the overall high accuracy is kept.

Description

Technical Field

[0001] This invention relates to a sample weight adjustment method, and more particularly to a real-time adaptive statistical adjustment method for sample weights in classification models. Background Technology

[0002] In machine learning classification tasks, the allocation of sample weights is a key factor affecting model performance. An ideal sample weighting strategy can guide the model to focus on samples with more information, thereby improving generalization ability. However, existing data often involves complex problems such as class imbalance, label noise, and dynamic changes in data distribution, which makes it difficult for traditional classification algorithms that assume all samples contribute equally to achieve ideal results. Summary of the Invention

[0003] This invention overcomes the shortcomings of the prior art and provides a method for real-time adaptive statistical adjustment of sample weights for classification models.

[0004] To achieve the above objectives, the technical solution adopted by this invention is: a real-time adaptive statistical adjustment method for sample weights in a classification model, comprising the following steps:

[0005] S1. Obtain the sample data of the current training batch and input the sample data into the main classification model. The main classification model processes the sample data and generates prediction result data.

[0006] S2. The real-time statistical tracking module receives the prediction result data and sample data, and updates the sample-level statistics and batch-level statistics based on the prediction result data.

[0007] S3. The weighted decision network receives the sample-level statistics and batch-level statistics, and calculates the real-time weight value for each sample.

[0008] S4. Update the parameters of the main classification model using a weighted loss function, and update the parameters of the main classification model using a backpropagation algorithm;

[0009] S5. Update the parameters of the weight decision network based on the update feedback of the main classification model.

[0010] In a preferred embodiment of the present invention, in S2, the process by which the real-time statistical tracking module updates the sample-level statistics and batch-level statistics includes:

[0011] S201. The current loss value of each sample is calculated using the cross-entropy loss function, wherein the current loss value is obtained by comparing the prediction result data and the sample data;

[0012] S202. Update the historical loss value sequence for each sample, add the current loss value to the historical loss value sequence of the corresponding sample, and maintain a sequence window of fixed length.

[0013] S203. Calculate the prediction consistency index and prediction uncertainty index for each sample; the prediction consistency index is obtained by calculating the cosine similarity between the prediction probability distribution of the current iteration and the prediction probability distribution of the previous iteration; the prediction uncertainty index is obtained by calculating the entropy of the prediction probability distribution of the current iteration or by calculating the complement of the maximum prediction probability.

[0014] S204. Calculate the sample quantity distribution of each category in the current batch. The sample quantity distribution is obtained by counting the sample quantity of each category and normalizing it. Calculate the average loss value of the current batch and update the batch-level statistics.

[0015] S205. Store the updated sample-level and batch-level statistics in memory and provide real-time data input for S3.

[0016] In a preferred embodiment of the present invention, in S3, the process of the weight decision network calculating the real-time weight value includes:

[0017] S301. Construct a multi-dimensional input feature vector that includes sample-level features and batch-level features;

[0018] S302. The multi-dimensional input feature vector is input into a weighted decision network, which includes an input layer, multiple hidden layers, and an output layer. The hidden layers use activation functions for non-linear transformations, and the output layer uses a sigmoid function to map the output value to... The interval is used to obtain the initial weight value for each sample;

[0019] S303. The initial weight values ​​are normalized by dividing the initial weight value of each sample by the sum of the initial weight values ​​of all samples and then multiplying by the batch size.

[0020] S304. Output the normalized real-time weight values ​​to S4 and calculate the weighted loss function.

[0021] In a preferred embodiment of the present invention, in S4, the process of updating the parameters of the main classification model using a weighted loss function includes:

[0022] S401. The loss value for each sample is calculated using the cross-entropy loss function, and the loss value is obtained by comparing the prediction result data and the sample data;

[0023] S402. The loss value of each sample is weighted according to the real-time weight value to obtain the weighted loss value;

[0024] S403. Sum the weighted loss values ​​of all samples to obtain the total weighted loss;

[0025] S404. Calculate the gradient of the weighted total loss with respect to the parameters of the main classification model using the backpropagation algorithm, and update the parameters of the main classification model using an optimization algorithm;

[0026] S405. The weighted total loss and parameter gradient are transmitted as update feedback to S5 to update the parameters of the weighted decision network.

[0027] In a preferred embodiment of the present invention, in S5, the process of updating the weight decision network parameters includes:

[0028] S501, Receive update feedback from S4 including the rate of change of the weighted total loss and the parameter gradient of the main classification model;

[0029] S502: Calculate the loss function of the weighted decision network, the loss function being based on the rate of change of the weighted total loss and the norm of the gradient of the main classification model parameters, and the product of the negative value of the rate of change of the weighted total loss and the norm of the parameter gradient is used as the loss value;

[0030] S503. Calculate the gradient of the loss function with respect to the weight decision network parameters using the backpropagation algorithm, and apply a conditional gradient stopping mechanism during the backpropagation process; the conditional gradient stopping mechanism dynamically determines whether to allow the gradient to continue backpropagating to some of the lower network layers of the main classification model based on the smoothness of the performance improvement of the main classification model.

[0031] S504. Update the weight decision network parameters using a stochastic gradient descent optimization algorithm with a smaller learning rate;

[0032] S505. Store the updated weight decision network parameters for weight calculation in the next training iteration.

[0033] In a preferred embodiment of the present invention, the adaptive target weight vector corresponds at least to a class balance target, a hard sample mining target, and a noise robustness target; when calculating the real-time weight value, the weight decision network weights and fuses the input features according to the target category with the corresponding coefficients in the adaptive target weight vector.

[0034] For the category balance objective, the weight decision network dynamically adjusts the weights based on the sample size distribution in the batch-level statistics;

[0035] For the target of difficult sample mining, the weight decision network identifies samples with consistently high loss and high prediction consistency as difficult samples based on the historical loss value sequence and prediction consistency index in the sample-level statistics, and increases the weight value to strengthen the classification boundary learning; for the target of noise robustness, the weight decision network identifies samples with low prediction consistency and large loss fluctuations as noise samples based on the prediction consistency index and volatility of the historical loss value sequence in the sample-level statistics; the multi-objective weight optimization strategy learns the balanced weights between different objectives through the hidden layer of the weight decision network to achieve adaptive weight allocation.

[0036] In a preferred embodiment of the present invention, the real-time statistical tracking module and the weight decision network adopt an alternating collaborative training mechanism with the main classification model; the alternating collaborative training mechanism includes: when the main classification model is updated, the parameters of the weight decision network remain fixed; when the weight decision network is updated, the parameters of the main classification model remain fixed; the alternating collaborative training is implemented through a dual time scale update rule, with the main classification model adopting a regular update rhythm and the weight decision network adopting a smaller learning rate update.

[0037] In a preferred embodiment of the present invention, the update frequency of the sample-level statistics and the batch-level statistics is synchronized with the training iteration; wherein, the sample-level statistics are updated after each training iteration, using the prediction results and sample data of the current batch; the batch-level statistics are updated after each training iteration, based on all sample data of the current batch; the real-time statistical tracking module maintains historical data through a sliding window mechanism.

[0038] In a preferred embodiment of the present invention, the weight decision network is a lightweight neural network, comprising an input layer, two hidden layers, and an output layer; wherein the number of nodes in the input layer is equal to the dimension of the multi-dimensional input feature vector; the number of nodes in the hidden layers are 64 and 32, respectively, and the ReLU activation function is used; the number of nodes in the output layer is 1, and the Sigmoid activation function is used.

[0039] In a preferred embodiment of the present invention, the real-time statistical tracking module updates the statistics using an incremental calculation method; the calculation of the weight decision network is performed in parallel with the forward propagation of the main classification model; and the weighted loss function calculation adopts vectorized operation and utilizes GPU acceleration processing.

[0040] This invention addresses the shortcomings of the prior art and has the following beneficial effects:

[0041] (1) This invention can sense changes in data distribution in real time and dynamically adjust sample weights, enabling the model to continuously optimize feature learning for minority class samples and difficult samples, thereby significantly improving the classification balance between categories while maintaining high overall accuracy. At the same time, by distinguishing between highly consistent difficult samples and low-consistency noisy samples in real time, it effectively suppresses the overfitting of the model to labeled noise and learns a more reasonable classification boundary.

[0042] (2) By introducing a real-time statistical tracking module and a weight decision network, and conducting alternating collaborative training with the main classification model, the traditional static or lagging weight allocation mode is changed. The weight decision network can generate the optimal sample weights in real time in each training iteration based on the multi-dimensional statistical features calculated from the latest model state, ensuring a close match between the weight strategy and the current training stage, and solving the technical problem that existing technologies cannot respond to training dynamics.

[0043] (3) This invention effectively controls the additional computational burden while ensuring the accuracy of weight adjustment by using a lightweight dual-path weight decision network and an alternating collaborative training mechanism. The forward propagation of the weight decision network can be parallelized with the main model computation, and the statistical tracking adopts incremental updates, avoiding complex meta-optimization or repeated training processes. This enables the invention to be efficiently applied to large-scale datasets and real-world industrial scenarios with limited computing resources.

[0044] (4) Through the multi-objective weight optimization strategy of the weight decision network, the three key objectives of class balance, difficult sample mining and noise robustness are uniformly and adaptively taken into account. Unlike traditional methods that only target a single problem, this invention enables the model to deal with multiple complex problems intertwined in real data at the same time, and intelligently balances the influence of each objective on the weights at different training stages, thereby greatly improving the overall stability and adaptability of the model in complex and non-ideal data environments. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart of a preferred embodiment of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein. Therefore, the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0049] like Figure 1 As shown, the real-time adaptive statistical adjustment method for sample weights in a classification model includes the following steps:

[0050] S1. Obtain the sample data of the current training batch and input the sample data into the main classification model. The main classification model processes the sample data and generates prediction result data.

[0051] S2. The real-time statistical tracking module receives prediction result data and sample data, and updates sample-level statistics and batch-level statistics based on the prediction result data.

[0052] Furthermore, in S2, the process by which the real-time statistical tracking module updates sample-level and batch-level statistics includes:

[0053] S201. The current loss value for each sample is calculated using the cross-entropy loss function. The current loss value is obtained by comparing the prediction result data with the sample data.

[0054] For a given sample, its loss value The calculation formula is as follows:

[0055] ;

[0056] in:

[0057] This represents the i-th sample in the batch.

[0058] This indicates the total number of categories.

[0059] It is the label of the true label of sample i in category c.

[0060] This is the prediction result data output by the main classification model, representing the predicted probability that sample i belongs to category c.

[0061] S202. Update the historical loss value sequence for each sample, add the current loss value to the historical loss value sequence of the corresponding sample, and maintain a sequence window of fixed length.

[0062] S203. Calculate the prediction consistency index and prediction uncertainty index for each sample; the prediction consistency index is obtained by calculating the cosine similarity between the prediction probability distribution of the current iteration and the prediction probability distribution of the previous iteration; the prediction uncertainty index is obtained by calculating the entropy of the prediction probability distribution of the current iteration or by calculating the complement of the maximum prediction probability.

[0063] Predictive Consistency Index The cosine similarity is obtained by calculating the predicted probability distribution between the current iteration t and the previous iteration t-1, as shown in the following formula:

[0064] ;

[0065] in:

[0066] and These represent the predicted probability distribution vectors of sample i in the t-th and t-1-th iterations, respectively.

[0067] “ "" represents the dot product of vectors. This represents the L2 norm of a vector. The range of values ​​is A larger value indicates that the model's prediction for that sample is more stable.

[0068] In another embodiment, the calculation of the prediction uncertainty index provides two mathematically and semantically complementary paths:

[0069] Entropy uncertainty: Based on information theory, the Shannon entropy of the predicted probability distribution of the model is calculated. Its mathematical expression is: This metric measures uncertainty from the perspective of overall distribution: entropy is highest when the probability is evenly distributed across all categories, indicating that the model is completely unable to determine the sample's classification; entropy is lowest when the probability is concentrated in a single category, indicating that the model is very certain. Entropy can sensitively capture the state where the model is "hesitant" about multiple categories simultaneously.

[0070] Confidence level uncertainty: calculated as This metric directly focuses on how well the model grasps the most likely class. Compared to entropy uncertainty, it places more emphasis on whether the "winner's" lead is significant. For example, in a three-class classification problem, the probability distribution... It exhibits high entropy uncertainty (because the probabilities of the two classes are close), but its confidence uncertainty is also high (because the maximum probability is only 0.49); while the distribution... Then both types of uncertainty are low.

[0071] In practice, the system can choose one or both indicators. When using both, a comprehensive uncertainty measure can be formed through a simple linear combination or by automatically learning the importance weights of the two indicators via a weighted decision network.

[0072] By combining the uncertainty index with the original loss value and prediction consistency index, a systematic sample classification and processing strategy matrix is ​​formed:

[0073] Sample type features Technical indicators Sample value judgment Weighting strategy Boundary difficulty samples High loss, high consistency, low uncertainty The model's judgments are stable but consistently wrong, remaining near the decision boundary, making it highly valuable for learning. Significantly increase the weights, forcing the model to invest more resources in learning, in order to drive boundary optimization. Potential noise samples High loss, low consistency, high uncertainty The model's wavering judgments and lack of confidence are likely due to mislabeling or feature anomalies. Significantly reduce weights to prevent the model from overfitting to erroneous signals and improve robustness. Typical easily distinguishable samples Low loss, high consistency, low uncertainty The model can reliably and confidently classify correctly, with clear and well-defined features. Maintain moderate or low weights to avoid the model "overlearning" on these samples and thus reducing efficiency. Fuzzy but easily distinguishable samples Low loss, low consistency, medium to high uncertainty The model can classify correctly but lacks confidence; the features may be atypical but belong to the same category. Applying appropriate weights helps the model learn intra-class diversity and improves its generalization ability. Emerging Model Samples Losses decreased from high to low, consistency increased from low to high, and uncertainty decreased from high to low. The representative model is learning new patterns, and its value changes over time. The weights are dynamically adjusted, with higher weights given in the early stages to accelerate learning, and then gradually normalized in the later stages.

[0074] This analytical framework enables the decision-making process of weighted decision networks to move from heuristic to systematic, providing the network with clear learning objectives.

[0075] Traditional methods primarily rely on loss values ​​or consistency to distinguish noise, but both have limitations: high-loss samples may be difficult samples rather than noise; samples with low consistency may be new patterns being learned. By incorporating an uncertainty metric, the system can more accurately identify true noise: samples simultaneously exhibiting high loss, low consistency, and high uncertainty are highly likely to be mislabeled or have corrupted features. Experiments show that this triple filtering mechanism can reduce the misclassification rate of noisy samples by more than 40%, significantly improving the model's performance in noisy data environments.

[0076] In traditional hard sample mining, high-loss samples are generally considered "hard," but these may contain noise and true boundary samples. This invention uses high consistency and low uncertainty to filter out truly hard boundary samples from high-loss samples. These samples are often located near the class decision boundary, and focusing on them can most effectively drive boundary optimization and improve the model's classification accuracy.

[0077] Predictive uncertainty essentially reflects the model's confidence in its predictions. By incorporating uncertainty into weight decisions, the system effectively introduces an implicit goal of confidence calibration during training: the model is encouraged to be more confident in its correct predictions (reducing uncertainty) while being more cautious about its incorrect predictions (increasing uncertainty). This helps alleviate the "overconfidence" problem often observed in deep neural networks and improves the reliability of the model's predictive probabilities.

[0078] S204. Calculate the sample size distribution of each category in the current batch. The sample size distribution is obtained by counting the sample size of each category and normalizing it. Also, calculate the average loss value of the current batch and update the batch-level statistics.

[0079] S205. Store the updated sample-level and batch-level statistics in memory and provide real-time data input for S3.

[0080] Furthermore, the real-time statistical tracking module and the weight decision network adopt an alternating collaborative training mechanism with the main classification model. The alternating collaborative training mechanism includes: when the main classification model is updated, the parameters of the weight decision network remain fixed; when the weight decision network is updated, the parameters of the main classification model remain fixed. The alternating collaborative training is implemented through a dual time scale update rule, with the main classification model adopting a regular update rhythm and the weight decision network adopting a smaller learning rate update.

[0081] The real-time statistical tracking module updates statistics using incremental calculation; the calculation of the weighted decision network is performed in parallel with the forward propagation of the main classification model; the weighted loss function calculation adopts vectorized operation and utilizes GPU acceleration.

[0082] S3. The weighted decision network receives sample-level statistics and batch-level statistics, and calculates the real-time weight value for each sample.

[0083] Furthermore, in S3, the process of the weighted decision network calculating real-time weight values ​​includes:

[0084] S301. Construct a multi-dimensional input feature vector that includes sample-level features and batch-level features;

[0085] Construct a feature vector for each sample The vector is composed of the following parts:

[0086] ;

[0087] Sample-level features are extracted from sample-level statistics, including:

[0088] Mean of the historical loss sequence: .

[0089] Variance of historical loss series: .

[0090] Trend slope of historical loss sequence: via linear regression The coefficients obtained from the fitting .

[0091] Predictive Consistency Index .

[0092] Batch-level features are replicated from batch-level statistics, including:

[0093] Normalized class number distribution vector .

[0094] average batch loss ,in This refers to the batch size.

[0095] S302. Input the multi-dimensional input feature vector into the weight decision network. The weight decision network includes an input layer, multiple hidden layers, and an output layer. The hidden layers use activation functions for non-linear transformations, and the output layer uses the sigmoid function to map the output values ​​to... The interval is used to obtain the initial weight value for each sample;

[0096] eigenvectors The forward propagation process of the input weight decision network can be represented as follows:

[0097] ;

[0098] ;

[0099] ;

[0100] in:

[0101] and These are the weight matrices and bias terms for each layer of the network.

[0102] It is an activation function.

[0103] It's the Sigmoid function, ensuring the output... exist Within the range.

[0104] S303. Normalize the initial weight values ​​by dividing the initial weight value of each sample by the sum of the initial weight values ​​of all samples and then multiplying by the batch size.

[0105] To prevent fluctuations in the overall weight magnitude and maintain relative relationships, the initial weights of all samples within a batch are normalized to obtain the real-time weight value for each sample. :

[0106] ;

[0107] in This is the batch size. This operation ensures... .

[0108] S304. Output the normalized real-time weight values ​​to S4 and calculate the weighted loss function.

[0109] Furthermore, the adaptive target weight vector corresponds at least to the class balance target, the hard sample mining target, and the noise robustness target; when calculating the real-time weight value, the weight decision network weights and fuses the input features according to the target category with the corresponding coefficients in the adaptive target weight vector.

[0110] For the category balance objective, the weight decision network dynamically adjusts the weights based on the sample size distribution in the batch-level statistics.

[0111] For the target of difficult sample mining, the weighted decision network identifies samples with consistently high loss and high prediction consistency as difficult samples based on the historical loss value sequence and prediction consistency index in the sample-level statistics, and increases the weight value to strengthen the learning of classification boundaries.

[0112] For the noise robustness objective, the weight decision network identifies samples with low prediction consistency and large loss fluctuations as noise samples based on the prediction consistency index in the sample-level statistics and the volatility of the historical loss value sequence. The multi-objective weight optimization strategy learns the balanced weights among different objectives through the hidden layer of the weight decision network to achieve adaptive weight allocation.

[0113] The update frequency of sample-level statistics and batch-level statistics is synchronized with the training iterations; sample-level statistics are updated after each training iteration, using the prediction results and sample data of the current batch; batch-level statistics are updated after each training iteration, based on all sample data of the current batch; the real-time statistical tracking module maintains historical data through a sliding window mechanism.

[0114] In another embodiment, suppose the weighted decision network needs to balance K optimization objectives, and ATWV is represented as a learnable parameter vector. During training, ATWV is updated along with other parameters of the weight decision network via gradient descent, but its update signal originates from a carefully designed meta-objective: maximizing the overall improvement of the main model across multiple dimensions.

[0115] ATWV standardization is achieved through the Softmax function:

[0116] ;

[0117] in Temperature is a parameter that controls the sharpness of the weight distribution. Lower... A higher value will make the weight distribution more extreme (dominated by a certain objective), and a higher value will result in a more extreme weight distribution. The value makes the distribution smoother (multi-objective equilibrium).

[0118] To achieve feature processing for target perception, the system designs a feature assignment matrix at the input of the weighted decision network. This matrix maps different dimensions of the input feature vector to the corresponding optimization objective. Specifically, this can be accomplished through a learnable attention mechanism:

[0119] ;

[0120] in It is a with A binary mask or soft attention vector of the same dimension identifies which feature dimensions primarily serve the k-th target; This indicates element-wise multiplication. It can be predefined (based on feature semantics) or used as a learnable parameter.

[0121] S4. Update the parameters of the main classification model using a weighted loss function, and then update the parameters of the main classification model using the backpropagation algorithm.

[0122] Furthermore, in S4, the process of updating the main classification model parameters using a weighted loss function includes:

[0123] S401. The cross-entropy loss function is used to calculate the loss value for each sample. The loss value is obtained by comparing the prediction result data and the sample data.

[0124] S402. Weight the loss value of each sample according to the real-time weight value to obtain the weighted loss value;

[0125] S403. Sum the weighted loss values ​​of all samples to obtain the total weighted loss;

[0126] Weighted total loss It is the final objective function that guides the update of the main model, and its calculation formula is:

[0127] ;

[0128] This formula reflects the sample weights. Its core role is to amplify the high The loss contribution, while suppressing unimportant or low . contributions.

[0129] S404. Calculate the gradient of the weighted total loss with respect to the parameters of the main classification model using the backpropagation algorithm, and update the parameters of the main classification model using the optimization algorithm.

[0130] S405. The weighted total loss and parameter gradient are transmitted as update feedback to S5 to update the parameters of the weight decision network.

[0131] S5. Update the parameters of the weight decision network based on the update feedback from the main classification model.

[0132] Furthermore, in S5, the process of updating the weight decision network parameters includes:

[0133] S501, Receive update feedback from S4 including the rate of change of the weighted total loss and the parameter gradient of the main classification model;

[0134] S502: Calculate the loss function of the weighted decision network. The loss function is based on the rate of change of the weighted total loss and the norm of the gradient of the main classification model parameters. The product of the negative value of the rate of change of the weighted total loss and the norm of the parameter gradient is used as the loss value.

[0135] Optimization objective of weighted decision networks Defined as:

[0136]

[0137] in:

[0138] and These are the weighted total losses of the previous iteration and the current iteration, respectively.

[0139] It approximates the rate at which the loss decreases (assuming a positive rate of decrease).

[0140] It is the norm of the gradient vector of the main classification model parameters, representing the magnitude of the model update.

[0141] Taking a negative sign means that the training of the weight network is to maximize This amount, namely, encourages the weight allocation strategy to simultaneously cause the main model loss to decrease rapidly and the update direction to be clear.

[0142] S503. Calculate the gradient of the loss function with respect to the weight decision network parameters using the backpropagation algorithm, and apply a conditional gradient stopping mechanism during the backpropagation process; the conditional gradient stopping mechanism dynamically determines whether to allow the gradient to continue backpropagating to some of the lower network layers of the main classification model based on the smoothness of the performance improvement of the main classification model.

[0143] In one embodiment, the system implements a three-level gradient routing control:

[0144] Level 0 (Complete Blocking): The default state, following standard alternating training, where the gradients of the main model and the weight network do not interfere with each other.

[0145] Level 1 (Partial Penetration): When it is detected that the main model learning has entered a plateau period ( (And for a period of time), allowing the gradients of the weighted network to backpropagate to the last 12 fully connected layers of the main model, slightly adjusting the high-level feature representations.

[0146] Level 2 (Depth Guidance): Only when a clear loss of optimization direction is detected (multiple metrics deteriorate simultaneously) is the gradient allowed to penetrate further into the terminal convolutional or attention layers of the feature extractor, guiding feature extraction to focus more on the currently important sample types.

[0147] To prevent gradient routing from causing training instability, the system has implemented multiple protection measures:

[0148] 1. Gradient clipping: Strict norm clipping of the permeability gradient.

[0149] 2. Learning rate decay: The learning rate of the penetration gradient is decayed additionally (e.g., multiplied by a coefficient of 0.1).

[0150] 3. Time Limit: After each gradient routing is enabled, it will continue for a maximum of a fixed number of iterations, and then be forced to return to level 0.

[0151] Weighted network update probability The calculation is a multi-factor decision-making process:

[0152] ;

[0153] in: The function processes the weighted total loss rate of change, returning a positive value when the loss decreases rapidly to encourage updates. The function processes the gradient norm of the main model, reflecting the activity of model updates; The function processes the entropy of the sample weight distribution, reflecting the stability of the weight distribution; The coefficients can be optimized through meta-learning or dynamically adjusted according to the training phase.

[0154] The system automatically adjusts the update strategy based on the training phase:

[0155] High-frequency update phase: In the early stages of training (the first 20% of rounds), a higher base update probability is set to quickly adapt to the initial training dynamics.

[0156] Balancing update phase: Mid-training (20%-80% of rounds), dynamically adjusted based on the above decision model, with an average update frequency of approximately 50%-70%.

[0157] Low-frequency update phase: In the later stages of training (the last 20% of epochs), the probability of basic updates is gradually reduced, focusing on fine-tuning the model rather than making significant changes to the weight strategy.

[0158] When deciding to skip weight network updates, the system does not simply reuse the old weights, but instead applies a smooth weight extrapolation:

[0159] ;

[0160] in The weight estimates are generated by a very lightweight (such as a single-layer perceptron) fast estimation network based on the current statistics. This is a smoothing coefficient. This mechanism ensures that even without updating the entire network, the weights can adequately reflect the current training state.

[0161] S504. Update the weight decision network parameters using a stochastic gradient descent optimization algorithm with a smaller learning rate;

[0162] S505. Store the updated weight decision network parameters for weight calculation in the next training iteration.

[0163] The weighted decision network is a lightweight neural network consisting of an input layer, two hidden layers, and an output layer. The number of nodes in the input layer is equal to the dimension of the multi-dimensional input feature vector. The number of nodes in the hidden layers are 64 and 32, respectively, and they use the ReLU activation function. The number of nodes in the output layer is 1, and it uses the Sigmoid activation function.

[0164] Based on the preferred embodiments of the present invention described above, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A method for real-time adaptive statistical adjustment of sample weights for classification models, characterized in that, Includes the following steps: S1. Obtain the sample data of the current training batch and input the sample data into the main classification model. The main classification model processes the sample data and generates prediction result data. S2. The real-time statistical tracking module receives the prediction result data and sample data, and updates the sample-level statistics and batch-level statistics based on the prediction result data. S3. The weighted decision network receives the sample-level statistics and batch-level statistics, and calculates the real-time weight value for each sample. S4. Update the parameters of the main classification model using a weighted loss function, and update the parameters of the main classification model using a backpropagation algorithm; S5. Update the parameters of the weight decision network based on the update feedback of the main classification model.

2. The real-time adaptive statistical adjustment method for sample weights in a classification model according to claim 1, characterized in that: In S2, the process by which the real-time statistical tracking module updates the sample-level statistics and batch-level statistics includes: S201. The current loss value of each sample is calculated using the cross-entropy loss function, wherein the current loss value is obtained by comparing the prediction result data and the sample data; S202. Update the historical loss value sequence for each sample, add the current loss value to the historical loss value sequence of the corresponding sample, and maintain a sequence window of fixed length. S203. Calculate the prediction consistency index and prediction uncertainty index for each sample; the prediction consistency index is obtained by calculating the cosine similarity between the prediction probability distribution of the current iteration and the prediction probability distribution of the previous iteration; the prediction uncertainty index is obtained by calculating the entropy of the prediction probability distribution of the current iteration or by calculating the complement of the maximum prediction probability. S204. Calculate the sample quantity distribution of each category in the current batch. The sample quantity distribution is obtained by counting the sample quantity of each category and normalizing it. Calculate the average loss value of the current batch and update the batch-level statistics. S205. Store the updated sample-level and batch-level statistics in memory and provide real-time data input for S3.

3. The real-time adaptive statistical adjustment method for sample weights in a classification model according to claim 2, characterized in that: In S3, the process of the weight decision network calculating real-time weight values ​​includes: S301. Construct a multi-dimensional input feature vector that includes sample-level features and batch-level features; S302. The multi-dimensional input feature vector is input into a weighted decision network, which includes an input layer, multiple hidden layers, and an output layer. The hidden layers use activation functions for non-linear transformations, and the output layer uses a sigmoid function to map the output value to... The interval is used to obtain the initial weight value for each sample; S303. The initial weight values ​​are normalized by dividing the initial weight value of each sample by the sum of the initial weight values ​​of all samples and then multiplying by the batch size. S304. Output the normalized real-time weight values ​​to S4 and calculate the weighted loss function.

4. The real-time adaptive statistical adjustment method for sample weights in a classification model according to claim 3, characterized in that: In S4, the process of updating the parameters of the main classification model using a weighted loss function includes: S401. The loss value for each sample is calculated using the cross-entropy loss function, and the loss value is obtained by comparing the prediction result data and the sample data; S402. The loss value of each sample is weighted according to the real-time weight value to obtain the weighted loss value; S403. Sum the weighted loss values ​​of all samples to obtain the total weighted loss; S404. Calculate the gradient of the weighted total loss with respect to the parameters of the main classification model using the backpropagation algorithm, and update the parameters of the main classification model using an optimization algorithm; S405. The weighted total loss and parameter gradient are transmitted as update feedback to S5 to update the parameters of the weighted decision network.

5. The real-time adaptive statistical adjustment method for sample weights in a classification model according to claim 4, characterized in that: In S5, the process of updating the weight decision network parameters includes: S501, Receive update feedback from S4 including the rate of change of the weighted total loss and the parameter gradient of the main classification model; S502: Calculate the loss function of the weighted decision network, the loss function being based on the rate of change of the weighted total loss and the norm of the gradient of the main classification model parameters, and the product of the negative value of the rate of change of the weighted total loss and the norm of the parameter gradient is used as the loss value; S503. Calculate the gradient of the loss function with respect to the weight decision network parameters using the backpropagation algorithm, and apply a conditional gradient stopping mechanism during the backpropagation process; the conditional gradient stopping mechanism dynamically determines whether to allow the gradient to continue backpropagating to some of the lower network layers of the main classification model based on the smoothness of the performance improvement of the main classification model. S504. Update the weight decision network parameters using a stochastic gradient descent optimization algorithm with a smaller learning rate; S505. Store the updated weight decision network parameters for weight calculation in the next training iteration.

6. The method for real-time adaptive statistical adjustment of sample weights for a classification model according to claim 1, characterized in that: The adaptive target weight vector corresponds at least to the class balance target, the hard sample mining target, and the noise robustness target; when calculating the real-time weight value, the weight decision network weights and fuses the input features according to the target category with the corresponding coefficients in the adaptive target weight vector. For the category balance objective, the weight decision network dynamically adjusts the weights based on the sample size distribution in the batch-level statistics; For the target of mining difficult samples, the weighted decision network identifies samples with consistently high loss and high prediction consistency as difficult samples based on the historical loss value sequence and prediction consistency index in the sample-level statistics, and increases the weight value to strengthen the learning of classification boundaries. For the noise robustness objective, the weighted decision network identifies samples with low prediction consistency and large loss fluctuations as noise samples based on the prediction consistency index in the sample-level statistics and the volatility of the historical loss value sequence. The multi-objective weight optimization strategy learns the balanced weights among different objectives through the hidden layer of the weight decision network, thereby achieving adaptive weight allocation.

7. The method for real-time adaptive statistical adjustment of sample weights for a classification model according to claim 1, characterized in that: The real-time statistical tracking module and weighted decision network adopt an alternating collaborative training mechanism with the main classification model; The alternating collaborative training mechanism includes: keeping the parameters of the weight decision network fixed when the main classification model is updated; When the weight decision network is updated, the parameters of the main classification model remain fixed; the alternating collaborative training is implemented through a dual time scale update rule, with the main classification model using a regular update rhythm and the weight decision network using a smaller learning rate.

8. The method for real-time adaptive statistical adjustment of sample weights for a classification model according to claim 1, characterized in that: The update frequency of the sample-level statistics and batch-level statistics is synchronized with the training iteration; the sample-level statistics are updated after each training iteration, using the prediction results and sample data of the current batch; the batch-level statistics are updated after each training iteration, based on all sample data of the current batch; the real-time statistical tracking module maintains historical data through a sliding window mechanism.

9. The method for real-time adaptive statistical adjustment of sample weights for a classification model according to claim 1, characterized in that: The weighted decision network is a lightweight neural network, comprising an input layer, two hidden layers, and an output layer. The number of nodes in the input layer is equal to the dimension of the multi-dimensional input feature vector. The number of nodes in the hidden layers are 64 and 32, respectively, and they use the ReLU activation function. The number of nodes in the output layer is 1, and it uses the Sigmoid activation function.

10. The method for real-time adaptive statistical adjustment of sample weights for a classification model according to claim 1, characterized in that: The real-time statistical tracking module updates statistics using incremental calculation; the calculation of the weighted decision network is performed in parallel with the forward propagation of the main classification model; the weighted loss function calculation adopts vectorized operation and utilizes GPU acceleration.

Images