A Dynamic Model Calibration Method and Apparatus Based on Simplex Equal Angle Tight Frame Classifier

Abstract

This disclosure provides a dynamic model calibration method and apparatus based on a simplex isoangular tight frame classifier. Input samples are converted into high-dimensional features through a shared feature extraction module. Two independent initial probability distribution vectors are generated using a standard trainable classifier and a confidence-adjustable module. These two probability distribution vectors are then weighted and fused. The calibration weight factor is dynamically adjusted to minimize the difference between the predicted confidence and the actual accuracy. Classification losses are calculated for each of the two probability distribution vectors, and these losses are weighted and fused based on the calibration weight factor to obtain the total loss function used to optimize the network. Finally, the outputs of the standard trainable classifier and the confidence-adjustable module are weighted and fused based on the trained calibration weight factor to generate the final predicted probability distribution. This disclosure is applied to image classification using deep neural networks, minimizing model calibration errors and improving model prediction accuracy, resulting in higher-quality prediction results.

Description

Technical Field

[0001] This disclosure relates to the field of deep neural network image classification technology, and in particular to a dynamic model calibration method and apparatus based on a simplex isoangular tight frame classifier. Background Technology

[0002] In recent years, deep neural networks have achieved remarkable success in various fields such as image classification, object detection, and natural language processing. However, reliability issues in practical applications have gradually emerged, particularly manifested in the discrepancy between the model's predicted probabilities and actual accuracy. This inconsistency between predicted probabilities and actual accuracy is called calibration error, which typically manifests as the model's overconfidence in its output. Calibration error poses a serious challenge in many applications that rely on model reliability, such as medical diagnosis and autonomous driving, as the model's misjudgment of uncertainty can lead to serious safety hazards.

[0003] To address this issue, improving the confidence calibration level of models has become an important direction in deep learning research. Existing model calibration methods are mainly divided into two categories: post-processing methods and training-time methods. Post-processing methods adjust or rescale the output probabilities after model training is completed without changing the model's internal features; training-time methods integrate calibration mechanisms during training to directly optimize the model's confidence estimation, thereby improving generalization performance. Recent studies have shown that training-time calibration methods are closely related to classifiers, and many training-time methods improve calibration performance by introducing weakly trained classification heads to reduce the risk of overfitting. However, these methods also have significant limitations. On the one hand, weak training may reduce the model's expressive power on the training set, thus affecting prediction accuracy; on the other hand, these methods perform poorly when dealing with out-of-distribution samples, especially in tasks requiring broad generalization ability, limiting the model's practical application.

[0004] Therefore, there is an urgent need for a novel training-time calibration method that avoids relying on weak training classifiers, while significantly reducing calibration errors and improving generalization ability while ensuring model accuracy, so as to overcome the limitations of existing methods in practical applications. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, this disclosure provides a dynamic model calibration method and apparatus based on a simplex isoangular tight frame classifier.

[0006] According to a first aspect of this disclosure, a dynamic model calibration method based on a simplex isoangular tight frame classifier is provided, characterized by comprising the following steps:

[0007] S1. Obtain the original input sample and convert the original input sample into a high-dimensional feature representation through the shared feature extraction module;

[0008] S2. Input the high-dimensional features into the standard trainable classifier and the confidence adjustable module respectively, and output two independent probability distribution vectors as the initial probability distribution vector results.

[0009] S3. Perform weighted fusion on the two initial probability distribution vectors, and dynamically adjust the calibration weight factor according to minimizing the difference between the prediction confidence and the actual accuracy of the fusion output.

[0010] S4. Calculate the classification loss of the two initial probability distribution vectors respectively, and use the calibration weight factor to weight and fuse the two losses to obtain the total loss function for training optimization.

[0011] S5. Based on the calibration weight factor, the output probability distribution of the standard trainable classifier and the output probability distribution of the confidence adjustment module are weighted and fused to generate the final classification result and output it.

[0012] Preferably, the confidence adjustable module in S2 consists of a fixed simplex isoangular tight frame classifier and an adapter. The fixed simplex isoangular tight frame classifier can adjust its output confidence range during the initialization phase. The adapter is located in front of the fixed simplex isoangular tight frame classifier and processes the input high-dimensional features to alleviate the conflict caused by the difference in feature requirements between the fixed simplex isoangular tight frame classifier and the standard trainable classifier.

[0013] The adapter employs a bottleneck architecture, comprising two convolutional layers. The first convolutional layer reduces dimensionality to compress features, while the second convolutional layer restores feature dimensionality to maintain consistency between the output and input.

[0014] The high-dimensional features are processed by the adapter and then input into the fixed simplex isoangular tight frame classifier, ultimately obtaining the output probability distribution vector of the confidence adjustable module;

[0015] Preferably, the fixed simplex isoangular tight frame classifier is obtained by randomly generating and fixing a simplex isoangular tight frame, wherein the fixed simplex isoangular tight frame is independent of the network's gradient propagation; for In the classification problem, the fixed simplex isoangular tight frame is defined as a... A set of vectors in space, which is composed of matrices The columns constitute, It refers to the dimension of the input features, and the calculation methods include:

[0016]

[0017] In the formula, , It is a scaling factor. , It is a partially orthogonal matrix, that is ,and , , It is the identity matrix. It is an all-1 vector; at this time, the confidence and scaling factor of the final predicted label output by the fixed simplex isoangular tight-frame classifier are... It is positively correlated with the number of categories. Negative correlation; scaling factor in the fixed simplex isoangular tight-frame classifier It was initialized to 1.0;

[0018] Preferably, the process of obtaining the two independent initial probability distribution vectors in S2 includes: obtaining the high-dimensional feature representation output by the shared feature extraction module; and passing the high-dimensional feature representation through the standard trainable classifier to generate the original probability distribution vector. The high-dimensional feature representation is passed through the confidence-adjustable module to generate a calibration probability distribution vector. Record the two sets of probability distribution vectors respectively;

[0019] Preferably, the weighted fusion process of the two initial probability distribution vectors in S3 includes:

[0020]

[0021] In the formula, This represents the result of mixing the two initial probability distribution vectors. It is a weighting factor that controls the contribution of the two initial probability distribution vectors;

[0022] Preferably, the process of dynamically adjusting the calibration weight factor in S3 includes: during training rounds In this process, the two initial probability distribution vectors of the records for that round are mixed using different weighting factors. By minimizing the difference between the prediction confidence and the actual accuracy, the search yields the result for the 1st round. Calibration weights for each training epoch The calculation methods include:

[0023]

[0024] In the formula, Indicates input sample and their corresponding real tags The dataset constituted It is a cautious hyperparameter used to balance the predictive accuracy and confidence of a model. and They represent the first time. The mixed output calculated based on the weight factor γ in each training epoch The prediction confidence and actual accuracy are calculated using the following methods:

[0025]

[0026]

[0027] In the formula, This is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise;

[0028] Preferably, the specific strategy for the total loss function in S4 includes: the total loss function consists of the original classification loss and the calibration classification loss, and the parameters of the shared feature extractor, the standard trainable classifier, and the adapter in the confidence-adjustable module. By minimizing the loss function of both the original classification loss and the calibration classification loss, and combining the calibration weight factor... Optimization is performed, and the calculation methods include:

[0029]

[0030] loss function The classification loss of the standard trainable classifier is represented by the following calculation method:

[0031]

[0032] loss function The classification loss of the confidence-adjustable module is represented by the following calculation method:

[0033]

[0034] Preferably, the process of generating the final classification result in S5 includes: during training, using the optimal model with the lowest loss function value selected through multiple training iterations and based on performance evaluation on the validation set, and obtaining the calibration weight factor for the corresponding iteration of the optimal model. Inference is performed by weighted fusion of the probability distribution vector output by the standard trainable classifier and the confidence adjustment module. The calculation method includes:

[0035]

[0036] In the formula, This is the probability distribution vector of the model's final output.

[0037] According to a second aspect of this disclosure, a dynamic model calibration apparatus based on a simplex isoangular tight frame classifier is provided, the apparatus comprising:

[0038] The shared feature extractor module is used to acquire the original input sample and convert the original input sample into a high-dimensional feature representation through the shared feature extraction module;

[0039] The initial probability distribution vector acquisition module is used to input the high-dimensional features into the standard trainable classifier and the confidence adjustable module respectively, and output two independent probability distribution vectors as the initial probability distribution vector results.

[0040] The calibration weight factor dynamic adjustment module is used to perform weighted fusion of the two initial probability distribution vectors and dynamically adjust the calibration weight factor according to minimizing the difference between the prediction confidence and the actual accuracy of the fusion output.

[0041] The total loss function determination module is used to calculate the classification loss of the two initial probability distribution vectors respectively, and use the calibration weight factor to weight and fuse the two losses to obtain the total loss function for training optimization.

[0042] The classification result weighted output module is used to perform weighted fusion of the output probability distribution of the standard trainable classifier and the output probability distribution of the confidence adjustment module based on the calibration weight factor, so as to generate and output the final classification result.

[0043] The principle of this disclosed technical solution is as follows:

[0044] During the training phase, input samples are converted into high-dimensional features through a shared feature extraction module. Then, two independent initial probability distribution vectors are generated using a standard trainable classifier and a confidence-adjustable module; these vectors are used for subsequent calibration. These two probability distribution vectors are weighted and fused, and the calibration weight factor is dynamically adjusted by minimizing the difference between the output prediction confidence and the actual accuracy. Classification losses are calculated for each of the two probability distribution vectors, and then weighted and fused based on the calibration weight factor to obtain the total loss function used to optimize the network. During the inference phase, the outputs of the standard trainable classifier and the confidence-adjustable module are weighted and fused based on the calibration weight factor obtained during training to generate the final predicted probability distribution.

[0045] The technical effects of this disclosure are as follows:

[0046] This disclosure utilizes the characteristic of a fixed simplex isoangular tight-frame classifier whose output confidence interval can be adjusted during the initialization phase to generate a low-confidence calibration probability distribution that contrasts with the overconfidence problem commonly found in the original probability distribution output by a standard trainable classifier, but with similar prediction results. A dynamic mixing module blends the overconfident original probability distribution with the low-confidence calibration probability distribution, and minimizes the calibration error by dynamically adjusting the weight factors. By combining the standard trainable classifier and the confidence-adjustable module, high-quality classification results are obtained. Using the method of this disclosure, calibration errors can be effectively reduced while improving model prediction accuracy, overcoming the limitations of existing classifier-based calibration methods in generalization ability, and thus improving the model's practical application performance. Attached Figure Description

[0047] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the present invention and are not intended to limit the scope of this disclosure. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0048] Figure 1 A flowchart of a dynamic model calibration method based on a simplex isoangular tight frame classifier provided in this disclosure;

[0049] Figure 2 This is a flowchart illustrating a dynamic model calibration method based on a simplex isoangular tight frame classifier provided in this disclosure.

[0050] Figure 3 This is a schematic diagram of a dynamic model calibration device based on a simplex isoangular tight frame classifier provided in this disclosure.

[0051] Figure 4 A t-SNE visualization of the last layer features of the dynamic model calibration method based on the simplex isoangular tight frame classifier provided in this disclosure on the CIFAR-10 dataset;

[0052] Figure 5 This diagram illustrates the class activation of a standard trainable classifier module and a confidence-adjustable module on the Tiny-ImageNet dataset for a dynamic model calibration device based on a simplex isoangular tight frame classifier provided in this disclosure. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0054] like Figure 1 and Figure 2 As shown, in one embodiment, a dynamic model calibration method based on a simplex isoangular tight frame classifier is provided, including the following steps:

[0055] S1. Obtain the original input sample and convert the original input sample into a high-dimensional feature representation through the shared feature extraction module;

[0056] The input samples are taken from the input samples and their corresponding real tags The training dataset constitutes ;

[0057] The input sample is fed into the shared feature extraction module to extract... The high-dimensional features of the shared feature extraction module consist of an input layer and multiple residual modules. The input layer includes convolutional kernels and max pooling layers. Each residual block includes two or three convolutional kernels, batch normalization (BN), and the ReLU activation function. All residual blocks are connected via shortcuts, directly adding the input to the output.

[0058] S2. Input the high-dimensional features into the standard trainable classifier and the confidence adjustable module respectively, and output two independent probability distribution vectors as the initial probability distribution vector results.

[0059] The high-dimensional features are input into a standard trainable classifier to generate the original probability distribution vector. The standard trainable classifier includes a global average pooling layer and a fully connected layer;

[0060] The high-dimensional features are input into the confidence-adjustable module to generate a calibration probability distribution vector. The confidence-adjustable module consists of an adapter and a simplex isoangular tight-frame classifier. The adapter includes two convolutional kernels, batch normalization (BN), and the ReLU activation function. The simplex isoangular tight-frame classifier includes a global average pooling layer and a fixed simplex isoangular tight-frame matrix. , It is the number of categories;

[0061] Record the two sets of probability distribution vectors respectively;

[0062] S3. Perform weighted fusion on the two initial probability distribution vectors, and dynamically adjust the calibration weight factor according to minimizing the difference between the prediction confidence and the actual accuracy of the fusion output.

[0063] In training rounds In this process, obtain two sets of initial probability distribution vectors for the records of this round;

[0064] The two initial probability distribution vectors are then mixed using different weighting factors:

[0065]

[0066] In the formula, This represents the result of mixing two initial probability distribution vectors. It is a weighting factor that controls the contributions of both.

[0067] Mixed output of computation The prediction confidence and actual accuracy are calculated using the following methods:

[0068]

[0069]

[0070] In the formula, and These represent mixed outputs. The prediction confidence and actual accuracy, This is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise;

[0071] By minimizing the difference between the prediction confidence and the actual accuracy, the search yields the [missing information]. Calibration weights for each training epoch The calculation methods include:

[0072]

[0073] In the formula, It is a cautious hyperparameter used to balance the predictive accuracy and confidence of the model;

[0074] S4. Calculate the classification loss of the two initial probability distribution vectors respectively, and use the calibration weight factor to weight and fuse the two losses to obtain the total loss function for training optimization.

[0075] Obtain the two sets of initial probability distribution vectors;

[0076] Calculate the original classification loss using the original probability distribution vector. The calculation methods include:

[0077]

[0078] The calibration classification loss is calculated using the calibration probability distribution vector. The calculation methods include:

[0079]

[0080] The total loss function is obtained by weighted fusion of the original classification loss and the calibrated classification loss. The model parameters are obtained by minimizing the two loss functions and incorporating the calibration weight factor. Optimization is performed, and the calculation methods include:

[0081]

[0082] in, These represent the parameters of the adapters in the shared feature extractor, the standard trainable classifier, and the confidence-adjustable module, respectively.

[0083] S5. Based on the calibration weight factor, the output probability distribution of the standard trainable classifier and the output probability distribution of the confidence adjustment module are weighted and fused to generate the final classification result and output it;

[0084] Using the optimal model and its corresponding calibration weight factors For inference, the probability distribution vectors output by the weighted fusion standard trainable classifier and the confidence adjustment module are calculated using the following methods:

[0085]

[0086] In the formula, This is the probability distribution vector of the model's final output.

[0087] In some embodiments of this disclosure, experiments were conducted on three datasets: CIFAR-10, CIFAR-100, and SVHN. The results obtained using the method of this disclosure were compared with those obtained using other training methods for model calibration in the prior art, particularly with the most relevant classifier-based methods (PLP, TST, and V-TST) in this disclosure. To objectively evaluate the comparison results, the results of each training-time method were calculated and presented using three metrics, as shown in Table 1. In the table, smaller values ​​for the objective evaluation metrics ECE and AECE indicate better calibration performance, while larger values ​​for ACC indicate higher classification accuracy. Table 1 shows that the calibration performance of the method of this disclosure consistently achieves the best results, and its classification accuracy is also improved compared to Vanilla, which does not use any training-time model calibration method.

[0088]

[0089] Table 1

[0090] This disclosure also provides experiments on out-of-distribution sample detection, training a model on one dataset (e.g., CIFAR-10) and testing it on other datasets (e.g., CIFAR-100 and SVHN). The results obtained using the methods of this disclosure are compared with those of other prior art training-time model calibration methods (PLP, TST, and V-TST) using classifier design. To objectively evaluate the comparison results, the results for each training-time model method are calculated and presented using two metrics, as shown in Table 2. In the table, a higher AUROC value indicates better generalization performance, and a lower FPR95 value indicates better generalization performance. As can be observed from Table 2, in some embodiments, the methods of this disclosure consistently achieve the best generalization performance.

[0091]

[0092] Table 2

[0093] In some embodiments, such as Figure 3 As shown, a dynamic model calibration device 100 based on a simplex isoangular tight frame classifier is provided, comprising:

[0094] The shared feature extractor module 10 is used to acquire the original input sample and convert the original input sample into a high-dimensional feature representation through the shared feature extraction module.

[0095] The initial probability distribution vector acquisition module 20 is used to input the high-dimensional features into the standard trainable classifier and the confidence adjustable module respectively, and output two independent probability distribution vectors as the initial probability distribution vector results.

[0096] The calibration weight factor dynamic adjustment module 30 is used to perform weighted fusion of the two initial probability distribution vectors and dynamically adjust the calibration weight factor according to minimizing the difference between the prediction confidence and the actual accuracy of the fusion output.

[0097] The total loss function determination module 40 is used to calculate the classification loss of the two initial probability distribution vectors respectively, and use the calibration weight factor to weight and fuse the two losses to obtain the total loss function for training optimization.

[0098] The classification result weighted output module 50 is used to perform weighted fusion of the output probability distribution of the standard trainable classifier and the output probability distribution of the confidence adjustment module based on the calibration weight factor, so as to generate and output the final classification result.

[0099] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made by those skilled in the art to the technical solutions of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A dynamic model calibration method based on a simplex isoangular tight-frame classifier, characterized in that, Includes the following steps: S1. Obtain the original input sample and convert the original input sample into a high-dimensional feature representation through the shared feature extraction module. The original input sample is taken from the CIFAR-10, CIFAR-100 or SVHN dataset. S2. Input the high-dimensional features into the standard trainable classifier and the confidence adjustable module respectively, and output two independent probability distribution vectors as the initial probability distribution vector results. S3. Perform weighted fusion on the two initial probability distribution vectors, and dynamically adjust the calibration weight factor according to minimizing the difference between the prediction confidence and the actual accuracy of the fusion output. S4. Calculate the classification loss of the two initial probability distribution vectors respectively, and use the calibration weight factor to weight and fuse the two losses to obtain the total loss function for training optimization. S5. Based on the calibration weight factor, the output probability distribution of the standard trainable classifier and the output probability distribution of the confidence adjustment module are weighted and fused to generate the final classification result and output it. The confidence-adjustable module in S2 consists of a fixed simplex isoangular tight frame classifier and an adapter. The fixed simplex isoangular tight frame classifier can adjust its output confidence range during the initialization phase. The adapter is located in front of the fixed simplex isoangular tight frame classifier and processes the input high-dimensional features to alleviate the conflict caused by the difference in feature requirements between the fixed simplex isoangular tight frame classifier and the standard trainable classifier. The adapter employs a bottleneck architecture, comprising two convolutional layers. The first convolutional layer reduces dimensionality to compress features, while the second convolutional layer restores feature dimensionality to maintain consistency between the output and input. The high-dimensional features are processed by the adapter and then input into the fixed simplex isoangular tight frame classifier, ultimately obtaining the output probability distribution vector of the confidence adjustable module; A fixed simplicial isoangular tight frame classifier is obtained by randomly generating and fixing a simplicial isoangular tight frame, wherein the fixed simplicial isoangular tight frame is independent of the network's gradient propagation; for In the classification problem, the fixed simplex isoangular tight frame is defined as a... A set of vectors in space, which is composed of matrices The columns constitute, It refers to the dimension of the input features, and the calculation methods include: ； In the formula, , It is a scaling factor. , It is a partially orthogonal matrix, that is ,and , , It is the identity matrix. It is an all-1 vector; at this time, the confidence and scaling factor of the final predicted label output by the fixed simplex isoangular tight-frame classifier are... It is positively correlated with the number of categories. Negative correlation; scaling factor in the fixed simplex isoangular tight-frame classifier It was initialized to 1.0; The process of dynamically adjusting the calibration weight factor as described in S3 includes: during training rounds In this process, the two initial probability distribution vectors of the records for that round are mixed using different weighting factors. By minimizing the difference between the prediction confidence and the actual accuracy, the search yields the result for the 1st round. Calibration weights for each training epoch The calculation methods include: ； In the formula, Indicates input sample and their corresponding real tags The dataset constituted It is a cautious hyperparameter used to balance the predictive accuracy and confidence of a model. and They represent the first time. The mixed output calculated based on the weight factor γ in each training epoch The prediction confidence and actual accuracy are calculated using the following methods: ； ； In the formula, This is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise.

2. The dynamic model calibration method based on the simplex isoangular tight frame classifier according to claim 1, characterized in that, The process of obtaining the two independent initial probability distribution vectors in S2 includes: obtaining the high-dimensional feature representation output by the shared feature extraction module; and passing the high-dimensional feature representation through the standard trainable classifier to generate the original probability distribution vector. The high-dimensional feature representation is passed through the confidence-adjustable module to generate a calibration probability distribution vector. Record the two sets of probability distribution vectors respectively.

3. The dynamic model calibration method based on the simplex isoangular tight frame classifier according to claim 2, characterized in that, The weighted fusion process of the two initial probability distribution vectors in S3 includes: ； In the formula, This represents the result of mixing the two initial probability distribution vectors. It is a weighting factor that controls the contribution of the two initial probability distribution vectors.

4. The dynamic model calibration method based on the simplex isoangular tight frame classifier according to claim 1, characterized in that, The specific strategy for the total loss function in S4 includes: the total loss function consists of the original classification loss and the calibration classification loss, and the parameters of the shared feature extractor, the standard trainable classifier, and the adapter in the confidence-adjustable module. By minimizing the loss function of both the original classification loss and the calibration classification loss, and combining the calibration weight factor... Optimization is performed, and the calculation methods include: ； loss function The classification loss of the standard trainable classifier is represented by the following calculation method: ； loss function The classification loss of the confidence-adjustable module is represented by the following calculation method: 。 5. The dynamic model calibration method based on the simplex isoangular tight frame classifier according to claim 4, characterized in that, The process of generating the final classification result in S5 includes: during training, using the optimal model with the lowest loss function value selected through multiple training iterations and based on performance evaluation on the validation set, and obtaining the calibration weight factor for the corresponding iteration of the optimal model. Inference is performed by weighted fusion of the probability distribution vector output by the standard trainable classifier and the confidence adjustment module. The calculation method includes: ； In the formula, This is the probability distribution vector of the model's final output.

6. A dynamic model calibration apparatus for implementing the method of any one of claims 1-5 based on a simplex isoangular tight frame classifier, the apparatus comprising: The shared feature extractor module is used to acquire the original input sample and convert the original input sample into a high-dimensional feature representation through the shared feature extraction module; The initial probability distribution vector acquisition module is used to input the high-dimensional features into the standard trainable classifier and the confidence adjustable module respectively, and output two independent probability distribution vectors as the initial probability distribution vector results. The calibration weight factor dynamic adjustment module is used to perform weighted fusion of the two initial probability distribution vectors and dynamically adjust the calibration weight factor according to minimizing the difference between the prediction confidence and the actual accuracy of the fusion output. The total loss function determination module is used to calculate the classification loss of the two initial probability distribution vectors respectively, and use the calibration weight factor to weight and fuse the two losses to obtain the total loss function for training optimization. The classification result weighted output module is used to perform weighted fusion of the output probability distribution of the standard trainable classifier and the output probability distribution of the confidence adjustment module based on the calibration weight factor, so as to generate and output the final classification result.

Images