A regression defect mitigation method for the evolution of deep learning table recognition models

Abstract

This invention discloses a regression defect mitigation method for the evolution of deep learning table recognition models. First, in the feature selection stage, the CONMI evaluation function combined with specific hyperparameters is used to screen new features. The new feature set, merged with the old feature set, serves as the model training data, achieving retraining based on contribution control. On one hand, feature contribution is calculated using expected gradients; on the other hand, an overall loss function incorporating cosine similarity and cross-entropy loss functions is designed to stabilize the contribution of old features and learn new features. The model is trained based on the new feature set through forward and backward propagation. Under the condition that the accuracy of the new version of the table recognition model meets the requirements, a feature evolution scenario is constructed, and the model evolution process is implemented according to the regression mitigation strategy. The accuracy of the new version of the table recognition model is used as the evaluation index to obtain the regression defect mitigation strategy for the evolution process. This invention helps improve the quality and reliability of deep learning table recognition models.

Description

Technical Field

[0001] This invention relates to the fields of software engineering and software testing, and in particular to the design of a regression defect mitigation method for the evolution of deep learning table recognition models. Background Technology

[0002] The relevant technical aspects of this invention are described below:

[0003] a) Table recognition model

[0004] Given the tremendous success of deep learning in fields such as image, audio, and text processing, researchers have shown great interest in extending deep learning techniques to data problems stored in tabular formats. With the advent of the big data era, the increasing volume of data and the continuous iteration of user characteristics have gradually revealed the limitations of traditional machine learning algorithms in processing such data. For example, when faced with massive amounts of e-commerce user shopping records (such as tables containing numerous columns of information like user ID, purchased product name, purchase time, and purchase amount), traditional machine learning algorithms may struggle to accurately extract purchasing preferences from users at different consumption levels and the correlations between products. The application of deep learning technology, however, has gradually gained widespread attention, as it can better handle such complex situations.

[0005] Deep learning table recognition models utilize deep learning technology to understand information within tables. These models can identify the structure, content, and relationships within tables, enabling more efficient processing and analysis of tabular data. Table data exhibits typical structured characteristics, composed of ordered rows and columns. Each row can be considered a sample, encompassing multi-dimensional information; each column represents a feature, describing a specific attribute of the sample. Taking an e-commerce user purchase behavior analysis table as an example, each row represents a user's purchase record, serving as an independent sample. It covers information such as the purchased product name, quantity, purchase time, and payment amount, comprehensively reflecting the user's consumption behavior. Each column corresponds to a feature; for example, the "Purchased Product Name" column specifically describes the category of products purchased by the sample (i.e., each user). Each sample also includes a categorical variable, such as whether the user is a high-value customer. Such models primarily consist of two modules: an information extraction module and a classification module. The information extraction module is primarily responsible for extracting meaningful key information from the raw tabular data. This typically involves using deep neural network structures, such as feedforward neural networks (FNNs) and convolutional neural networks (CNNs), to effectively capture the structural and semantic information within the tables. The classification module mainly maps the extracted information to predefined categories, thereby achieving data classification.

[0006] In practical applications, developers first perform feature selection when training deep learning table recognition models. Given that the original data may contain a large number of redundant or irrelevant features, such as temporary promotional activity markers that appear incidentally and have no substantial impact on user classification in e-commerce user purchase behavior analysis, these need to be removed using specialized feature selection methods to reduce data dimensionality and complexity. The selected effective features and corresponding categorical variables are then used to train the model, and the resulting model can then make predictions.

[0007] b) Evolution of Deep Learning Model Features

[0008] In the development of deep learning systems, continuous evolution and updates are necessary to meet ever-changing product requirements or fix user-reported defects. Feature-driven evolution has become increasingly prevalent in this process. Given a deep learning model... and the feature set used Adjust by adding, deleting, or replacing existing features. To obtain And by using To improve the model by retraining or fine-tuning it. To obtain a new model This patent will Called the original model, Called The regression model. In this field, adding new features is a common practice, which uses the introduction of new features to regress deep learning systems; this is known as feature-driven model evolution. It is defined as follows: given the old feature set... A deep learning model that has been trained Then collect a new set of features The regression process is defined as being based on Improve by retraining and fine-tuning the model. Developers need to improve model performance while mitigating regression defects caused by evolution.

[0009] c) Regression error of deep learning table recognition models

[0010] For a classification task, given a k-class classification model and an input set Model Input Mapping to a k-dimensional vector This includes the confidence score for each class. The class with the highest confidence is represented as And this is considered a prediction result. Given a deep learning model The original version, its regression model Given a test input x and its corresponding true label y. If The label of x was correctly predicted, but Prediction error, i.e. Therefore, x triggers a regression defect. Furthermore, this patent defines x as the regression fault trigger input and the erroneous behavior it triggers. Given the continuous iteration and optimization of deep learning models, and the extremely subtle nature of their regression defects, detecting and fixing deep learning regression defects is crucial for the ecological security of software systems.

[0011] d) Regression mitigation in deep learning table recognition models

[0012] Regression defects have not received widespread attention in deep learning systems; however, mitigating them is crucial for ensuring the safety and reliability of continuous iteration in deep learning systems. Mitigation of regression defects can be viewed as reducing the regression defects introduced by model iteration. Specifically, given an initial model... Its regression model and a test set The definition of regression defect mitigation is in Train a new model based on Make the regression defect ratio of this model few.

[0013] Current research on mitigating regression defects in deep learning systems is limited, with only a few works focusing on this area. However, these methods face several challenges in practical applications: for example, retraining may lead to performance degradation, post-processing techniques are ineffective with out-of-distribution test inputs, and model training itself consumes significant time and computational resources. Therefore, effectively reducing newly introduced regression defects during model iteration has become a more meaningful and challenging problem.

[0014] In feature evolution scenarios, when a tabular model learns new features, it may ignore the knowledge gained from old features and their significant contributions, leading to regression defects. (In this paper, "knowledge" specifically refers to the cognitive system constructed by the model for that feature, i.e., the contribution of that feature to the model's prediction. For example, in a deep learning tabular model for predicting house prices, there are two features: house area and house age. If the model finds, based on past training and feedback, that the house area feature has a greater weight in influencing the final prediction result than the house age feature in each prediction, it means that the house area feature contributes more than the house age feature, indicating that the model is more inclined to use the house area feature for prediction.) Secondly, if the model fails to capture effective information from new features and instead relies excessively on old features, this will adversely affect the model's generalization ability and may lead to regression defects. To mitigate these problems, ideally, the model should achieve a balance between learning knowledge from new and old features. This patent designs a tabular model regression defect mitigation method, TRFmitigator (hereinafter, "this patent" and "TRFMmitigator" have the same meaning, both referring to the mitigation method proposed in this patent). TRFMitigator designs a feature selection method that effectively selects new feature sets to ensure that no potentially regressive features are introduced. Simultaneously, it designs a reasonable memory mechanism for the knowledge of old features, introduces interpretability methods to accurately quantify the contribution of old features to the model's decisions, and applies contribution constraints during training to ensure that the tabular model's knowledge of old features remains unaffected, achieving robust and continuous evolution. This reduces the number of regressive defects introduced during the evolution of deep learning tabular models. Experiments demonstrate that this method can effectively alleviate regressive defects introduced in feature-driven model evolution scenarios.

[0015] Currently, there is limited research on mitigating regression defects in deep learning tabular models, with only a few works focusing on the problem of repairing regression defects in tabular models. Furthermore, most existing methods suffer from the following shortcomings:

[0016] 1) Significant drop in accuracy: Some methods retrain the model by exploiting the generated regression defects. While this method can reduce regression defects, it can also lead to a significant drop in the overall accuracy of the model. This is because the retraining process may alter the effective feature representations originally learned by the model, thereby affecting its generalization ability.

[0017] 2) Insufficient consideration of feature-driven evolution scenarios: Existing methods often do not adequately consider the specific scenario of feature-driven evolution. As new features are introduced and model dependencies deepen, the model may unintentionally reduce its attention to old features, changing the weight of these features in decision-making. When learning new features, the model may partially or completely forget previously learned knowledge, potentially leading to serious regression defects and threatening the model's stability.

[0018] 3) Hinders continuous iteration and updating of tabular models: Some methods require integrating the predictions of multiple versions of the model (referring to models before and after evolution) for decision-making. This not only increases computational complexity but also hinders continuous iteration and updating of the model. In the rapidly evolving field of deep learning, flexible adaptation to new data and feature changes is crucial, and these methods cannot meet this requirement.

[0019] In practical applications, regression defects in deep learning table models are often only discovered after deployment. Repairing them at this point is not only extremely difficult but also wastes significant resources. Unlike repair tasks, mitigation tasks can be performed during model iteration. That is, before the new model is fully trained, regression defects can be effectively mitigated by preprocessing data and optimizing the model training process. Therefore, this patent proposes a novel method to mitigate regression defects in the evolution of deep learning table models, thereby improving model stability and generalization ability. Specifically, this patent proposes a regression defect mitigation method based on feature selection and contribution control. This method introduces additional processing strategies during training to strictly control changes in feature contribution, ensuring that the knowledge learned by the deep learning model from old features is not excessively destroyed during evolution, thus effectively mitigating regression defects in feature-driven model evolution scenarios. Summary of the Invention

[0020] To address the shortcomings of the existing technologies, this invention proposes a regression defect mitigation method for the evolution of deep learning table recognition models. Based on a designed deep learning framework, the method performs feature selection on the table recognition model, and uses the selected new feature set to achieve model retraining based on contribution control. Under the constructed feature evolution scenario, regression defect mitigation based on regression mitigation strategy is achieved.

[0021] A regression defect mitigation method for deep learning table recognition model evolution includes:

[0022] S1, Design a feature selection algorithm for the evolution of the table recognition model. The algorithm takes the newly added feature set by the user as the input, performs feature selection suitable for subsequent model training based on the feature forward search algorithm, removes redundant features, and obtains a new feature set.

[0023] S2, merge the obtained new feature set with the old feature set, and use the merged new feature set as model training data. Calculate the feature contribution of the new feature set based on the expected gradient to achieve feature retraining with contribution control. Calculate the cosine similarity of the feature contribution on the new / old model. In the tabular model training process, introduce a new loss function to control the change in feature contribution. Combine the cosine similarity and the loss function to obtain the overall loss function. Use the overall loss function to maintain the contribution of the old features while learning new features. Train the model based on the new feature set through forward propagation and backward propagation.

[0024] S3. Randomly select the total number of features as the original features to train the initial table recognition model. Then, combine the remaining features as new features with the original features to train a new version of the table recognition model. Continue until the accuracy of the new version of the table recognition model is higher than that of the initial table recognition model to construct the feature evolution scenario.

[0025] S4, in the feature evolution scenario, the model evolution process is realized according to the regression mitigation strategy. The accuracy of the new version of the table recognition model is used as the evaluation constraint. The regression defect mitigation strategy of the evolution process is that the lower the negative flip rate and the relative negative flip rate, the smaller the regression defect of the evolution process. The regression defect is repaired according to this strategy.

[0026] In some implementations, the feature selection based on the feature forward search algorithm for subsequent model training further includes: combining the feature forward search algorithm with Pearson correlation coefficient and normalized mutual information to comprehensively describe the dependency relationship between features and categorical variables, where the categorical variable is the class to which each sample belongs; calculating the feature selection evaluation function CONMI value between the features of the new feature set to be processed and the features of the selected feature set to evaluate the dependency relationship of features; selecting the new feature with the largest CONMI value to add to the selected feature set, until the CONMI value of all features is less than or equal to 0 or all new features have been selected; the expression of the feature selection evaluation function CONMI is as follows:

[0027]

[0028] Where λ represents the hyperparameter, NMI is the normalized mutual information used to measure the nonlinear correlation between two variables, and CO is the absolute value of the Pearson correlation coefficient.

[0029] In some implementations, the introduction of a new loss function to control the change in feature contribution during the training of the tabular model further includes: introducing a monitoring mechanism during the training of the tabular model, that is, in each iteration, calculating the difference between the contribution of each feature in the new version of the model and the contribution in the old version of the model through a similarity calculation method, and converting the difference into a loss function, guiding the model through backpropagation to reduce the difference.

[0030] In some implementations, S2 further includes retraining the model based on feature contribution control of the new feature set, calculating the cosine similarity of the feature contribution on the new / old model, and obtaining the change of the old feature contribution during the evolution of the feature contribution vector between different model versions.

[0031] In some implementations, S2 further includes combining cosine similarity and cross-entropy loss functions to obtain an overall loss function, using the overall loss function to maintain the contribution of old features while learning new features.

[0032] In some implementations, the overall loss function The expression is as follows:

[0033]

[0034] In the formula, x represents a single sample. This is an older version model. For the new version model, Features In the old version model Contribution to For the true label of sample x, For sample x in the model On the predicted label, This is the cross-entropy loss function commonly used during model training.

[0035] Compared with the prior art, the beneficial technical effects and technical progress achieved by the present invention are as follows:

[0036] 1) The CONMI evaluation function was adopted, which combines Pearson correlation coefficient (linear) and normalized mutual information (non-linear) to comprehensively describe the dependency between features and categorical variables. This allows for better removal of irrelevant or redundant features, thereby improving model performance and accuracy.

[0037] 2) After the new feature set is fused with the original old features, in order to ensure that the contribution of the old features in the model is not affected by the new features, a new loss function is introduced. An overall loss function including cosine similarity and cross-entropy loss function is designed to stabilize the contribution of the old features and learn the new features.

[0038] 3) The strategy for mitigating regression defects in the evolutionary process is that the lower the negative flip rate and the relative negative flip rate, the smaller the regression defects in the evolutionary process. The regression defects are repaired according to this strategy. Therefore, it performs well in reducing and repairing regression defects, which helps to improve the quality and reliability of deep learning table recognition models and has important application value in the field of deep learning table recognition.

[0039] 4) During training, the model can learn and optimize based on these selected high-quality features, avoiding interference and overfitting risks caused by irrelevant or redundant features. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the overall process of a regression defect mitigation method for deep learning table recognition model evolution according to the present invention;

[0041] Figure 2 Design a flowchart for the loss function;

[0042] Figure 3 This is a schematic diagram illustrating the performance of repairing regression defects under different distributions;

[0043] Figure 4 This is a schematic diagram illustrating the implementation process of a regression defect mitigation method for deep learning table recognition model evolution according to the present invention. Detailed Implementation

[0044] The present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0045] like Figure 1 As shown, the present invention provides a regression defect mitigation method for the evolution of deep learning table recognition models, which specifically includes the following steps:

[0046] Step 1: Design an algorithm for feature selection in the evolution of the table recognition model. This algorithm uses a feature forward search algorithm to select features before training the deep learning model, removing redundant features to obtain a new feature set. The feature set filtered by this algorithm can be directly input into the table recognition model (e.g., feedforward neural network (FNN) and convolutional neural network (CNN)). The input to this algorithm is the user-added feature set. After processing, the algorithm will filter out features that are detrimental to subsequent model training, retaining useful features, and finally obtaining the feature set. The table recognition model is used to process tabular data, which consists of rows and columns arranged in an ordered manner. Each row can be considered a sample, covering multi-dimensional information; each column represents a feature, used to describe the attribute of a sample in a certain dimension. The table recognition model is often a feedforward neural network or a convolutional neural network. Specifically, this algorithm is based on the feature forward search algorithm, combined with Pearson correlation coefficient (linear) and normalized mutual information (non-linear) to comprehensively describe the dependency relationship between features and categorical variables, where the categorical variable is the class to which each sample belongs. This algorithm uses the feature selection evaluation function CONMI to evaluate feature dependencies. This includes calculating the CONMI value between the features in the new feature set and the features in the selected feature set (columns in the table), selecting the new feature with the largest CONMI value and adding it to the selected feature set, until the CONMI value of all features is less than or equal to 0 or all new features have been selected. This process removes irrelevant or redundant features, resulting in a new feature set. Specifically, when the maximum CONMI value between the new feature set and the features in the selected feature set is greater than 0, the algorithm performs feature selection processing, calculates the CONMI value between each feature in the new feature set and the features in the selected feature set, and then selects the new feature with the largest CONMI value and adds it to the feature set. The hyperparameter λ used to combine the two is set to 0.5 in this method. As new features are gradually introduced, subsequent features will not have a positive impact on the model. Therefore, this patent uses a forward search algorithm to filter out unnecessary feature subsets. The filtering process ends when the CONMI value of the remaining feature subset is less than 0. The expression for CONMI is as follows:

[0047]

[0048] Where λ represents the hyperparameter, NMI is the normalized mutual information used to measure the nonlinear correlation between two variables, and CO is the absolute value of the Pearson correlation coefficient.

[0049] The hyperparameter λ is used to balance the weights of normalized mutual information (NMI) and the absolute value of Pearson correlation coefficient (CO) in the evaluation function, which measures the linear correlation between the two variables.

[0050] The feature forward search algorithm in this step specifically includes: calculating the CONMI value between the features in the new feature set and the features in the selected feature set; selecting the new feature with the largest CONMI value and adding it to the selected feature set; and continuing until the CONMI value of all features is less than or equal to 0 or all new features have been selected. The algorithm proceeds to feature selection when the largest CONMI value between the features in the new feature set and the features in the selected feature set is greater than 0. Specifically, the process is as follows: first, calculate the CONMI value between each feature in the new feature set and the features in the selected feature set; then, select the new feature with the largest CONMI value and check if its CONMI value is greater than 0. If it is greater than 0, add the new feature to the selected feature set, update the selected feature set, and delete the feature from the new feature set. Finally, return the selected feature set as the feature selection result. The final feature set obtained after the above rigorous feature selection process is then input into a deep learning table recognition model for training.

[0051] Step 2: Merge the new feature set obtained in Step 1 with the old feature set. Use the merged new feature set as model training data. Retrain the model based on feature contribution control using the new feature set. Calculate the feature contribution based on the expected gradient. In the table model training process, introduce a new loss function to control the change in feature contribution: In the initialization phase, use the weights of the original table recognition model to assign weights to the new table recognition model to be trained, ensuring the effective transfer of old knowledge. Subsequently, in the model training process, introduce a monitoring mechanism. In each iteration, calculate the difference between the contribution of each feature in the new version model and the contribution in the old version model using a similarity calculation method. Transform this difference into a loss function and guide the model to reduce this difference through backpropagation. This ensures that the table recognition model does not weaken its understanding of old features when learning new features. This balanced learning strategy aims to maintain the stability of the deep learning model during its evolution, avoiding performance fluctuations or regression defects caused by the introduction of new features, thus ensuring the stability of the deep learning model during its evolution.

[0052] Step 2.1: Retrain the model based on feature contribution control for the new feature set. The specific process is as follows:

[0053] To accurately quantify the contribution of old features to model predictions and ensure their stability during model updates, an advanced interpretability approach is employed. This approach helps users understand the model's internal workings by conducting in-depth analysis of the model's internal structure, exploring the dependence of the model's predictions on each feature, and further transforming the calculated dependence on each feature into a quantifiable contribution metric.

[0054] Specifically, the calculation of feature contribution based on the expected gradient is implemented as follows:

[0055] A feature contribution calculation method based on expected gradients was adopted: During model training, the gradient of each feature relative to the model output was calculated one by one using the chain rule through backpropagation. These gradient values ​​were then expected to yield the expected gradient of each feature's contribution. The expected gradient not only intuitively reflects the model's dependence on each feature but also serves as an effective indicator for evaluating feature contribution. The larger the expected gradient value of a feature, the more significant its role in the decision-making of the deep learning table model. Therefore, this method fully utilizes the differentiability of deep learning models and effectively combines it with the backpropagation mechanism.

[0056] Step 2.2: Calculate the cosine similarity between the feature contributions on the old and new models to obtain the change in the contribution of the old features during the evolution of the feature contribution vector across different model versions. Specifically, cosine similarity is commonly used to measure the similarity between two vectors. It is computationally efficient, unaffected by vector length, and has strong versatility. The higher the cosine similarity, the closer the feature's contribution is between the old and new model versions, and the better its stability. By applying cosine similarity, the stability of features can be effectively constrained during model evolution.

[0057] Step 2.3: Introduce the original cross-entropy loss function to control the change in feature contribution, ensuring that the model can learn new features during the evolution process and improve overall performance. Using the new feature set obtained, the algorithm can train the model through forward propagation and back propagation. On the training set, the weight parameters are continuously updated iteratively through back propagation until the difference between the model's prediction results and the actual results on the test set tends to stabilize.

[0058] The cross-entropy loss function is designed with the constraint of ensuring that the contribution of old features to the model's predictions remains stable during training. Two core requirements are considered: computational efficiency and differentiability. Efficiency ensures that the loss value can be calculated quickly during model training, avoiding unnecessary computational burden. Differentiability ensures that the model parameters can be effectively adjusted and optimized using methods such as gradient descent.

[0059] like Figure 2 The diagram illustrates the design process of this loss function. To ensure that the contribution of old features does not deviate significantly during the prediction process, TRFmitigator designs a new loss function. The expression is as follows:

[0060]

[0061] In the formula, x represents a single sample. This is an older version model. For the new version model, Features In the old version model Contribution to For the true label of sample x, For sample x in the model The predicted labels on the screen. This represents the cross-entropy loss function commonly used during model training. This loss function ensures that the model learns relevant knowledge about new features, thereby improving the overall accuracy.

[0062] Cosine similarity is chosen as the metric to ensure that the contribution vector of old features remains as constant as possible during model evolution. A higher cosine similarity indicates that the contributions of old features are closer across different model versions. By applying cosine similarity, this loss function effectively constrains the deep learning model to maintain the stability of old feature contributions during evolution, avoiding excessive changes. To ensure that the deep learning model can learn new features while maintaining the contributions of old features, the original cross-entropy loss function is also introduced. By combining cosine similarity constraints and cross-entropy loss function, the overall loss function is obtained. This design aims to ensure that the model can effectively utilize old features while efficiently learning new features during evolution, thereby improving the overall performance of the model.

[0063] Step 3: Construct a feature evolution scenario, which specifically includes: First, randomly extracting 1-a% of the total number of features as the original features, and training the initial table recognition model. Then, the remaining 'a%' features are used as newly added features and combined with the original features to train a new version of the table recognition model. Require The accuracy rate is higher than To ensure the effectiveness of the feature evolution scenario; otherwise, the scenario will be discarded and features will be reselected, and the above process will be repeated.

[0064] Step 4: Implement the model evolution process based on regression mitigation strategies, specifically including: [The text abruptly ends here, likely due to an incomplete sentence or missing information.] The accuracy rate is used as an evaluation metric, denoted as . To measure the effectiveness of regression defect mitigation, two metrics were used: negative flip rate (NFR) and relative negative flip rate (NFR). rel The theelative negative flip rate (NFR) measures the number of regression failures triggered during model evolution. NFR represents the proportion of test inputs in the test set that trigger regression failures. NFR and NFR... relThe lower the value, the fewer regression failures are caused during the regression process. This means that fewer previously correctly classified samples are misclassified due to model updates or improvements, thus proving the effectiveness of the regression mitigation strategy.

[0065] The specific implementation details are as follows:

[0066] In step 1, high-dimensional data often contains a lot of noise and irrelevant information, which may lead to overfitting of the trained model and poor performance on new data. Feature selection is the process of choosing the most valuable subset of features for model building and prediction from the numerous features of the original data. For example, in an e-commerce dataset containing many user behavior features, if there are some features that are not substantially related to the user's purchase decision, such as the time a user spends on an irrelevant page, retaining these features may interfere with the model's learning of truly important factors, causing the model to be biased in predicting user purchase behavior. After removing these features through feature selection, the model can more accurately capture the relationship between key features such as the user's historical purchase records and browsed product categories and purchase behavior, thereby improving the accuracy of prediction. TRFMitigator uses feature selection techniques to reduce the dimensionality of the feature space and remove irrelevant or redundant features, thereby improving model performance and accuracy. This technique uses the CONMI evaluation function as the evaluation criterion for feature selection. The CONMI function combines the Pearson correlation coefficient (linear) and normalized mutual information (non-linear) to comprehensively describe the dependency between features and categorical variables.

[0067] In step 2, the contribution calculation function is chosen: TRFMitigator uses the expected gradient scheme to calculate the feature contribution. The calculated feature contribution can be used to measure the importance of the feature to the model's prediction. This method enhances the interpretability of deep learning models by integrating axiomatic attribution priors, extending the Integrated Gradients method and eliminating the need for predefined baseline inputs, thus addressing many key limitations of existing attribution techniques. By integrating over the dataset rather than relying on a single baseline, the expected gradient provides a more comprehensive and generalized approach to understanding feature contributions under different inputs. Simultaneously, this method follows important interpretability axioms, such as completeness and realization invariance, ensuring that the attribution method is both comprehensive and applicable to different model implementations. This characteristic is crucial for achieving high-quality attribution, accurately reflecting the importance of input features in model predictions. The value of the expected gradient clearly ranks the importance of different features. In a complex deep learning tabular model, there may be multiple input features, such as user age, purchase frequency, and the number of product categories browsed in a model analyzing e-commerce user purchasing behavior. The expected gradient can quantify the degree of influence of each feature on the final purchase decision, thereby determining which features play a key role in the model's decision-making process and which are relatively minor.

[0068] To verify the effectiveness of this invention, the experimental results were scientifically analyzed. This invention proposes a novel deep learning table recognition model regression defect mitigation method (TRFMitigator). All implementations are in Python, using TensorFlow 2.3.0 as the high-level deep learning library for the front end. This patent was implemented on an Intel(R) Xeon(R) machine with 125Gb RAM, a CPU model E5-2640 v4 with 40 cores and a frequency of 2.40GHz, running Ubuntu 18.04.

[0069] This invention analyzes datasets and corresponding deep learning models across 14 evolutionary scenarios as experimental subjects. These datasets cover a variety of application scenarios and task types to ensure the broad applicability of the experimental results. Since feature evolution scenario data is relatively common in industrial applications but not open source, a series of open-source datasets were selected to simulate the feature evolution process. Specifically, this paper constructs a total of 14 scenarios, including 12 manually constructed regression scenarios (ID: 1–12) and 2 real-world scenarios (ID: 13–14). All models and datasets used are listed in Table 1, including ID number, dataset name and its abbreviation, model structure, size of training, validation, and test sets, number of classes, accuracy of each model version, and number of features used in each model version. These datasets have been widely used in previous deep learning tabular model tasks, covering multiple fields such as mobile phone price range prediction and disease diagnosis. To ensure the wide applicability of the datasets, datasets with fewer than 50 likes on Kaggle or fewer than 10 stars on GitHub were filtered out. To prevent overfitting, datasets with fewer than 500 samples and fewer than 8 features were excluded. In existing open-source datasets, data publishers typically provide a comprehensive and carefully collected set of features, rather than features collected in previous stages. Therefore, in the experiment, to simulate the feature evolution process, a% of the total features were removed, and these removed features were treated as newly added features. The remaining 1-a% of features were then used as the original features for training the previous version of the model. Specifically, the feature evolution scenario was constructed as follows: First, 1-a% of the total number of features was randomly selected as the original features, and an initial model was trained. Then, the remaining 'a%' features are used as newly added features and combined with the original features to train the model. Require The accuracy rate is higher than This is to ensure the effectiveness of the feature evolution scenario; otherwise, the scenario will be discarded and features will be reselected, and the above process will be repeated.

[0070] During model evolution, balancing the improvement in model accuracy with the introduction of regression defects is crucial. Therefore, evaluation metrics were selected from existing work, including one metric for measuring accuracy improvement and three metrics for measuring regression defects. Accuracy is often used to quantify model performance; in this invention, the model... The accuracy rate is used as an evaluation metric, denoted as . To measure the effectiveness of regression defect mitigation, two widely used metrics in existing studies were employed: negative flip rate (NFR) and relative negative flip rate (NFR). relThese metrics are used to measure the number of regression failures triggered during model evolution. NFR represents the proportion of test inputs in the test set that trigger regression failures. Since the error rate (1-ACC) is an upper limit of NFR, comparing regressions between models with different error rates becomes challenging. Therefore, this invention introduces NFR... rel Its denominator reflects the result of The expected error rate on a correctly predicted subset of samples, used to measure the proportion of incorrect predictions caused by regression. NFR and NFR rel A lower value indicates fewer regression faults during the regression process, meaning fewer misclassifications of previously correctly classified samples due to model updates or improvements, thus demonstrating the effectiveness of the regression mitigation strategy. This invention further analyzes the performance of the regression fault mitigation method on test inputs and regression faults in the test set under different data distributions. The original version of the regression fuzzing tool was used. and its regression model Test inputs that trigger regression failures are generated. Then, the models obtained through different fixation methods are evaluated to determine whether they can correctly predict these inputs. The accuracy of the generated test inputs is denoted as Fix. This metric can be seen as the accuracy of the regression model against the generated inputs. Table 1 shows the dataset and model information.

[0071] Table 1

[0072]

[0073] The dataset size is recorded in the form (a / b / c), where a / b / c represent the size of the training set / validation set / test set, respectively.

[0074] By comparing the method of this invention (TRFMitigator) with state-of-the-art regression defect repair methods, the effectiveness of TRFMitigator in 14 different scenarios was evaluated. Since ensemble methods and other methods differ significantly in nature—ensemble methods estimate uncertainty through the predictions of multiple models, providing more reliable predictions, but with higher computational complexity and less conducive to continuous model iteration; while other methods directly repair the original regression model, with lower computational complexity, but may be less accurate or reliable than ensemble methods in some cases—this invention evaluates ensemble methods and other methods separately to more comprehensively assess the performance of TRFMitigator.

[0075] a) Comparison of the present invention's TRFMitigator method with non-integrated methods

[0076] In 14 scenarios, TRFmitigator achieved the best accuracy improvement in 78.5% (11 / 14) of the scenarios, Finetuning performed best in only 7.14% (1 / 14) of the scenarios, while NeuRecover and FS did not achieve the best results in any of the 14 scenarios. Furthermore, in all scenarios, TRFmitigator significantly outperformed all the comparison methods in terms of regression failure rate (NFR), reducing NFR by 89.64%, 83.89%, 95.71%, and 125.23% compared to DT, FS, Finetuning, and NeuRecover, respectively. The experimental results demonstrate that TRFmitigator can maintain accuracy while mitigating regression failures. The effectiveness of TRFMitigator in repairing regression faults under different distributions was also evaluated. In all scenarios, TRFMitigator significantly outperformed other comparative methods in terms of fix, with an average repair rate of 84.60%. Compared with FS, Finetuning, and NeuRecover, TRFMitigator improved by 19.43%, 68.95%, and 200.98%, respectively.

[0077] This invention also evaluated the effectiveness of TRFmitigator in repairing regression faults under different distributions. In all scenarios, TRFmitigator significantly outperformed other comparative methods in terms of fixation, with an average repair rate of 84.60%. Compared to FS, Finetuning, and NeuRecover, TRFmitigator improved by 19.43%, 68.95%, and 200.98%, respectively. Specifically, as... Figure 3 As shown in scenarios 2 and 11, the X-axis represents the confidence distribution of regression defects on the model obtained through direct training, and the Y-axis represents the repair rate of these inputs. It can be seen that the repair rate of TRFMitigator is higher than other methods across different confidence intervals, indicating that TRFMitigator can stably repair regression defects with various confidence ranges. This is mainly because TRFMitigator addresses the root cause of regression defects in feature evolution scenarios and enhances the robustness of the model. In contrast, the NeuRecover method can only repair regression defects near the decision boundary, indicating that its performance is limited when encountering regression defects deviating from the training data. The above experimental results demonstrate that TRFMitigator is significantly effective in repairing regression defects under different distributions.

[0078] b) Comparison of TRFitigator and non-integrated methods

[0079] The average results of the present invention's TRFmitigator method combined with three advanced ensemble methods across 14 scenarios are shown in Table 3. In these 14 scenarios, TRFmitigator achieved the best accuracy improvement in 57.14% (8 / 14) of the scenarios, while Dropout and Scaling achieved the best results in 7.14% (1 / 14) and 21.42% (4 / 14) of the scenarios, respectively. Simultaneously, TRFmitigator achieved the lowest NFR in 78.57% (11 / 14) of the scenarios, reducing the NFR by 53.59%, 49.80%, and 21.77% compared to Dropout, Perturbation, and Scaling, respectively. TRFmitigator's repair rate was higher than other methods across different confidence intervals, indicating that TRFmitigator can stably repair regression defects with various confidence ranges. In contrast, ensemble methods can only repair regression defects near the decision boundary, mainly because ensemble models are susceptible to overconfident decisions by individual models. The above experimental results demonstrate the effectiveness of TRFitigator.

[0080] It should be noted that although the present invention has been shown and described with reference to specific exemplary embodiments thereof, those skilled in the art should understand that the present invention is not limited to the above embodiments, and all modifications thereof fall within the scope of protection of the present invention.

Claims

1. A regression defect mitigation method for deep learning table recognition model evolution, characterized in that, The method comprises the following steps: S1, a feature selection algorithm design for table recognition model evolution is performed, a user-added feature set is taken as the input of the algorithm, a feature selection suitable for subsequent model training is performed based on a feature forward search algorithm, redundant features are removed, and a new feature set is obtained; S2, the new feature set obtained is merged with an old feature set, the merged new feature set is taken as model training data, a feature contribution degree is calculated based on an expected gradient, a feature retraining with contribution degree control is realized, a cosine similarity of the contribution degree of the feature on the new / old model is calculated, a new loss function is introduced in the table model training process to control the change of the feature contribution degree, a total loss function is obtained by combining the cosine similarity and the loss function, the old feature contribution degree is maintained by using the total loss function, and the new feature is learned, the model is trained through a forward propagation and a backward propagation based on the new feature set; S3, a total feature quantity is randomly extracted as an original feature, an initial table recognition model is trained, then the remaining features are combined with the original features as newly added features, a new version of the table recognition model is trained, and the feature evolution scenario is constructed until the accuracy of the new version of the table recognition model is higher than that of the initial table recognition model; S4, in the feature evolution scenario, a model evolution process is realized according to a regression mitigation strategy, the accuracy of the new version of the table recognition model is taken as an evaluation constraint, and the evolution process regression defect mitigation strategy is obtained; the lower the negative flip rate and the relative negative flip rate are, the smaller the evolution process regression defect is, and the regression defect is repaired according to the strategy.

2. The regression defect mitigation method for deep learning table recognition model evolution according to claim 1, wherein, The feature selection suitable for subsequent model training based on the feature forward search algorithm further comprises: the feature forward search algorithm is combined with a Pearson correlation coefficient and a normalized mutual information to comprehensively describe the dependency relationship between the features and the category variable, wherein the category variable is the classification to which each sample belongs; a feature selection evaluation function CONMI value between the new feature set to be processed and the features in the selected feature set is calculated, the dependency relationship of the features is evaluated, the new feature with the maximum CONMI value is selected and added to the selected feature set, and the CONMI values of all features are less than or equal to 0 or the new features are completely selected until the new features are completely selected; and an expression of the feature selection evaluation function CONMI is as follows: Wherein, λ represents a hyperparameter, NMI is a normalized mutual information used to measure the nonlinear correlation between two variables, and CO is an absolute value of a Pearson correlation coefficient.

3. The regression defect mitigation method for deep learning table recognition model evolution according to claim 1, wherein, The new loss function introduced in the table model training process to control the change of the feature contribution degree further comprises: a monitoring mechanism is introduced in the table model training process, that is, in each iteration process, the difference between the contribution degree of each feature in the new version of the model and the contribution degree in the old version of the model is calculated by a similarity calculation method, and the difference is converted into a loss function, the model is guided by the way of back propagation, and the difference is reduced.

4. The regression defect mitigation method for deep learning table recognition model evolution according to claim 1, wherein, The S2 further comprises model retraining based on feature contribution degree control of the new feature set, and the cosine similarity of the contribution degree of the features on the new / old model is calculated to obtain the change of the old feature contribution degree in the evolution process of the feature contribution degree vector between different model versions.

5. The regression defect mitigation method for deep learning table recognition model evolution according to claim 1, wherein, The overall loss function The expression of the overall loss function is as follows: In the formula, x represents a single sample. This is an older version model. For the new version model, Features In the old version model Contribution to For the true label of sample x, For sample x in the model On the predicted label, This is the cross-entropy loss function commonly used during model training.

Images