Regression defect detection method and system for medical table prediction model evolution

Abstract

The invention discloses a regression defect detection method and system for medical table prediction model evolution, and the method comprises the steps: constructing an evolution scene from an initial deep learning medical table prediction model to a new-version deep learning medical table prediction model, executing a variation feature selection strategy, executing a variation operator selection strategy, and carrying out the regression defect detection of the medical table prediction model evolution. Generating medical table data after variation; inputting the varied medical table data into the deep learning medical table prediction model of the new version, and performing regression defect detection in the regression evolution process of the deep learning medical table prediction model of the new version; and obtaining a defect index detection value of the evolved new-version deep learning medical table prediction model, and realizing model evolution regression defect detection. According to the method, high-efficiency and high-quality regression defect detection under the regression evolution scene of the medical table prediction model based on ground-to-deep learning is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of software engineering and software testing, and in particular to a detection method for a deep learning-oriented medical table prediction model. BACKGROUND

[0002] The deep learning medical table prediction model architecture mainly includes two core functional modules: a medical information extraction module and a diagnosis classification module. The medical information extraction module uses a feedforward neural network (FNN) or a convolutional neural network (CNN) to decode the potential pathological association patterns from the original table numerical matrix. This composite network can simultaneously process discrete classification variables and continuous detection values to effectively capture the structure and semantic information in the medical table. The diagnosis classification module maps the abstract feature vector to the disease coding system, and through an attention weighting mechanism, it strengthens the decision weight of key indicators, thereby realizing the classification of medical table data.

[0003] Due to the continuous iteration and optimization of deep learning medical table prediction models, regression defects are extremely hidden in the use process of the model, and therefore detecting regression defects is crucial for the ecological safety of software systems. Existing deep learning model regression defect detection methods mainly include two types: fuzzy testing technology-based methods and adversarial attack-based methods. However, existing methods mainly focus on defect detection of specific version models rather than regression defects of deep learning medical table prediction models, and their detection effects are poor and unstable.

[0004] During the iteration process of deep learning medical table prediction models, how to effectively detect introduced regression defects becomes a challenge with important application value. SUMMARY

[0005] The present application aims to propose a regression defect detection method and system for medical table prediction model evolution, which introduces medical table feature contribution degree calculation and mutation feature selection strategy during the evolution detection of deep learning medical table models, and designs a mutation operator selection strategy to dynamically select multiple mutation operators, thereby realizing regression defect detection based on medical table feature selection fuzzy testing in the deep learning medical table prediction model regression evolution scenario.

[0006] The present application achieves the above-mentioned technical effects by using the following technical solutions:

[0007] In a first aspect, the present application proposes a regression defect detection method for medical table prediction model evolution, which includes:

[0008] S1, constructing an evolution scenario from an initial deep learning medical table prediction model to a new version of the deep learning medical table prediction model, the evolution scenario including inputting original medical table features and the new version of the medical table prediction model, performing a variation feature selection strategy on the original medical table features, and obtaining to-be-varied medical table features;

[0009] S2, obtaining original medical table data corresponding to the to-be-varied medical table features, performing a variation operator selection strategy on the original medical table data, obtaining a variation operator, performing variation processing on the original medical table data by using the variation operator, and generating varied medical table data;

[0010] S3, inputting the varied medical table data into the new version of the deep learning medical table prediction model, constructing a regression evolution scenario from the new version of the deep learning medical table prediction model to the initial deep learning medical table prediction model, and performing regression defect detection on the regression evolution scenario;

[0011] S4, obtaining an evaluation index detection value of the regression-evolved new version of the deep learning medical table prediction model, and realizing regression defect detection of the new version of the deep learning medical table prediction model.

[0012] In some embodiments, the performing of the variation feature selection strategy in S1 further includes: selecting the original medical table features as screening objects of the to-be-varied medical table features, respectively calculating two contribution degrees including an expected value of a contribution degree of each original medical table feature to the initial deep learning medical table prediction model and an expected value of a contribution degree of each original medical table feature to the new version of the deep learning medical table prediction model, performing normalization processing on the two contribution degree score expected values, calculating a contribution degree difference, and preferentially selecting a to-be-varied medical table feature with a large contribution degree difference.

[0013] In some embodiments, the expected value of the contribution degree of each original medical table feature to the initial deep learning medical table prediction model and the expected value of the contribution degree of each original medical table feature to the new version of the deep learning medical table prediction model are as follows:

[0014]

[0015] wherein f i represents each original medical table feature, represents a target input (i.e., medical table data) in each original medical table feature, represents a baseline input in each original medical table feature, a represents an interpolation parameter, x represents the target input in the original medical table data, and x' represents the baseline input in the original medical table data, represents the initial deep learning medical table prediction model Or a new version of the medical form prediction model p D (x′) represents the probability of the baseline input in the original medical form data. The change in the target input representing the features of the original medical form (obtained by adding noise). Represents the corresponding model The amount of change in the output of medical form data, This represents the path along the baseline input x′ in the original medical table data to the target input x in the original medical table data for the model. Characteristics of each original medical form By integrating the gradient along this path, we obtain the features of each original medical table. The model throughout the mutation process The cumulative effect of the output, This indicates the probability of different baseline inputs appearing in the medical form data for the feature. The importance of the factors is weighted and averaged to reflect the characteristics. Its importance within the overall distribution of medical table data;

[0016] The contribution difference is calculated as follows:

[0017]

[0018] in, Indicates the original medical table features f i Predictive model for the new version of medical forms The expected value of the contribution after normalization. Indicates the original medical table features f i For the initial deep learning medical table prediction model The expected value of the contribution after normalization. This represents the expected gradient contribution score.

[0019] In some implementations, S2's step of using the mutation operator to perform mutation processing on the medical table data corresponding to the medical table features to be mutated further includes, based on the type of medical table data variable, adjusting the size and range of the medical table features using three mutation operators—Gaussian noise operator, uniform noise operator, or multiplicative noise operator—for continuous variables, and adjusting the size and range of the medical table features using a label random operator for discrete variables; the expression for the mutation operator is as follows:

[0020] The expression for the Gaussian noise operator is:

[0021] The expression for the uniform noise operator is:

[0022] The multiplication noise operator expression is:

[0023] The label random operator expression is:

[0024]

[0025] wherein x represents a target input in original medical table data, represents noise, according to a probability conforming to a distribution, μ is a mean value of the original medical table data, ~ represents conforming to a distribution, represents a Gaussian distribution function, μ represents a mean value of medical table data, σ 2 represents a variance of medical table data, for example, a mean value and a variance of 100 pieces of data, ~ represents a meaning of conforming to a distribution), -σ 2 represents a negative variance of medical table data, σ 2 represents a variance of medical table data, represents a random distribution function, v rand represents a value randomly selected from a label (a label of certain discrete data) candidate set, y represents a current label candidate value, represents whether the randomly selected label needs to be mutated, 1 represents that mutation is needed, v rand represents a set of v rand , Uniform(V\{y}) represents a value of a uniform distribution function of a union set V excluding y, V represents a union set of v rand and y, y' represents a label random mutation value.

[0026] In some embodiments, the mutation operator selection strategy adopts a probability-based strategy selection, including calculating a priority score of a mutation operator, obtaining a probability of selection of each mutation operator, and performing priority selection of the mutation operator according to the probability of selection of each mutation operator;

[0027] The priority score of the mutation operator is as follows:

[0028]

[0029] wherein #Trigger_rf represents a number of test cases generated to trigger a prediction difference between multi-version models, #Select_count represents a total number of times that the mutation operator is selected in the fuzzing process, #Trigger_dirij_inputs represents a number of inputs generated to trigger a regression defect in a direction y i →y j , λ represents a regulation parameter of the operator selection strategy to a prediction bias.

[0030] In some embodiments, the regression defect detection of the regression evolution of S3 further comprises, with the prediction deviation of the mutated medical table data in the regression evolution as the adjustment direction, realizing dynamic adjustment of the mutation operator selection probability: the prediction deviation further comprises a prediction difference and a prediction direction, so that the probability of the test case being predicted correctly on the initial deep learning medical table prediction model is large, and the probability of being predicted incorrectly on the new version of the deep learning medical table prediction model is large, and the prediction confidence of the test case on the new version of the deep learning medical table prediction model is reduced.

[0031] In some embodiments, the expression of the prediction difference is:

[0032]

[0033] wherein, represents a mutation operator, d represents a balance parameter of two prediction differences, represents a new version of a deep learning medical table prediction model the confidence of the prediction of the baseline input x' in the original medical table data, represents an initial deep learning medical table prediction model the confidence of the prediction of the baseline input x' in the original medical table data, represents an initial deep learning medical table prediction model and a new version of a deep learning medical table prediction model the prediction difference of the baseline input x', represents a new version of a deep learning medical table prediction model the prediction difference before and after mutation; the present application is to detect regression defects, and it is hoped that errors can be detected more on the new model.

[0034] In some embodiments, the prediction direction is to trigger different regression defects as the prediction direction on any two different target inputs x i and x j in the original medical table data on the new version of the model deep learning medical table prediction model ; as shown in the following formula:

[0035]

[0036] wherein, represents an initial deep learning medical table prediction model the prediction of the target input x i in the original medical table data, represents a new version of a deep learning medical table prediction model Prediction of a target input x in original medical form data i .

[0037] In some embodiments, the method further comprises evaluating the medical form data after variation that fails to trigger the regression defect in combination with diversity and authenticity, deciding whether the medical form data after variation is added to the seed pool according to the evaluation result, combining the added medical form feature data with the original set of features to be varied, and dynamically adjusting the set of features to be varied.

[0038] In a second aspect, the present application provides a regression defect detection system for medical form prediction model evolution, which comprises a variation feature selection module, a variation operator selection module, a model evolution regression defect detection module, and an evaluation module; wherein:

[0039] The variation feature selection module obtains original medical form data corresponding to the medical form features to be varied, executes a variation operator selection strategy on the original medical form data, obtains a variation operator, performs variation processing on the original medical form data using the variation operator, and generates medical form data after variation;

[0040] The variation feature selection module obtains original medical form data corresponding to the medical form features to be varied, executes a variation operator selection strategy on the original medical form data, obtains a variation operator, performs variation processing on the original medical form data using the variation operator, and generates medical form data after variation;

[0041] The model evolution regression defect detection module inputs the medical form data after variation into the new version of the deep learning medical form prediction model, constructs a regression evolution scenario from the new version of the deep learning medical form prediction model to the initial deep learning medical form prediction model, and performs regression defect detection on the regression evolution scenario;

[0042] The evaluation module obtains evaluation index detection values of the regression-evolved new version of the deep learning medical form prediction model, and realizes regression defect detection of the new version of the deep learning medical form prediction model.

[0043] Compared with the prior art, the present application can achieve the following beneficial technical effects:

[0044] 1. In the detection process, the contribution degree calculation of the medical form features to be varied and the selection strategy of the features to be varied are introduced, the regression defect detection of the selected set of features to be varied is realized based on multiple variation operators and seed pool maintenance, the features to be varied are effectively selected, the features with large changes in model cognition are preferentially varied, and high-efficiency and high-quality regression defect detection is realized in the constructed medical form feature evolution scenario.

[0045] 2. A diversity mutation operator and dynamic operator selection strategy are designed for medical form data, and a variety of mutation operators are selected for regression defect fuzzy testing to increase the detection space and diversity of regression defects, effectively trigger more diverse regression defects, and ensure that regression defects are detected to the maximum extent in terms of quantity and diversity, and the medical form feature set is constrained and expanded;

[0046] 3. A seed pool maintenance based on variational autoencoder is designed to maintain the seed pool space, dynamically adjust the medical form features to be mutated, further improve the diversity of regression defect detection, and ensure the authenticity of regression defect detection;

[0047] 4. The method can detect errors in the new version of the deep learning medical form prediction model. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 FIG. 1 is a schematic diagram of the medical form prediction model evolution regression defect detection method of the present application.

[0049] Figure 2 FIG. 3 is a variation feature selection strategy technology roadmap.

[0050] Figure 3 FIG. 4 is a schematic diagram of the medical form prediction model evolution regression defect detection method of the present application.

[0051] Figure 4 FIG. 5 is a medical form prediction model evolution regression defect detection system module diagram of the present application. DETAILED DESCRIPTION

[0052] The present application will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely intended to explain the present application and are not intended to limit the present application.

[0053] Example 1 Figure 1 The medical form prediction model evolution regression defect detection method of the present application is shown, which specifically includes the following steps:

[0054] Step 1. In the evolution scenario, a deep learning oriented medical form prediction model evolution algorithm is executed according to the features to be mutated, and medical form data and a new version of the medical form prediction model are used as inputs of the algorithm;

[0055] Specifically, the process of obtaining the new version of the medical form prediction model includes: first, randomly extract 1-m% of the total number of new medical form features to generate an original medical form feature set, use the original medical form feature set as an initial seed, and train an initial deep learning medical form prediction model Then the remaining m% features are added as new medical table feature set, combined with the initial seed, to train a new version of the medical table prediction model Requirements The prediction accuracy is higher than To ensure the effectiveness of the medical table feature evolution scenario; otherwise, discard the scenario and reselect the features, repeat the above process; this process is the evolution process from the deep learning-oriented medical table prediction model to the new version of the medical table prediction model;

[0056] The input of the deep learning-oriented medical table prediction model is the original medical table feature, which is the column of the medical table corresponding to the label category; the medical table feature has corresponding original medical table data, that is, the row of the medical table, that is, the data corresponding to the feature label column; including numerical medical table features and categorical medical table features, the continuous variable is a numerical medical table feature, such as price, weight, etc., which triggers potential defects by adjusting the size or range of the numerical value of these medical table data. The categorical variable is a discrete medical table feature, such as color, information, etc., where information such as blood type has optional values A, B, O; the output of the model is the predicted deep learning-oriented medical table, which specifically includes the typical structured medical table features presented in the form of a table matrix, for example: the horizontal dimension represents independent case records, covering multi-dimensional information; the vertical dimension corresponds to a specific biomarker indicator, which is used to describe the attributes of a patient's indicator; the deep learning medical table prediction model is used to identify the structure, content and relationship of the medical table, and more effectively process and analyze medical table data;

[0057] The medical table prediction model of the application may be established based on a deep learning network such as a feedforward neural network (FNN) and a convolutional neural network (CNN), for example.

[0058] Figure 2 A variation feature selection strategy technology roadmap is shown, specifically, the variation feature selection strategy includes: calculating the contribution degree of each medical table feature, quantifying the influence of the medical table feature on the model prediction result, and selecting a set of features to be mutated by contribution degree normalization difference sorting; the contribution degree refers to the relationship between the medical table feature set selected for the deep learning-oriented medical table prediction model evolution and the generation of regression defects; specifically, according to the two requirements of being able to quickly and accurately evaluate the influence of each medical table feature on the model prediction and being able to calculate the contribution degree of the feature to each model, the expected gradient algorithm is selected to calculate the expected value of the contribution degree of different medical table features to the model prediction, and the medical table feature set more related to the generation of regression defects is located, After processing, the medical table features to be mutated selected by this module will more explainably trigger regression defects.

[0059] The model interpretability technology is used to calculate the contribution degree of each feature, quantify the influence of the medical table data features on the prediction result of the model, and select a set of to-be-varied features by contribution degree normalization difference sorting;

[0060] Step 2. According to the to-be-varied features and the variation requirement, a variation operator is selected and designed, the size and range of the medical table data features are adjusted by using the variation operator, the error degree of the medical table data is increased, the set of to-be-varied features is varied, new medical feature data is generated, and various variation operators are used for continuous medical feature data and discrete medical feature data respectively to detect the regression defects of the deep learning medical table prediction model. In the detection process, the variation operator selection strategy is designed, and the prediction difference and direction of the medical feature data in the model evolution are used as the guidance to dynamically adjust the variation operator selection probability in the regression defect detection process of the deep learning medical table prediction model. Specifically, it includes the variation feature selection strategy, the variation operator, and the variation operator selection strategy design of the deep learning medical table prediction model evolution.

[0061] Step 2.1. Design a variation feature selection strategy: select the original medical table features as the screening object of the to-be-varied medical table features, calculate the contribution degree of each original medical table feature to the initial deep learning medical table prediction model and the contribution degree of each original medical table feature to the new version of the deep learning medical table prediction model, perform normalization processing on the two contribution degree score expectation values, calculate the contribution degree difference, and sort according to the contribution degree difference. The to-be-varied medical table features are preferentially selected according to the contribution degree difference. That is, in the training stage, the medical table features are used to train two models, and the features that have a great influence on the model prediction are selected as the to-be-varied medical table features. The contribution degree difference is calculated, and each prediction category (the initial deep learning medical table prediction model to the new version of the deep learning medical table prediction model) is applied to the new data in the prediction stage. The trained model outputs the prediction category to which the model belongs.

[0062] Step 2.2, mutation operator design, in this step, two types of mutation operators are used, namely continuous variable mutation operator and discrete variable mutation operator, to perform mutation operation; the continuous variable mutation operator is mainly suitable for numerical medical table feature data, such as price, weight, etc., by adjusting the size or range of these numerical values to trigger potential defects. Discrete variable mutation operator is for classification or enumeration type medical table feature data, such as color, information, etc., by changing the value of these attributes to generate regression defects. According to the above classification, a total of 4 kinds of mutation operators are implemented, including 3 continuous variable mutation operators and 1 discrete variable mutation operator, a total of 4 mutation operators. Continuous variable data for three continuous variable mutation operators: i, continuous variable Gaussian noise (Gaussian Noise) mutation operator: add noise based on Gaussian distribution N(μ,σ) on continuous features, where μ and σ represent the mean and standard deviation of the distribution, respectively. ii, continuous variable uniform noise (uniform Noise) mutation operator: for continuous variables, randomly add noise that obeys uniform distribution in the interval [-δ,+δ]. δ determines the maximum range that the noise value can change, which means that the noise value will be uniformly distributed in the range from -δ to +δ. iii, continuous variable multiplication noise (Multiplicative Noise) mutation operator: for continuous variables, multiply the data by a factor randomly selected from a distribution, such as Gaussian distribution N(μ,σ). Mutation operator for discrete variable data: iv, discrete variable label random (Label Random Replace) mutation operator: randomly select another label to replace the existing label in the value range of discrete features. The specific expression is:

[0063] The Gaussian noise operator expression is:

[0064] The uniform noise operator expression is:

[0065] The multiplication noise operator expression is:

[0066] The label random operator expression is:

[0067]

[0068] where x represents the target input in the original medical table data, ∈ represents noise, according to the probability of conforming to a distribution, μ is the mean of the original medical table data, ~ represents conforming to the distribution, is the Gaussian distribution function, μ represents the mean of the medical table data, σ 2 represents the variance of the medical table data, for example, the mean and variance of 100 data, ~ represents the meaning of conforming to the distribution), -σ2 denotes the negative medical table data variance, σ 2 denotes the medical table data variance, denotes the random distribution function, v rand denotes the value of the randomly selected label (the label of a certain discrete data) candidate set, y denotes the current label candidate value, denotes whether the randomly selected label should be mutated, 1 denotes that it should be mutated, v rand denotes v rand , Uniform(V\{y}) denotes the uniform distribution function value of the union set V excluding y, and v rand and y, and y' denotes the randomly mutated label value.

[0069] Step 2.3, a mutation operator selection strategy is designed; specifically, the design idea of the mutation operator selection strategy is: if a mutation operator can frequently amplify the prediction difference of the seed between different versions of deep learning medical table prediction models, the selection probability of the operator should be increased; 2) if a mutation operator can frequently trigger a regression defect in a certain direction, the mutation operator should be preferentially selected when the regression defect in the direction needs to be triggered. Based on these two ideas, a probability-based mutation operator selection strategy is designed. The strategy first calculates the priority score of each mutation operator, then converts the priority score of each mutation operator into a regression defect triggering probability, and selects the next mutation operator according to the regression defect triggering probability; specifically, the prediction bias of the mutated medical table data in the regression evolution is taken as the adjustment direction to realize dynamic adjustment of the mutation operator selection probability: the prediction bias further includes a prediction difference and a prediction direction, so that the test case is more likely to be predicted correctly on the initial deep learning medical table prediction model, but more likely to be predicted incorrectly on the new version of the deep learning medical table prediction model, and the prediction confidence of the test case on the new version of the deep learning medical table prediction model is reduced;

[0070] Step 3, input the mutated medical table data into the new version of the deep learning medical table prediction model, perform regression evolution from the new version of the deep learning medical table prediction model to the initial deep learning medical table prediction model, and perform regression defect detection on the regression evolution. The scenario of the medical table prediction model evolution based on the deep learning network constructed in step 1 includes: performing the mutation medical table feature selection strategy designed in step 2.1 to select a medical table feature set more related to the generation of regression defects as a medical table feature set to be mutated; using the medical table feature set to be mutated obtained in step 2 as a candidate test set for subsequent mutation operations to perform regression defect detection; including using the mutation operator selection strategy designed in step 2.3 to select the next mutation operator according to the regression defect trigger probability, and using the selected mutation operator to perform mutation operation on the medical table feature to be mutated.

[0071] Step 4, determine the evaluation index, obtain the evaluation index detection value of the regression evolved new version of the deep learning medical table prediction model, and realize the regression defect detection of the new version of the deep learning medical table prediction model: the larger the detection value, the better the regression defect detection effect of the new version of the deep learning medical table prediction model; including: designing a regression defect detection strategy, and performing regression defect detection on the regression evolution process of the new version of the medical table prediction model according to the regression defect detection strategy, the specific process including: taking the number of detected regression defects, the number of regression defect seed triggers, and the number of unique regression defects on the new version of the deep learning medical table prediction model as evaluation indexes, respectively denoted as RFI, Seed, and RF. The higher the value of RFI, the more regression defects detected in the model, the higher the value of Seed, the more seeds in the initial medical table feature data that can be used to trigger regression defects, and the higher the value of RF, the more unique regression defects detected, thereby proving the effectiveness of the regression defect detection.

[0072] Specifically, in step 2.1, the contribution of each new medical table feature to the model prediction is calculated based on the model interpretability technology, and the influence of each new medical table feature on the prediction result of the deep learning medical table prediction model is quantified; the new medical table features are sorted in descending order of the normalized contribution difference value of each new medical table feature set, and the new medical table features with high ranking are selected as the medical table features to be mutated and form a medical table feature set to be mutated; the medical table features to be mutated selected by this strategy will more likely trigger regression defects.

[0073] For each test input x and its corresponding feature set For each new medical table feature f i The contribution of each new medical table feature f to the new version of the deep learning medical table prediction model is calculated

[0074] For the original deep learning medical table prediction model and regression model respectively calculate the contribution of any medical table feature f i to and is and Considering that the original deep learning medical table prediction model does not utilize the new medical table feature set for reasoning, the mutation operation on the new medical table feature set will not affect the prediction results of the original deep learning medical table prediction model, so only the original medical table feature set needs to be selected as the screening object of the feature subset. At the same time, the contribution of the original medical table feature to the regression defect prediction model needs to be normalized. Through the normalization operation, it can be ensured that the sum of the contribution of all old features on the same test case x is 1, so as to better compare the contribution difference between different features. The regression defect prediction model is the regression defect prediction model of the original deep learning medical table prediction model. Thus, the contribution difference of each original medical table feature in the deep learning-oriented medical table prediction model and its regression defect prediction model can be calculated. For each original medical table feature f i , the contribution difference is represented by the following formula (1):

[0075]

[0076] Where the left side represents the normalized contribution difference of the original medical table feature f i , and the right side represents the normalized contribution of the feature f i to , represents the contribution of f i to . The higher the score, the greater the change in the corresponding original medical table contribution, and the greater the impact on regression defect detection.

[0077]

[0078] Where the left side represents the normalized contribution of the feature f i to , and the right side represents the expected gradient contribution score. Through the normalization operation, it can be ensured that the sum of the contribution of all original medical table features on the same test input x is 1, so as to better compare the contribution difference between different features. The contribution calculation formula is as follows:

[0079]

[0080] Where the left side is the formula abbreviation, and the right side is the full formula.

[0081] For the model The contribution score is calculated by the expected gradient formula as follows:

[0082]

[0083] Where f i represents each original medical table feature, represents the target input in each original medical table feature (i.e., medical table data), represents the baseline input in each original medical table feature, a represents the interpolation parameter, x represents the target input in the original medical table data, and x' represents the baseline input in the original medical table data, represents the initial deep learning medical table prediction model or the new version of the medical table prediction model p D (x') represents the probability of the baseline input in the original medical table data, represents the change in the target input of the original medical table feature (obtained by adding noise), represents the output change of the medical table data corresponding to the model , represents the path along the baseline input x' in the original medical table data to the target input x in the original medical table data, The integral of the gradient of each original medical table feature is obtained by integrating the gradient along this path, and the integral of the gradient of each original medical table feature is the cumulative impact of the model output during the entire variation process, thereby measuring its importance, represents the weighted average of the importance of the feature according to the probability of the occurrence of different baseline inputs in the medical table data, so as to more comprehensively and accurately reflect the importance of the feature under the entire medical table data distribution;

[0084] Where the target input x in the original medical table data refers to a certain data containing all features, and is the target input of the original medical table feature f i , that is, is part of x, for example, x is a target input composed of 5 features, and is a feature fi The corresponding input is specifically as follows:

[0085] The target input x is in the format: [x: f1, f2, f3, f4,.f5]

[0086] The specific data is: [x: 0.1, 0.2, 0.3, 0.4, 0.5], then is 0.1, is 0.2

[0087] The baseline input is specifically described: the original medical table data has different categories, assuming that the model is a ten-class problem, then there are ten categories, (1) define a baseline input for each category: for each category c (assuming there are C categories in total), select the

mean value of the category

[0088] 2. Calculate the data distribution: for each baseline input x_c^′, the probability distribution p D (x_c^′) of the baseline in the data set needs to be calculated, that is, the proportion of samples belonging to category c in the data. (3) Weighted calculation of expected gradient: when calculating the expected gradient, each baseline input x_c^′ is weighted, and the weight is p D (x_c^′), and the data integral corresponding to each baseline input x_c^′ will affect the feature contribution value of the category.

[0089] Specifically, in step 2.1, the algorithm process of the variation medical feature selection strategy specifically includes: calculating the contribution of each original medical table feature f i to the prediction model of the original medical table and the regression prediction model is and At the same time, the contribution difference of the original medical feature to the regression defect prediction model is normalized, and the score of the feature is higher, indicating that the feature contribution difference is larger, and the influence on the regression defect detection is greater. The contribution difference value of the feature is sorted in descending order to reveal which features are most likely to cause regression defects when the medical table is varied or disturbed. The selection result of the variation medical table feature is the feature subset F s with the top contribution difference, and the number of feature subsets is limited to no more than K.

[0090] The specific implementation details are as follows:

[0091] The high dimensionality of the initial medical form feature data determines that not every medical form feature contributes equally to the deep learning medical form prediction model, which may lead to a lack of targeting in detecting regression defects, thereby reducing the detection effect. The purpose of feature selection is to select a set of the most valuable features to be mutated for regression defect detection from numerous medical form features. For example, in a medical form containing a large amount of information, if the test data obtained by the user for a cold examination has no substantial correlation with the user's diabetes examination, resulting in deviation of the deep learning medical form prediction model in the prediction result, and after screening the medical form features through feature selection, the regression defects can be more effectively detected. The present technology adopts an expected gradient explainability technology, comprehensively considers various possible baseline conditions in the data set, and weights the integrated gradient according to the probability of different baseline inputs in the data set to more comprehensively and accurately reflect the importance of the features under the entire data distribution.

[0092] Specifically, the strategy enhances the explainability of the deep learning model by integrating the axiomatized attribution prior, follows important explainability axioms such as completeness and implementation invariance by integrating on the data set rather than relying on a single baseline, ensures that the attribution method is comprehensive and applicable to different deep learning medical form model implementations, and considers the data distribution P D (x′) is integrated for all possible baseline inputs x′, rather than relying on a single baseline input, but comprehensively considering various possible baseline conditions in the data set, and weighting the integrated gradient according to the probability of different baseline inputs in the data set. In this way, the importance of the features under the entire data distribution can be more comprehensively and accurately reflected, and inaccurate attribution caused by improper selection of a single baseline can be avoided.

[0093] Specifically, in step 2.3, the mutation operator selection strategy realizes a probability-based strategy selection, and the specific process is as follows: the following formula is used to calculate the priority score of the mutation operator. The formula consists of two parts: the first part represents the ability of the mutation operator to generate test cases that can amplify the prediction difference between different versions of the model, and the second part represents the ability of the mutation operator to trigger the direction of the current required regression defect. Wherein #Trigger_rf represents the number of test cases generated to trigger the prediction difference between different versions of the model, #Select_count represents the total number of times the mutation operator is selected in the fuzzing process, and #Trigger_dirij_inputs represents the number of inputs generated to trigger the regression defect in the direction y i →y j If the generated test case set currently triggers the regression defect direction y i →y jIf the number of inputs is small, the mutation operator that triggers the direction should be selected to increase the probability of triggering the direction. Therefore, the second part is introduced to improve the probability of selecting the mutation operator that triggers the current required regression defect direction. It is worth noting that after obtaining the priority score of each mutation operator, it needs to be converted into a probability to represent the probability of selecting each mutation operator. Finally, the priority score of each mutation operator is divided by the sum of the priority scores of all mutation operators to obtain the probability of selecting each mutation operator;

[0094] The priority score is as follows:

[0095]

[0096] Where λ represents the regulation parameter of the operator selection strategy to the prediction deviation, and the range is (0, 1).

[0097] Specifically, in the step 2.3, wherein the prediction difference is defined as follows: the test case after the test input x is subjected to the mutation operation of the mutation operator R is x', and the original deep learning medical table prediction model The confidence of the real class predicted by the test case x' after the mutation operation is represented as Regression model The confidence of the real class of the test case x' after the mutation operation is represented as Original model The confidence of the real class of the test input x is represented as Regression model The confidence of the real class of the test input x is represented as

[0098] The front part of the following formula represents the hope that the generated test case is more easily predicted correctly by the original model , and more easily predicted incorrectly by the regression model The rear part represents the hope of reducing the prediction confidence of the newly generated test case on the regression model . The larger the value is, the more easily the mutation operator R generates the test case that triggers the regression defect.

[0099]

[0100] Wherein, represents the mutation operator, d represents the balance parameter of the two prediction differences, represents the new version of the deep learning medical table prediction model The confidence of the baseline input x' in the original medical table data predicted, represents the initial deep learning medical table prediction model the confidence of the baseline input x' prediction in the original medical form data, represents the initial deep learning medical form prediction model and the new version of deep learning medical form prediction model the prediction difference of the baseline input x', represents the new version of deep learning medical form prediction model the prediction difference before and after mutation; in order to realize the detection of regression defects, therefore, the present application hopes to detect more errors on the new model.

[0101] The regression defect trigger direction is defined as follows: represents the original model the predicted label of the test input x, represents the regression model the predicted class of x. Assuming x i and x j are both y, if and x i and x j show different error behaviors, and then they are likely to trigger different regression defects.

[0102] The prediction direction is that the predicted classes of any two different target inputs x i and x j in the original medical form data on the new version of model deep learning medical form prediction model are not the original class, and the expression is:

[0103]

[0104] and

[0105]

[0106] wherein, represents the initial deep learning medical form prediction model the prediction (label class) of the target input x i in the original medical form data, represents the new version of deep learning medical form prediction model the prediction (label class) of the target input x i in the original medical form data.

[0107] In the regression defect detection process, a maintenance mechanism based on a variational autoencoder is designed for the seed pool. The medical table feature data after mutation that fails to trigger the regression defect is evaluated. The medical table feature data after mutation is decided to be added to the seed pool through diversity and authenticity evaluation. The medical table feature data added is combined with the original mutation feature set to dynamically adjust the mutation feature set. The error correction capability is further increased. The steps in the process of generating new medical table data are shown.

[0108] Figure 3 The technical route of the regression defect detection method for deep learning medical table prediction model evolution of the application is shown.

[0109] Embodiment two: Figure 4 The regression defect detection system for deep learning medical table prediction model evolution of the application is shown. The system is used to realize the regression defect detection method for deep learning medical table prediction model evolution of the application in embodiment one. The system includes a mutation feature selection module 100, a mutation operator selection module 200, a model evolution regression defect detection module 300, and an evaluation module 400. Wherein:

[0110] The mutation feature selection module obtains the original medical table data corresponding to the medical table feature to be mutated. The mutation operator selection strategy is performed on the original medical table data to obtain the mutation operator. The original medical table data is mutated using the mutation operator to generate the mutated medical table data.

[0111] The mutation feature selection module obtains the original medical table data corresponding to the medical table feature to be mutated. The mutation operator selection strategy is performed on the original medical table data to obtain the mutation operator. The original medical table data is mutated using the mutation operator to generate the mutated medical table data.

[0112] The model evolution regression defect detection module inputs the mutated medical table data into the new version of the deep learning medical table prediction model. The regression evolution scenario from the new version of the deep learning medical table prediction model to the initial deep learning medical table prediction model is constructed. The regression defect detection is performed on the regression evolution scenario.

[0113] The evaluation module obtains the evaluation index detection value of the regression evolution new version of the deep learning medical table prediction model to realize the regression defect detection of the new version of the deep learning medical table prediction model. The larger the detection value is, the better the regression defect detection effect of the new version of the deep learning medical table prediction model is.

[0114] In order to verify the effectiveness of the present application, the experimental results are analyzed scientifically. The present application proposes a novel deep learning medical table prediction model regression defect detection method (DTFUzz), all implementations use Python language, and uses TensorFlow2.3.0 as a high-level deep learning library in the front end. The present application is implemented on an Intel(R) Xeon(R) machine with 125 Gb RAM, the CPU model is E5-2640v4, with 40 cores, the frequency is 2.40 GHz, and the operating system used by the machine is Ubuntu18.04.

[0115] The present application analyzes 9 evolution scenarios of medical table data sets and their corresponding deep learning medical table prediction models as experimental objects. These medical table data sets cover a variety of application scenarios and task types to ensure the broad applicability of the experimental results. Since in the medical field, the data of the table feature evolution scenario is common but not open source, a series of open source data sets are selected to simulate the medical table feature evolution process, including fetal health classification, patient treatment classification and diabetes diagnosis data set. Specifically, the present application manually constructs a total of 9 regression scenarios (ID: 1~9). All deep learning medical table prediction models and medical table data sets used are listed in Table 1. Including ID number, data set name and its abbreviation, model structure, training set, validation set and test set size, class number, accuracy of each model version and number of features used in each model version. These data sets have been widely used in previous deep learning table model tasks, covering multiple data sets in the medical field such as disease diagnosis. In order to ensure the broad use of the data set, the data set with less than 50 likes on Kaggle or less than 10 stars on GitHub is filtered out. In order to prevent model overfitting, data sets with less than 500 samples and less than 8 features are excluded. In existing open source data sets, the data publisher usually provides a complete and carefully collected complete feature set, rather than the features collected in the previous stage. Therefore, in the experiment, in order to simulate the feature evolution process, m% of the total features are removed, and these removed features are regarded as newly added features. Then, the remaining 1-m% features are used as the original features of the previous version of the trained model. Specifically, the medical table feature evolution scenario is constructed according to the following steps: first, randomly extract 1-m% of the total number of features as original features, and train the initial model Then, the remaining m% features are used as newly added features, combined with the original features to train the model Requirements The accuracy is higher than To ensure the effectiveness of the feature evolution scenario; otherwise, the scenario will be discarded and the feature will be reselected, repeating the above process. It is crucial to maximize the detection of regression defects during the evolution of deep learning medical table prediction models. Therefore, three evaluation indicators for measuring regression defects were selected from existing work, which are used to measure the number, diversity, and triggerability of regression faults caused during the evolution of deep learning medical table prediction models. RFI represents the number of medical table data triggering regression defects in the detection results; Seed represents the number of seeds triggering regression defects in the initial medical table data; RF represents the number of unique regression defects detected by the test input triggering regression defects, that is, the count of [initial seed, defect direction] triplets. The higher the value of RFI, the more regression defects detected during the evolution of medical table data; the higher the value of Seed, the more seeds that can trigger regression defects mined by the regression defect detection method for the initial medical table data; the higher the value of RF, the more unique regression defects detected during the evolution of medical table data, thereby proving the effectiveness of the regression detection strategy. The present application further analyzes the performance of the regression defect mitigation method in handling test inputs triggering regression faults and regression defects in the test set under different data distributions. As shown in Table 1, the data set to be tested and model information.

[0116] Table 1

[0117]

[0118] The size of the data set is recorded in the form of (a / b / c), where a / b / c represents the size of the training set / verification set / test set, respectively. By comparing the present application method (DTFuzz) with the most advanced regression defect detection method, the effectiveness of DTFuzz on 9 different medical scenarios is evaluated. Although there is currently no technology specifically for deep learning medical table prediction model regression defect detection methods, in order to more comprehensively evaluate the performance of DTFuzz, the present application will be evaluated simultaneously with the two most advanced defect detection methods PGD and LowProFool that can be adapted.

[0119] The comparison of the method DTFuzz with two advanced adversarial attack methods PGD and LowProFool is analyzed in 9 scenes according to three evaluation indexes respectively. Under the RFI index, DTFuzz achieves the best regression defect detection effect in 100% (9 / 9) scenes, the number of detected regression defects is 11960, the number of detected regression defects of PGD is 67, and the number of detected regression defects of LowProFool is 109, and the effect of DTFuzz is improved by 17751% and 10872% compared with PGD and LowProFool; under the Seed index, DTFuzz achieves the best regression defect detection effect in 100% (9 / 9) scenes, and the number of seeds that can be successfully used to detect regression defects in the test set is 333, the number of seeds of PGD is 67, and the number of detected regression defects of LowProFool is 83, and the effect of DTFuzz is improved by 397% and 301% compared with PGD and LowProFool; under the RF index, DTFuzz achieves the best regression defect detection effect in 100% (9 / 9) scenes, the number of detected regression defect behaviors is 358, the number of detected regression defect behaviors of PGD is 67, and the number of detected regression defect behaviors of LowProFool is 109, and the effect of DTFuzz is improved by 434% and 228% compared with PGD and LowProFool; the above experimental results show that compared with the existing regression defect detection method of the deep learning medical table data prediction model, DTFuzz has significant effectiveness in the number of detected regression defects, the ability to mine initial seeds of detected regression defects, and the number of unique regression defects.

[0120] It should be noted that although the present application has been shown and described with reference to certain exemplary embodiments thereof, it is to be understood that the present application is not limited to the above-described embodiments and that various changes in form and details can be made therein without departing from the spirit and scope of the present application.

[0121] The above only describes the embodiments of the present application and is not intended to limit the present application. Those skilled in the art can make various modifications and changes to the present application without departing from the spirit and scope of the present application. Those skilled in the art can make various any modifications, equivalent replacements or variations, etc. without departing from the spirit and scope of the present application, and any modifications, equivalent replacements or variations, etc. made within the spirit and principles of the present application fall within the protection scope of the present application defined by the appended claims.

Claims

1. A regression defect detection method for medical form prediction model evolution, characterized in that, The method comprises: S1, constructing an evolution scenario from an initial deep learning medical table prediction model to a new version of the deep learning medical table prediction model, inputting original medical table features and the new version of the medical table prediction model into the evolution scenario, performing a variation feature selection strategy on the original medical table features to obtain to-be-varied medical table features; S2, obtaining original medical table data corresponding to the to-be-varied medical table features, performing a variation operator selection strategy on the original medical table data to obtain a variation operator, and performing variation processing on the original medical table data by using the variation operator to generate varied medical table data; S3, inputting the varied medical table data into the new version of the deep learning medical table prediction model, constructing a regression evolution scenario from the new version of the deep learning medical table prediction model to the initial deep learning medical table prediction model, and performing regression defect detection on the regression evolution scenario; S4, obtaining an evaluation index detection value of the regression-evolved new version of the deep learning medical table prediction model to realize regression defect detection of the new version of the deep learning medical table prediction model.

2. The method of claim 1, wherein the medical table prediction model evolution is a regression defect detection method. The variation feature selection strategy of S1 further comprises: selecting the original medical table features as the screening object of the to-be-varied medical table features, respectively calculating two contribution degrees including an expected value of the contribution degree of each original medical table feature to the initial deep learning medical table prediction model and an expected value of the contribution degree of each original medical table feature to the new version of the deep learning medical table prediction model, performing normalization processing on the two contribution degree score expected values, calculating a contribution degree difference, and preferentially selecting a to-be-varied medical table feature with a large contribution degree difference.

3. The method of claim 2, wherein the medical table prediction model evolution is a regression defect detection method. The expected value of the contribution degree of each original medical table feature to the initial deep learning medical table prediction model and the expected value of the contribution degree of each original medical table feature to the new version of the deep learning medical table prediction model are as follows: where f i represents each original medical form feature, represents the target input in each original medical form feature, represents the baseline input in each original medical form feature, a represents an interpolation parameter, x represents the target input in the original medical form data, and x' represents the baseline input in the original medical form data, represents an initial deep learning medical form prediction model or a new version of the medical form prediction model p D (x') represents the probability of the baseline input in the original medical form data, represents the change amount of the target input of the original medical form feature, represents the output change amount of the medical form data corresponding to the model , represents the cumulative influence of the model on the entire variation process of the output, the integral of the gradient of each original medical form feature along the path from the baseline input x' in the original medical form data to the target input x in the original medical form data, by integrating the gradient along this path, the cumulative influence of the model on the entire variation process of the output, represents the weighted average of the importance of the feature according to the probability of the occurrence of different baseline inputs in the medical form data, reflecting the importance of the feature under the entire medical form data distribution; The contribution degree difference is calculated as follows: wherein, denotes the original medical form feature f i a new version of the medical form prediction model the normalized contribution expectation value of the new version of the medical form prediction model, denotes the original medical form feature f i an initial deep learning medical form prediction model the normalized contribution expectation value of the initial deep learning medical form prediction model, denotes the expected gradient contribution score.

4. The method of claim 1, wherein the medical table prediction model evolution is a regression defect detection method. The variation processing of the medical table data corresponding to the to-be-varied medical table features by using the variation operator of S2 further comprises: according to the variable type of the medical table data, adjusting the size and range of the medical table features by using three kinds of variation operators of Gaussian noise operator, uniform noise operator or multiplication noise operator for continuous variables, and adjusting the size and range of the medical table features by using a label random operator for discrete variables; The variation operator expression is as follows: The Gaussian noise operator expression is: The uniform noise operator expression is: The multiplication noise operator expression is: The tag random operator expression is: wherein, x represents a target input in original medical table data, represents noise, represents a probability in accordance with a distribution, represents a mean of original medical table data, represents in accordance with a distribution, represents a Gaussian distribution function, represents a mean of medical table data, represents a variance of medical table data 2 represents a Gaussian distribution function, represents a mean of medical table data, represents a variance of medical table data 2 represents a negative variance of medical table data, represents a variance of medical table data 2 represents a variance of medical table data, represents a random distribution function, v rand represents a value randomly selected from a label candidate set, y represents a current label candidate value, represents a label variation randomly selected, V rand represents a set of v rand , Uniform(V\{y}) represents a value of a uniform distribution function of a set V except y, V represents a set of v rand and y, y' represents a label variation value randomly. 5.The deep learning oriented medical table prediction model evolution regression defect detection method of claim 1, wherein, The variation operator selection strategy adopts a probability-based strategy selection mode, specifically comprising: calculating a priority score of the variation operator to obtain a probability of selection of each variation operator, and preferentially selecting the variation operator according to the probability of selection of each variation operator; the priority score of the variation operator is as follows: where #Trigger_rf represents the number of generated test cases triggering the multi-version model inter-prediction difference, #Select_count represents the total number of times the mutation operator is selected in the fuzzing process, #Trigger_dirij_inputs represents the generated trigger direction y i →y j the number of inputs of the regression defect, and λ represents the regulation parameter of the operator selection strategy on the prediction deviation.

6. The method of claim 1, wherein the medical table prediction model evolution is a regression defect detection method. The regression defect detection on the regression evolution of S3 further includes, taking the prediction deviation of the mutated medical form data in the regression evolution as the adjustment direction, realizing dynamic adjustment of the mutation operator selection probability: the prediction deviation further includes a prediction difference and a prediction direction, so that the probability of the test case being predicted correctly on the initial deep learning medical form prediction model is large, while the probability of being predicted incorrectly on the new version of the deep learning medical form prediction model is large, and the prediction confidence of the test case on the new version of the deep learning medical form prediction model is reduced.

7. The method of claim 1, wherein the medical table prediction model evolution is a regression defect detection method. The expression of the prediction difference is: wherein, denotes a mutation operator, d denotes a balancing parameter of two prediction differences, denotes a new version of a deep learning medical table prediction model a confidence of a prediction for a baseline input x' in the original medical table data, denotes an initial deep learning medical table prediction model a confidence of a prediction for a baseline input x' in the original medical table data, denotes an initial deep learning medical table prediction model and a new version of a deep learning medical table prediction model a prediction difference for a baseline input x', denotes a new version of a deep learning medical table prediction model a prediction difference before and after mutation.

8. The method of claim 1, wherein the medical table prediction model evolution is a regression defect detection method. The prediction direction: in the new version of the model deep learning medical table prediction model The target input x in any two different original medical table data i And x j Trigger different regression defects as the prediction direction; as follows: in, This represents the initial deep learning medical table prediction model. The target input x in the original medical form data i The prediction This indicates the new version of the deep learning medical table prediction model. The target input x in the original medical form data i The prediction.

9. The method of claim 1, wherein the medical table prediction model evolution is a regression defect detection method. The method further includes combining diversity and authenticity evaluation of medical form data after mutation that fails to trigger regression defects, deciding whether the mutated medical form data is added to the seed pool according to the evaluation result, combining the added medical form feature data with the original to-be-mutated medical form feature set, and realizing dynamic adjustment of the to-be-mutated medical form feature set.

10. A regression defect detection system for evolving a medical table prediction model implementing the method of any one of claims 1 to 9, characterized in that, The system includes a mutation feature selection module, a mutation operator selection module, a model evolution regression defect detection module, and an evaluation module; wherein: The mutation feature selection module obtains original medical form data corresponding to to-be-mutated medical form features, executes a mutation operator selection strategy on the original medical form data, obtains a mutation operator, performs mutation processing on the original medical form data using the mutation operator, and generates mutated medical form data; The mutation feature selection module obtains original medical form data corresponding to to-be-mutated medical form features, executes a mutation operator selection strategy on the original medical form data, obtains a mutation operator, performs mutation processing on the original medical form data using the mutation operator, and generates mutated medical form data; The model evolution regression defect detection module inputs the mutated medical form data into the new version of the deep learning medical form prediction model, constructs a regression evolution scenario from the new version of the deep learning medical form prediction model to the initial deep learning medical form prediction model, and performs regression defect detection on the regression evolution scenario; The evaluation module obtains evaluation index detection values of the regression evolution of the new version of the deep learning medical form prediction model, and realizes regression defect detection of the new version of the deep learning medical form prediction model.

Images