Rapid Shapley value estimation method based on stratified sampling

By adopting fast Shapley value estimation methods with hierarchical sampling and non-parametric test in deep neural networks, the calculation complexity problem of Shapley value estimation in high-dimensional samples is solved, and more accurate interpretation results are achieved, filling the field of fault diagnosis. Explain the gap in the evaluation of the accuracy of the results.

CN119047589APending Publication Date: 2024-11-29CHONGQING UNIV
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411183588.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

The high computational complexity of the Shapley value estimation method in the prior art limits its application in high-dimensional sample interpretation, especially when deep neural networks are used for fault diagnosis, traditional methods have problems with error and computational complexity.

Method used

The fast Shapley value estimation method based on stratified sampling was used to reduce the number of features and combine non-parametric testing methods to reduce the computational complexity, and the Shapley value of each feature was estimated by Monte Carlo sampling and Mann-Whitney test.

Benefits of technology

Faster and accurate Shapley value estimation in high-dimensional samples is achieved, filling the gap in quantitative evaluation of the accuracy of interpretation results in the field of fault diagnosis and providing better interpretation results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119047589A_ABST
    Figure CN119047589A_ABST
Patent Text Reader

Abstract

The invention relates to a rapid Shapley value estimation method based on stratified sampling, and belongs to the technical field of fault diagnosis. According to the method, firstly, an improved evaluation index is defined, the contribution degree of features is measured through the distance between output of a target neuron and output of a reference neuron, and the nonlinear attribute of SoftMax is better met. And secondly, the potential relationship is measured by using the interaction relationship among the members, and the Shapley value of each feature is estimated through stratified sampling and a nonparametric test method, so that the calculation complexity is effectively reduced. Finally, the invention provides an explanation result accuracy quantitative evaluation method, the accuracy of the explanation result is evaluated through activation and inactivation characteristics, and a quantitative index is provided for evaluation of the explanation method. Experimental results show that compared with traditional SHAP, the method is more suitable for explanation of high-dimensional samples and has a better explanation result, and the blank of an explanation result accuracy quantitative evaluation method in the field of fault diagnosis is filled.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of fault diagnosis and relates to a fast Shapley value estimation method based on stratified sampling. Background Art

[0002] Compared with signal processing-based mechanical health monitoring methods, deep neural networks (DNNs) have the characteristics of higher accuracy, stronger adaptability, and lower requirements for expert knowledge. Therefore, deep neural networks have been widely used in mechanical health monitoring. However, the uninterpretability of deep neural networks limits their application in the engineering field.

[0003] Explainable artificial intelligence (XAI) is proposed to reveal the decision-making process of deep neural networks to improve the credibility of deep learning. XAI investigations can be simply divided into two areas. One approach is to change the DNN structure. For example, the convolution operation can be replaced by multi-level dictionary learning or wavelet kernels, which helps to understand the intermediate features learned by the DNN. Implanting a constraint module can also produce the same explanation results. If the module is designed based on physical knowledge, it will force the DNN to learn the required physical properties. If the module is designed by referring to the attention mechanism, the module can tell the model what to pay attention to. However, except for the attention module, all methods may inhibit the generalization ability of the DNN and only return local explanations.

[0004] On the other hand, the attention module performs well for positive contributions but ignores negative contributions. Another option is to build an explanation method. Tool-based methods are more intuitive and applicable than structure-based explanation methods due to model independence. In general, methods in this area can be regarded as attribution explanations; therefore, many researchers focus on this area of ​​research. Backpropagation process in DNN, class activation mapping (CAM), Grad-CAM, Grad-CAM++, and layer-by-layer relevance propagation (LRP) are proposed to generate saliency maps. Locally interpretable model agnostic explanations (LIME) and Shapley values ​​(SHAP) are constructed using mask techniques to reveal the contribution of each feature.

[0005] Similar to the attention mechanism, CAM-based methods may lose features with negative contributions. LPR is insensitive to the target class, while the strategy for generating superpixels in LIME is difficult to determine. In contrast, SHAP, which originates from game theory, is the only attribution explanation method that satisfies the properties of linearity, redundancy, symmetry, and efficiency. However, the high computational complexity of SHAP seriously hinders its application, especially in the interpretation of high-dimensional samples. To address this problem, a variety of methods have been proposed, such as Shap-Deep, Shap-Grad, Shap-Kernel, HierarchicalSHAP, SHEAR, and FastSHAP. These methods simplify the calculation process of SHAP or estimate the value of Shapley using simple models, but may produce large errors. In order to effectively implement SHAP and eliminate the above problems, sampling-based methods such as Sobol sequences and orthogonal spherical codes, as well as Neyman allocation and empirical Bernstein bounds have been proposed. Methods based on Sobol sequences and orthogonal spherical codes will further increase the computational complexity. Neyman allocation and Bernstein constraints only augment the sample once without any checks. In addition, both methods are sensitive to the initial sample size. When deep neural networks are used for fault diagnosis, there are problems with the sampling-based Shapley value estimation method because its training samples are original vibration signals.

[0006] Therefore, there is still a lack of a fast and effective Shapley value estimation method, especially for high-dimensional sample interpretation. Summary of the invention

[0007] In view of this, the object of the present invention is to provide a fast Shapley value estimation method based on stratified sampling. The computational complexity of SHAP is reduced by reducing the number of features. The Shapley value of each feature is estimated by combining stratified sampling and non-parametric test methods.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A fast Shapley value estimation method based on stratified sampling, the method comprising the following steps:

[0010] Problem definition: When interpreting a fault diagnosis network using SHAP, the output of the model f(·) is considered as all participants x = {x 1 ,x 2 ,…,x n} cooperation results, where x i represents the ith participant, whose contribution is calculated as:

[0011]

[0012] Where N is the set formed by any combination of all participants, and S represents the set that does not contain member x. i The number of elements in the set; N\{x i} means to remove participant x i The collection formed later; represents all the i}; ∪ represents the union operation; (·)! represents the factorial operation; the computational complexity of the Shapley value is exponentially related to the number of members;

[0013] Improved evaluation index: Suppose there is a classification model, when the input sample x = {x 1 ,x 2 ,…,x i ,…,x n} The output is [1,2,10,1]; replace member x with zero value i , the model output is [2,3,8,4]; this result shows that member x i It is a positive influence in the sample; the output value distance between the target neuron and other neurons is used to evaluate the contribution of each member; first, according to the serious nonlinear property of SoftMax, the output result of SoftMax is mainly affected by the distance between the maximum value and the second largest value; the evaluation index is relaxed to the distance between the output of the target neuron and the maximum value of the output of the remaining neurons, which satisfies the property of Shapley value validity:

[0014]

[0015] The sum of the contributions of all members is equal to the overall cooperation result; the reference neuron is fixed by the result of the model under the original sample input and the contribution of each member is evaluated by the distance between its output and the target neuron output; member x i The contribution of is (10-2)-(8-3)=3; rewrite equation (1) as:

[0016]

[0017] Where dis(S) = f T (S)-f Ref (S), subscript f T (·) and f Ref (·) represent the outputs of the target neuron and the reference neuron, respectively;

[0018] Fast Shapley value estimation: The interaction relationship between two players is used to measure the potential relationship between players. The mathematical formula is:

[0019] B(x i ,x j )=φf (x i ,x j |N ij )-φ f (x i |N i )-φ f (x j |N j ) (4)

[0020] Where x j represents the jth member; N ij Indicates that it belongs to N\{x i ,x j}The set of all subsets; φ f (x i |N ij ) represents member x i In the set N ij Shaplye value under; if two members belong to an alliance relationship, then this relationship is established in any level of alliance; consider N\{x i ,x j}; from N, N\{x i}、N\{x j} and N\{x i ,x j} The relationship between members is measured in four aspects to reduce the computational complexity; Formula (4) is rewritten as:

[0021]

[0022] When B(x i ,x j )>0, member x i and x j It is positive collaboration; otherwise, it is negative collaboration; only the member x is retained. i The mathematical formula for the most important alliances is:

[0023]

[0024] where abs(·) represents the absolute value operation; after obtaining the candidate alliance, member x i The Shapley value calculation formula is:

[0025]

[0026] According to the comparative sampling method, equation (7) can be rewritten as:

[0027]

[0028] Where V represents the ignored member; Monte Carlo sampling method is used to calculate equation (8); the summation term in equation (8) is expanded as:

[0029]

[0030] Formula (9) indicates that the weight of each sub-item is different, which means that it is unreasonable to directly apply the Monte Carlo method. Stratified sampling is used to estimate Formula (9);

[0031] First, Take the summation term in equation (9) as an example; this term indicates that when S contains only two members, member x i contribution; by the properties of permutations and combinations, this term has |S i |×(S i |-1) / 2! combinations, which indicates that the item is considered as a member x i The expected contribution when there are only two alliance members; rewrite equation (9) as:

[0032]

[0033] In the formula If there are q members in S, member x i ; if the overall mean can be accurately estimated based on the sample, then avoid traversing all possible combinations of elements in S;

[0034] The mean error is assessed by checking whether the sample reflects the population; the Mann-Whitney test is used to check whether the two samples come from the same distribution. The steps are as follows:

[0035] ① Randomly select two groups of samples from the whole, the number of which is n 1 and n 2 , member x i The contribution of cooperation is denoted as Ω 1 and Ω 2 ;

[0036] ②Establish test hypothesis; H0: Ω 1 and Ω 2 Derived from the same distribution; H1: Ω 1 and Ω 2 Originated from different distributions;

[0037] ③Write Ω 1 and Ω 2 The elements in are sorted in ascending order and the sorting results are recorded; data of the same size are given the same sorting level and the level is the average of the first and last values ​​of the same data;

[0038] ④ Calculate the rank sum of the two sets of data based on the recorded results, recorded as R1 and R2 respectively; calculate the minimum statistic of the two sets of data; the mathematical formula is:

[0039]

[0040] ⑤Calculate the z-value based on the minimum statistic:

[0041]

[0042] ⑥Substitute the z-value into the normal distribution to obtain the p-value;

[0043] If p < 0.05, H0 is rejected and H1 is accepted; this indicates that the two groups of samples are from different distributions, and the estimated mean cannot represent the whole, so the sample needs to be expanded; the combination of all previously drawn samples constitutes a new sample, and the same number of samples as the new sample are drawn from the whole for Mann-Whitney test; if the test passes, it is considered that the sample can represent the whole; the sample mean is proposed to represent the whole mean;

[0044] The overall pseudo code is as follows:

[0045] S1: Input: model to be explained f(·) and x to be explained = {x 1 ,x 2 ,…,x n}, hyperparameter α = 60

[0046] S2: Output: φ f (x)

[0047] S3: When i=1,2,…,n, loop execution of S4~S21:

[0048] S4:B(x i ,:)←Eq.(5)

[0049] S5:S i ←Eq.(6)

[0050] S6: When q=0,1,…,|S i |When executing S6~S19 in a loop:

[0051] S7: k = 0, Ω_T = []

[0052] S8: When k≤5, execute loop S3

[0053] S9:

[0054] S10:

[0055] S11: Ω_T←[Ω_init∪Ω_T]

[0056] S12:Ω_F←Ω_T(1:half),Ω_L←Ω_T(half+1:end)

[0057] S13: z←Eq.(12)

[0058] S14: Calculate p-value

[0059] S15: If p < 0.05:

[0060] S16: Then k=k+1, α=30

[0061] S17: Execute the next loop of the current loop;

[0062] S18: Otherwise

[0063] S19: jump out of the current loop;

[0064] S20:

[0065] S21:φ f (x i )←Eq.(10)

[0066] Among them, R(S i ,q,2 k ×α) represents the i Randomly select q non-repeating elements from and repeat the process 2 k ×α times; ← means assigning the output result on the right side to the left side; then enter the calculated member x i Shapley value under cooperation with samples; It means that the input of f(·) is Ω_T(1:half) and Ω_T(half+1:end) represent the first half and the second half of the elements in Ω_T respectively;

[0067] Sort the explanation results in descending order; activate the input features according to the index order obtained from the sorting results; inactivate the input features according to the index order obtained from the sorting results to obtain the negative impact of losing positive features; activating positive contribution features will increase the distance between the target neuron output and the reference neuron; inactivating these features will suppress the distance; combining the two perspectives to obtain the final evaluation index:

[0068] M(P β )=dis(P β ×data)-dis((1-P β )×data) (13)

[0069] Where data is the input sample, P β is a vector whose elements are {0,1}, and the subscript β indicates that there are β members in the sorting result that are in the activated state; from formula (13), we know that M(P β ) value, the more accurate the interpretation result.

[0070] The beneficial effects of the present invention are as follows: the fast Shapley value estimation method designed by the present invention is more suitable for the interpretation of high-dimensional samples and has better interpretation results than the traditional SHAP. At the same time, the interpretation result evaluation method proposed by the present invention fills the gap in the field of fault diagnosis that there is no quantitative evaluation method for the accuracy of interpretation results.

[0071] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0073] Figure 1 It is a system diagram of the present invention;

[0074] Figure 2 Output results for several post hoc interpretation methods;

[0075] Figure 3 Quantitative evaluation results of the interpretation accuracy constructed in the present invention are obtained for the output results of several post hoc interpretation methods. DETAILED DESCRIPTION

[0076] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0077] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0078] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0079] Problem definition: When interpreting a fault diagnosis network using SHAP, the output of the model f(·) is considered as all participants x = {x 1 ,x 2 ,…,x n} cooperation results. Participant x i The contribution of

[0080]

[0081] Where N is the set formed by any combination of all participants, S and |S| represent the set that does not contain member x. i The set and the number of elements corresponding to the set; represents all the i}. It can be seen that the computational complexity of the Shapley value is exponentially related to the number of members. When the vibration signal is used as the input sample, its dimension is often high. Therefore, SHAP cannot be directly used to explain this type of model. On the other hand, formula (1) only focuses on the output of the target class neurons. In the multi-classification task based on SoftMax, the classification probability is mainly based on the distance between the maximum element and other elements. Therefore, formula (1) is not optimal for performing multi-classification tasks based on SoftMax.

[0082] Improved evaluation index: Assume that there is a classification model, in the input sample x = {x 1 ,x 2 ,…,x i ,…,x n Replace member x with zero value. i, the model output is [2,3,8,4]. This result shows that member x i In the sample, it shows a positive impact. According to the traditional SHAP contribution evaluation index, at this time, member x i The contribution of this member is 10-8=2. On the other hand, according to the nonlinear properties of SoftMax, the contribution of this member should be (10-2)-(8-4)=4. This is significantly different from the contribution in traditional SHAP. Therefore, the present invention intends to use the output value distance between the target neuron and other neurons to evaluate the contribution of each member. First of all, according to the serious nonlinear properties of SoftMax, it can be seen that the output result of SoftMax is mainly affected by the distance between the maximum value and the second largest value. Therefore, the evaluation index is relaxed to the distance between the output of the target neuron and the maximum value of the outputs of the remaining neurons. However, it is still unreasonable to use this indicator directly. The Shapley value needs to meet the property of effectiveness.

[0083]

[0084] That is, the sum of the contributions of all members is equal to the overall cooperation result. The highly nonlinear properties of the neural network result in that the neuron corresponding to the maximum value in the output of the remaining neurons is not fixed. Therefore, the validity of the Shapley value cannot be met. In this regard, the present invention intends to fix the reference neuron through the results of the model under the original sample input and use the distance between its output and the target neuron output to evaluate the contribution of each member. Under this condition, member x in the above simple case i The contribution of is (10-2)-(8-3)=3. Therefore, rewrite equation (1) as

[0085]

[0086] Where dis(S) = f T (S)-f Ref (S), subscripts T and Ref denote target neuron and reference neuron, respectively.

[0087] Fast Shapley value estimation method: The best way to reduce the complexity of SHAP calculation is to eliminate invalid or unimportant participants. Therefore, the present invention intends to use the interaction relationship between two players to measure the potential relationship between players. The mathematical formula is:

[0088] B(x i ,x j )=φ f (x i ,x j |N ij )-φ f (x i |N i )-φf (x j |N j ) (4)

[0089] Where Nij represents the i ,x j Theoretically, if two members belong to an alliance relationship, then this relationship is valid in alliances of any level. This means that we need to consider N\{x i ,x j}. This operation still cannot be performed when the sample dimension is high. To this end, the present invention proposes to start from N, N\{x i}、N\{x j} and N\{x i ,x j} and other four aspects to measure the relationship between members to reduce the computational complexity. Therefore, formula (4) can be rewritten as

[0090]

[0091] When B(x i ,x j )>0, member x i and x j It is positive collaboration; on the contrary, it is negative collaboration. At the same time, Wang pointed out that the most important alliances can be considered to estimate the Shapley value. Based on this, the present invention only retains the member x i The mathematical formula for the most important alliances is:

[0092]

[0093] After obtaining the candidate alliance, member x i The Shapley value calculation formula is:

[0094]

[0095] However, the error of directly calculating the Shapley value according to formula (7) is large. Therefore, according to the comparative sampling method, formula (7) is rewritten as

[0096]

[0097] In the formula, V represents the ignored member. There are two methods to implement the calculation of formula (8). One method is to traverse S i The possible permutations and combinations of all members in can obtain a more accurate estimation result, but its computational complexity is high. Another method is to estimate the Shapley value by Monte Carlo sampling. The present invention proposes to use Monte Carlo sampling to calculate formula (8). First, expand the summation term in formula (8) to

[0098]

[0099] Formula (9) indicates that the weight of each sub-item is different, which means that it is unreasonable to directly apply the Monte Carlo method. Therefore, stratified sampling is used to estimate Formula (9).

[0100] First, Take the summation term in equation (9) as an example. The data of this term is when S contains only two members x i The contribution of . According to the properties of permutations and combinations, this term has |S i |×(S i |-1) / 2! combinations, which indicates that this item can be regarded as a member x i The expected contribution when there are only two alliance members. Therefore, equation (9) can be rewritten as

[0101]

[0102] In the formula When S species consists of q members, member x i On the other hand, formula (10) also shows that if the overall mean can be accurately estimated based on the sample, it is possible to avoid traversing all possible combinations of elements in S.

[0103] To achieve this goal, the present invention intends to evaluate the mean error by checking whether the sample can reflect the population. Due to the nonlinearity of the neural network, the overall distribution of the contribution of each level of alliance is unknown, so the present invention intends to use the Mann-Whitney test to check whether two samples come from the same distribution. The main steps are as follows:

[0104] 1. Randomly select two groups of samples from the whole to calculate their contribution and record them as Ω 1 and Ω 2 .

[0105] 2. Establish the test hypothesis. H0: Ω 1 and Ω 2 Derived from the same distribution; H1: Ω 1 and Ω 2 Comes from different distributions.

[0106] 3. Set Ω 1 and Ω 2 The elements in are sorted from small to large and the sorting results are recorded. It should be noted that data of the same size are given the same sorting level and the level is the average of the first and last values ​​of the same data.

[0107] 4. Calculate the rank sum of the two sets of data based on the recorded results, and record them as R1 and R2 respectively. Calculate the minimum statistic of the two sets of data. The mathematical formula is:

[0108]

[0109] 5. Calculate the z-value based on the minimum statistic:

[0110]

[0111] 6. Substitute the z-value into the normal distribution to obtain the p-value.

[0112] If p < 0.05, then H0 is rejected and H1 is accepted. This indicates that the two groups of samples are drawn from different distributions, and the estimated mean cannot represent the whole, and sample expansion is needed. In the present invention, a new sample is formed by combining all previously drawn samples, and the same number of samples as the new sample are drawn from the whole for the Mann-Whitney test. Otherwise, it is considered that the sampled sample can represent the whole. In order to further improve the accuracy of mean estimation, the present invention intends to use the mean of the drawn samples to represent the overall mean. The overall pseudo code of the method proposed in the present invention is as follows:

[0113] The overall pseudo code is as follows:

[0114]

[0115]

[0116] Quantitative evaluation method for the accuracy of interpretation results: To date, there is no public dataset and model that can be used as a benchmark for evaluating the accuracy of interpretation results. Therefore, it is difficult to compare different interpretation methods. Therefore, the interpretation results are first sorted in descending order. Then, the input features are activated according to the index order obtained from the sorting results; the contribution index designed by the present invention is used to evaluate the contribution of the activated features. On the other hand, the input features are inactivated according to the index order obtained from the sorting results to obtain the negative impact of losing positive features. Activating positive contribution features will increase the distance between the target neuron output and the reference neuron. Inactivating these features can suppress the distance. Combining the two perspectives, the final evaluation index is

[0117] M(P β )=dis(P β ×data)-dis((1-P β )×data) (13)

[0118] Where data is the input sample, P βis a vector whose elements are {0,1}, and the subscript β indicates that there are β members in the sorting result that are in the activated state. From formula (13), we can see that M(P β ) value, the more accurate the interpretation result.

[0119] Figure 1 It is a system diagram of the present invention. The present invention intends to use the vibration signal generated by the test bench combined with the ResNet18 model to illustrate the effectiveness of the proposed method, and compare it with methods such as Shap-Deep, Shap-Grad, Shap-Kernel and FastShap to highlight the advancement of the method of the present invention. The data set contains vibration signals with a sampling frequency of 10khz. The fault types in this data set are the same as those in NEPU. In this test, the motor speed is fixed at 896rpm. Similarly, different loads (0kN, 1kN, 2kN, 3kN) are applied radially to the test bearing through the loading system. In this experiment, the sliding window method is used to construct samples, and the window width and moving step size are both 1024 points. 70% are randomly selected as training sets, and the rest are used as test sets. During training, Adam is used as the optimizer, and the learning rate, number of training rounds, and batch size distribution are set to 0.01, 100, and 256.

[0120] The interpretation results of the method proposed by the present invention and the comparative method are as follows Figure 2 As shown. The method proposed in the present invention, Shap-Deep and Shap-Grad believe that the fault impulse response in the input sample is the reasoning basis of ResNet18. However, a careful comparison reveals that there are certain differences between these results. The method proposed in the present invention believes that most elements in the impulse response show positive contributions, while in the interpretation results of Shap-Deep and Shap-Grad, more elements in the fault response show negative contributions. Compared with these three interpretation results, it is difficult to observe the fault feature response in the interpretation results of the other two methods, and it is very difficult to understand. It can be seen that the interpretation effects of Shap-Kernel and FastShap are poor.

[0121] In order to further highlight the advanced nature of the method constructed by the present invention, the interpretation results of each method were evaluated using the interpretation accuracy quantitative evaluation method purchased by the present invention. The results are as follows: Figure 3 As shown. The levels of the interpretation results of Shap-Deep and Shape-Grad are comparable, which is consistent with the interpretation results in the previous text. At the same time, the interpretation results of Shap-Kernel and FastShap are the first, which is also consistent with the interpretation results in the previous text. Compared with the four comparative interpretation results, the method constructed by the present invention achieved the highest score. This phenomenon shows that the interpretation results of the method constructed by the present invention are the most accurate, highlighting the advanced nature of the method of the present invention.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A fast Shapley value estimation method based on stratified sampling, characterized in that: The method comprises the following steps: Problem definition: When interpreting a fault diagnosis network using SHAP, the output of the model f(·) is considered as all participants x = {x1, x2, …, x n } cooperation results, where x i represents the ith participant, whose contribution is calculated as: Where N is the set formed by any combination of all participants, and S represents the set that does not contain member x. i The number of elements in the set; N\{x i } means to remove participant x i The collection formed later; represents all the i }; ∪ represents the union operation; (·)! represents the factorial operation; the computational complexity of the Shapley value is exponentially related to the number of members; Improved evaluation index: Suppose there is a classification model, when the input sample x={x1,x2,…,x i ,…,x n } The output is [1,2,10,1]; replace member x with zero value i , the model output is [2,3,8,4]; this result shows that member x i It is a positive influence in the sample; the output value distance between the target neuron and other neurons is used to evaluate the contribution of each member; first, according to the serious nonlinear property of SoftMax, the output result of SoftMax is affected by the distance between the maximum value and the second largest value; the evaluation index is relaxed to the distance between the output of the target neuron and the maximum value of the output of the remaining neurons, satisfying the property of Shapley value validity: The sum of the contributions of all members is equal to the overall cooperation result; the reference neuron is fixed by the result of the model under the original sample input and the contribution of each member is evaluated by the distance between its output and the target neuron output; member x i The contribution of is (10-2)-(8-3)=3; rewrite equation (1) as: Where dis(S) = f T (S)-f Ref (S), subscript f T (·) and f Ref (·) represent the outputs of the target neuron and the reference neuron, respectively; Fast Shapley value estimation: The interaction relationship between two players is used to measure the potential relationship between players. The mathematical formula is: Where x j represents the jth member; N ij Indicates that it belongs to N\{x i ,x j }The set of all subsets; φ f (x i |N ij ) indicates that member xi is in the set N ij Shaplye value under; if two members belong to an alliance relationship, then this relationship is established in any level of alliance; consider N\{x i ,x j }; from N, N\{x i }、N\{x j } and N\{x i ,x j } The relationship between members is measured in four aspects to reduce the computational complexity; Formula (4) is rewritten as: B(x i ,x j )=f(N)-f(N\x i ,x j )-[f(N)-f(N\x i )]-[f(N)-f(N\x j )] =dis(N\x i )+dis(N\x j )-dis(N\x i ,x j )-dis(N) (5) When B(x i ,x j )>0, member x i and x j It is positive collaboration; otherwise, it is negative collaboration; only the member x is retained. i The mathematical formula for the most important alliances is: where abs(·) represents the absolute value operation; after obtaining the candidate alliance, member x i The Shapley value calculation formula is: According to the comparative sampling method, equation (7) can be rewritten as: Where V represents the ignored member; Monte Carlo sampling method is used to calculate equation (8); the summation term in equation (8) is expanded as: Formula (9) indicates that the weight of each sub-item is different, which means that it is unreasonable to directly apply the Monte Carlo method. Stratified sampling is used to estimate Formula (9); First, for Analyze the summation term in equation (9); this term indicates that when S contains only two members, member x i contribution; by the properties of permutations and combinations, this term has |S i |×(S i |-1) / 2! combinations, which indicates that the item is considered as a member x i The expected contribution when there are only two alliance members; rewrite equation (9) as: In the formula If there are q members in S, member x i ; if the overall mean can be accurately estimated based on the sample, then avoid traversing all possible combinations of elements in S; The mean error is assessed by checking whether the sample reflects the population; the Mann-Whitney test is used to check whether the two samples come from the same distribution. The steps are as follows: ① Randomly select two groups of samples from the whole, the number of which is n1 and n2 respectively, and the member x i The contributions under their collaboration are recorded as Ω1 and Ω2; ② Establish the test hypothesis; H0: Ω1 and Ω2 come from the same distribution; H1: Ω1 and Ω2 come from different distributions; ③ Sort the elements in Ω1 and Ω2 from small to large and record the sorting results; data of the same size are given the same sorting level and the level is the average of the first and last order of the same data; ④ Calculate the rank sum of the two sets of data based on the recorded results, recorded as R1 and R2 respectively; calculate the minimum statistic of the two sets of data; the mathematical formula is: ⑤Calculate the z-value based on the minimum statistic: ⑥Substitute the z-value into the normal distribution to obtain the p-value; If p < 0.05, H0 is rejected and H1 is accepted; this indicates that the two groups of samples are from different distributions, and the estimated mean cannot represent the whole, so the sample needs to be expanded; the combination of all previously drawn samples constitutes a new sample, and the same number of samples as the new sample are drawn from the whole for Mann-Whitney test; if the test passes, it is considered that the sample can represent the whole; the sample mean is proposed to represent the whole mean; The overall pseudo code is as follows: S1: Input: model to be explained f(·) and x to be explained = {x1, x2, …, x n }, hyperparameter α = 60 S2: Output: φ f (x) S3: When i=1,2,…,n, loop execution of S4~S21: S4:B(x i ,:)←Eq.(5) S5:S i ←Eq.(6) S6: When q=0,1,…,|S i |When executing S6~S19 in a loop: S7: k = 0, Ω_T = [] S8: When k≤5, execute loop S3 S9: S10: S11: Ω_T←[Ω_init∪Ω_T] S12:Ω_F←Ω_T(1:half),Ω_L←Ω_T(half+1:end) S13: z←Eq.(12) S14: Calculate p-value S15: If p < 0.05: S16: Then k=k+1, α=30 S17: Execute the next loop of the current loop; S18: Otherwise S19: jump out of the current loop; S20: S21: f f (x i )←Eq.(10) Among them, R(S i ,q,2 k ×α) represents the i Randomly select q non-repeating elements from and repeat the process 2 k ×α times; ← means assigning the output result on the right side to the left side; then enter the calculated member x i Shapley value under cooperation with samples; It means that the input of f(·) is Ω_T(1:half) and Ω_T(half+1:end) represent the first half and the second half of the elements in Ω_T respectively; Sort the explanation results in descending order; activate the input features according to the index order obtained from the sorting results; inactivate the input features according to the index order obtained from the sorting results to obtain the negative impact of losing positive features; activating positive contribution features will increase the distance between the target neuron output and the reference neuron; inactivating these features will suppress the distance; combining the two perspectives to obtain the final evaluation index: M(P β )=dis(P β ×data)-dis((1-P β )×date) (13) Where data is the input sample, P β is a vector whose elements are {0,1}, and the subscript β indicates that there are β members in the sorting result that are in the activated state; from formula (13), we know that M(P β ) value, the more accurate the interpretation result.

Citation Information

Cited By

  • Multi-agent reinforcement learning algorithm efficiency interpretable evaluation method and system based on Shapley additive interpretation

    CN120524843A

  • Interpretable evaluation method and system for the effectiveness of multi-agent reinforcement learning algorithms based on Shapley additive interpretation

    CN120524843B