Formation rapid interpretable method and system of trusted neural network
By employing formal methods and game theory optimization, a decision tree-based neural network model is constructed, which solves the problems of imprecise explanation and high complexity in existing technologies, and achieves a fast, rigorous, and consistent explanation.
Patent Information
- Application Number
- CN202410976566.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-20
- Publication Date
- 2024-11-19
AI Technical Summary
Existing neural network interpretation methods lack logical rigor and spatiotemporal stability, and are highly complex, failing to provide a decision interpretation that is 100% faithful to the model.
Formal methods are used to compile and simplify neural network models, and formal interpretations are constructed using logical variables and decision trees. Game theory methods are then combined to optimize the interpretation process, achieving a fast and rigorous interpretation.
It provides a logically rigorous and spatiotemporally stable explanation, which can be completed within 0.001 seconds, and the explanation results are completely consistent with the model, making it user-friendly.
Smart Images

Figure CN118982045A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of neural network decision technology, and in particular relates to a formalized, fast and interpretable method and system for a credible neural network. Background Art
[0002] Existing neural networks are black boxes, and it is impossible to give semantic explanations for the decisions they make.
[0003] Existing neural network interpretable methods basically use the idea of approximate fitting to seek explanations for decisions, which are not stable in time and space. Different interpretation methods may give different explanations for the same model, and the same interpretation method may give different explanations for the same model when run at different times. This is because these interpretable methods themselves are not based on formal logic methods, cannot dig out decision logic that is 100% faithful to the interpreted model, and do not have logical reliability and stability in time and space.
[0004] The few existing logically rigorous interpretable machine learning methods based on formal methods often have extremely complex computational processes and can easily reach exponential complexity due to combinatorial explosion. Summary of the invention
[0005] The purpose of the present invention is to provide a formalized, fast and interpretable method and system for a credible neural network.
[0006] The technical solution to realize the present invention is: a formalized fast interpretable method for a credible neural network, comprising:
[0007] Step 1: Get a neural network model that makes decisions in real life, and extract the eigenvalues and parameter information of the model, including the weights, biases, and activation functions of the neural network;
[0008] Step 2: Extract the formal explanation from the neural network module, specifically:
[0009] S1, compile the input feature values of the neural network model obtained in step 1 into logical variables using a formal method;
[0010] S2, based on the logical variables obtained in the previous step, compile the model into a formalized logical expression. The specific method is: multiply the weight value matrix of each layer of neurons in the neural network model to obtain the final weight value; multiply the bias value matrix of each layer of neurons to obtain the final bias value; then, based on the input-output mapping relationship of the activation function of each layer, obtain the threshold-based linear function L corresponding to the neural network model;
[0011] S3, the threshold condition of the linear function L is represented as a decision tree DT from the input to the output mapping space i, the specific method is as follows:
[0012] a) Let the threshold condition be the root node of the decision tree;
[0013] b) Select an eigenvalue sequence x1, x2, ..., x n , traverse all possible values of the feature value in order, each possible value is a path starting from the root node, and the child node reached by each path is the threshold condition after bringing in the situation represented by each path;
[0014] c) Starting from each child node obtained in the previous step, according to the order of the eigenvalues selected in the previous step, x1, x2, ..., x n , traverse each subsequent eigenvalue in turn, traverse each value of the eigenvalue, as a path, and the child node reached is the threshold condition after the value is taken in;
[0015] d) Repeat step c) until the last eigenvalue of the eigenvalue sequence selected in b), and the child node obtained at this time is the leaf node of the decision tree, and the value is the decision result {y1, y2, ..., y n ];
[0016] e) At this point, the construction of the decision tree is completed;
[0017] S4, each path p from the root node to the leaf node of the decision tree represents a formal explanation of a decision;
[0018] S5, classify all paths from the root node to the leaf node of the decision tree according to the values of its leaf nodes.
[0019] S6, based on the path classification obtained in the previous step, merge the paths of each category into similar items and simplify them to PrimeImplicant;
[0020] S7, the decision root cause obtained by simplification in the previous step is expressed in natural language and returned to the user.
[0021] Furthermore, in step 2, a formal explanation is extracted from the neural network module, which can also be:
[0022] S1, compile the input feature values of the neural network model obtained in step 1 into logical variables using a formal method;
[0023] S2, based on the logical variables obtained in the previous step, compile the model into a formalized logical expression. The specific method is: multiply the weight value matrix of each layer of neurons in the neural network model to obtain the final weight value; multiply the bias value matrix of each layer of neurons to obtain the final bias value; then, based on the input-output mapping relationship of the activation function of each layer, obtain the threshold-based linear function L corresponding to the neural network model;
[0024] S3, each threshold condition of the linear function L is transformed into a form Con without negative terms on both sides of the inequality sign according to the sign shift; the transformed threshold condition Con is regarded as the two sides of the two-party adversarial game, and the two sides of the inequality sign are defined as the positive side P and the negative side N respectively; the eigenvalues contained in the two sides are ranked from large to small and The weights corresponding to each eigenvalue are and The maximum values are and The value ranges of both parties are [L p , H p ] and [L n , H n ];
[0025] S4, first determine the value Target of one party, and then determine the strategy for the other party to win at the lowest cost; for the opponent, start first and calculate the value Target of the opponent first n , and then remove the other eigenvalue weights in turn, And round up the quotient Get the winning strategy that considers only one value:
[0026] S5, for the value obtained in the previous step if calculate The value of Target is used as the new target value n1 ;
[0027] S6, the new target value Target obtained in the previous step n1 , remove the second eigenvalue weight of the square in turn And round up the quotient Get the winning strategy considering two values:
[0028] S7, and so on, we can obtain winning strategies that consider three, four, or any finite positive integer number.
[0029] Based on the same inventive concept, the present invention also provides a formalized fast interpretable system of a credible neural network, which is used to implement the above method. The system includes:
[0030] In the first module, a machine learning model that makes decisions in real life is obtained, and the feature values and parameter information of the model are extracted, including the weights, biases, and activation functions of the neural network.
[0031] The second module is used to extract formal explanations from machine learning models, specifically:
[0032] Compile the input feature values of the machine learning model obtained in the first module into logical variables using formal methods;
[0033] Compile the model into a formalized logical expression. The specific method is as follows: multiply the weight value matrix of each layer of neurons in the neural network model Model to obtain the final weight value; multiply the bias value matrix of each layer of neurons to obtain the final bias value; then, according to the input-output mapping relationship of the activation function of each layer, obtain the threshold-based linear function L corresponding to the neural network model;
[0034] The threshold condition of the linear function L is represented as a decision tree DT from the input to the output mapping space i ;
[0035] Each path p from the root node to the leaf node of the decision tree represents a formal explanation of a decision;
[0036] The obtained decision tree path p is classified according to the value of its leaf node, and all paths from the root node to the leaf node of the decision tree are classified;
[0037] Combine similar items of each type of path and simplify it to Prime Implicant;
[0038] The simplified decision root cause is expressed in natural language and returned to the user.
[0039] Furthermore, the second module, which is used to extract formal explanations from machine learning models, can also be implemented in the following way:
[0040] Compile the input feature values of the machine learning model obtained in the first module into logical variables using formal methods;
[0041] Compile the model into a formalized logical expression. The specific method is as follows: multiply the weight value matrix of each layer of neurons in the neural network model Model to obtain the final weight value; multiply the bias value matrix of each layer of neurons to obtain the final bias value; then, according to the input-output mapping relationship of the activation function of each layer, obtain the threshold-based linear function L corresponding to the neural network model;
[0042] Each threshold condition of the linear function L is transformed into a form Con without negative terms on both sides of the inequality sign by shifting the sign. The transformed threshold condition Con is regarded as the two sides of the two-party adversarial game, and the two sides of the inequality sign are defined as the affirmative side P and the negative side N respectively. The eigenvalues of the two sides are ranked from large to small. and The weights corresponding to each eigenvalue are and The maximum values are and The value ranges of both parties are [L p , H p ] and [L n , H n ];
[0043] First determine the value Target of one party, and determine the strategy for the other party to win at the lowest cost; for the opponent, start first and calculate the value Target of the opponent first. n , and then remove the other eigenvalue weights in turn, And round up the quotient Get the winning strategy that considers only one value:
[0044] For the value obtained in the previous step if calculate The value of Target is used as the new target value n1 ;
[0045] The new target value Target obtained in the previous step n1 , remove the second eigenvalue weight of the square in turn And round up the quotient Get the winning strategy considering two values:
[0046] By analogy, we can obtain winning strategies that consider three, four, or any finite positive integer number.
[0047] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the computer program is loaded into the processor.
[0048] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.
[0049] A computer program product comprises a computer program, which implements the steps of the above method when executed by a processor.
[0050] Compared with the prior art, the present invention has the following significant advantages:
[0051] The present invention uses formal methods to automatically compile, simplify, and extract PrimeImplicant explanations of the neural network model, and automatically translates them into natural language explanations and returns them to the user;
[0052] Formalization is verifiable, ensuring that the explanations given are logically rigorous and complete, and 100% faithful to the model;
[0053] Faster, simplifying the NP-hard problem of traditional brute force traversal of decision trees to a task that can be completed in 0.001 seconds;
[0054] It is user-friendly and provides semantically complete explanations, which can accurately explain the decisions provided by the model in the simplest language. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 A diagram to formalize the basic process of interpretable machine learning.
[0056] Figure 2 The main process diagram is shown in Figure 2.
[0057] Figure 3 A flowchart for compiling knowledge and extracting formal explanations for machine learning models such as neural networks. DETAILED DESCRIPTION
[0058] The present invention will be further described below in conjunction with specific examples. The following examples are only used to more clearly illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention.
[0059] The present invention is a novel neural network model interpretable system. By formalizing and simplifying the neural network model, taking game as the decision-making process model of the model, and taking game tree traversal as the interpretation method of the model decision, a logically rigorous and complete interpretation of the neural network model is obtained, thereby realizing a credible neural network model.
[0060] The interpretable system has two working modes, one is the global interpretable mode and the other is the local interpretable mode. The global interpretable mode explains the decision rules of the entire model, which is independent of the specific instance; the local interpretable mode explains the root cause of the decision made by the model on the specific instance based on the input. In the global interpretable mode, the system accepts the model to be explained and outputs the root cause described in natural language; in the local interpretable mode, the system receives the model to be explained and an instance to be explained, and outputs the logically rigorous and simplest formal explanation of the decision made by the model on the instance, i.e. the root cause.
[0061] The interpretable system mainly does two parts of work, namely, compiling the neural network model and simplifying the compiled logical expression. When interpreting the neural network, the system will convert the model into a formalized logical expression, such as a threshold-based linear function (L), based on the model's parameters, such as feature variables, weights, biases, activation functions, etc., and then find the boundary input value (BIV) when the output of the function (L) changes by traversing the input-output mapping space of the function (L), which is a formal explanation of the decision made by the model.
[0062] The purpose of the present invention is to complete a practical solution for a formalized interpretable neural network, open the black box of the neural network model, complete the extraction and representation of the decision rules of the neural network model within a limited time, for human experts to review, and realize a credible neural network. The existing neural network interpretable methods basically use the idea of approximate fitting to seek explanations for decisions. They are not explanations obtained by formal methods, and cannot dig out decision logic that is 100% faithful to the interpreted model, and do not have logical reliability and spatiotemporal stability. The existing few logically rigorous interpretable machine learning methods based on formal methods often have extremely complex calculation processes. They are limited by combinatorial explosions and can easily reach exponential complexity, or can only be used for simple neural networks with Boolean eigenvalues. For deep neural network models, the amount of calculation of traversing the function (L) is too large, and it is easy to encounter combinatorial explosion problems. The present invention proposes a method for quickly calculating the boundary input value (BIV), which can directly calculate the boundary input value (BIV) when the output of the function (L) changes, without traversing all possible input and outputs. This is also the biggest innovation of the present invention.
[0063] Another highlight of the present invention is that it can provide a formal explanation for neural networks of continuous variables.
[0064] The basic idea of the formal machine learning interpretable method is to regard the machine learning model as a chip, model its neurons and internal architecture into (potentially very complex) logic circuits through abstraction and mapping, and then formally verify it using the method of formally verifying logic circuits to obtain the root cause of each decision, such as Figure 1 The explanation obtained by this method is logically sufficient and complete, absolutely rigorous, and stable in time and space, which means that no matter when, under what circumstances, or how many times it is repeated, the explanation is absolutely correct and stable.
[0065] The schematic diagram and framework diagram of the present invention are respectively as follows Figure 1 to Figure 3 shown.
[0066] Step 1. Obtain a machine learning model (Model) for making decisions in real life, whose functions include but are not limited to classifiers (classifiers), and models include but are not limited to neural networks and models based on neural networks, such as deep learning, RNN, transformer, etc., and extract the feature values and parameter information of the model, such as the weights, bias, activation function, etc. of the neural network.
[0067] Step 2. Extract formal explanations from the neural network model. The flowchart is as follows Figure 3 shown.
[0068] S1, compile the input feature values of the model obtained in the previous step into logical variables using formal methods. Specifically including but not limited to:
[0069] If it is a continuous variable, it becomes a multi-valued logical variable after discretization;
[0070] Or, after being discretized and Booleanized, it becomes a Boolean logic variable;
[0071] S2, based on the logical variables obtained in the previous step, compile the model Model into a formalized logical expression, including but not limited to the threshold-based linear function (L). The specific method is: multiply the weight value matrix of each layer of neurons in the neural network model (Model) to obtain the final weight value; multiply the bias value matrix of each layer of neurons to obtain the final bias value. Then, based on the input-output mapping relationship of the activation function of each layer, the threshold-based linear function (L) corresponding to the neural network model is obtained.
[0072] The threshold-based linear function is formally defined as follows:
[0073] Definition 1 (threshold-based linear function): If the range of a function is a finite value f(x)∈{y1,y2,...,y n}, n∈N, the domain of f(x) is And there is where ω i is a constant, which is x i weight; if f(x) has the following form:
[0074]
[0075] Among them, T1, T2, ..., T n is a constant, representing the threshold, then f(x) is called a linear function based on the threshold. It is called the threshold condition.
[0076] S3, then the threshold condition of the function (L) is represented as a decision tree (DTi ), the specific method is as follows:
[0077] a) Let the threshold condition be the root node of the decision tree;
[0078] b) Select an eigenvalue sequence, such as x1, x2, ..., x n , traverse all possible values of the feature value in order, each possible value is a path starting from the root node, and the child node reached by each path is the threshold condition after the situation represented by each path is brought in. For example, x1∈[1,3], then traverse the three situations of x1=1, x1=2, and x1=3 in turn, that is, the three paths starting from the root node, and the corresponding three child nodes are respectively substituted with x1=1, x1=2, and x1=3 The formula obtained later
[0079] c) Starting from each child node obtained in the previous step, according to the order of the eigenvalues selected in the previous step, x1, x2, ..., x n , traverse each subsequent eigenvalue in turn, traverse each value of the eigenvalue as a path, and the child node reached is the threshold condition after the value is taken in. For example, in this example, x2 should be traversed next, such as x2∈[1,5]. Starting from the three child nodes obtained in the previous step, each child node has 5 paths x2=1, x2=2, ... x2=5, and the child nodes reached are the threshold conditions after the path value is substituted.
[0080] d) Repeat step c) until the last eigenvalue of the eigenvalue sequence selected in b), and the child node obtained at this time is the leaf node of the decision tree, and the value is the decision result {y1, y2, ..., y n ].
[0081] e) At this point, the construction of the decision tree is completed.
[0082] S4, each path (p) from the root node to the leaf node of the decision tree represents a formal explanation of a decision.
[0083] S5, the obtained decision tree path (ρ) is classified according to the value of its leaf node (i.e. the final decision value). For example, for a classifier with decision values of True and False, all paths of the decision tree can be divided into two categories: “paths with a final value of True” and “paths with a final value of False”. ", and "Paths ending in False ".make (in, is the path ending in True, is the path with the ending False), we can know is the input interval for which all decisions of the model are True, The input interval for which all decisions of the model are False.
[0084] S6, based on the path classification obtained in the previous step, merge the paths of each category into similar items and simplify them to PrimeImplicant. Simplify to the extreme Will Simplify to the extreme This is the Prime Implicant explanation of the model and the root cause of the decision.
[0085] S7, the decision root cause obtained by simplification in the previous step is expressed in natural language and returned to the user.
[0086] The present invention also provides a solution, which replaces step 2 with the following step 3. The two steps are parallel, and you can choose either step 2 or step 3. The selection criteria of the two methods are: when the model is simple (few layers, few neurons, simple architecture), you can use step 2 or step 3. When the model scale is large (greater than a set threshold), using step 2 will lead to combinatorial explosion and cannot be completed within a limited time under realistic conditions. At this time, step 3 should be used, which can be completed quickly within a limited time.
[0087] Step 3. Another method to extract formal explanations from neural network models, a variation of S3~S6 in step 2.
[0088] S1~S2 are the same as S1~S2 in step 2.
[0089] S3. Transform each threshold condition of the function (L) into a form (Con) without negative terms on both sides of the inequality sign. The transformed threshold condition (Con) is regarded as the two sides of the two-party adversarial game, and the two sides of the inequality sign are defined as the positive side (P) and the negative side (N). The eigenvalues of the two sides are ranked from large to small. and The weights corresponding to each eigenvalue are and The maximum values are and The value ranges of both parties are [L p , H p ] and [L n , H n ].
[0090] S4. First determine the value of one side (Target), and then determine the strategy of the other side to win at the lowest cost. For example, the opponent starts first, first calculate the value of the opponent (Targetn ), and then remove the other eigenvalue weights in turn And round up the quotient Get a winning strategy that considers only one value
[0091] S5. Get the value from the previous step if calculate The value of n1 ).
[0092] S6. The new target value (Target n1 ), remove the second eigenvalue weight of the square in turn And round up the quotient Get the winning strategy considering two values
[0093] S7. By analogy, we can obtain winning strategies that consider three, four, or any finite positive integer number of strategies.
[0094] It can be proved that the winning strategy obtained by this method can directly get the boundary of the model game range, which is the formal interpretation of the model.
[0095] The present invention is described in detail below with reference to the accompanying drawings and embodiments.
[0096] Example
[0097] Application of data pattern recognition scenarios in Web-SQL Injection network intrusion detection.
[0098] 27 eigenvalues of the real Web-SQL Injection data were selected and discretized using the MDLP discretization method in step 2 S1. An MLP neural network model was trained with an average accuracy AUC value of 0.93. The model has three layers, with 27, 100, and 1 neurons respectively, and the activation function is the identity function. According to step 3 S3, the eigenvalues of the model can be divided into:
[0099] Proponent:
[0100] Opposing side:
[0101] After step 2 S2, the threshold-based linear function of the model is obtained as follows:
[0102]
[0103] If the remaining steps of step 2 are used, there will be a combinatorial explosion. It is necessary to calculate 7×6×8×8×7×7×6×3×7×7×11×6×7×11×9×8×13×10×10×11×11×7×10×10×2×12=605463345439599700000000000 times. The final decision tree requires so many branches, which is impossible to construct in practice.
[0104] But if we use step 3, we first normalize the threshold condition in formula [1] as follows:
[0105]
[0106] Threshold conditions
[0107]
[0108] According to S3 to S5 in step 3, the boundary of the model game interval can be obtained within the complexity of O(1), that is, the formal interpretation of the model, for example:
[0109] Explanation 1: If the eigenvalue of a network flow is reversed or or or or or or or At the same time, if the other negative values are 1, no matter what values the positive values are, the model will determine that this network flow is a Web SQL Injection intrusion.
[0110] This explanation calculation takes 0.0001 seconds.
[0111] Explanation 2: If a network flow has an inverse square in its eigenvalues At the same time, #≤4, or at the same time Or at the same time Or at the same time Or at the same time At the same time, if the other negative feature values are all 1, then no matter what value the positive side takes, the model will determine that this network flow is a Web SQL Injection intrusion.
[0112] This explanation calculation takes 0.0002 seconds.
[0113] The present invention can express the complex rules hidden in the model in clear and concise natural language, and at the same time ensure accuracy, rigor and 100% compliance with the original model.
Claims
1. A formalized fast interpretable method for credible neural networks, characterized by: include: Step 1: Get a neural network model that makes decisions in real life, and extract the eigenvalues and parameter information of the model, including the weights, biases, and activation functions of the neural network; Step 2: Extract the formal explanation from the neural network module, specifically: S1, compile the input feature values of the neural network model obtained in step 1 into logical variables using a formal method; S2, based on the logical variables obtained in the previous step, compile the model into a formalized logical expression. The specific method is: multiply the weight value matrix of each layer of neurons in the neural network model to obtain the final weight value; multiply the bias value matrix of each layer of neurons to obtain the final bias value; then, based on the input-output mapping relationship of the activation function of each layer, obtain the threshold-based linear function L corresponding to the neural network model; S3, the threshold condition of the linear function L is represented as a decision tree DT from the input to the output mapping space i , the specific method is as follows: a) Let the threshold condition be the root node of the decision tree; b) Select an eigenvalue sequence x1, x2, ..., x n , traverse all possible values of the feature value in order, each possible value is a path starting from the root node, and the child node reached by each path is the threshold condition after bringing in the situation represented by each path; c) Starting from each child node obtained in the previous step, according to the order of the eigenvalues selected in the previous step, x1, x2, ..., x n , traverse each subsequent eigenvalue in turn, traverse each value of the eigenvalue, as a path, and the child node reached is the threshold condition after the value is taken in; d) Repeat step c) until the last eigenvalue of the eigenvalue sequence selected in b), and the child node obtained at this time is the leaf node of the decision tree, and the value is the decision result {y1,y2,...,y n }; e) At this point, the construction of the decision tree is completed; S4, each path p from the root node to the leaf node of the decision tree represents a formal explanation of a decision; S5, classify all paths from the root node to the leaf node of the decision tree according to the values of its leaf nodes. S6, based on the path classification obtained in the previous step, merge the paths of each category into similar items and simplify them to PrimeImplicant; S7, the decision root cause obtained by simplification in the previous step is expressed in natural language and returned to the user.
2. A formalized fast interpretable method for a credible neural network according to claim 1, characterized in that: The input feature values of the neural network model obtained in step 1 are compiled into logical variables using a formal method. Specifically, if it is a continuous variable, it becomes a multi-valued logical variable after discretization; or, it becomes a Boolean logical variable after discretization and Booleanization.
3. A formalized fast interpretable method for a credible neural network according to claim 1, characterized in that: The threshold-based linear function is formally defined as follows: If the range of a function is finite, f(x)∈{y1,y2,…,y n }, n∈N, the domain of f(x) is And there is where ω i is a constant, which is x i weight; if f(x) has the following form: Among them, T1, T2, ..., T n is a constant, representing the threshold, then f(x) is called a linear function based on the threshold. It is called the threshold condition.
4. A formalized fast interpretable method for a credible neural network according to claim 1, characterized in that: Step 2: Extract the formal explanation from the neural network module, which can also be: S1, compile the input feature values of the neural network model obtained in step 1 into logical variables using a formal method; S2, based on the logical variables obtained in the previous step, compile the model into a formalized logical expression. The specific method is: multiply the weight value matrix of each layer of neurons in the neural network model to obtain the final weight value; multiply the bias value matrix of each layer of neurons to obtain the final bias value; then, based on the input-output mapping relationship of the activation function of each layer, obtain the threshold-based linear function L corresponding to the neural network model; S3, transforming each threshold condition of the linear function L into a form Con without negative terms on both sides of the inequality sign according to the sign shift; The transformed threshold condition Con is regarded as the two sides of the two-party adversarial game, and the two sides of the inequality sign are defined as the affirmative side P and the negative side N respectively; the eigenvalues of the two sides are sorted from large to small. and The weights corresponding to each eigenvalue are and The maximum values are and The value ranges of both parties are [L p , H p ] and [L n , H n ]; S4, first determine the value Target of one party, and then determine the strategy for the other party to win at the lowest cost; for the opponent, start first and calculate the value Target of the opponent first n , and then remove the other eigenvalue weights in turn, And round up the quotient Get the winning strategy that considers only one value: S5, for the value obtained in the previous step if calculate The value of Target is used as the new target value n1 ; S6, the new target value Target obtained in the previous step n1 , remove the second eigenvalue weight of the square in turn And round up the quotient Get the winning strategy considering two values: S7, and so on, we can obtain winning strategies that consider three, four, or any finite positive integer number.
5. A formalized fast interpretable system for credible neural networks, characterized by: For implementing the method described in any one of claims 1 to 3, the system comprises: In the first module, a machine learning model that makes decisions in real life is obtained, and the feature values and parameter information of the model are extracted, including the weights, biases, and activation functions of the neural network. The second module is used to extract formal explanations from machine learning models, specifically: Compile the input feature values of the machine learning model obtained in the first module into logical variables using formal methods; Compile the model into a formalized logical expression. The specific method is as follows: multiply the weight value matrix of each layer of neurons in the neural network model Model to obtain the final weight value; multiply the bias value matrix of each layer of neurons to obtain the final bias value; then, according to the input-output mapping relationship of the activation function of each layer, obtain the threshold-based linear function L corresponding to the neural network model; The threshold condition of the linear function L is represented as a decision tree DT from the input to the output mapping space i ; Each path p from the root node to the leaf node of the decision tree represents a formal explanation of a decision; The obtained decision tree path p is classified according to the value of its leaf node, and all paths from the root node to the leaf node of the decision tree are classified; Combine similar items of each type of path and simplify it to Prime Implicant; The simplified decision root cause is expressed in natural language and returned to the user.
6. A formalized fast interpretable system for a credible neural network according to claim 5, characterized in that: The second module is used to extract formal explanations from machine learning models, which can also be implemented as follows: Compile the input feature values of the machine learning model obtained in the first module into logical variables using formal methods; Compile the model into a formalized logical expression. The specific method is as follows: multiply the weight value matrix of each layer of neurons in the neural network model Model to obtain the final weight value; multiply the bias value matrix of each layer of neurons to obtain the final bias value; then, according to the input-output mapping relationship of the activation function of each layer, obtain the threshold-based linear function L corresponding to the neural network model; Each threshold condition of the linear function L is converted into a form Con without negative terms on both sides of the inequality sign by shifting the terms according to the sign; The transformed threshold condition Con is regarded as the two sides of the two-party adversarial game, and the two sides of the inequality sign are defined as the affirmative side P and the negative side N respectively; the eigenvalues of the two sides are sorted from large to small. and The weights corresponding to each eigenvalue are and The maximum values are and The value ranges of both parties are [L p , H p ] and [L n , H n ]; First determine the value Target of one party, and determine the strategy for the other party to win at the lowest cost; for the opponent, start first and calculate the value Target of the opponent first. n , and then remove the other eigenvalue weights in turn, And round up the quotient Get the winning strategy that considers only one value: For the previous step, the value if calculate The value of Target is used as the new target value n1 ; The new target value Target obtained in the previous step n1 , remove the second eigenvalue weight of the square in turn And round up the quotient Get the winning strategy considering two values: By analogy, we can obtain winning strategies that consider three, four, or any finite positive integer number.
7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into a processor, the steps of the method according to any one of claims 1 to 4 are implemented.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 4 are implemented.
9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 4 are implemented.
