An interpretable neural network optimization method and system based on bounded orthogonal constraints

Abstract

The application discloses an explainable neural network optimization method and system based on bounded orthogonality constraints, and the method comprises the following steps: obtaining a training data set; training neural network model parameters by using the training data set; wherein the neural network model parameters comprise neural network model weight parameters based on boundedness constraints and same-layer neural network weight parameters based on orthogonality constraints; a loss function of an optimization target is constructed based on the boundedness constraints and the orthogonality constraints, and the neural network model parameters are optimized until the loss function converges to update the neural network model parameters to obtain a trained neural network model. The application can obtain a neural network model with strong explainability, the model can realize high-precision backtracking reconstruction and saliency map generation, and the classification performance of the model is improved, so that the trust degree of applying a black box model in a high-reliability industry such as automatic driving is improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of neural network optimization, and in particular to an interpretable neural network optimization method and system based on bounded orthogonal constraints. BACKGROUND

[0002] The invention of deep neural networks (DNNs) has facilitated almost all artificial intelligence applications. However, the black-box nature of DNNs hinders people's understanding of the internal working principles of the network. Industries such as autonomous driving and quantitative finance that require high reliability need interpretable models. DNNs are usually used as a kind of universal mapping function to connect the input data space and the hidden variable space. Therefore, interpretability is usually considered from two mapping directions: (1) forward direction (understandability): refers to the ability of the model to produce an intuitive understanding of how each module transforms the input into the hidden variable; (2) backward direction (transparency): refers to the ability to reversely transform the hidden variable to the original space. Only when both understandability and transparency are possessed can the model have interpretability.

[0003] The properties of neural networks are highly related to their corresponding optimization problems. The basic stochastic gradient descent (SGD) optimizer generally updates the parameters along the direction of the random gradient. Later optimizers (such as RMSProp and Adam) calculate adaptive gradients by using second-order moment estimation and sliding average. Other researches focus on using flat optimization target surfaces to improve generalization performance. In addition, optimization constraints also affect the properties of the model. Widely used L1 and L2 regularization obtains better generalization by filtering out redundant parameters. AdamW improves generalization by separating weight decay and training objectives. Recent researches use decoupled constraints to improve the interpretability of the model by forcing each filter to represent a specific data pattern. Transform invariance constraints then appear to improve the robustness of interpretability. However, these methods usually face the trade-off between performance and interpretability, and are difficult to generalize to different model structures.

[0004] Many applications desire a deep understanding of the internal mechanism of deep neural networks, which has driven the development of explainable methods in the past decade. These methods are divided into two categories, black-box and white-box methods, according to whether the internal neuron state is considered. Black-box explanation focuses on the external behavior of the model. Methods based on saliency assign saliency to the pixels that have the greatest impact on the model's prediction using activation maps, gradient maps, or perturbation maps. Proxy-based methods approximate the input / output relationship by simple proxy models, such as linear models, Shapley values, and probabilistic models. Although these methods can achieve good results, the black-box nature still hinders further understanding of the internal mechanism of the model. Therefore, white-box methods are needed to provide a deep understanding of the deep network. However, existing white-box methods are usually based on unfounded assumptions. Backtracking-based methods usually assume that each filter represents a specific pattern and can reconstruct the input feature by weighted summation; decomposition-based methods believe that the overall feature can be linearly decomposed at reference points; hybrid model-based methods assist in understanding the internal mechanism by decoupling transparent rules such as decision trees, additive models, and entropy rules; and some methods propose to regularize network parameters by decoupling and invariance constraints. However, existing research has not formed a formal mathematical definition of explainability, so there is a lack of theoretical guarantee for the definition of the relationship between these assumptions and explainability. SUMMARY

[0005] The present application aims to at least partially solve one of the problems in the related art.

[0006] Existing explainability work usually improves model explainability by observing or interfering with the internal state of the neural network, but these methods are generally based on special assumptions and lack theoretical guarantees for the correlation between their assumptions and target explainability. Therefore, the present application proposes an explainable neural network optimization method based on bounded orthogonal constraints by giving a mathematical definition of neural network explainability and deriving its sufficient conditions, the purpose of the present application is to realize the explainability of black-box deep neural networks. Aligning the parameters of deep feedforward neural networks with semantic understanding, revealing the internal reasoning logic of black-box models, and monitoring the model state with the help of visualization, improving the trust of applying black-box models in high-reliability industries such as autonomous driving, quantitative finance, etc.

[0007] The second aspect of the present application is to propose an explainable neural network optimization system based on bounded orthogonal constraints.

[0008] To achieve the above purpose, the present application proposes an explainable neural network optimization method based on bounded orthogonal constraints, which includes:

[0009] Obtaining a training data set;

[0010] training a neural network model parameter by using the training data set, wherein the neural network model parameter comprises a neural network model weight parameter based on a boundedness constraint and a same-layer neural network weight parameter based on an orthogonality constraint;

[0011] constructing a loss function of an optimization objective based on the boundedness constraint and the orthogonality constraint, and optimizing the neural network model parameter until the loss function converges to update the neural network model parameter to obtain a trained neural network model.

[0012] The boundedness-orthogonality-constrained interpretable neural network optimization method implemented by the present application can further have the following additional technical features:

[0013] Further, after obtaining the trained neural network model, the method further comprises:

[0014] inputting image data into the trained neural network model, and obtaining a plurality of most significant channels of a top-layer feature according to a maximum activation value of each channel of the image data;

[0015] performing a reconstruction task on each channel to obtain a plurality of reconstruction results, and binarizing all the reconstruction results by using a preset threshold to obtain a mask set;

[0016] weighting and summing all the masks in the mask set according to the activation value to obtain a final saliency map.

[0017] Further, the method further comprises initializing the neural network model parameter to be trained and pre-processing the training data set.

[0018] Further, a constrained optimization problem is obtained based on the boundedness-orthogonality constraint:

[0019]

[0020]

[0021] a constrained optimization problem is converted into an unconstrained problem by using a Lagrange multiplier method:

[0022]

[0023] wherein, and are weight and bias parameters, respectively, is a row vector of the weight, and X is the training data set.

[0024] Further, the update strategy of the neural network model parameter is:

[0025]

[0026] wherein, alpha represents a learning rate, and lambda represents a constraint coefficient.

[0027] To achieve the above object, another aspect of the present application provides an interpretable neural network optimization system based on bounded orthogonality constraint, comprising:

[0028] a data acquisition module configured to acquire a training data set;

[0029] a model training module configured to train neural network model parameters using the training data set; wherein the neural network model parameters comprise neural network model weight parameters based on boundedness constraint and same-layer neural network weight parameters based on orthogonality constraint;

[0030] a parameter updating module configured to construct a loss function of an optimization objective based on the boundedness constraint and the orthogonality constraint, and optimize the neural network model parameters until the loss function converges to update the neural network model parameters to obtain a trained neural network model.

[0031] The interpretable neural network optimization method and system based on bounded orthogonality constraint according to the embodiments of the present application, by initializing and preprocessing the neural network model and the training data; next, using the Bort optimizer to optimize the neural network model until the training loss function converges, respectively using the boundedness and orthogonality constraints to improve the understandability and transparency of the model; finally obtaining a neural network model with strong interpretability, which can realize high-precision backtracking reconstruction and saliency map generation, and at the same time improve the classification performance of the model, thereby improving the trust degree of applying black box model in high-reliability industries such as autonomous driving.

[0032] Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and / or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0033] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0034] Figure 1 A flowchart of the interpretable neural network optimization method based on bounded orthogonality constraint according to the embodiments of the present application;

[0035] Figure 2 A schematic diagram of the interpretable neural network optimization method based on bounded orthogonality constraint according to the embodiments of the present application;

[0036] Figure 3 A schematic diagram of using the model to realize feature backtracking reconstruction or saliency map generation according to the embodiments of the present application;

[0037] Figure 4 A structural schematic diagram of the interpretable neural network optimization system based on bounded orthogonality constraint according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0039] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the protection scope of the present application.

[0040] The bounded orthogonality constraint-based interpretable neural network optimization method and system proposed according to the embodiments of the present application will be described below with reference to the accompanying drawings.

[0041] Figure 1 The flowchart of the bounded orthogonality constraint-based interpretable neural network optimization method according to the embodiments of the present application.

[0042] As shown in Figure 1 , the method includes but is not limited to the following steps:

[0043] S1, obtaining a training data set;

[0044] S2, training neural network model parameters using the training data set; wherein the neural network model parameters include bounded constraint-based neural network model weight parameters and orthogonality constraint-based same-layer neural network weight parameters;

[0045] S3, constructing a loss function of an optimization objective based on the bounded constraint and the orthogonality constraint, and optimizing the neural network model parameters until the loss function converges to update the neural network model parameters to obtain a trained neural network model.

[0046] Further, the present application proposes a bounded orthogonality constraint-based interpretable neural network optimization method, which can realize monitoring and constraint of the black box model in combination with the visualization method, and improve the trust degree of the black box model for high-reliability industry users, Figure 2 A schematic diagram of the principle of the present application, as shown in Figure 2As shown, firstly, aiming at a deep neural network, the application proposes a constrained optimization problem for improving the interpretability of a neural network in combination with a bounded orthogonal constraint. In order to solve the problem efficiently, the application proposes a Bort optimizer, which converts the constrained problem into an unconstrained problem by using a Lagrange multiplier method, and only needs to add a gradient correction term to a conventional gradient descent algorithm to ensure the boundedness and orthogonality of the model weights. Then, for the initialized model and data, the Bort optimizer is used for optimization until the loss function converges to complete training. The trained model can realize high-precision backtracking reconstruction and saliency map generation, and the performance of the classification task can also be improved synchronously. After training, the application further realizes the monitoring of the internal state of the black box model by using a saliency map visualization method, so as to reveal the complex mechanism inside the model. The Bort optimizer proposed by the application can realize the optimization training of various fully connected neural networks, can improve the model performance and interpretability, and is a general trust improvement method for monitoring and constraining the internal space of a model.

[0047] Further, the specific process of the interpretable neural network optimization method based on the bounded orthogonal constraint can be as follows.

[0048] S101, definitions of interpretability and transparency are given.

[0049] For a multi-layer fully connected network (MLP) of l layers, the application records all the training data as and records each layer of hidden variables as The overall model f can be regarded as a composite mapping Each layer f i is a fully connected layer z i with an activation function i (z i-1 ) = σ (W i z i-1 + b i ), where and are weight and bias parameters respectively, and σ represents an activation function. The application mainly analyzes the interpretability from the forward and backward directions.

[0050] For forward projection, information flows from input z i-1 to output z i . For an interpretable neural network, the application considers that the row vector of the weight should look similar to a semantic data pattern, and this property is defined as interpretability, that is,

[0051]

[0052] where Z denotes a set of semantic data patterns.

[0053] For backward reconstruction, the direction considers the output z i How to reconstruct the input from the output where g denotes the reconstruction operation. If the reconstruction operation can be performed layer by layer, and finally reconstructs the input data x with no more than an error of ∈, the invention refers to this property as ∈-transparency, i.e., there exists a reconstruction algorithm g satisfying

[0054] S102, obtaining a bounded orthogonal sufficient condition for interpretability.

[0055] For interpretability, previous interpretability methods [7-8, 11, 22-24] usually assume that the activation value is a measure of the similarity between the input pattern and the corresponding weight However, according to the definition of interpretability in S101, the weight with a higher activation value is often not similar to the input pattern, because The large norm also leads to a significant activation value. Therefore, it is necessary to constrain in a bounded closed ball, i.e.

[0056] where C w is a constant

[0057] The invention denotes the norm of z i-1 as C z , then the inner product has an upper bound According to the Cauchy-Schwarz inequality, the activation value takes the maximum value if and only if there exists a non-negative real number k such that The boundedness constraint ensures that a large activation value represents a high similarity between the corresponding weight and the input pattern, which is a sufficient condition for interpretability.

[0058] For transparency, in a feedforward neural network, each weight corresponds to a specific pattern. Many methods consider that a linear combination of these weights can reconstruct the input, i.e.

[0059]

[0060] where denotes the projection (i.e., the inner product) of z i-1 on , denotes the reconstructed input. Replacing the g function in the transparency definition, we can obtain the following optimization problem to obtain the optimal reconstruction:

[0061]

[0062] where p z represents the distribution of z. The present application minimizes the following objective function by letting the first derivative with respect to z be zero. Considering that transparency should be independent of the data, the present application obtains W T W = I, i.e. the orthogonality constraint. This constraint ensures optimality in the sense of linear combination reconstruction, and is therefore a sufficient condition for transparency.

[0063] S103, optimizing network weights using Bort optimizer.

[0064] First, the neural network to be trained is initialized, and the training set data is preprocessed. Combining the constraint condition proposed in S102 with the conventional gradient descent optimization, the following constrained optimization problem can be obtained:

[0065]

[0066]

[0067] It is easy to see that the orthogonality condition has already included the boundedness condition, so the above problem is converted into an unconstrained problem by using the Lagrange multiplier method as follows:

[0068]

[0069] By solving the gradient of the above optimization objective, the present application proposes the update strategy of the Bort optimizer as follows:

[0070]

[0071] where α represents the learning rate and λ represents the constraint coefficient. By selecting appropriate learning rate α and coefficient λ, the model parameters are updated iteratively by the gradient descent algorithm until the loss function of the training is reduced to the convergence stage.

[0072] S104, using the model to realize feature backtracking reconstruction or saliency map generation.

[0073] Specifically, for the trained model, the bounded orthogonality of its weights can be used to realize backtracking reconstruction and saliency map generation, as shown in Figure 3 .

[0074] For the backtracking reconstruction task, given a certain layer output feature z i , first use the backpropagation algorithm with guidance to obtain s i , and then use the linear combination method to reconstruct This process can be performed layer by layer from top to bottom until the reconstructed input data

[0075] For the saliency map generation task, similar to the backtracking reconstruction task, first, the top n most significant channels of the maximum activation value of each channel are selected i of the top n most significant channels Then the reconstruction algorithm is performed on each channel to obtain n reconstruction results respectively Then select a suitable threshold to binarize all reconstruction results to obtain a mask set Finally, the activation value is used to weight and sum all the masks to obtain the final saliency map. The saliency map reveals the attention mechanism of the black box model, which helps users understand the reasoning logic of the model and improves their trust in the black box model.

[0076] In summary, the present application imposes boundedness constraints on neural network weights to improve understandability. The activation value in the neural network is an observation index for measuring the correlation between data features and model weights. However, since the activation value is the inner product of data features and model weights, the inner product is not a strict index for measuring similarity. The present application eliminates the influence of weight modulus length information on similarity by imposing boundedness constraints on model weights, so that the physical meaning of the inner product converges to the cosine distance. Orthogonality constraints are imposed on the same layer neural network weights to improve transparency. Many model backtracking methods assume that input features can be reconstructed by linear combination of weight vectors, without considering the coupling between different weight vectors. The present application eliminates the coupling between weights by imposing orthogonality constraints on the same layer model weights, ensuring the optimality of feature reconstruction under linear combination. The Bort optimizer is proposed by combining boundedness and orthogonality constraints. Adding boundedness and orthogonality constraints directly in the model optimization process will increase the difficulty of optimization. In order to efficiently solve the optimization of constrained problems, the present application proposes the Bort optimizer, which adds a new gradient correction term to the original gradient direction using the Lagrange multiplier method, and converts the constrained problem into an unconstrained problem. The optimizer can improve the explainability of the black box model, and monitor and constrain the internal state of the black box model through visualization, thereby improving the user's trust in the black box model.

[0077] The explainable neural network optimization method based on bounded orthogonality constraints according to the embodiments of the present application can realize high-precision backtracking reconstruction and saliency map generation, while improving the classification performance of the model, thereby improving the trust in the application of the black box model in high-reliability industries such as autonomous driving.

[0078] To achieve the above embodiments, as Figure 4 shown, the present embodiment also provides an explainable neural network optimization system 10 based on bounded orthogonality constraints. The system 10 includes a data acquisition module 100, a model training module 200, and a parameter updating module 300.

[0079] The data acquisition module 100 is used to acquire a training data set.

[0080] a model training module 200 configured to train neural network model parameters using a training dataset, wherein the neural network model parameters comprise neural network model weight parameters based on a boundedness constraint and same-layer neural network weight parameters based on an orthogonality constraint;

[0081] a parameter updating module 300 configured to construct a loss function of an optimization objective based on the boundedness constraint and the orthogonality constraint, and to optimize the neural network model parameters until the loss function converges to update the neural network model parameters to obtain a trained neural network model.

[0082] Further, after obtaining the trained neural network model, the system 10 further comprises a data reconstruction module configured to:

[0083] input image data into the trained neural network model, and obtain a plurality of channels of a top feature most significant according to maximum activation values of each channel of the image data;

[0084] perform a reconstruction task on each channel to obtain a plurality of reconstruction results, and binarize all the reconstruction results using a preset threshold to obtain a mask set;

[0085] weight and sum all the masks in the mask set according to activation values to obtain a final saliency map.

[0086] Further, the system 10 further comprises a preprocessing module configured to initialize the neural network model parameters to be trained and to preprocess the training dataset.

[0087] Further, a constrained optimization problem is obtained based on the boundedness and orthogonality constraints:

[0088]

[0089]

[0090] The constrained optimization problem is converted into an unconstrained problem using a Lagrange multiplier method:

[0091]

[0092] wherein, and are weight and bias parameters, respectively, is a row vector of the weight, and X is the training dataset.

[0093] Further, the updating strategy of the neural network model parameters is:

[0094]

[0095] Wherein, a represents a learning rate, and λ represents a constraint coefficient.

[0096] The interpretable neural network optimization system based on the bounded orthogonal constraint according to the embodiment of the present application can realize high-precision backtracking reconstruction and saliency map generation, and improve the classification performance of the model, thereby improving the trust degree of applying the black box model in high-reliability industries such as automatic driving.

[0097] It should be noted that the computer readable medium described above in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of computer readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, an RF (radio frequency) or the like, or any suitable combination of the above.

[0098] The computer readable medium described above can be contained in the electronic device described above; or can exist separately and not be assembled into the electronic device. The computer readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to execute the determination method of the sediment content in the flowing water body described above.

[0099] Computer program code for carrying out operations of the present disclosure can be written in any one or more programming languages or combinations of languages including object or visual programming languages such as Java, Smalltalk, C++ or conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0100] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples, without contradiction.

[0101] In addition, the terms "first", "second", etc. are used only for the purpose of description and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise explicitly specified.

[0102] Any process or method descriptions or descriptions of the flow diagrams described herein or otherwise described in the specification can be understood as representing the modules, segments, or portions of code that include executable instructions for performing specific logical functions or steps in the processes, and the scope of the preferred embodiments of the present application includes additional implementation in which the functions performed by the described blocks are performed in a different order, including substantially simultaneously, or in reverse order, as will be understood by those skilled in the art.

[0103] The logic and / or steps represented in the flowcharts and / or described herein, for example, can be considered as a sequence of instructions to implement logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device, such as a computer-based system, processor- based system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a computer- readable storage medium or a computer-readable signal medium. The computer- readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include the following: an electrical connection having one or more wires (electrical connections), a portable computer diskette (a magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and stored in a computer memory.

[0104] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, the various steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and in another embodiment, any of the following technologies, known in the art, or their combinations can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0105] Those skilled in the art can understand that all or part of the steps carried out by the above-mentioned embodiments can be completed by programs instructing related hardware, and the programs can be stored in a computer-readable storage medium. When the programs are executed, they include one or a combination of the steps of the method embodiments.

[0106] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing module, or each of the units can be physically present separately, or two or more units can be integrated in one module. The integrated module can be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0107] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. An interpretable neural network optimization method based on bounded orthogonal constraints, characterized in that, The method comprises the following steps: obtaining a training data set; training neural network model parameters by using the training data set; wherein the neural network model parameters comprise neural network model weight parameters based on a boundedness constraint and same-layer neural network weight parameters based on an orthogonality constraint; constructing a loss function of an optimization objective based on the boundedness constraint and the orthogonality constraint, and optimizing the neural network model parameters until the loss function converges to update the neural network model parameters to obtain a trained neural network model; wherein after obtaining the trained neural network model, the method further comprises: inputting image data into the trained neural network model, and obtaining a plurality of most significant channels of top-layer features according to maximum activation values of each channel of the image data; performing a reconstruction task on each channel to obtain a plurality of reconstruction results respectively, and binarizing all the reconstruction results by using a preset threshold to obtain a mask set; performing weighted summation on all the masks in the mask set according to activation values to obtain a final saliency map; obtaining a constrained optimization problem based on the constraint condition of the bounded orthogonality constraint: converting the constrained optimization problem into an unconstrained problem by using a Lagrange multiplier method: wherein, and are weight and bias parameters, respectively, is a row vector of weights, is a training data set.

2. The method of claim 1, wherein, The method further comprises initializing the neural network model parameters to be trained and pre-processing the training data set.

3. The method of claim 1, wherein, The update strategy for updating the neural network model parameters is: wherein denotes a learning rate, denotes a constraint coefficient.

4. An interpretable neural network optimization system based on bounded orthogonal constraints, the system comprising: a neural network model; a bounded orthogonal constraint module configured to apply a bounded orthogonal constraint to the neural network model; and a training module configured to train the neural network model with the bounded orthogonal constraint. comprises: a data acquisition module configured to obtain a training data set; a model training module configured to train neural network model parameters by using the training data set; wherein the neural network model parameters comprise neural network model weight parameters based on a boundedness constraint and same-layer neural network weight parameters based on an orthogonality constraint; a parameter update module configured to construct a loss function of an optimization objective based on the boundedness constraint and the orthogonality constraint, and optimize the neural network model parameters until the loss function converges to update the neural network model parameters to obtain a trained neural network model; wherein after obtaining the trained neural network model, the system further comprises a data reconstruction module configured to: input image data into the trained neural network model, and obtain a plurality of most significant channels of top-layer features according to maximum activation values of each channel of the image data; perform a reconstruction task on each channel to obtain a plurality of reconstruction results respectively, and binarize all the reconstruction results by using a preset threshold to obtain a mask set; perform weighted summation on all the masks in the mask set according to activation values to obtain a final saliency map; obtain a constrained optimization problem based on the constraint condition of the bounded orthogonality constraint: convert the constrained optimization problem into an unconstrained problem by using a Lagrange multiplier method: wherein, and are weight and bias parameters, respectively, is a row vector of weights, is a training data set.

5. The system of claim 4, wherein, The system further comprises a pre-processing module configured to initialize the neural network model parameters to be trained and pre-process the training data set.

6. The system of claim 4, wherein, The update strategy for updating the neural network model parameters is: wherein denotes a learning rate, denotes a constraint coefficient.

Images