Model Stability Explanation Method, Device, Equipment and Storage Medium

Abstract

The present application discloses a method, apparatus, device, and storage medium for model stability interpretation. The method includes: when receiving a model stability interpretation instruction, determining a model to be interpreted and determining the condition numbers associated with the matrices of each layer of the model to be interpreted; determining the input data of the model to be interpreted and determining perturbation data for perturbing the input data; determining a stability interpretation result of the model to be interpreted based on the condition numbers of the model to be interpreted and the perturbation data, so as to optimize the model to be interpreted based on the stability interpretation result. Before deploying an artificial intelligence algorithm model, the present application accurately verifies and checks the application reliability of the artificial intelligence algorithm model, avoids potential safety hazards caused by directly deploying a model with low stability, and avoids application risks of the artificial intelligence algorithm model, solving the technical problem in the prior art that the artificial intelligence algorithm model is directly deployed, resulting in application risks of the artificial intelligence algorithm model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology in financial technology (Fintech), and particularly to a method, device, equipment, and storage medium for explaining model stability. Background Art

[0002] With the continuous development of financial technology, especially Internet technology finance, more and more technologies are applied in the financial field. However, the financial industry also poses higher requirements for technologies. For example, the financial industry has higher requirements for explaining model stability.

[0003] When an artificial intelligence algorithm model confronts adversarial attack samples (adversarial samples refer to samples with specific noise data incorporated), it is extremely prone to misjudgment. For example, a face recognition program may misclassify A with added noise data as C. Currently, existing artificial intelligence algorithm models are directly deployed, ignoring the potential safety hazards caused by misjudgment. Obviously, this will pose risks to the application reliability of artificial intelligence algorithm models. Summary of the Invention

[0004] The main objective of this application is to provide a method, device, equipment, and storage medium for explaining model stability, aiming to solve the technical problem in the prior art that artificial intelligence algorithm models are directly deployed, resulting in application risks of artificial intelligence algorithm models.

[0005] To achieve the above objective, this application provides a method for explaining model stability, which includes:

[0006] When receiving a model stability explanation instruction, determine the model to be explained and determine the condition numbers associated with the matrices of each layer of the model to be explained;

[0007] Determine the input data of the model to be explained and determine the perturbation data for perturbing the input data;

[0008] Based on the condition numbers of the model to be explained and the perturbation data, determine the stability explanation result of the model to be explained, so as to optimize the model to be explained based on the stability explanation result.

[0009] Optionally, the step of, when receiving a model stability explanation instruction, determining the model to be explained and determining the condition numbers associated with the matrices of each layer of the model to be explained includes:

[0010] When receiving a model stability explanation instruction, determine the model to be explained and determine the condition numbers associated with the matrices of each layer of the model to be explained;

[0011] Wherein, the condition numbers are determined by the maximum eigenvalues of the matrices of each layer in the model to be explained.

[0012] Optionally, the step of determining the input data of the model to be explained and determining the perturbation data for perturbing the input data includes:

[0013] Determine the input data of the model to be explained, and extract perturbation correlation information from the model stability explanation instruction;

[0014] Based on the perturbation correlation information and the input data, determine the perturbation data of the model to be explained.

[0015] Optionally, the step of determining the perturbation data of the model to be explained based on the perturbation correlation information and the input data includes:

[0016] Extract perturbation amplitude information from the perturbation correlation information;

[0017] Based on the perturbation amplitude information, the input data, and the preset noise data generation rule, generate the perturbation data of the model to be explained.

[0018] Optionally, the step of determining the perturbation data of the model to be explained based on the perturbation correlation information and the input data includes:

[0019] Extract perturbation direction information from the perturbation correlation information;

[0020] Based on the perturbation direction information, the input data, and the preset noise data generation rule, generate the perturbation data of the model to be explained.

[0021] Optionally, the step of determining the stability explanation result of the model to be explained based on the condition number of the model to be explained and the perturbation data, and optimizing the model to be explained based on the stability explanation result includes:

[0022] Determine the correlation degree between the perturbation data and the input data, and determine the correlation relationship between the correlation degree and the condition number of the model to be explained;

[0023] If the correlation relationship is that the correlation degree between the perturbation data and the input data is large and the condition number of the model to be explained is large, determine the model to be explained

[0024] Optionally, the model to be explained is a face recognition model to be explained;

[0025] The step of determining the stability explanation result of the model to be explained based on the condition number of the model to be explained and the perturbation data, and optimizing the model to be explained based on the stability explanation result includes:

[0026] Determine the face recognition stability interpretation result of the face recognition model to be explained based on the condition number of the face recognition model to be explained and the perturbation data.

[0027] This application also provides a model stability interpretation device, which includes:

[0028] A first determination module, configured to determine a model to be explained and determine the condition number associated with each layer matrix of the model to be explained when receiving a model stability interpretation instruction;

[0029] A second determination module, configured to determine the input data of the model to be explained and determine the perturbation data for perturbing the input data;

[0030] A third determination module, configured to determine the stability interpretation result of the model to be explained based on the condition number of the model to be explained and the perturbation data, so as to optimize the model to be explained based on the stability interpretation result.

[0031] Optionally, the first determination module includes:

[0032] A first determination unit, configured to determine a model to be explained and determine the condition number associated with each layer matrix of the model to be explained when receiving a model stability interpretation instruction;

[0033] Wherein, the condition number is determined by the maximum eigenvalue of each layer matrix in the model to be explained.

[0034] Optionally, the second determination module includes:

[0035] A second determination unit, configured to determine the input data of the model to be explained and extract perturbation association information from the model stability interpretation instruction;

[0036] A third determination unit, configured to determine the perturbation data of the model to be explained based on the perturbation association information and the input data.

[0037] Optionally, the second determination unit includes:

[0038] An extraction subunit, configured to extract perturbation amplitude information from the perturbation association information;

[0039] A generation subunit, configured to generate the perturbation data of the model to be explained based on the perturbation amplitude information, the input data, and a preset noise data generation rule.

[0040] Optionally, the generation subunit is used to implement:

[0041] Extract the perturbation direction information from the perturbation association information;

[0042] Generate the perturbation data of the model to be explained based on the perturbation direction information, the input data, and the preset noise data generation rule.

[0043] Optionally, the third determination module includes:

[0044] A fourth determination unit, configured to determine the correlation between the perturbation data and the input data, and determine the correlation relationship between the correlation and the condition number of the model to be explained;

[0045] A fifth determination unit, configured to determine the model to be explained if the correlation relationship is that the correlation between the perturbation data and the input data is large and the condition number of the model to be explained is large.

[0046] Optionally, the model to be explained is a face recognition model to be explained;

[0047] The third determination module includes:

[0048] A sixth determination unit, configured to determine the face recognition stability explanation result of the face recognition model to be explained based on the condition number of the face recognition model to be explained and the perturbation data.

[0049] The present application further provides a model stability explanation device, which is an entity node device. The model stability explanation device includes: a memory, a processor, and a program of the model stability explanation method stored on the memory and executable on the processor. When the program of the model stability explanation method is executed by the processor, the steps of the model stability explanation method as described above can be implemented.

[0050] The present application further provides a storage medium, on which a program for implementing the above model stability explanation method is stored. When the program of the model stability explanation method is executed by a processor, the steps of the model stability explanation method as described above are implemented.

[0051] The present application further provides a computer program product, including a computer program, which when executed by a processor implements the steps of the above model stability explanation method.

[0052] The present application provides a method, apparatus, device and storage medium for explaining model stability. Compared with the prior art where artificial intelligence algorithm models are directly deployed, which causes application risks of artificial intelligence algorithm models, in the present application, before deploying an artificial intelligence algorithm model, the stability and reliability of the model are tested. Specifically, when receiving a model stability explanation instruction, the condition numbers associated with the matrices of each layer of the model to be explained are determined, and the input data of the model to be explained and the perturbation data for perturbing the input data are determined. Furthermore, based on the condition numbers of the model to be explained and the perturbation data, the stability explanation result of the model to be explained is determined, so as to optimize the model to be explained based on the stability explanation result. That is, in the present application, by determining the influence of the perturbation data corresponding to the input data on the condition numbers of the model, the stability of the model to be explained is determined and explained, and then the model to be explained is optimized. That is, before deploying an artificial intelligence algorithm model, the application reliability of the artificial intelligence algorithm model is accurately verified and checked, so as to avoid potential safety hazards caused by directly deploying a model with low stability, and avoid application risks of artificial intelligence algorithm models, thus solving the technical problem in the prior art that artificial intelligence algorithm models are directly deployed, resulting in application risks of artificial intelligence algorithm models. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or in the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0055] Figure 1 It is a schematic flowchart of the first embodiment of the model stability explanation method of the present application;

[0056] Figure 2 It is a schematic flowchart of the refinement steps for determining the input data of the model to be explained and the perturbation data for perturbing the input data in the model stability explanation method of the present application;

[0057] Figure 3 It is a schematic diagram of the device structure of the hardware operating environment related to the solution of the embodiment of the present application.

[0058] The implementation, functional features and advantages of the object of the present application will be further described in conjunction with the embodiments with reference to the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] It should be understood that the specific embodiments described herein are merely used to explain the present application and are not used to limit the present application.

[0060] An embodiment of the present application provides a method for explaining model stability. In the first embodiment of the method for explaining model stability of the present application, refer to Figure 1 , the method for explaining model stability includes:

[0061] Step S10, when receiving a model stability explanation instruction, determine the model to be explained, and determine the condition number associated with each layer matrix of the model to be explained;

[0062] Step S20, determine the input data of the model to be explained, and determine the perturbation data for perturbing the input data;

[0063] Step S30, based on the condition number of the model to be explained and the perturbation data, determine the stability explanation result of the model to be explained, so as to optimize the model to be explained based on the stability explanation result.

[0064] The specific steps are as follows:

[0065] Step S10, when receiving a model stability explanation instruction, determine the model to be explained, and determine the condition number associated with each layer matrix of the model to be explained;

[0066] In this embodiment, it should be noted that the method for explaining model stability can be applied to a model stability explanation system, and this model stability explanation system belongs to a model stability explanation device. For the model stability explanation system, there is a model to be explained built in, or the model stability explanation system can call the model to be explained that needs stability explanation of other components, so as to determine the model to be explained and determine the condition number associated with each layer matrix of the model to be explained when receiving a model stability explanation instruction.

[0067] In this embodiment, the specific application scenario can be:

[0068] The trained model needs to be deployed to the application systems of customers or enterprises. Before deployment, it is necessary to perform stability interpretation on the trained model to ensure that customers or enterprises can clarify the stability of the model, so as to decide whether to adopt the trained model or reject it. Specifically, for example, if the trained model is a face recognition model and it needs to be deployed to the transportation system to recognize users at traffic intersections and determine whether the users at traffic intersections violate traffic rules, such as running red lights. Before deploying the face recognition model to the transportation system, it is necessary to first determine the stability of the face recognition model. That is, due to the fact that artificial intelligence algorithm models are extremely prone to misjudgment when dealing with adversarial attack samples (adversarial samples refer to samples with specific noise data). For example, the face recognition program misidentifies A with added noise data as C. Currently, existing artificial intelligence algorithm models are directly deployed, ignoring the security risks caused by misjudgment. Obviously, determining the stability of the face recognition model first will enhance the secure application of the artificial intelligence algorithm model.

[0069] When receiving a model stability interpretation instruction, determine the model to be interpreted and determine the condition numbers associated with the matrices of each layer of the model to be interpreted. Among them, the triggering method of the model stability interpretation instruction can be:

[0070] Method 1: The tester touches or clicks to trigger the model stability interpretation instruction on the model stability interpretation system to determine the model to be interpreted;

[0071] Among them, the model stability interpretation instruction includes the interpretation time, the interpretation method, and the selected model to be interpreted.

[0072] Among them, the interpretation methods include local interpretation methods and global interpretation methods. The partial interpretation method interprets the model stability from a certain direction (positive perturbation direction or negative perturbation direction), and the global interpretation method interprets the model stability from all directions.

[0073] Method 2: On the model stability interpretation system, when receiving the model to be interpreted sent by other components, trigger the model stability interpretation instruction.

[0074] In this embodiment, the condition numbers associated with the matrices of each layer of the model to be interpreted are also determined. Among them, after the model to be interpreted is determined, the condition numbers of the model to be interpreted are determined based on the model to be interpreted. In this embodiment, the scenario of thinking about determining the model stability of the model to be interpreted based on the condition numbers is as follows:

[0075] During the acquisition process of radar signals, it is necessary to stabilize the radar signals for subsequent analysis. The process of stabilizing the radar signals adopts the condition number. Based on this, in this embodiment, it is considered to use the condition number to interpret the model to be explained.

[0076] Specifically, the condition number is a measure of the sensitivity of the solution of the linear equation Ax = b to errors or uncertainties in b. The mathematical definition is that the condition number of matrix B is equal to the product of the norm of B and the norm of the inverse of B, that is, cond(A) = ‖A‖·‖the inverse of A‖.

[0077] Specifically, the steps of determining the model to be explained and the condition numbers associated with the matrices of each layer of the model to be explained when receiving a model stability interpretation instruction include:

[0078] Step S11, when receiving a model stability interpretation instruction, determine the model to be explained and determine the condition numbers associated with the matrices of each layer of the model to be explained;

[0079] Among them, the condition number is determined by the maximum eigenvalue of the matrices of each layer in the model to be explained.

[0080] In this embodiment, if A is the model to be explained, the condition number of A is ||A||2||A -1 |2, which is also called the condition number of model A.

[0081] Specifically, ||.||2 represents the induced two-norm. In this embodiment, the induced two-norm of model A is the sum of the maximum eigenvalues of each layer matrix, or optionally, the induced two-norm of model A is the sum of the average eigenvalues of each layer matrix.

[0082] Specifically, for example, if the model to be explained is a fully connected type model, the matrices of each layer can be 512*512. If the model to be explained is a convolutional type model, the matrices of different layers of this fully connected type model can be 64*3*3, 32*3*3, 128*3*3, etc.

[0083] Specifically, in this embodiment, the process of obtaining the condition number can be:

[0084] Assume that the model to be explained is A, and A can be a deep model, which is not specifically limited. Assume AX = b, where the data x is the input and b is the model output. If the perturbation is delta (δ), then the perturbed solution:

[0085] A(x + δx) = b + δb

[0086] From the above equation, we can get:

[0087] δ = A-1 * δb

[0088] Based on the properties of the induced second norm, it can be further obtained that:

[0089] ||δx||2 <= ||A-1||2||δb||2

[0090] ||b||2 <= ||A||2||x||2

[0091] From this, it can be obtained that:

[0092]

[0093] At this time, considering the influence of δA, the system of equations becomes

[0094] (A + δA)(x + δx) = b

[0095] From the above equation, it can be obtained that: δx = -A-1δ(x + δx)

[0096] That is

[0097]

[0098] where ||A||2||A -1 ||2 is called the condition number of model A.

[0099] Step S20, determine the input data of the model to be explained, and determine the perturbation data for perturbing the input data;

[0100] In this embodiment, after obtaining the condition number, determine the input data of the model to be explained, and determine the perturbation data for perturbing the input data. In this embodiment, perturbation refers to data with introduced noise, and this noise data refers to data that causes the model to misidentify or disrupt the model's identification. For example, if the input data is car loan data to be processed, the perturbation data can be mortgage data. In this embodiment, after obtaining the condition number, determine the input data of the model to be explained, and generate the perturbation data of the input data. In this embodiment, the input data is sample data. Specifically, for example, the input data can be the consumption data of users, or the input data can be the face recognition data of users, or the input data can be the loan data of users. In this embodiment, the specific input data corresponds to the type of the model to be explained.

[0101] In this embodiment, it should be noted that the specific process of generating perturbation data (preset noise data generation rule) based on the input data can be:

[0102] The input data is input into the model to be explained to obtain an output result. It should be noted that the actual result of the input data is known. Therefore, based on the output result and the actual result, the loss value of the loss function of the model to be explained can be obtained. Based on the partial derivative of the corresponding loss value of the loss function, the weight correction values of the matrices in different layers of the model to be explained can be derived. Based on these weight correction values, the parameters of different matrix layers of the model to be explained can be corrected. Regarding the input data as the parameters of the input layer, the corrected data of the input data can thus be derived. Based on the inverse process of this corrected data, the perturbed data can be obtained.

[0103] Among them, the steps of determining the input data of the model to be explained and determining the perturbed data for perturbing the input data include:

[0104] Step S21, determine the input data of the model to be explained, and extract perturbation correlation information from the model stability explanation instruction;

[0105] In this embodiment, the perturbation correlation information is extracted from the model stability explanation instruction. This perturbation correlation information includes information on the number of perturbation factors, that is, it is determined whether there are multiple or single perturbation correlation information. In addition, this perturbation correlation information also includes specific perturbation factors.

[0106] Step S22, based on the perturbation correlation information and the input data, determine the perturbed data of the model to be explained.

[0107] Based on the perturbation correlation information and the input data, determine the perturbed data of the model to be explained, where the perturbation factor can specifically be a weight factor or a gradient factor, etc.

[0108] The step of determining the perturbed data of the model to be explained based on the perturbation correlation information and the input data includes:

[0109] Step A1, extract perturbation amplitude information from the perturbation correlation information;

[0110] In this embodiment, the perturbation amplitude information is extracted from the perturbation correlation information. If the perturbation factor is a gradient, the change value of the gradient is the perturbation amplitude information. If the perturbation factor is a weight, the change value of the weight is the perturbation amplitude information.

[0111] Step A2, based on the perturbation amplitude information, the input data, and the preset noise data generation rule, generate the perturbed data of the model to be explained.

[0112] Specifically, based on the gradient perturbation amplitude information, the input data, and the preset noise data generation rule, generate the gradient perturbation data of the model to be explained, or based on the weight perturbation amplitude information, the input data, and the preset noise data generation rule, generate the weight perturbation data of the model to be explained.

[0113] Step S30, based on the condition number of the model to be explained and the perturbation data, determine the stability explanation result of the model to be explained, so as to optimize the model to be explained based on the stability explanation result.

[0114] After obtaining the condition number of the model to be explained and the perturbation data, determine the stability explanation result of the model to be explained based on the linkage between the condition number of the model to be explained and the perturbation data.

[0115] The step of determining the stability explanation result of the model to be explained based on the condition number of the model to be explained and the perturbation data, so as to optimize the model to be explained based on the stability explanation result, includes:

[0116] Step S31, determine the correlation degree between the perturbation data and the input data, and determine the correlation relationship between the correlation degree and the condition number of the model to be explained;

[0117] In this embodiment, determine the correlation degree between the perturbation data and the input data. The correlation degree between the perturbation data and the input data can be a large correlation degree or a small correlation degree.

[0118] Among them, if the perturbation factor is the gradient, the change value of the gradient is small, the correlation degree between the perturbation data and the input data is small; if the perturbation factor is the gradient, the change value of the gradient is large, the correlation degree between the perturbation data and the input data is large; if the perturbation factor is the weight, the change value of the weight is small, the correlation degree between the perturbation data and the input data is small; if the perturbation factor is the weight, the change value of the weight is large, the correlation degree between the perturbation data and the input data is large.

[0119] Step S32, if the correlation relationship is that the correlation degree between the perturbation data and the input data is large and the condition number of the model to be explained is large, determine that the stability result of the model to be explained is that the stability of the model to be explained is poor, so as to optimize the model to be explained based on the stability explanation result of the poor stability of the model to be explained.

[0120] If the correlation relationship is that the correlation degree between the perturbation data and the input data is large and the condition number of the model to be explained is large, determine that the stability result of the model to be explained is that the stability of the model to be explained is poor, so as to optimize the model to be explained based on the stability explanation result of the poor stability of the model to be explained. δ

[0121] According to the above formula (1), it can be known that the relationship between the condition number and the interference and the input x. When a very small interference will produce a large perturbation, the reliability of model A is very poor, and it is very easy to be attacked by adversarial samples. That is to say, the larger the condition number, the worse the reliability of model A. If the correlation relationship is that the degree of correlation between the perturbation data and the input data is large, and the condition number of the model to be explained is small, it is determined that the stability result of the model to be explained is that the stability of the model to be explained is good.

[0122] In this embodiment, if the stability of the model to be explained is poor, the step of training the model to be explained is returned. Specifically, the weight parameters of the model to be explained are continuously adjusted to obtain a model to be explained with good stability.

[0123] The present application provides a method, device, equipment and storage medium for model stability explanation. Compared with the prior art where artificial intelligence algorithm models are directly deployed, resulting in application risks of artificial intelligence algorithm models, in the present application, before deploying an artificial intelligence algorithm model, the stability and reliability of the model are tested. Specifically, when a model stability explanation instruction is received, the condition number associated with each layer matrix of the model to be explained is determined, the input data of the model to be explained is determined, and the perturbation data for perturbing the input data is determined. Furthermore, based on the condition number of the model to be explained and the perturbation data, the stability explanation result of the model to be explained is determined, so as to optimize the model to be explained based on the stability explanation result. That is, in the present application, by determining the influence of the perturbation data corresponding to the input data on the condition number of the model, the stability of the model to be explained is determined and explained. That is, before deploying an artificial intelligence algorithm model, the application reliability of the artificial intelligence algorithm model is accurately verified and checked, so as to avoid potential safety hazards caused by directly deploying a model with low stability, and to avoid application risks of artificial intelligence algorithm models, solving the technical problem in the prior art that artificial intelligence algorithm models are directly deployed, resulting in application risks of artificial intelligence algorithm models.

[0124] Further, based on the first embodiment of the present application, another embodiment of the present application is provided. In this embodiment, the step of determining the perturbation data of the model to be explained based on the perturbation correlation information and the input data includes:

[0125] Step B1, extracting perturbation direction information from the perturbation correlation information;

[0126] In this embodiment, if the perturbation factor is the gradient, when the gradient becomes smaller, the perturbation direction is positive, and when the gradient becomes larger, the perturbation direction is negative. If the perturbation factor is the weight, when the weight becomes smaller, the perturbation direction is positive, and when the weight becomes larger, the perturbation direction is negative.

[0127] Step B2: Generate the perturbation data of the model to be explained based on the perturbation direction information, the input data, and the preset noise data generation rule.

[0128] To generate the perturbation data of the model to be explained based on the perturbation direction information, the input data, and the preset noise data generation rule, specifically, it can be to generate the perturbation data of the model to be explained based on the positive perturbation gradient, the input data, and the preset noise data generation rule, or to generate the perturbation data of the model to be explained based on the negative perturbation gradient, the input data, and the preset noise data generation rule, or it can be to generate the perturbation data of the model to be explained based on the positive perturbation weight, the input data, and the preset noise data generation rule, or to generate the perturbation data of the model to be explained based on the negative perturbation weight, the input data, and the preset noise data generation rule.

[0129] In this embodiment, extract the perturbation direction information from the perturbation correlation information; generate the perturbation data of the model to be explained based on the perturbation direction information, the input data, and the preset noise data generation rule. In this embodiment, the perturbation data of the model to be explained is accurately generated.

[0130] Further, based on the first embodiment of the present application, another embodiment of the present application is provided. In this embodiment, the model to be explained is a face recognition model to be explained;

[0131] The step of determining the stability explanation result of the model to be explained based on the condition number of the model to be explained and the perturbation data, and optimizing the model to be explained based on the stability explanation result includes:

[0132] Step C1: Determine the face recognition stability explanation result of the face recognition model to be explained based on the condition number of the face recognition model to be explained and the perturbation data.

[0133] In this embodiment, it is applied to the stability explanation of the face recognition model. Specifically, determine the face recognition stability explanation result of the face recognition model to be explained based on the condition number of the face recognition model to be explained and the perturbation data. This face recognition stability explanation result includes that the face recognition is stable or unstable. In this embodiment, if the face recognition is unstable, then return to the step of training the face recognition model to be explained. Specifically, continuously adjust the weight parameters of the face recognition model to be explained, etc., to obtain a face recognition model to be explained with good stability.

[0134] In this embodiment, the face recognition stability explanation result of the face recognition model to be explained is determined based on the condition number of the face recognition model to be explained and the perturbation data. In this embodiment, the stability of the face recognition model is accurately explained.

[0135] Refer to Figure 3 , Figure 3 which is a schematic diagram of the device structure of the hardware operating environment involved in the solution of the embodiment of the present application.

[0136] As Figure 3 shown, the model stability explanation device may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between the processor 1001 and the memory 1005. The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0137] Optionally, the model stability explanation device may further include a rectangular user interface, a network interface, a camera, an RF (Radio Frequency) circuit, sensors, an audio circuit, a WiFi module, and so on. The rectangular user interface may include a display screen and an input sub-module such as a keyboard. Optionally, the rectangular user interface may further include a standard wired interface and a wireless interface. The network interface may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0138] Those skilled in the art can understand that Figure 3 the model stability explanation device structure shown in

[0139] As Figure 3 shown, in the memory 1005 as a storage medium, there may be included an operating system, a network communication module, and a model stability explanation program. The operating system is a program for managing and controlling the hardware and software resources of the model stability explanation device, and supports the operation of the model stability explanation program and other software and / or programs. The network communication module is used to realize the communication between the components inside the memory 1005, and the communication with other hardware and software in the model stability explanation system.

[0140] In Figure 3In the model stability interpretation device shown, the processor 1001 is configured to execute the model stability interpretation program stored in the memory 1005 to implement the steps of the model stability interpretation method described in any one of the above.

[0141] The specific implementation manners of the model stability interpretation device of the present application are basically the same as those of the embodiments of the above model stability interpretation method, and will not be elaborated herein.

[0142] The present application further provides a model stability interpretation device, and the model stability interpretation device includes:

[0143] A first determination module, configured to determine a model to be interpreted and determine the condition numbers associated with the matrices of each layer of the model to be interpreted when receiving a model stability interpretation instruction;

[0144] A second determination module, configured to determine the input data of the model to be interpreted and determine the perturbation data for perturbing the input data;

[0145] A third determination module, configured to determine the stability interpretation result of the model to be interpreted based on the condition number of the model to be interpreted and the perturbation data, so as to optimize the model to be interpreted based on the stability interpretation result.

[0146] Optionally, the first determination module includes:

[0147] A first determination unit, configured to determine a model to be interpreted and determine the condition numbers associated with the matrices of each layer of the model to be interpreted when receiving a model stability interpretation instruction;

[0148] Wherein, the condition number is determined by the maximum eigenvalue of the matrices of each layer in the model to be interpreted.

[0149] Optionally, the second determination module includes:

[0150] A second determination unit, configured to determine the input data of the model to be interpreted and extract perturbation correlation information from the model stability interpretation instruction;

[0151] A third determination unit, configured to determine the perturbation data of the model to be interpreted based on the perturbation correlation information and the input data.

[0152] Optionally, the second determination unit includes:

[0153] An extraction subunit, configured to extract perturbation amplitude information from the perturbation correlation information;

[0154] A generation subunit, configured to generate the perturbation data of the model to be interpreted based on the perturbation amplitude information, the input data, and a preset noise data generation rule.

[0155] Optionally, the generating subunit is configured to:

[0156] Extract perturbation direction information from the perturbation correlation information;

[0157] Generate perturbation data for the model to be explained based on the perturbation direction information, the input data, and a preset noise data generation rule.

[0158] Optionally, the third determination module includes:

[0159] A fourth determination unit, configured to determine the correlation degree between the perturbation data and the input data, and determine the correlation relationship between the correlation degree and the condition number of the model to be explained;

[0160] A fifth determination unit, configured to, when the correlation relationship is that the correlation degree between the perturbation data and the input data is large and the condition number of the model to be explained is large, determine the model to be explained

[0161] Optionally, the model to be explained is a face recognition model to be explained;

[0162] The third determination module includes:

[0163] A sixth determination unit, configured to determine a face recognition stability explanation result of the face recognition model to be explained based on the condition number of the face recognition model to be explained and the perturbation data.

[0164] The specific implementation manners of the model stability explanation device of the present application are basically the same as those of the embodiments of the above model stability explanation method, and will not be elaborated herein.

[0165] The embodiments of the present application provide a storage medium, and the storage medium stores one or more programs, and the one or more programs can also be executed by one or more processors to implement the steps of the model stability explanation method described in any one of the above.

[0166] The specific implementation manners of the storage medium of the present application are basically the same as those of the embodiments of the above model stability explanation method, and will not be elaborated herein.

[0167] The present application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the steps of the above model stability explanation method.

[0168] The specific implementation manners of the computer program product of the present application are basically the same as those of the embodiments of the above model stability explanation method, and will not be elaborated herein.

[0169] It should be noted that, in this document, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device comprising that element.

[0170] The serial numbers of the above-mentioned embodiments of the present invention are only for description and do not represent the superiority or inferiority of the embodiments.

[0171] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0172] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A method for explaining model stability, characterized in that The model stability interpretation method includes: When receiving a model stability interpretation instruction, determining the model to be interpreted, and determining the condition number associated with each layer matrix of the model to be interpreted, where the condition number is obtained by calculating the sum of the maximum eigenvalues of each layer matrix in the model to be interpreted, and the model to be interpreted is a face recognition model to be interpreted; Determining the input data of the model to be interpreted, and extracting perturbation correlation information from the model stability interpretation instruction; Based on the perturbation correlation information and the input data, determining the perturbation data of the model to be interpreted; Based on the condition number of the model to be interpreted and the perturbation data, determining the stability interpretation result of the model to be interpreted, so as to optimize the model to be interpreted based on the stability interpretation result; The step of determining the stability interpretation result of the model to be interpreted based on the condition number of the model to be interpreted and the perturbation data, so as to optimize the model to be interpreted based on the stability interpretation result includes: Determining the correlation degree between the perturbation data and the input data, and determining the correlation relationship between the correlation degree and the condition number of the model to be interpreted; If the correlation relationship is that the correlation degree between the perturbation data and the input data is large and the condition number of the model to be interpreted is large, determining that the stability result of the model to be interpreted is that the stability of the model to be interpreted is poor, so as to optimize the model to be interpreted based on the stability interpretation result of the poor stability of the model to be interpreted.

2. The model stability explanation method according to claim 1, wherein The step of determining the perturbation data of the model to be interpreted based on the perturbation correlation information and the input data includes: Extracting perturbation amplitude information from the perturbation correlation information; Based on the perturbation amplitude information, the input data, and a preset noise data generation rule, generating the perturbation data of the model to be interpreted.

3. The method for explaining model stability according to claim 1, wherein, The step of determining the perturbation data of the model to be interpreted based on the perturbation correlation information and the input data includes: Extracting perturbation direction information from the perturbation correlation information; Based on the perturbation direction information, the input data, and a preset noise data generation rule, generating the perturbation data of the model to be interpreted.

4. The model stability explanation method according to claim 1, characterized in that The step of determining the stability interpretation result of the model to be interpreted based on the condition number of the model to be interpreted and the perturbation data, so as to optimize the model to be interpreted based on the stability interpretation result includes: Based on the condition number of the face recognition model to be interpreted and the perturbation data, determining the face recognition stability interpretation result of the face recognition model to be interpreted, so as to optimize the model to be interpreted based on the face recognition stability interpretation result.

5. A model stability explanation device, characterized in that, The model stability interpretation device includes: A first determination module, configured to determine the model to be interpreted when receiving a model stability interpretation instruction, and determine the condition number associated with each layer matrix of the model to be interpreted, where the condition number is obtained by calculating the sum of the maximum eigenvalues of each layer matrix in the model to be interpreted, and the model to be interpreted is a face recognition model to be interpreted; A second determination module, where the second determination module includes: A second determination unit, configured to determine the input data of the model to be interpreted, and extract perturbation correlation information from the model stability interpretation instruction; A third determination unit, configured to determine perturbation data of the model to be explained based on the perturbation association information and the input data; A third determination module, configured to determine a stability explanation result of the model to be explained based on the condition number of the model to be explained and the perturbation data, so as to optimize the model to be explained based on the stability explanation result; The third determination module includes: A fourth determination unit, configured to determine the correlation degree between the perturbation data and the input data, and determine the correlation relationship between the correlation degree and the condition number of the model to be explained; A fifth determination unit, configured to determine that the stability result of the model to be explained is that the stability of the model to be explained is poor when the correlation relationship is that the correlation degree between the perturbation data and the input data is large and the condition number of the model to be explained is large, so as to optimize the model to be explained based on the stability explanation result that the stability of the model to be explained is poor.

6. A model stability explanation device, characterized in that, The model stability explanation device includes: a memory, a processor, and a program stored on the memory for implementing the model stability explanation method, The memory is used for storing the program for implementing the model stability explanation method; The processor is used for executing the program for implementing the model stability explanation method to implement the steps of the model stability explanation method according to any one of claims 1 to 4.

7. A storage medium, characterized in that, A program for implementing the model stability explanation method is stored on the storage medium, and the program for implementing the model stability explanation method is executed by the processor to implement the steps of the model stability explanation method according to any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images