Feature dimension importance analysis method, device, equipment and readable storage medium

Abstract

The present invention discloses a method, device, equipment and readable storage medium for analyzing the importance of feature dimensions, the method comprising: inputting an original sample into a target machine learning model to obtain an original output result, and modifying the original output result to obtain a modified result; back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain a modified sample; performing data comparison on each feature dimension of the modified sample and the original sample, and obtaining an importance analysis result of each feature dimension based on the comparison result. The present invention proposes an effective way to analyze the importance of each feature dimension of machine learning model input data.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a feature dimension importance analysis method, device, equipment and readable storage medium. Background Art

[0002] In the era of big data, artificial intelligence has become an essential tool in the industrial, financial and scientific research fields to help people explore and explain the facts behind the data. On the other hand, although artificial intelligence will give corresponding prediction results, most artificial intelligence models and methods are regarded as black box models due to the upper limit of human's ability to understand abstract things. One problem with black box models is that the role of each feature dimension of the model input data on the model prediction results, that is, its importance is unknown. However, in practical applications, it is often necessary to understand the importance of each feature dimension in order to better apply artificial intelligence models. Moreover, with the continuous improvement of computer computing power, storage capacity and artificial intelligence algorithm performance, the feature dimensions of model input samples are also growing. Therefore, how to judge the importance of each feature dimension has become an urgent problem to be solved. Summary of the invention

[0003] The main purpose of the present invention is to provide a feature dimension importance analysis method, device, system and readable storage medium, aiming to solve the problem of how to judge the importance of each feature dimension of artificial intelligence model input samples.

[0004] To achieve the above object, the present invention provides a feature dimension importance analysis method, the method comprising the following steps:

[0005] Inputting the original sample into the target machine learning model to obtain an original output result, and modifying the original output result to obtain a modified result;

[0006] Back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain a modified sample;

[0007] Data comparison is performed on each feature dimension of the modified sample and the original sample, and importance analysis results of each feature dimension are obtained based on the comparison results.

[0008] Optionally, the modified result is a counterexample result corresponding to the original sample.

[0009] The step of back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain a modified sample includes:

[0010] Back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain an alternative sample;

[0011] Inputting the candidate sample into the target machine learning model to obtain an alternative result, and detecting whether the alternative result constitutes a counterexample result of the original sample;

[0012] If it is detected that the candidate result constitutes a counterexample result of the original sample, the candidate sample is used as a modified sample;

[0013] If it is detected that the alternative result does not constitute a counterexample result of the original sample, the alternative result is used as the new original output result, and the step is performed again based on the new original output result: the difference between the modified result and the original output result is back-propagated in the target machine learning model to modify the original sample to obtain the alternative sample.

[0014] Optionally, the step of back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain the modified sample includes:

[0015] Calculating the difference between the modified result and the original output result;

[0016] Calculate the gradient value of each feature dimension of the original sample according to the difference value using the back propagation method;

[0017] The data of each feature dimension of the original sample is modified according to the gradient value to obtain a modified sample.

[0018] Optionally, the step of performing data comparison on each feature dimension of the modified sample and the original sample and obtaining an importance analysis result of each feature dimension based on the comparison result includes:

[0019] Calculate the difference between the data of each feature dimension in the modified sample and the data of the corresponding feature dimension in the original sample to obtain the data difference of each feature dimension;

[0020] Each of the data differences is compared with a preset threshold of the corresponding feature dimension, and a feature dimension whose data difference is greater than the preset threshold is determined as an important feature, so as to obtain an importance analysis result of each feature dimension.

[0021] Optionally, the step of modifying the original output result to obtain a modified result includes:

[0022] receiving a modification instruction for modifying the original output result;

[0023] The result carried in the modification instruction is used as the modification result of the original output result.

[0024] Optionally, when the original output result is a probability value, the step of modifying the original output result to obtain a modified result includes:

[0025] Detecting whether the original output result is greater than a preset classification threshold;

[0026] If the original output result is greater than the preset classification threshold, a probability value is selected from a first interval as the modification result, wherein the first interval is an interval greater than or equal to zero and less than the preset classification threshold.

[0027] If the original output result is less than or equal to the preset classification threshold, a probability value is selected from the first interval as the modified result, wherein the second interval is an interval less than or equal to one and greater than the preset classification threshold.

[0028] Optionally, the importance analysis result is a result indicating important features in each feature dimension, the target machine learning model is a risk credit model, and after the step of performing data comparison on each feature dimension of the modified sample and the original sample and obtaining the importance analysis result of each feature dimension based on the comparison result, the step further includes:

[0029] According to the importance analysis result, detecting whether the data corresponding to the important features in the sample data of the user to be predicted is abnormal data;

[0030] If the data corresponding to the important feature is abnormal data, weakening the data corresponding to the important feature;

[0031] The weakened sample data is input into the target machine learning model to obtain the risk credit result.

[0032] To achieve the above object, the present invention provides a feature dimension importance analysis device, the device comprising:

[0033] An input module, used to input the original sample into the target machine learning model to obtain the original output result, and modify the original output result to obtain the modified result;

[0034] A modification module, used for back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain a modified sample;

[0035] The comparison module is used to compare the data of each feature dimension of the modified sample and the original sample, and obtain the importance analysis result of each feature dimension based on the comparison result.

[0036] To achieve the above-mentioned purpose, the present invention also provides a feature dimension importance analysis device, which includes: a memory, a processor, and a feature dimension importance analysis program stored in the memory and executable on the processor, and the feature dimension importance analysis program, when executed by the processor, implements the steps of the feature dimension importance analysis method described above.

[0037] In addition, to achieve the above-mentioned purpose, the present invention also proposes a computer-readable storage medium, on which a feature dimension importance analysis program is stored. When the feature dimension importance analysis program is executed by a processor, the steps of the feature dimension importance analysis method as described above are implemented.

[0038] In the present invention, the original output result is obtained by inputting the original sample into the target machine learning model, the original output result is modified to obtain the modified result, the difference between the modified result and the original output result is back-propagated in the target machine learning model to modify the original sample to obtain the modified sample, and the data of each feature dimension of the original sample and the modified sample are compared to obtain the importance analysis result of each feature dimension. By modifying the output result, the change of the output result is reflected on the input sample by back-propagation, so that the influence of each feature dimension on the output result can be obtained according to the data change of each feature dimension of the input sample, and then the importance analysis result of each feature dimension can be obtained. That is, the present invention proposes an effective way to analyze the importance of each feature dimension of the input data of the machine learning model, and can obtain accurate analysis results, and then can improve the use efficiency of the machine learning model according to the accurate analysis results. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of the structure of the hardware operating environment involved in the embodiment of the present invention;

[0040] Figure 2 It is a flowchart of the first embodiment of the feature dimension importance analysis method of the present invention;

[0041] Figure 3 A schematic diagram of a process of modifying and obtaining counterfactual samples according to various embodiments of the present invention;

[0042] Figure 4 It is a functional schematic module diagram of a preferred embodiment of the feature dimension importance analysis device of the present invention.

[0043] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0045] like Figure 1 As shown, Figure 1 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.

[0046] It should be noted that the feature dimension importance analysis device in the embodiment of the present invention can be a smart phone, a personal computer, a server and other devices, and no specific limitation is made here.

[0047] like Figure 1 As shown, the feature dimension importance analysis device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0048] Those skilled in the art will understand that Figure 1 The device structure shown in does not constitute a limitation on the feature dimension importance analysis device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0049] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a feature dimension importance analysis program. The operating system is a program that manages and controls the hardware and software resources of the device, and supports the operation of the feature dimension importance analysis program and other software or programs. Figure 1 In the device shown, the user interface 1003 is mainly used to communicate data with the client; the network interface 1004 is mainly used to establish a communication connection with the server; the processor 1001 can be used to call the feature dimension importance analysis program stored in the memory 1005, and perform the following operations:

[0050] Inputting the original sample into the target machine learning model to obtain an original output result, and modifying the original output result to obtain a modified result;

[0051] Back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain a modified sample;

[0052] Data comparison is performed on each feature dimension of the modified sample and the original sample, and importance analysis results of each feature dimension are obtained based on the comparison results.

[0053] Furthermore, the modified result is a counterexample result corresponding to the original sample.

[0054] The step of back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain a modified sample includes:

[0055] Back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain an alternative sample;

[0056] Inputting the candidate sample into the target machine learning model to obtain an alternative result, and detecting whether the alternative result constitutes a counterexample result of the original sample;

[0057] If it is detected that the candidate result constitutes a counterexample result of the original sample, the candidate sample is used as a modified sample;

[0058] If it is detected that the alternative result does not constitute a counterexample result of the original sample, the alternative result is used as the new original output result, and the step is performed again based on the new original output result: the difference between the modified result and the original output result is back-propagated in the target machine learning model to modify the original sample to obtain the alternative sample.

[0059] Furthermore, the step of back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain the modified sample includes:

[0060] Calculating the difference between the modified result and the original output result;

[0061] Calculate the gradient value of each feature dimension of the original sample according to the difference value using the back propagation method;

[0062] The data of each feature dimension of the original sample is modified according to the gradient value to obtain a modified sample.

[0063] Furthermore, the step of performing data comparison on each feature dimension of the modified sample and the original sample and obtaining the importance analysis result of each feature dimension based on the comparison result includes:

[0064] Calculate the difference between the data of each feature dimension in the modified sample and the data of the corresponding feature dimension in the original sample to obtain the data difference of each feature dimension;

[0065] Each of the data differences is compared with a preset threshold of the corresponding feature dimension, and a feature dimension whose data difference is greater than the preset threshold is determined as an important feature, so as to obtain an importance analysis result of each feature dimension.

[0066] Furthermore, the step of modifying the original output result to obtain a modified result includes:

[0067] receiving a modification instruction for modifying the original output result;

[0068] The result carried in the modification instruction is used as the modification result of the original output result.

[0069] Further, when the original output result is a probability value, the step of modifying the original output result to obtain a modified result includes:

[0070] Detecting whether the original output result is greater than a preset classification threshold;

[0071] If the original output result is greater than the preset classification threshold, a probability value is selected from a first interval as the modification result, wherein the first interval is an interval greater than or equal to zero and less than the preset classification threshold.

[0072] If the original output result is less than or equal to the preset classification threshold, a probability value is selected from the first interval as the modified result, wherein the second interval is an interval less than or equal to one and greater than the preset classification threshold.

[0073] Further, the importance analysis result is a result indicating important features in each feature dimension, the target machine learning model is a risk credit model, and after the step of performing data comparison on each feature dimension of the modified sample and the original sample and obtaining the importance analysis result of each feature dimension based on the comparison result, the processor 1001 can also be used to call the feature dimension importance analysis program stored in the memory 1005 to perform the following operations:

[0074] According to the importance analysis result, detecting whether the data corresponding to the important features in the sample data of the user to be predicted is abnormal data;

[0075] If the data corresponding to the important feature is abnormal data, weakening the data corresponding to the important feature;

[0076] The weakened sample data is input into the target machine learning model to obtain the risk credit result.

[0077] Based on the above structure, various embodiments of the feature dimension importance analysis method are proposed.

[0078] Reference Figure 2 , Figure 2 This is a flowchart of the first embodiment of the feature dimension importance analysis method of the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than here. The execution subjects of each embodiment of the feature dimension importance analysis method of the present invention may be devices such as smart phones, personal computers, and servers. For ease of description, the execution subjects are omitted in the following embodiments. In this embodiment, the feature dimension importance analysis method includes:

[0079] Step S10, inputting the original sample into the target machine learning model to obtain an original output result, and modifying the original output result to obtain a modified result;

[0080] In this embodiment, importance analysis can be performed on each feature dimension of the input data of the target machine learning model, wherein the target machine learning model can be a commonly used machine learning model, for example, various types of neural network models. Sample data is data used as input data of the target machine learning model, and at least one sample data can be obtained as a basis for analysis. It should be understood that the larger the amount of sample data used for importance analysis, the more reliable and accurate the analysis results obtained. Since the processing process for each sample data is similar, a sample data is used as an example for explanation.

[0081] The acquired sample data is used as the original sample, and the original sample is input into the target machine learning model to obtain the original output result, that is, the target machine learning model is used to process the input original sample, and a prediction result is output, and the prediction result is used as the original output result (indicating the output result corresponding to the original sample). For example, the target machine learning model is a risk credit model used to process user data and predict whether to provide a loan to the user. The acquired user sample data is input into the target machine learning model as the original sample. After being processed by the machine learning model, a loan prediction result is obtained. The loan prediction result indicates whether a loan is provided to the user. If the output result is a probability value, a probability greater than 0.5 indicates a loan, and a probability less than 0.5 indicates no loan. The loan prediction result is used as the original output result.

[0082] After obtaining the original output result, the original output result can be modified to obtain a modified result. Among them, modifying the original output result means: when the output data of the target machine learning model is a continuous value, the original output result is modified to other values ​​within the continuous value range except the original output result. For example, when the output data is a probability value, its corresponding continuous value range is 0 to 1. If the original output result is 0.2, the original output result can be modified to other probability values ​​within 0 to 1 except 0.2, for example, modified to 0.4; when the output data of the target machine learning model is a discrete value, the original output result is modified to other values ​​within the discrete value range except the original output result. For example, when the discrete value range of the output data is {0, 1, 2}, and the original output result is 0, the original output result can be modified to 1 or 2.

[0083] The modification method may be to randomly select a modification result from the optional modification values. For example, in the above discrete numerical value example, one may be randomly selected from 1 and 2 as the modification result of the original output result 0. The modification method may also be to select the modification result from the optional modification values ​​according to a preset rule. For example, in the above continuous numerical value example, a rule is preset: when the original output result is less than 0.5, each modification is modified in the direction of increase, for example, each time it is increased by 0.1; when the original output result is not less than 0.5, each modification is modified in the direction of decrease, for example, each time it is decreased by 0.1.

[0084] It should be noted that for the original output result of a sample data, multiple different modifications can be made to perform multiple analyses, and then the final importance analysis result can be obtained based on the results of the multiple analyses. For the sake of ease of description, this embodiment takes one modification as an example.

[0085] Furthermore, the step of modifying the original output result to obtain a modified result in step S10 includes:

[0086] Step S101, receiving a modification instruction for modifying the original output result;

[0087] Step S102: Using the result carried in the modification instruction as the modification result of the original output result.

[0088] The modification method may also be based on a modification instruction. Specifically, a modification instruction for modifying the original output result may be received, the modification instruction carries a result, and the result is used as the modification result of the original output result. The modification instruction may be triggered by the user, for example, an interactive interface is provided for the user to input the modification result, and the user inputs the modification result in the interactive interface to trigger the modification instruction.

[0089] Step S20, back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain a modified sample;

[0090] There is a difference between the modified result and the original output result, and the difference can be back-propagated in the target machine learning model, that is, the difference is used as an error, and the error is back-propagated in the target machine learning model. Backpropagation (BP) is the abbreviation of "error back propagation", which is a common method used in combination with optimization methods (such as gradient descent method) to train artificial neural networks. In this embodiment, referring to the existing process of updating model parameters according to the backpropagation method, the data of each feature dimension of the original sample can be modified by the backpropagation method, that is, the original backpropagation method is that the input data is fixed and the parameters are updated by error propagation. In this embodiment, the model parameters are fixed and the input data is updated by error propagation. Through error backpropagation, the updated value of the data of each feature dimension in the original sample is calculated, that is, a new sample is obtained, and the sample is used as a modified sample of the original sample.

[0091] Step S30, performing data comparison on each feature dimension of the modified sample and the original sample, and obtaining importance analysis results of each feature dimension based on the comparison results.

[0092] The data of each feature dimension of the modified sample obtained is changed compared with the data of the corresponding feature dimension in the original sample. The data of each feature dimension of the modified sample and the original sample can be compared, that is, the data of the same feature dimension in the two samples are compared, and the importance analysis results of each feature dimension are obtained based on the results of the pairwise comparison.

[0093] Specifically, taking a feature dimension as an example, if the difference between the data of this feature dimension in the two samples is larger, it means that the modification of the output result causes a larger change in the data of this feature dimension, which in turn means that the change in the data of this feature dimension causes the change in the output result, which means that the feature dimension has a greater impact on the output result of the target machine learning model, that is, the more important the feature dimension is; conversely, if the difference between the data of this feature dimension in the two samples is smaller, it means that the modification of the output result causes a smaller change in the data of this feature dimension, which in turn means that the change in the output result is not necessarily caused by the change in the data of this feature dimension, which means that the feature dimension has a smaller impact on the output result of the target machine learning model, that is, the less important the feature dimension is.

[0094] Furthermore, the step S30 includes:

[0095] Step S301, calculating the difference between the data of each feature dimension in the modified sample and the data of the corresponding feature dimension in the original sample, to obtain the data difference of each feature dimension;

[0096] Step S302, comparing each of the data differences with a preset threshold of the corresponding feature dimension, and determining the feature dimension whose data difference is greater than the preset threshold as an important feature, so as to obtain the importance analysis result of each feature dimension.

[0097] The difference between the data of each feature dimension in the modified sample and the data of the corresponding feature dimension in the original sample can be calculated, that is, the difference of the data of the same feature dimension in the two samples is calculated to obtain the data difference corresponding to each feature dimension. A threshold can be set for each feature dimension. If the data difference corresponding to a feature dimension is greater than the threshold corresponding to the feature dimension, the feature dimension is determined to be an important feature, otherwise, the feature dimension is determined to be an unimportant feature.

[0098] It should be noted that when the original output result of the original sample is modified multiple times, the importance of each feature dimension can be determined by synthesizing the analysis results corresponding to each modification. For example, in n modifications, when the number of times the data difference corresponding to a feature dimension is greater than the corresponding threshold is greater than m times, the feature dimension is determined as an important feature, otherwise, it is determined as a non-important feature, where m is less than or equal to n. Similarly, when multiple sample data are used for analysis, the importance of each feature dimension can be determined by synthesizing the analysis results corresponding to each sample data. For example, for a sample data, each sample data corresponds to the difference of each feature dimension. If the number of times the data difference corresponding to a feature dimension is greater than the corresponding threshold is greater than b, the feature dimension is determined as an important feature, otherwise, it is determined as a non-important feature, where b is less than or equal to a. By synthesizing the final importance analysis result based on the analysis results obtained by multiple modifications, the reliability and accuracy of the final analysis result can be improved. Similarly, the final importance analysis result is obtained based on the analysis results corresponding to multiple sample data, which is based on a large amount of data and also improves the reliability and accuracy of the final analysis result.

[0099] In this embodiment, the original output result is obtained by inputting the original sample into the target machine learning model, the original output result is modified to obtain the modified result, the difference between the modified result and the original output result is back-propagated in the target machine learning model to modify the original sample to obtain the modified sample, and the data of each feature dimension of the original sample and the modified sample are compared to obtain the importance analysis results of each feature dimension. By modifying the output result, the change of the output result is reflected on the input sample by back-propagation, so that the influence of each feature dimension on the output result can be obtained according to the data change of each feature dimension of the input sample, and then the importance analysis results of each feature dimension can be obtained. That is, this embodiment proposes an effective way to analyze the importance of each feature dimension of the input data of the machine learning model, and can obtain accurate analysis results, and then can improve the efficiency of using the machine learning model according to the accurate analysis results.

[0100] Further, based on the above first embodiment, a second embodiment of the feature dimension importance analysis method of the present invention is proposed. In this embodiment, step S20 includes:

[0101] Step S201, back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain an alternative sample;

[0102] Furthermore, in this embodiment, the modified result may be a counter-example result corresponding to the original sample. The counter-example result refers to a result that is opposite in fact to the output result of the original sample. For example, if the output result corresponding to the original sample is to determine that a loan is granted to the target user, then the counter-example result is to determine that a loan is not granted to the target user.

[0103] In the target machine learning model, the difference between the modified result and the original output result is back-propagated to obtain the updated value of each feature dimension data of the original sample, that is, to obtain a new sample, which can be used as an alternative sample first.

[0104] Step S202, inputting the candidate sample into the target machine learning model to obtain a candidate result, and detecting whether the candidate result constitutes a counterexample result of the original sample;

[0105] The candidate sample is input into the target machine learning model for processing to obtain a prediction result, and the prediction result is used as the candidate result, that is, the output result corresponding to the candidate sample. It can be detected whether the candidate result constitutes a counterexample result of the original sample. Specifically, when the output data of the target machine learning model is a probability value, different probability intervals represent different factual results. It can be determined whether the output results corresponding to the candidate result and the original sample are in different intervals. If they are in different intervals, it is determined that the candidate result constitutes a counterexample result of the original sample, otherwise it is determined that the candidate result does not constitute a counterexample result of the original sample. For example, a threshold is set in advance based on industry experience, such as 0.5. The probability value output by the standard machine learning model is greater than 0.5, indicating that the user is loaned, and not greater than 0.5, indicating that the user is not loaned. The output result corresponding to the original sample is 0.2. When the candidate result is greater than 0.5, it is determined that the candidate result constitutes a counterexample result of the original sample, otherwise it does not constitute a counterexample result of the original sample. When the output data of the machine learning model is a discrete value, different discrete values ​​represent different factual results. It can be determined whether the alternative result is different from the original sample. If they are different, it means that the alternative result constitutes a counterexample result of the original sample. If they are the same, it is determined that it does not constitute a counterexample result of the original sample. For example, the target machine learning model outputs 1 to indicate that a loan is made to the user, and outputs 0 to indicate that a loan is not made to the user. The output result corresponding to the original sample is 0. Then, when the alternative result is 1, it is determined that the alternative result constitutes a counterexample result of the original sample. Otherwise, it does not constitute a counterexample result of the original sample.

[0106] Step S203: if it is detected that the candidate result constitutes a counterexample result of the original sample, the candidate sample is used as a modified sample;

[0107] Step S204: If it is detected that the alternative result does not constitute a counterexample result of the original sample, the alternative result is used as the new original output result, and the step is performed again based on the new original output result: the difference between the modified result and the original output result is back-propagated in the target machine learning model to modify the original sample to obtain an alternative sample.

[0108] If it is detected that the alternative result constitutes a counterexample result of the original sample, the alternative sample will be used as the final modified sample.

[0109] If it is detected that the alternative result does not constitute a counter-example result of the original sample, the alternative result is used as the new original output result, and step S201 and subsequent steps are performed again on the basis of the new original output result. That is, if the modified alternative sample does not constitute a counter-example result of the original sample, the alternative result is used as the new original output result, and the difference between the new original output result and the modified result is back-propagated in the target machine learning model to obtain the updated values ​​of each feature dimension of the original sample, that is, a new sample is obtained, and the sample is used as an alternative sample again, and the alternative result of the alternative sample is detected to constitute a counter-example result of the original sample. If not, the alternative result of the alternative sample is used as the new original output result, and this cycle is repeated until the alternative result of the alternative sample constitutes a counter-example result of the original sample, and the alternative sample is used as the final modified sample.

[0110] That is, in this embodiment, after multiple cycles of iteration, the output result corresponding to the modified sample finally obtained is the counterexample result of the output result of the original sample. By comparing and analyzing the data of the final modified sample and the original sample in each feature dimension, the importance analysis results of each feature dimension can be obtained more accurately. Specifically, taking a feature dimension as an example, the greater the difference between the data of the feature dimension in the two samples, the greater the impact of the data change of the feature dimension on the output result of the machine learning model, and the impact is that the output result becomes a completely opposite result in fact, which can better illustrate the importance of the feature dimension, and also make the final importance analysis result more reliable and accurate.

[0111] Further, in one embodiment, step S20 includes:

[0112] Step S205, calculating the difference between the modified result and the original output result;

[0113] Step S206, calculating the gradient value of each feature dimension of the original sample according to the difference value using a back propagation method;

[0114] Step S207: modify the data of each feature dimension of the original sample according to the gradient value to obtain a modified sample.

[0115] The process of back-propagating the difference between the modified result and the original output result in the target machine learning model to obtain the modified sample can be specifically: first calculate the error between the modified result and the original output result; then refer to the method of calculating the gradient value of the model parameter according to the error in the back-propagation method, and obtain the gradient value corresponding to each feature dimension of the original sample according to the error calculation; then refer to the method of updating the model parameter according to the gradient value, and update (i.e. modify) the data of each feature dimension of the original sample according to the gradient value to obtain a new sample, and use the new sample as the modified sample. Alternatively, use the new sample as an alternative sample, and iterate in a loop according to steps S201 to 204 to obtain the final modified sample.

[0116] like Figure 3 As shown, it is a schematic diagram of a process of modifying and obtaining a counterfactual sample according to an embodiment of the present invention, wherein the counterfactual sample is a modified sample whose output result finally obtained constitutes a counterexample result of the original sample. Figure 3 In the method, the original output result is modified to obtain the modified result constituting the counterexample result of the original sample; the error between the modified result and the original output result is calculated, the gradient value is calculated using the error, the original sample is modified to obtain a new sample (the new counterfactual sample in the figure), and it is judged whether the sample meets the counterfactual requirement, that is, it is judged whether the output result of the sample constitutes the counterexample result of the output result of the original sample; if the counterfactual requirement is met, the new sample can be output as a counterfactual sample, so that the user can intuitively know the difference between the data of each feature dimension of the counterfactual sample and the original sample, and the importance of each feature dimension can be analyzed based on the difference; if the counterfactual requirement is not met, the output result of the new sample can be used as the new original output result, and the error between the modified result and the original output result is calculated, and the gradient value is calculated to modify the sample; this cycle is repeated until the counterfactual sample is obtained; further, the data of each feature dimension of the counterfactual sample and the original sample are compared to obtain the importance analysis result of each feature dimension.

[0117] Further, based on the above-mentioned first and / or second embodiments, a third embodiment of the feature dimension importance analysis method of the present invention is proposed. In this embodiment, when the original output result is a probability value, the step of modifying the original output result to obtain a modified result in step S10 includes:

[0118] Step S103, detecting whether the original output result is greater than a preset classification threshold;

[0119] Furthermore, in this embodiment, in order to modify and obtain a modified result constituting a counterexample result of the original sample, the modification method may also be:

[0120] When the original output result is a probability value, that is, when the output result of the target machine learning model is a probability value, a classification threshold can be set in advance. When the output result is greater than the classification threshold, it indicates a fact result, and when the output result is not greater than the classification threshold, it indicates another fact result. For example, when the output result is greater than the classification threshold, it indicates that a loan is made to the user, and when the output result is not greater than the classification threshold, it indicates that a loan is not made to the user.

[0121] Check whether the original output result is greater than the preset classification threshold.

[0122] Step S104: if the original output result is greater than the preset classification threshold, a probability value is selected from a first interval as the modification result, wherein the first interval is an interval greater than or equal to zero and less than the preset classification threshold.

[0123] If the original output result is greater than the preset classification threshold, a probability value is selected from the first interval as the modified result, wherein the first interval is an interval greater than or equal to zero and less than the preset classification threshold, such as if the preset classification threshold is 0.5, then the first interval is [0, 0.5). Selecting a probability value from the first interval can be randomly selected.

[0124] Step S105, if the original output result is less than or equal to the preset classification threshold, a probability value is selected from the first interval as the modified result, wherein the second interval is an interval less than or equal to one and greater than the preset classification threshold.

[0125] If the original output result is less than or equal to the preset classification threshold, a probability value is selected from the second interval as the modification result, wherein the second interval is an interval less than or equal to one and greater than the preset classification threshold. For example, if the preset classification threshold is 0.5, the second interval is (0.5, 1]. Selecting a probability value from the second interval can be randomly selected.

[0126] In this embodiment, a modified result constituting a counterexample result of the original sample is obtained by modification, and a modified sample of the original sample is obtained by back-propagating the difference between the modified result and the original output result. By comparing the data of each feature dimension of the original sample and the modified sample, the role played by each feature dimension in making the output result become a result that is actually completely opposite can be analyzed, thereby obtaining a more accurate importance analysis result of each feature dimension. For example, when the original output result is to determine not to lend to the user, and the modified result is to determine to lend to the user, the change in the data of each feature dimension of the modified sample relative to the original sample reflects the role played by each feature dimension in changing the result of not lending to the user to the result of lending to the user; that is, the greater the change in the data corresponding to the feature dimension, it means that it is precisely because of the data modification of the feature dimension that the model prediction result changes from not lending to lending to lending, that is, the feature dimension is an important factor in determining whether to lend, and thus the feature dimension can be determined to be an important feature.

[0127] Furthermore, in one embodiment, the importance analysis result is a result indicating important features in each feature dimension, the target machine learning model is a risk credit model, and after step S30, the method further includes:

[0128] Step S40, based on the importance analysis result, detecting whether the data corresponding to the important features in the sample data of the user to be predicted is abnormal data;

[0129] In this embodiment, the target machine learning model may be a risk credit model, which can be used to predict whether to provide a loan to a user. The importance analysis result may be a result indicating the important features in each feature dimension, that is, which feature dimensions are important features. Then, when using the target machine learning model to predict whether to provide a loan to the user to be predicted, the data corresponding to the important features in the sample data of the user to be predicted can be obtained first. For example, according to the importance analysis result, the deposit amount is determined to be an important feature, and then the deposit amount in the sample data of the user to be predicted can be extracted.

[0130] For the data corresponding to the extracted important features, detect whether the data is abnormal data. Specifically, an abnormal threshold can be pre-set for the important features, and the abnormal threshold can be set to be relatively large. When it is detected that the data corresponding to the extracted important features is greater than the abnormal threshold, the data is determined to be abnormal data, otherwise, the data is determined to be normal data.

[0131] Step S50, if the data corresponding to the important feature is abnormal data, weakening the data corresponding to the important feature;

[0132] Step S60, input the weakened sample data into the target machine learning model to obtain the risk credit result.

[0133] If the data corresponding to the important feature is abnormal data, the data can be weakened. Specifically, the data can be reduced to a normal range, such as reduced to a range smaller than the above abnormal threshold. The weakened sample data is input into the target machine learning model to obtain the risk credit result, and whether to provide a loan to the user can be determined based on the risk credit result. Specifically, when the risk credit result is a discrete value, whether to provide a loan to the user can be determined based on the pre-defined relationship between each discrete value and whether to provide a loan; when the prediction result is a probability value, whether to provide a loan to the user can be determined based on the pre-defined relationship between each probability interval and whether to provide a loan.

[0134] In this embodiment, when the data corresponding to an important feature is large, since the important feature has a greater impact on the output result of the target machine learning model, it may make the output result of the target machine learning model remain unchanged no matter how the data of other feature dimensions change, that is, the machine learning model ignores the role of data of other feature dimensions. In order to prevent users from forging the data of important features and causing the machine learning model to output the result of granting a loan to the user, the data corresponding to the important features is weakened so that the machine learning model can comprehensively judge whether to grant a loan to the user based on the data of other feature dimensions, thereby obtaining a more reliable prediction result.

[0135] In addition, the embodiment of the present invention also provides a feature dimension importance analysis device, referring to Figure 4 , the device comprises:

[0136] An input module 10 is used to input the original sample into the target machine learning model to obtain the original output result, and to modify the original output result to obtain the modified result;

[0137] A modification module 20, configured to back-propagate the difference between the modification result and the original output result in the target machine learning model to modify the original sample to obtain a modified sample;

[0138] The comparison module 30 is used to compare the data of each feature dimension of the modified sample and the original sample, and obtain the importance analysis result of each feature dimension based on the comparison result.

[0139] Furthermore, the modification result is a counterexample result corresponding to the original sample, and the modification module 20 includes:

[0140] A first modification unit, configured to back-propagate a difference between the modification result and the original output result in the target machine learning model to modify the original sample to obtain an alternative sample;

[0141] A first detection unit, configured to input the candidate sample into the target machine learning model to obtain a candidate result, and detect whether the candidate result constitutes a counterexample result of the original sample;

[0142] A first determining unit, configured to use the candidate sample as a modified sample if it is detected that the candidate result constitutes a counterexample result of the original sample;

[0143] A loop unit is used to, if it is detected that the alternative result does not constitute a counterexample result of the original sample, use the alternative result as the new original output result, and then execute the step based on the new original output result: back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain the alternative sample.

[0144] Furthermore, the modification module 20 includes:

[0145] A first calculation unit, used for calculating the difference between the modified result and the original output result;

[0146] A second calculation unit, used for calculating the gradient value of each feature dimension of the original sample according to the difference value in accordance with the back propagation method;

[0147] The second modification unit is used to modify the data of each feature dimension of the original sample according to the gradient value to obtain a modified sample.

[0148] Furthermore, the comparison module 30 includes:

[0149] A third calculation unit is used to calculate the difference between the data of each feature dimension in the modified sample and the data of the corresponding feature dimension in the original sample to obtain the data difference of each feature dimension;

[0150] The comparison unit is used to compare each of the data differences with the preset threshold of the corresponding feature dimension, and determine the feature dimension whose data difference is greater than the preset threshold as an important feature to obtain the importance analysis result of each feature dimension.

[0151] Furthermore, the input module 10 comprises:

[0152] A receiving unit, configured to receive a modification instruction for modifying the original output result;

[0153] The second determining unit is used to use the result carried in the modification instruction as the modification result of the original output result.

[0154] Further, when the original output result is a probability value, the input module 10 includes:

[0155] A second detection unit, used to detect whether the original output result is greater than a preset classification threshold;

[0156] The first selection unit is used to select a probability value from a first interval as the modification result if the original output result is greater than the preset classification threshold, wherein the first interval is an interval greater than or equal to zero and less than the preset classification threshold.

[0157] The second selection unit is used to select a probability value from the first interval as the modified result if the original output result is less than or equal to the preset classification threshold, wherein the second interval is an interval less than or equal to one and greater than the preset classification threshold.

[0158] Furthermore, the importance analysis result is a result indicating important features in each feature dimension, the target machine learning model is risk credit, and the device further includes:

[0159] A detection module, used to detect whether the data corresponding to the important features in the sample data of the user to be predicted is abnormal data according to the importance analysis result;

[0160] A weakening module, used for weakening the data corresponding to the important features if the data corresponding to the important features are abnormal data;

[0161] The input module 10 is also used to input the weakened sample data into the target machine learning model to obtain the risk credit result.

[0162] The expanded content of the specific implementation of the feature dimension importance analysis device of the present invention is basically the same as the various embodiments of the above-mentioned feature dimension importance analysis method, and will not be elaborated here.

[0163] In addition, an embodiment of the present invention further proposes a computer-readable storage medium, on which a feature dimension importance analysis program is stored. When the feature dimension importance analysis program is executed by a processor, the steps of the feature dimension importance analysis method described below are implemented.

[0164] The various embodiments of the feature dimension importance analysis device and computer-readable storage medium of the present invention may all refer to the various embodiments of the feature dimension importance analysis method of the present invention, and will not be described in detail here.

[0165] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0166] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0167] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the embodiments of the present invention.

[0168] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for analyzing the importance of characteristic dimensions of a risk credit model, characterized in that: The method is applied to a feature dimension importance analysis device, and the method comprises the following steps: Inputting user data as original samples into a target machine learning model to obtain an original output result, and modifying the original output result to obtain a modified result, wherein the target machine learning model is a risk credit model for processing user data and predicting whether to provide a loan to the user, wherein modifying the original output result means that when the output data of the target machine learning model is a continuous value, the original output result is modified to a value other than the original output result within a continuous value range, and when the output data of the target machine learning model is a discrete value, the original output result is modified to a value other than the original output result within a discrete value range; Back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain a modified sample; Data comparison is performed on each characteristic dimension of the modified sample and the original sample, and importance analysis results of each characteristic dimension of the user data are obtained based on the comparison results. The importance analysis results reflect the degree of influence of each characteristic dimension on the risk credit result predicted by the risk credit model.

2. The characteristic dimension importance analysis method for the risk credit model according to claim 1 is characterized in that: The modified result is the counterexample result corresponding to the original sample. The step of back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain a modified sample includes: Back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain an alternative sample; Inputting the candidate sample into the target machine learning model to obtain an alternative result, and detecting whether the alternative result constitutes a counterexample result of the original sample; If it is detected that the candidate result constitutes a counterexample result of the original sample, the candidate sample is used as a modified sample; If it is detected that the alternative result does not constitute a counterexample result of the original sample, the alternative result is used as the new original output result, and the step is performed again based on the new original output result: the difference between the modified result and the original output result is back-propagated in the target machine learning model to modify the original sample to obtain the alternative sample.

3. The characteristic dimension importance analysis method for the risk credit model according to claim 1 is characterized in that: The step of back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain a modified sample includes: Calculating the difference between the modified result and the original output result; Calculate the gradient value of each feature dimension of the original sample according to the difference value using the back propagation method; The data of each feature dimension of the original sample is modified according to the gradient value to obtain a modified sample.

4. The characteristic dimension importance analysis method for the risk credit model according to claim 1 is characterized in that: The step of performing data comparison on each feature dimension of the modified sample and the original sample and obtaining the importance analysis result of each feature dimension based on the comparison result comprises: Calculate the difference between the data of each feature dimension in the modified sample and the data of the corresponding feature dimension in the original sample to obtain the data difference of each feature dimension; Each of the data differences is compared with a preset threshold of the corresponding feature dimension, and a feature dimension whose data difference is greater than the preset threshold is determined as an important feature, so as to obtain an importance analysis result of each feature dimension.

5. The characteristic dimension importance analysis method for the risk credit model according to claim 1 is characterized in that: The step of modifying the original output result to obtain a modified result comprises: receiving a modification instruction for modifying the original output result; The result carried in the modification instruction is used as the modification result of the original output result.

6. The characteristic dimension importance analysis method for the risk credit model according to claim 1, characterized in that: When the original output result is a probability value, the step of modifying the original output result to obtain a modified result includes: Detecting whether the original output result is greater than a preset classification threshold; If the original output result is greater than the preset classification threshold, a probability value is selected from a first interval as the modified result, wherein the first interval is an interval greater than or equal to zero and less than the preset classification threshold; If the original output result is less than or equal to the preset classification threshold, a probability value is selected from a second interval as the modified result, wherein the second interval is an interval less than or equal to one and greater than the preset classification threshold.

7. The method for analyzing the importance of characteristic dimensions of a risk credit model according to any one of claims 1 to 6, characterized in that: The importance analysis result is a result indicating important features in each feature dimension, the target machine learning model is a risk credit model, and after the step of performing data comparison on each feature dimension of the modified sample and the original sample and obtaining the importance analysis result of each feature dimension based on the comparison result, the method further includes: According to the importance analysis result, detecting whether the data corresponding to the important features in the sample data of the user to be predicted is abnormal data; If the data corresponding to the important feature is abnormal data, weakening the data corresponding to the important feature; The weakened sample data is input into the target machine learning model to obtain the risk credit result.

8. A device for analyzing the importance of characteristic dimensions of a risk credit model, characterized in that: The device comprises: An input module, used for inputting user data as original samples into a target machine learning model to obtain an original output result, and modifying the original output result to obtain a modified result, wherein the target machine learning model is a risk credit model for processing user data and predicting whether to provide a loan to the user, wherein modifying the original output result means that when the output data of the target machine learning model is a continuous value, the original output result is modified to a value other than the original output result within a continuous value range, and when the output data of the target machine learning model is a discrete value, the original output result is modified to a value other than the original output result within a discrete value range; A modification module, used for back-propagating the difference between the modified result and the original output result in the target machine learning model to modify the original sample to obtain a modified sample; A comparison module is used to compare the data of each characteristic dimension of the modified sample and the original sample, and obtain the importance analysis results of each characteristic dimension of the user data based on the comparison results, and the importance analysis results reflect the influence of each characteristic dimension on the risk credit result predicted by the risk credit model.

9. A device for analyzing the importance of characteristic dimensions of a risk credit model, characterized in that: The device for analyzing the importance of characteristic dimensions of risk credit models comprises: a memory, a processor, and a program for analyzing the importance of characteristic dimensions of risk credit models stored in the memory and executable on the processor. When the program for analyzing the importance of characteristic dimensions of risk credit models is executed by the processor, the steps of the method for analyzing the importance of characteristic dimensions of risk credit models as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a characteristic dimension importance analysis program for a risk credit model, and when the characteristic dimension importance analysis program for a risk credit model is executed by a processor, the steps of the characteristic dimension importance analysis method for a risk credit model as described in any one of claims 1 to 7 are implemented.

Images