Model explainability information generation method and apparatus, electronic device, and storage medium

Abstract

The application provides a model explainability information generation method and device, electronic equipment and a storage medium. The model explainability information generation method comprises: obtaining a first sample set used for training a target model; determining feature importance information according to the first sample set and model parameters of the target model; obtaining a second sample set comprising the feature importance information and each sample in the first sample set; performing dimension reduction processing on the second sample set to obtain a dimension reduction processing result; mapping the plurality of features to a two-dimensional coordinate system as new coordinate axes according to the dimension reduction processing result to obtain a mapping processing result; and obtaining model explainability information of the target model according to the mapping processing result. The present scheme can enhance the explainability of the model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a model explainability information generation method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the wide application of AI (Artificial Intelligence) solutions in the fields of finance, regulation, justice, medical treatment and education, how to improve the direct understanding of the working mechanism of the model and open the black box of artificial intelligence becomes increasingly important. In the industrial scene, understanding the output result of the model and the calculation method of the explanation of the AI model is very helpful for maintenance personnel.

[0003] At present, there are many methods for explaining the AI model, such as visual exploration, feature-based and sample-based.

[0004] However, after dimension reduction of multiple features in the sample based on these methods, the reduced features do not have enough explainability to the model. SUMMARY

[0005] Therefore, the embodiments of the present application provide a model explainability information generation method and device, electronic equipment and storage medium, which can enhance the explainability of the model.

[0006] According to a first aspect of the embodiments of the present application, a model explainability information generation method is provided, including: obtaining a first sample set used for training a target model, wherein the first sample set includes multiple samples, and each sample includes multiple features; determining feature importance information according to the first sample set and model parameters of the target model, wherein the feature importance information is used to indicate feature importance of the multiple features; obtaining a second sample set including the feature importance information and each sample in the first sample set; performing dimension reduction processing on the second sample set to obtain a dimension reduction processing result, wherein the dimension reduction processing result is used to indicate a position of each sample in the second sample set in a two-dimensional coordinate system, and the two-dimensional coordinate system takes two dimensions after dimension reduction as coordinate axes; mapping the multiple features as new coordinate axes into the two-dimensional coordinate system respectively according to the dimension reduction processing result to obtain a mapping processing result; and obtaining model explainability information of the target model according to the mapping processing result.

[0007] In a possible implementation, the determining the feature importance information according to the first sample set and the model parameters of the target model comprises: when the target model is a logistic regression model, determining a plurality of coefficients corresponding to a plurality of features included in the samples in the first sample set in the model parameters of the target model, and determining an absolute value corresponding to each of the coefficients as the feature importance information; when the target model is a tree-based model, determining a plurality of occurrence frequency information of a plurality of features included in the samples in the first sample set in a tree node in the model parameters of the target model, and respectively determining the occurrence frequency information corresponding to each of the features as the feature importance information; when the target model is a deep neural network model, determining a plurality of Shapley additive explanation values corresponding to a plurality of features included in the samples in the first sample set in the model parameters of the target model, and determining each of the plurality of Shapley additive explanation values as the feature importance information.

[0008] In a possible implementation, the method used in the dimension reduction processing comprises a multidimensional scaling analysis method or a principal component analysis method.

[0009] In a possible implementation, the method further comprises: inputting the samples in the first sample set into the target model to obtain a prediction result output by the target model; and according to the prediction result, marking the mapping points corresponding to the samples in the first sample set in the two-dimensional coordinate system.

[0010] In a possible implementation, the obtaining the model interpretability information of the target model according to the mapping processing result comprises: determining a projection value of the samples in the second sample set on the new coordinate axis according to the mapping processing result; and determining the projection value of the samples in the second sample set on the new coordinate axis as the model interpretability information of the target model.

[0011] In a possible implementation, the target model comprises an industrial predictive maintenance model.

[0012] According to a second aspect of the embodiments of the present application, a model explainability information generation apparatus is provided, comprising: a first obtaining module configured to obtain a first sample set used for training a target model, wherein the first sample set comprises a plurality of samples, and each sample comprises a plurality of features; a determining module configured to determine feature importance information according to the first sample set and model parameters of the target model, wherein the feature importance information is used to indicate feature importance of the plurality of features; a second obtaining module configured to obtain a second sample set comprising the feature importance information and each sample in the first sample set; a processing module configured to perform dimension reduction processing on the second sample set to obtain a dimension reduction processing result, wherein the dimension reduction processing result is used to indicate a position of each sample in the second sample set in a two-dimensional coordinate system, and the two-dimensional coordinate system takes two dimensions after dimension reduction as coordinate axes; a mapping module configured to map the plurality of features as new coordinate axes into the two-dimensional coordinate system according to the dimension reduction processing result to obtain a mapping processing result; and a third obtaining module configured to obtain model explainability information of the target model according to the mapping processing result.

[0013] According to a third aspect of the embodiments of the present application, an electronic device is provided, comprising: a processor, a communication interface, a memory and a communication bus, the processor, the memory and the communication interface complete communication with each other through the communication bus; the memory is used to deposit at least one executable instruction, and the executable instruction makes the processor execute the operation corresponding to the model explainability information generation method provided in the first aspect.

[0014] According to a fourth aspect of the embodiments of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores computer instructions, and the computer instructions make the processor execute the operation corresponding to the model explainability information generation method provided in the first aspect when executed by the processor.

[0015] According to a fifth aspect of the embodiments of the present application, a computer program product is provided, and the computer program product is tangibly stored on a computer readable medium and comprises computer executable instructions, and the computer executable instructions make at least one processor execute the model explainability information generation method provided in the first aspect when executed.

[0016] The above technical solution first obtains a first sample set for training the target model. Based on the first sample set and the model parameters of the target model, the importance information of features is determined. Then, by merging the feature importance information with the samples in the first sample set, a second sample set is obtained. Next, the second sample set undergoes dimensionality reduction processing to obtain the dimensionality reduction result. Based on the dimensionality reduction result, multiple features are respectively used as new coordinate axes and mapped onto two-dimensional coordinate axes to obtain the mapping result. Finally, based on the mapping result, the interpretability information of the target model is obtained. Since the samples in the second sample set have multiple features, by performing dimensionality reduction processing on multiple features and mapping multiple features as new coordinate axes onto two-dimensional coordinate axes to obtain the mapping result, the interpretability information of the target model can be obtained based on the mapping result. Based on the model interpretability information, the importance of features and samples to the target model can be determined, thereby improving the interpretability of the model. Attached Figure Description

[0017] Figure 1 This is a flowchart of a method for generating model interpretability information according to an embodiment of this application;

[0018] Figure 2 This is a schematic diagram illustrating the use of multidimensional scaling analysis in one embodiment of this application;

[0019] Figure 3 This is a flowchart of a method for generating model interpretability information provided in another embodiment of this application;

[0020] Figure 4 This is a schematic diagram of a model interpretability information generation apparatus provided in one embodiment of this application;

[0021] Figure 5 This is a schematic diagram of an electronic device provided in one embodiment of this application.

[0022] List of reference numerals in the attached diagram:

[0023] 100: Method for generating model interpretability information; 500: Device for generating model interpretability information.

[0024] 201: Main unit temperature; 202: Lubricating oil pressure; 203: Effective value of vibration at the axial measuring point of the main unit.

[0025] 401: First Acquisition Module; 402: Determination Module; 403: Second Acquisition Module

[0026] 404: Processing module; 405: Mapping module; 406: Third acquisition module

[0027] 500: Electronic device; 502: Processor; 504: Communication interface

[0028] 506: memory 508: communication bus 510: program

[0029] P: feature importance point M: sample point

[0030] 101: Obtain a first sample set for training a target model

[0031] 102: Determine feature importance information according to the first sample set and model parameters of the target model

[0032] 103: Obtain a second sample set including feature importance information and each sample in the first sample set

[0033] 104: Perform dimension reduction processing on the second sample set to obtain a dimension reduction processing result

[0034] 105: According to the dimension reduction processing result, map a plurality of features as new coordinate axes to a two-dimensional coordinate system respectively to obtain a mapping processing result

[0035] 106: Obtain model explainability information of the target model according to the mapping processing result

[0036] 301: Input the samples in the first sample set into the target model to obtain a prediction result output by the target model

[0037] 302: According to the prediction result, mark the mapping points corresponding to the samples in the first sample set in the two-dimensional coordinate system DETAILED DESCRIPTION

[0038] With the wide application of AI solutions in various fields, it becomes more and more important to open the black box of artificial intelligence by improving the direct understanding of the working mechanism of the model. Especially in the industrial scene, it is very helpful for maintenance personnel to understand the output result of the model and the calculation method of the explanation of the AI model. At present, there are many methods to explain the AI model, such as visual exploration, feature-based, sample-based, etc. However, after dimension reduction of multiple features in the sample based on these methods, the dimension-reduced features do not have enough explainability to the model.

[0039] In the embodiment of the present application, the feature importance information is determined according to the first sample set and the model parameters of the target model, a second sample set including the feature importance information and each sample in the first sample set is obtained, the second sample set is processed by dimension reduction to obtain a dimension reduction processing result, the plurality of features are respectively mapped to a two-dimensional coordinate system as new coordinate axes according to the dimension reduction processing result to obtain a mapping processing result, and the model interpretability information of the target model is obtained according to the mapping processing result. Since the plurality of features in the second sample set are mapped to the two-dimensional coordinate system to obtain the mapping processing result of the plurality of features corresponding to the new coordinate axes relative to the two-dimensional coordinate system, the model interpretability information of the target model is obtained, and according to the model interpretability information, the importance of each sample and feature to the target model can be determined, thereby improving the interpretability of the model.

[0040] The model interpretability information generation method, device, electronic equipment and storage medium provided in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0041] Model explainability information generation method

[0042] Figure 1 is a flowchart of the model interpretability information generation method of one embodiment of the present application. As shown in Figure 1 , the model interpretability information generation method 100 includes steps 101 to 106:

[0043] Step 101, obtaining a first sample set for training a target model.

[0044] The first sample set is the sample set for training the target model. The plurality of samples of the first sample set and the prediction results corresponding to the plurality of samples are input into the target model, the input plurality of samples are taken as variables of the target model, and the prediction results corresponding to the plurality of samples are taken as results corresponding to the variables of the target model. By continuously inputting sample variables, the parameters of the target model are adjusted to determine the final parameters of the target model, and then the target model is generated.

[0045] The first sample set for training the target model is obtained. The first sample set includes a plurality of samples, and each sample includes a plurality of features. For example, a sample set includes at least 3 samples, and each sample has three features.

[0046] In one example, the target model is a warning model for predicting whether a compressor is working normally, and the first sample set for training the warning model is shown in Table 1.

[0047] Table 1

[0048]

[0049] From the above table 1, each sample in the first sample set includes three features, and the three features are host temperature, lubricating oil pressure, and host axial measurement point vibration effective value. Taking sample 1 as an example, the value corresponding to the host temperature is 85℃, the value corresponding to the lubricating oil pressure is 20Mpa, and the value corresponding to the host axial measurement point vibration effective value is 5m / s 2 .

[0050] Step 102, determining feature importance information according to the first sample set and the model parameters of the target model.

[0051] The feature importance information is used to indicate the feature importance of the plurality of features, and the feature importance refers to the influence of a certain feature on the target model.

[0052] The feature importance information is a floating point value distributed in [0, +∞), and the greater the value in the feature importance information of a feature, the more important the feature is to the target model, and the value closer to 0 indicates no influence on the target model.

[0053] In one example, based on the first sample set shown in table 1, the feature importance information is shown in the following table 2.

[0054] Table 2

[0055] Feature name Feature importance information Lubricating oil pressure 5 Main engine temperature 2.5 Main engine axial point vibration effective value 1 … …

[0056] As shown in the above table 2, the importance information corresponding to the lubricating oil pressure is 5, the importance information corresponding to the host temperature is 2.5, and the importance information corresponding to the host axial measurement point vibration effective value is 1. As can be seen, the importance information corresponding to the lubricating oil pressure is greater than the importance information corresponding to the host temperature, and the importance information corresponding to the host temperature is greater than the importance information corresponding to the host axial measurement point vibration effective value, which indicates that, relative to the host temperature and the host axial measurement point vibration effective value, the lubricating oil pressure is more important to the target model, i.e. the influence of the lubricating oil pressure on the target model is greater.

[0057] Step 103, obtaining a second sample set including the feature importance information and each sample in the first sample set.

[0058] The feature importance information is added to the first sample set as a sample to obtain the second sample set, so the second sample set has one more sample than the first sample set, and the sample includes the feature importance information corresponding to the feature.

[0059] In one example, based on the first sample set shown in table 1 and the feature importance information shown in table 2, the second sample set is shown in the following table 3.

[0060] Table 3

[0061]

[0062] As shown in Table 3 above, the second sample set includes not only the samples in the first sample set, but also samples encoded as sample importance information. These samples record the sample importance information corresponding to features such as lubricating oil pressure, main engine temperature, and effective value of vibration at the main engine axial measuring point.

[0063] Step 104: Perform dimensionality reduction on the second sample set to obtain the dimensionality reduction result.

[0064] The dimensionality reduction result is used to indicate the position of each sample in the second sample set in a two-dimensional coordinate system, with the two dimensions after dimensionality reduction as the coordinate axes. By performing dimensionality reduction on the second sample set, the multi-dimensional features included in the samples in the second sample set can be reduced to two-dimensional features. This allows us to determine the values ​​of each sample in the second sample set in the two feature dimensions after dimensionality reduction, thus obtaining an indication of the position of each sample in the second sample set in a two-dimensional coordinate system with the two feature dimensions after dimensionality reduction as the coordinate axes. In one example, the coordinate system with the two dimensions after dimensionality reduction as the coordinate axes is a Cartesian coordinate system.

[0065] In one example, after dimensionality reduction of the second sample set shown in Table 3 above, we obtain... Figure 2 The dimensionality reduction result is shown. Figure 2 As shown, the multiple dimensions of the samples in the second sample set are reduced to dimensions x and y. The three new coordinate axes corresponding to the three features in the three degrees of freedom directions are host temperature 201, lubricating oil pressure 202, and effective value of vibration of the host axial measuring point 203. The higher the absolute value of the projection of the sample point in the second sample set onto the new coordinate axis, the more important the sample is to that feature. After dimensionality reduction, feature importance points P will also be displayed. The higher the absolute value of the projection of feature importance point P onto any of the three new coordinate axes, the higher the importance of the feature corresponding to that new coordinate axis.

[0066] Step 105: Based on the dimensionality reduction result, map the multiple features as new coordinate axes to the two-dimensional coordinate system to obtain the mapping result.

[0067] After dimensionality reduction, multiple features in the second sample set are projected onto a two-dimensional coordinate system as new coordinate axes. By determining the absolute value of the projection of the feature importance points among the multiple features onto the new coordinate axes corresponding to the feature, the mapping result is obtained.

[0068] Step 106: Based on the mapping processing results, obtain the model interpretability information of the target model.

[0069] In an implementable example, in the mapping processing result, the higher the absolute value of the feature importance point in the plurality of features in the projection of the corresponding new coordinate axis of the feature, the more important the feature. According to the mapping processing result, the model explainability information of the target model is generated.

[0070] In the embodiments of the present application, first, a first sample set used for training the target model is acquired, feature importance information is determined according to the first sample set and the model parameters of the target model, then a second sample set including the feature importance information and each sample of the first sample set is acquired, next, the second sample set is subjected to dimension reduction processing to obtain a dimension reduction processing result, according to the dimension reduction processing result, the plurality of features are respectively mapped to a two-dimensional coordinate system as new coordinate axes to obtain a mapping processing result, and finally, according to the mapping processing result, the model explainability information of the target model is obtained. As can be seen, after the plurality of features in the second sample set are mapped to the two-dimensional coordinate system as new coordinate axes to obtain the mapping processing result, the model explainability information of the target model is obtained, the plurality of features are mapped to the two-dimensional coordinate system, the feature importance point in the plurality of features and the absolute value of the projection of the corresponding new coordinate axis of the feature importance point are determined, so as to know which feature and sample have stronger influence on the target model, thereby improving the explainability of the target model.

[0071] In a possible implementation manner, the target model in step 102 in the model explainability information generation method 100 can be different models:

[0072] When the target model is a logistic regression model, a plurality of coefficients corresponding to a plurality of features included in the samples in the first sample set in the model parameters of the target model are determined, and the absolute value of each coefficient is determined as the feature importance information.

[0073] The method for determining the model parameters of the target model is different when the target model is different. When the target model is a logistic regression model, each feature in the samples in the first sample set corresponds to a coefficient, and the absolute values of the plurality of parameters are determined as the feature importance information. For example, there are 3 features in the samples in the first sample set, the coefficient corresponding to feature 1 is 2, the absolute value of the coefficient 2 is determined as the feature importance information of feature 1, the coefficient corresponding to feature 2 is -3, the absolute value of the coefficient -3 is 3, which is determined as the feature importance information of feature 2, and the coefficient corresponding to feature 3 is 3, the absolute value of the coefficient 3 is 3, which is determined as the feature importance information of feature 3.

[0074] When the target model is a tree-based model, the occurrence frequency information of the plurality of features included in the samples in the first sample set at the tree nodes in the model parameters of the target model is determined, and the occurrence frequency information corresponding to each feature is determined as the feature importance information.

[0075] When the target model is a tree-based model, the plurality of times information corresponding to the plurality of features included in the sample in the first sample set appearing at the nodes of the tree is determined, and the plurality of times information is determined as the feature importance information. For example, the sample in the first sample set has four features, the times information of feature A appearing at the nodes of the tree is 3, the times 3 is determined as the feature importance information of feature A; the times information of feature B appearing at the nodes of the tree is 1, the times 1 is determined as the feature importance information of feature B; the times information of feature C appearing at the nodes of the tree is 2, the times 2 is determined as the feature importance information of feature C; and the times information of feature D appearing at the nodes of the tree is 5, the times 5 is determined as the feature importance information of feature D.

[0076] When the target model is a deep neural network model, the plurality of Shapley values corresponding to the plurality of features included in the sample in the first sample set in the model parameters of the target model are determined, and each Shapley value is determined as the feature importance information.

[0077] When the target model is a deep neural network model, a prediction value is generated for each sample in the first sample set, the Shapley value assigned to each feature in the sample is the Shapley value assigned to each feature, and the Shapley value assigned to each feature may be different or the same, and then the Shapley value corresponding to each feature is determined as the feature importance information. For example, the sample in the first sample set has three features, the Shapley value corresponding to feature a is 0.5, 0.5 is determined as the feature importance information of feature a; the Shapley value corresponding to feature b is 1, 1 is determined as the feature importance information of feature b; and the Shapley value corresponding to feature c is 2.5, 2.5 is determined as the feature importance information of feature c.

[0078] In the embodiments of the present application, different target models use different determination methods to determine the feature importance information, and different determination methods are used to determine the feature importance information that is more consistent with the target model, so as to obtain more accurate feature importance information.

[0079] In a possible implementation, the method used in the dimension reduction processing can be a multidimensional scaling analysis method or a principal component analysis method.

[0080] The multidimensional scaling analysis method (multidimensional scaling, MDS) is based on the similarity of the second sample set, calculates the positions of the sample data points and the feature data points in the K-dimensional space, and requires that after the high-dimensional data is converted into low-dimensional data, the relative position relationship of the sample data points and the feature data points remains unchanged.

[0081] Principal Components Analysis (PCA) is a method of mapping data in a high-dimensional second sample set to a low-dimensional space through a certain linear projection, and expecting the information amount of data in the projected dimension to be maximum, so as to use less data dimension while retaining more characteristics of the original data points.

[0082] In the embodiments of the present application, the second sample set is processed by using the multidimensional scaling analysis method or the principal component analysis method to obtain sample points and feature importance points P of the second sample set projected in the two-dimensional coordinate system, so as to determine the importance of the sample to the feature and the importance of the feature according to the absolute value of the projection of the sample point on the new coordinate axis and the absolute value of the projection of the feature importance point P on any new coordinate axis in the new coordinate axis, so as to determine the influence of the sample and the feature on the target model.

[0083] Figure 3 is a flowchart of the model interpretability information generation method of another embodiment of the present application. As shown in Figure 3 , the model interpretability information generation method 100 further includes steps 301 to 302 in addition to steps 101 to 106:

[0084] Step 301, inputting the samples in the first sample set into the target model to obtain the prediction results output by the target model.

[0085] The first sample set is input into the target model, and the plurality of features in the first sample set are predicted. Based on the above example, it is predicted whether the compressor is working normally. The prediction results of different features in the three different samples of whether the compressor is working normally are high risk, low risk and medium risk respectively, so as to obtain the prediction results of the features of different samples in the first sample set.

[0086] Step 302, according to the prediction results, marking the mapping points of the samples in the first sample set in the two-dimensional coordinate system.

[0087] Based on the above example, after the second sample set is processed by dimension reduction, the mapping points of each sample in the second sample set in the two-dimensional coordinate system are marked.

[0088] Based on the existing method, after the samples in the first sample set are mapped in the two-dimensional coordinate system, there is not enough interpretability, and it is not intuitive for users to use.

[0089] By using the method of marking the mapping points, the intuitiveness of the mapping points is improved.

[0090] The marking method can be marked by different colors or different symbols, etc. For example, the sample points in the above strength are marked by different colors, the sample points with high risk are marked in red, the sample points with low risk are marked in blue, and the sample points with medium risk are marked in orange.

[0091] In the embodiment of the present application, the samples in the first sample set are predicted by the target model to obtain the prediction results, and the mapping points corresponding to the samples in the first sample set are marked according to the prediction results. Through the marked mapping points, the prediction results of the sample points can be directly viewed in the two-dimensional coordinate system, and the user can easily obtain information.

[0092] In a possible implementation, step 106 in the model explainability information generation method 100 includes the following sub-steps:

[0093] According to the mapping processing result, the projection value of the sample in the second sample set on the new coordinate axis is determined.

[0094] After the dimension reduction processing of the samples in the second sample set, the plurality of samples are projected into sample points on the two-dimensional coordinate axis, and the sample points correspond to the corresponding projection value on each new coordinate axis. For example, Figure 2 As shown in the figure, the projections of the sample point M on the three new coordinate axes are: the projection value on the lubricating oil pressure 202 is 15, the projection value on the host temperature 201 is 1, and the projection value on the host shaft direction measurement point vibration effective value 203 is -8.

[0095] The projection value of the sample in the second sample set on the new coordinate axis is determined as the model explainability information of the target model.

[0096] According to the projection value of the sample in the second sample set on the new coordinate axis, the higher the absolute value of the projection value, the more important the sample is to the feature, thereby determining the model explainability information of the target model.

[0097] In the embodiment of the present application, according to the mapping processing result, the projection value of the sample in the second sample set on the new coordinate axis is determined, and then the projection value is determined as the model explainability information of the target model. The plurality of sample data in the second sample set is converted into a fixed numerical value, i.e. the projection value, thereby facilitating the user to obtain the model explainability information by reading the projection value, and improving the explainability of the target model.

[0098] In a possible implementation, the target model includes an industrial predictive maintenance model.

[0099] In industrial manufacturing, if the downtime of a device exceeds 1 hour, the loss can be as high as hundreds of thousands of dollars. Through predictive maintenance of industrial equipment, the occurrence of hidden troubles can be avoided.

[0100] The industrial data are high-dimensional and large-capacity data. In an example, the high dimensionality can be that the number of features is greater than 50, and the large capacity can be that the number of samples is greater than 10,000.

[0101] In the embodiments of the present application, by limiting the target model to be an industrial predictive maintenance model, the application scenario of the present application is determined, so that the method of the present application is applicable to the industrial scenario, and the interpretability of the target model is improved.

[0102] Model explainability information generation device

[0103] Based on the model interpretability information generation method 100 in the above embodiments, an embodiment of the present application provides a model interpretability information generation device. Figure 5 is a schematic diagram of the model interpretability information generation device according to an embodiment of the present application. As shown in Figure 5 The model interpretability information generation device 400 includes the following modules:

[0104] The first acquisition module 401 is configured to acquire a first sample set used for training a target model, wherein the first sample set includes a plurality of samples, and each sample includes a plurality of features.

[0105] The determination module 402 is configured to determine feature importance information according to the first sample set and model parameters of the target model, wherein the feature importance information is used to indicate feature importance of the plurality of features.

[0106] The second acquisition module 403 is configured to obtain a second sample set including the feature importance information and each sample in the first sample set.

[0107] The processing module 404 is configured to perform dimension reduction processing on the second sample set to obtain a dimension reduction processing result, wherein the dimension reduction processing result is used to indicate a position of each sample in the second sample set in a two-dimensional coordinate system, and the two-dimensional coordinate system takes two dimensions after dimension reduction as coordinate axes.

[0108] The mapping module 405 is configured to map the plurality of features as new coordinate axes into the two-dimensional coordinate system according to the dimension reduction processing result to obtain a mapping processing result.

[0109] The third acquisition module 406 is configured to obtain model interpretability information of the target model according to the mapping processing result.

[0110] In the embodiment of the present application, the first acquisition module 401 acquires a first sample set trained by the target model, and the determination module 402 determines the feature importance information according to the first sample set and the model parameters of the target model; the second acquisition module 403 further obtains a second sample set by merging the feature importance information and each sample in the first sample set; the processing module 404 performs dimension reduction processing on the second sample set to obtain a dimension reduction processing result; the mapping module 405 maps the plurality of features as new coordinate axes into a two-dimensional coordinate axis according to the dimension reduction processing result to obtain a mapping processing result; and the third acquisition module 406 obtains the model explainability information of the target model according to the mapping processing result. Since the features of the samples in the second sample set are multiple, the plurality of features are subjected to dimension reduction processing, and the plurality of features are mapped as new coordinate axes into a two-dimensional coordinate axis to obtain a mapping processing result, so that the model explainability information of the target model is obtained according to the mapping processing result, and the model explainability information is used to determine which feature is important to the sample, thereby improving the explainability of the model.

[0111] Electronic device

[0112] Referring to Figure 5 is a schematic diagram of an electronic device provided by an embodiment of the present application. The specific implementation of the electronic device is not limited by the embodiments of the present application. As shown in Figure 5 , the electronic device can include a processor 502, a communications interface 504, a memory 506, and a communications bus 508. Among them:

[0113] The processor 502, the communications interface 504, and the memory 506 complete mutual communication through the communications bus 508.

[0114] The communications interface 504 is configured to communicate with other electronic devices or servers.

[0115] The processor 502 is configured to execute the program 510, and specifically can execute the related steps in any of the preceding model explainability information generation method embodiments.

[0116] Specifically, the program 510 can include program code including computer operation instructions.

[0117] The processor 502 can be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement one or more embodiments of the present application. The one or more processors included in the smart device can be the same type of processor, such as one or more CPUs, or different types of processors, such as one or more CPUs and one or more ASICs.

[0118] The memory 506 is configured to store a program 510. The memory 506 can include a high-speed RAM memory, and can further include a non-volatile memory, such as at least one disk memory.

[0119] The program 510 can be specifically configured to enable the processor 502 to perform the model explainability information generation method in any of the foregoing embodiments.

[0120] The specific implementation of each step in the program 510 can refer to the corresponding description in the corresponding steps and units in any of the foregoing model explainability information generation method embodiments, and will not be described herein. It can be clearly understood by those skilled in the art that, for the convenience and brevity of description, the specific working process of the device and the module described above can refer to the corresponding process description in the foregoing method embodiments, and will not be described herein.

[0121] In the embodiments of the present application, first, a first sample set used for training a target model is acquired, feature importance information is determined according to the first sample set and model parameters of the target model, and then a second sample set including the feature importance information and each sample of the first sample set is acquired. Next, the second sample set is processed in dimension reduction to obtain a dimension reduction processing result. According to the dimension reduction processing result, a plurality of features are respectively mapped to a two-dimensional coordinate system as new coordinate axes to obtain a mapping processing result. Finally, according to the mapping processing result, model explainability information of the target model is obtained. As can be seen, after the plurality of features in the second sample set are mapped to the two-dimensional coordinate system as new coordinate axes to obtain the mapping processing result, the model explainability information of the target model is obtained. The plurality of features are mapped to the two-dimensional coordinate system, and the feature importance point in the plurality of features and the absolute value of the projection of the new coordinate axis corresponding to the feature importance point are determined, so as to know which feature has stronger influence on the target model, thereby improving the explainability of the target model.

[0122] Computer storage medium

[0123] The embodiments of the present application also provide a computer readable storage medium storing instructions for causing a machine to perform the model explainability information generation method described herein. Specifically, a system or apparatus equipped with a storage medium on which a software program code for implementing the functions of any of the above embodiments is stored can be provided, and the computer (or CPU or MPU) of the system or apparatus is caused to read out and execute the program code stored in the storage medium.

[0124] In this case, the program code read from the storage medium itself can implement the functions of any of the above embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present application.

[0125] Embodiments of the storage medium for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as a CD-ROM, a CD-R, a CD-RW, a DVD-ROM, a DVD-RAM, a DVD-RW, a DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0126] Furthermore, it should be clear that not only the program code read by the computer, but also the operating system or the like operating on the computer based on the instructions of the program code can perform part or all of the actual operations to implement the functions of any of the above embodiments.

[0127] Furthermore, it should be understood that the program code read from the storage medium is written into a memory provided in an expansion board inserted into the computer or a memory provided in an expansion module connected to the computer, and then part or all of the actual operations are performed by a CPU or the like mounted on the expansion board or the expansion module based on the instructions of the program code to implement the functions of any of the above embodiments.

[0128] Computer program product

[0129] The embodiments of the present application also provide a computer program product tangibly stored on a computer readable medium and including computer executable instructions that, when executed, cause at least one processor to perform the model explainability information generation method provided by each of the above embodiments. It should be understood that each of the schemes in the present embodiment has the corresponding technical effects in the above method embodiments, which will not be described here.

[0130] It should be noted that not all steps and modules in the above processes and system structure diagrams are necessary, and some steps or modules can be omitted according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in each of the above embodiments can be a physical structure or a logical structure, that is, some modules can be implemented by the same physical entity, or some modules can be implemented by multiple physical entities, or can be implemented by some components in multiple independent devices together.

[0131] In each of the above embodiments, a hardware module can be implemented mechanically or with electrical means. For example, a hardware module can include dedicated circuitry or logic that is permanently configured to perform certain operations. A hardware module can also include programmable logic or circuitry (e.g., as encompassed in a general-purpose processor or other programmable processor) that is temporarily configured by software to perform certain operations. The specific manner in which a hardware module is implemented (i.e., mechanically or with permanently configured circuitry, or temporarily configured circuitry) can depend on cost and time considerations.

[0132] The application has been described in detail through the accompanying drawings and preferred embodiments, but the application is not limited to these disclosed embodiments. As can be known by those skilled in the art based on the above-described multiple embodiments, more embodiments of the application can be obtained by combining the code review means in the above-described different embodiments, and these embodiments are also within the protection scope of the application.

Claims

1. A method for generating model interpretability information (100), comprising: Obtain a first sample set for training the target model, wherein the first sample set includes multiple samples, and each sample includes multiple features; Based on the first sample set and the model parameters of the target model, feature importance information is determined, wherein the feature importance information is used to indicate the feature importance of the multiple features; Obtain a second sample set including the feature importance information and each sample in the first sample set; The second sample set is subjected to dimensionality reduction processing to obtain the dimensionality reduction processing result, wherein the dimensionality reduction processing result is used to indicate the position of each sample in the second sample set in a two-dimensional coordinate system, the two-dimensional coordinate system being used as coordinate axes after dimensionality reduction; Based on the dimensionality reduction result, the multiple features are respectively used as new coordinate axes and mapped onto the two-dimensional coordinate system to obtain the mapping result; Based on the mapping processing results, the interpretability information of the target model is obtained.

2. The method according to claim 1, wherein, The step of determining feature importance information based on the first sample set and the model parameters of the target model includes: When the target model is a logistic regression model, multiple coefficients corresponding to multiple features included in the first sample set in the model parameters of the target model are determined, and the absolute value corresponding to each coefficient is determined as the feature importance information; When the target model is a tree-based model, the occurrence frequency information of multiple features included in the first sample set in the model parameters of the target model in the tree nodes is determined, and the occurrence frequency information corresponding to each feature is determined as the feature importance information. When the target model is a deep neural network model, the multiple Shapley sum interpretation values ​​corresponding to multiple features included in the first sample set are determined in the model parameters of the target model, and each of the Shapley sum interpretation values ​​is determined as the feature importance information.

3. The method according to claim 1, wherein, The dimensionality reduction process employs methods including multidimensional scaling analysis or principal component analysis.

4. The method according to claim 1, wherein, The method further includes: The samples in the first sample set are input into the target model to obtain the prediction results output by the target model; Based on the prediction results, the mapping points corresponding to the samples in the first sample set are marked in the two-dimensional coordinate system.

5. The method according to claim 1, wherein, The step of obtaining the model interpretability information of the target model based on the mapping processing result includes: Based on the mapping processing result, determine the projection value of the sample in the second sample set on the newly added coordinate axis; The projection values ​​of the samples in the second sample set onto the newly added coordinate axis are determined as the model interpretability information of the target model.

6. The method according to any one of claims 1-5, wherein, The target model includes an industrial predictive maintenance model.

7. A model interpretability information generation device (500), comprising: The first acquisition module (501) is used to acquire a first sample set for training the target model, wherein the first sample set includes multiple samples, and each sample includes multiple features; The determining module (502) is used to determine feature importance information based on the first sample set and the model parameters of the target model, wherein the feature importance information is used to indicate the feature importance of the plurality of features; The second acquisition module (503) is used to obtain a second sample set including the feature importance information and each sample in the first sample set; The processing module (504) is used to perform dimensionality reduction processing on the second sample set to obtain the dimensionality reduction processing result, wherein the dimensionality reduction processing result is used to indicate the position of each sample in the second sample set in a two-dimensional coordinate system, the two-dimensional coordinate system using the two dimensions after dimensionality reduction as coordinate axes; The mapping module (505) is used to map the multiple features as new coordinate axes to the two-dimensional coordinate system according to the dimensionality reduction processing result, so as to obtain the mapping processing result; The third acquisition module (506) is used to obtain the model interpretability information of the target model based on the mapping processing result.

8. An electronic device (600), comprising: The processor (602), the communication interface (604), the memory (606), and the communication bus (608) communicate with each other through the communication bus (608). The memory (606) is used to store at least one executable instruction that causes the processor (602) to perform the operation corresponding to the model interpretability information generation method (100) as described in any one of claims 1-6.

9. A computer-readable storage medium storing computer instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1-6.

10. A computer program product tangibly stored on a computer-readable medium and comprising computer-executable instructions that, when executed, cause at least one processor to perform the method according to any one of claims 1-6.

Images