Insights from deep learning models using provenance data

Abstract

A method, computer system, and computer program product for generating deep learning model insights using provenance data are provided. Embodiments of the invention may include collecting provenance data. Embodiments of the invention may include generating model insights based on the collected provenance data. Embodiments of the invention may include generating a training model based on the generated model insights. Embodiments of the invention may include reducing the size of the training model. Embodiments of the invention may include creating a final trained model.

Description

Technical Field

[0001] The present invention relates generally to the field of computing, and more particularly to machine learning. Background Art

[0002] Deep learning models have been used in many fields and can be considered as black boxes in terms of understanding the model behavior. Various trained models process data and provide different results with different accuracy. Producing understandable results and accurate classifications has advanced the field of machine learning by providing precise and unbiased output data to be used in various fields and industries. Summary of the invention

[0003] Embodiments of the present invention disclose a method, computer system, and computer program product for generating deep learning model insights using provenance data. Embodiments of the present invention may include collecting provenance data. Embodiments of the present invention may include generating model insights based on the collected provenance data. Embodiments of the present invention may include generating a training model based on the generated model insights. Embodiments of the present invention may include reducing the size of the training model. Embodiments of the present invention may include creating a final trained model. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] These and other objects, features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments read in conjunction with the accompanying drawings. The various features of the drawings are not drawn to scale because the illustrations are for clarity, so as to assist those skilled in the art in understanding the present invention in conjunction with the detailed description. In the drawings:

[0005] Figure 1 illustrates a networked computer environment according to at least one embodiment;

[0006] Figure 2 is an example of a block diagram for collecting provenance data in each iteration according to at least one embodiment;

[0007] Figure 3 is an operational flow diagram illustrating a process for building an analytical model using collected provenance data in accordance with at least one embodiment;

[0008] Figure 4 is an operational flow diagram illustrating a process for building an inference model using collected provenance data in accordance with at least one embodiment;

[0009] Figure 5 According to at least one embodiment Figure 1 A block diagram of the internal and external components of a computer and server depicted in;

[0010] Figure 6According to an embodiment of the present disclosure, Figure 1 A block diagram of an illustrative cloud computing environment of a computer system depicted in; and

[0011] Figure 7 According to the embodiment of the present disclosure Figure 6 A block diagram of the functional layers of an illustrative cloud computing environment. DETAILED DESCRIPTION

[0012] Detailed embodiments of the claimed structures and methods are disclosed herein; however, it is understood that the disclosed embodiments are merely illustrative of the claimed structures and methods that may be embodied in various forms. However, the present invention may be embodied in many different forms and should not be construed as limited to the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete and will fully convey the scope of the invention to those skilled in the art. In the specification, details of well-known features and techniques may be omitted to avoid unnecessary obfuscation of the presented embodiments.

[0013] As mentioned earlier, deep learning models are used in many fields and can be considered a black box in terms of understanding the model behavior. Various trained models process data and provide different results with varying degrees of accuracy. Producing understandable results and accurate classifications has advanced the field of machine learning by providing precise and unbiased output data that can be used in various fields and industries.

[0014] Deep learning models can collect and analyze large amounts of data across various fields and can be used to solve real-world problems. However, understanding model training, model analysis, model behavior, model advantages, and model reasoning may be limited by the quality of the data used in deep learning training models. Typically, the training model is analyzed based on the data output from the final trained model rather than the origin data. Existing methods for pruning deep neural networks may analyze the final trained network, for example, by calculating the sum of the absolute kernel weights of the filters, and if the sum is less than a threshold, the filter is removed and the entire network is retrained. One disadvantage of existing methods may include that, given the analysis of the final trained model, there is no insight into the filter weights and how the filter weights converge. Another disadvantage may include using additional overhead to retrain the model after pruning the model. Another disadvantage may include that pruning filters with low weights does not guarantee that irrelevant filters have been removed, and therefore relevant or important filters may be removed.

[0015] Additionally, due to the large amount of data being analyzed and the amount of resources used to train the machine learning model, further limitations may be placed on the training process. Therefore, it may be advantageous to use provenance data to generate insights into the machine learning model, among other things. More specifically, to generate insights that will improve the accuracy of the deep learning model in producing unbiased results and to generate insights into why the deep learning model produced inaccurate results. Additionally, the model size may be reduced by determining which nodes to prune at each iteration or after multiple iterations, thereby providing greater bandwidth resources for analysis and training of the deep learning model.

[0016] Deep learning is a type of machine learning that can classify information based on training data. The training data can be structured data or unstructured data. Structured data can include highly organized data, such as spreadsheets, relational databases, or data stored in fixed fields. Unstructured data can include data that is unorganized and has an unconventional internal structure, such as portable document formats (PDFs), images, presentations, web pages, video content, audio content, emails, word processing documents, or multimedia content. Deep learning may also be related to or referred to as hierarchical learning or deep structured learning.

[0017] Deep learning can map inputs, classify data, interpret data sets, and provide data outputs for one or more layers of data. Each layer of data can be represented as a node. Nodes can also be referred to as neurons or artificial neurons. Deep learning can detect similarities in data that may be labeled or unlabeled. For example, deep learning can act as supervised learning, unsupervised learning, or semi-supervised learning. Supervised learning can use labeled data sets to train ML models. Unsupervised learning can use all unlabeled data to train ML models. Semi-supervised learning can use both labeled and unlabeled data sets to train ML models. A deep learning model can provide, for example, a graph output that can be generated as nodes and edges related to the domain-specific taxonomy being learned.

[0018] The exemplary embodiments described below provide systems, methods, and program products for deep learning. As such, embodiments of the present invention have the ability to improve the technical field of deep learning by generating insights from deep learning models that utilize provenance data collected during the training phase. More specifically, deep learning training data collected and analyzed using provenance data at each iteration will be used to build analytical models, reduce model size, and build inference models.

[0019] According to one embodiment, the analytical model can use the provenance data collected during the training phase to generate insights about the strength of the model to reduce the model size and detect the reasons for the failure of test samples. The insights generated from the analytical model can be used to reduce the size of the deep learning model. The insights generated from the analytical model can also be used to build an inference model. The inference model can analyze or generate insights about model behavior, test input sample failures, and model inference failures.

[0020] Neural networks can be a component of deep learning. Neural networks can be related to or referred to as deep networks or deep neural networks. Neural networks can interpret, label, and classify raw data, such as unstructured data. Neurons in a deep neural network can combine input data and assign weights to the input data based on the significance level of what the neural network is learning in order to classify the data. The deeper the neural network, the more layers of neurons or nodes the input data passes through. Neurons, nodes, and filters can be considered interchangeable terms. Neurons can represent receiving input data, generating input weights and associating the input weights with the data, and then determining by calculation where the data should continue or advance in the network before being classified. Each layer of neurons can train data based on the output layer of the previous layer.

[0021] According to one embodiment, provenance data may be collected, stored, and analyzed to train a deep learning model. Provenance data is collected and analyzed during the training phase of a deep learning model to provide reasoning and insights for various modeling tasks. Insights for the learning model are generated based on the provenance data at each iteration during the training process. The training process may include one or more iterations. Provenance data may include data collected based on training data that passes through a layer of a deep model. Each layer that the training data passes through may provide provenance data at each iteration. For example, an image used as training data passes through a model, and learned weights, activation values, and gradients generated by the model at each layer and each iteration. The learned weights, learned activation values, and learned gradients of the training data at each iteration may be stored as provenance data. Analysis of the provenance data at each iteration may include using the training image, using the activation values, and using the calculated gradients to filter and assign weights at neuronal connections. Collecting and storing provenance data at each iteration generates insights at one iteration, and over time, may collectively create a highly accurate training model after multiple iterations.

[0022] In one embodiment, the collected provenance data can be used to build an analytical model to generate insights about the deep learning model during model training. The analytical model can evaluate neuron weight convergence, the number of neuron weights that converge, the speed of neuron convergence, neuron weight fluctuations, neuron learning characteristics, the learning history of each neuron, which sets of neurons are related, neuron weight lineages, and nodes that have not been used. The analytical model evaluation provides insights on how to optimize the deep learning network.

[0023] The provenance data can be analyzed across different iterations. The quality of the provenance data can be determined by analyzing the convergence of neuron weights and how quickly the neuron weights converge. Non-convergence of weights may mean that the training data is high quality data or higher quality data, for example, the class data is balanced, or there is little or no blur or noise in the image data. Accordingly, the more the weights converge, the less provenance data may be needed for further training. Fast convergence of neuron weights indicates that the provenance data is quality data. Alternatively, slower convergence of neuron weights indicates lower quality of the training data, such as a small number of samples, imbalanced classes, or little data variation.

[0024] Analyzing provenance data for filter weight convergence and changes in each iteration can provide information that allows filters to be prioritized and filters to be pruned. Appropriately determining which filters can be removed without losing the integrity of the model can help reduce the size of the deep learning model. Neuron weight fluctuations can be analyzed to determine whether model parameters are updated correctly. Model parameters can include, for example, the momentum and learning rate of the model. If the weights of neurons fluctuate highly, even after multiple iterations, weight oscillation problems may occur. Weight oscillations may indicate that the data is insufficient for model training.

[0025] Analyzing neuron correlations can determine how features are combined to provide a more complex representation of the origin data at each iteration. Neuron correlation analysis can also help understand if neurons are learning using different filters. If the neural network learns similar filters, it can indicate that the complexity of the data is low or simple, so that a small network is sufficient to learn the data patterns of the training data. Low complexity of the data can mean, for example, little or no noise in the data, fewer data categories are present, less data variation is present, no occlusions are present, and no background information is present. With a smaller network and simplified data, some redundant filters can be removed, or the number of layers or nodes in the network can be reduced.

[0026] Analysis of the learning history of each neuron can be accomplished by generating snapshots or video clips for each neuron. The snapshots or video clips illustrate the characteristics of neuron learning over time and can highlight areas of activation on an image. For example, the provenance data is used to create a model snapshot that is taken at each timestamp to immediately show how activations evolve during training. The visualizations obtained from the snapshots of the provenance data taken at each timestamp are accurate descriptions, not estimates of backpropagation.

[0027] Snapshots can produce image files in multiple formats, such as graphical interface format (gif), joint photographic experts group (jpeg), portable network graphics (png), or tagged image file format (tiff). Collecting provenance data and creating model snapshots of neuron history at each iteration can provide insights into the training model. Model snapshots can provide insights by analyzing data about weights, activations, and changes between iterations. Insights can help prioritize neurons or filters to prune, thereby conserving resources while improving accuracy.

[0028] The neuron weight spectrum may include generating a spectrum for each neuron weight. For example, at each step, the deep learning program may generate input weights for other neurons that are back-propagated. The applied transformations and input images may also be generated. The data used to update the weights is data that can be used to generate the input weights, applied transformations, and input images.

[0029] In one embodiment, the insights generated from analyzing the model can be used to reduce the size of the deep learning model. At each iteration during training, the size of the deep learning model may be reduced. If it is determined that certain neurons have learned similar filters, redundant filters can be pruned or removed from the network. In addition, analyzing the change in weights at each iteration can allow pruning of higher weighted filters. For example, given two different neuron weights, if one weight is greater than the other, and both the higher weight and the lower weight are decreasing with each iteration, the higher weighted filter can be deleted because the higher weighted filter is becoming less relevant due to the increase in the number of samples with each iteration.

[0030] An average deep network may include, for example, 8 layers, 96 filters of size 11×11, 100,000 images, and 100,000 iterations. At each iteration, the stored data may include the weights of all neurons and connections, and for each image, the regions activated by each neuron. The total number of stored values ​​may become large and continue to grow, for example, starting at 10B and getting larger.

[0031] The reduction in model size can provide less computation, which can result in less runtime required to gain further insights about the trained model or less runtime required for the deep learning model. Training can start with a large number of nodes, and a large number of nodes can represent the model size. The model size can vary and can depend on the amount of training data for a specific domain. Various model sizes can be executed and run independently to evaluate model characteristics. For example, changing the model size and trying different filters requires running model training independently, which may not be feasible in a practical machine learning setting. In the case of using provenance data, the knowledge gained from various model sizes and various filters can be used to reduce the size of the deep learning model during model training.

[0032] In one embodiment, an inference model may be constructed to generate insights about a deep learning model using provenance data. An inference model may also be constructed to further prune data, test query samples, and find out why input test samples may fail on the model. An inference model may evaluate or utilize insights generated from an analytical model to infer the behavior of a deep learning model. An inference model may help detect non-optimal or poor performance of a model by analyzing model parameters. For example, an inference model may detect poor performance in a model by analyzing model parameters using provenance data and by using the constructed analytical model. Model parameters may include: data distribution, adversarial samples, corrupted samples, underfitting, overfitting, model training issues, or sampling issues.

[0033] Analysis of inference models related to adversarial examples may include, for example, feeding misclassified examples into the model and evaluating model performance and model behavior. Using provenance data, a graph may be generated to illustrate which class samples are responsible for activating nodes at different iterations. More specifically, a graph may be generated to illustrate which portion of a class sample image is responsible for the activation of a node. The graph may provide insights into which nodes have learned class samples or portions of class samples.

[0034] Underfitting can reduce model performance due to the inability of the machine learning model to accurately depict the input data or data trends. Overfitting can reduce model performance by providing a large amount of training data to the machine learning model so that the model learns too many details (such as noise and inaccurate data). Underfitting or overfitting of the training model can be determined by analyzing the node activations at different layers and different iterations. For example, if the nodes in the higher layer hierarchy are activated for different category samples at the initial iteration, but are not activated for different category samples at later iterations, there may be underfitting or overfitting. In addition, if the hierarchy of node activations is not saturated from multiple classes to a single class, or if the category set samples that activate the node do not have any visual similarity, there may also be underfitting or overfitting. If fine-grained category samples are used, and the node activation reaches saturation and still maps the test sample to the wrong category, then the test sample is likely to belong to the adversarial category. Fine-grained category samples may include category set samples with visual similarities. The saturation of node activations can be, for example, nodes activated for a single category.

[0035] The inference model can also analyze model training issues related to sampling issues. The inference model can identify sampling strategy issues by using images that will activate nodes or images that will be responsible for activating nodes in order to capture patterns in node activations. For example, if samples of the same class activate different sets of nodes, this information can help determine if a single class has multiple distributions or if there is a problem with the sampling strategy.

[0036] refer to Figure 1 , depicts an exemplary networked computer environment 100 according to one embodiment. The networked computer environment 100 may include a computer 102 having a processor 104 and a data storage device 106, which is capable of running a software program 108 and a deep learning program 110a. The networked computer environment 100 may also include a server 112, which is capable of running a deep learning program 110b that can interact with a database 114 and a communication network 116. The networked computer environment 100 may include multiple computers 102 and servers 112, only one of which is shown. The communication network 116 may include various types of communication networks, such as a wide area network (WAN), a local area network (LAN), a telecommunications network, a wireless network, a public switched network, and / or a satellite network. It should be understood that Figure 1 This is merely an illustration of one implementation and does not imply any limitations on the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made based on design and implementation requirements.

[0037] The client computer 102 can communicate with the server computer 112 via a communication network 116. The communication network 116 may include connections such as wires, wireless communication links, or fiber optic cables. Figure 5 As discussed above, the server computer 112 may include internal components 902a and external components 904a, respectively, and the client computer 102 may include internal components 902b and external components 904b, respectively. The server computer 112 may also operate in a cloud computing service model, such as software as a service (SaaS), analysis as a service (AaaS), platform as a service (PaaS), blockchain as a service (BaaS), or infrastructure as a service (IaaS). The server 112 may also be located in a cloud computing deployment model, such as a private cloud, a community cloud, a public cloud, or a hybrid cloud. The client computer 102 may be, for example, a mobile device, a phone, a personal digital assistant, a netbook, a laptop computer, a tablet computer, a desktop computer, or any type of computing device capable of running programs, accessing a network, and accessing a database 114. According to various implementations of the present embodiment, the deep learning program 110a, 110b may interact with a database 114, which may be embedded in various storage devices, such as, but not limited to, a computer / mobile device 102, a networked server 112, or a cloud storage service.

[0038] According to the present embodiment, a user using the client computer 102 or the server computer 112 can use the deep learning program 110a, 110b (respectively) to build an analytical model that can generate insights by analyzing the provenance data collected during the deep learning training. The insights generated from the analytical model can be further used to reduce the size of the deep learning model and understand why the deep learning model fails. Figures 2 to 4 Explain deep learning methods in more detail.

[0039] Reference now Figure 2 , depicts an example block diagram of collecting provenance data in each iteration used by a deep learning program 110a, 110b, according to at least one embodiment.

[0040] The training data 202 may include input data, including both structured and unstructured data. For example, the training data 202 may include images and documents stored on one or more databases. Public databases may be used, and private databases may be used with appropriate access rights. For example, a sample of the training data 202 may include various animal species obtained from public databases for training models. An example of a private database may include images and documents from a hospital repository used to train models related to medical images or medical diagnosis. Another example of a private database may include images and documents from other industries, such as finance, law, agriculture, automotive, or government. The training data 202 may be obtained based on accessibility.

[0041] The deep neural network architecture may have multiple layers 204a-n. Each layer 204a-n may have multiple neurons or nodes. Layers 204a-n may be various types of layers, such as convolutional layers, pooling layers, Relu layers, linear layers, or fully connected layers. A loss layer 206 may be after the last layer 204n. The loss layer 206 may calculate model strength and gradients. The loss layer 206 may backpropagate gradients to previous layers such as layer 204n to update model weights. Each iteration of the model has a forward pass and a backward pass. After each iteration, the collected data may include origin data 208a-n. The origin data 208a-n may include the training images used in the iteration, the weights learned, activation values ​​and which nodes were activated, and gradients.

[0042] Reference now Figure 3 , depicts an operational flow diagram illustrating an exemplary analytical model building process 300 used by the deep learning programs 110a, 110b, according to at least one embodiment. The analytical model building process 300 can be created by using the provenance data collected during the model training phase to generate insights regarding model strength.

[0043] At 302, provenance data is collected. Provenance data is collected by passing the training data 202 through the deep model layers and by storing the provenance data and the training data at each iteration, e.g., as previously described in Figure 2 As shown in . For example, training data retrieved from a public database related to a specific animal species is collected and passed through the deep model layer in the deep network, and the origin data is collected on the training data for that iteration. At each iteration, the origin data and the training data are stored in the database.

[0044] At 304, model insights are generated using the collected provenance data. Model insights are obtained by analyzing neuron weights, neuron learning characteristics, neuron history and lineage, related neurons, and unused neurons. The number of neuron weights that have converged can be analyzed, for example, to find problems with the collected provenance data or to find more data that can be used to train the model in another iteration. The neuron weight analysis can include evaluating the speed at which the neuron weights converge, and evaluating the number of iterations taken to identify the accuracy of the collected data and the accuracy of the trained algorithm. For example, training data related to an animal species and the collected provenance data were evaluated by the analytical model, and the neuron weights converged quickly after 5 iterations. Therefore, the rapid convergence of the neuron weights can indicate that the training data for the specified animal species is quality data.

[0045] Neuron weight analysis may also include evaluating the weights of neurons. If neurons or filters fluctuate widely, it may be necessary to modify or change parameters, such as by changing momentum. Neuron weight analysis may include comparing the weights of neurons across iterations and plotting graphs to visually illustrate neuron behavior. Neuron learning feature analysis may include identifying whether neurons are learning the same features. Filters may be added or reduced at each layer, for example, if neurons are learning the same filters, those filters on that particular layer may be pruned.

[0046] Neuron learning history analysis for each neuron can be evaluated so that snapshots or video clips are generated to highlight features that the neuron is learning over a period of time. The snapshots can additionally highlight which neuron regions are activated while generating insights into the collected provenance data. The snapshots can also provide visualizations of how the neural network is operating, for example, an image file can be generated for the user to view a representation of the neuron's learning history.

[0047] Neuron correlation analysis can assess which sets of neurons are correlated and can provide understanding of deep neural networks and how features are combined to give more complex representations at each iteration. Neuron lineage analysis can be generated or produced for each neuron weight and the input weights of other back-propagated neurons at each iteration.

[0048] At 306, the model size is reduced using the generated model insights. The model insights generated at step 304 may be used to reduce the model size. At each iteration, the provenance data is analyzed and the network size may be pruned or reduced. After the network size is reduced, model training will continue on the reduced network. Model insights that may be used to reduce the network size may include insights about neuron weights, neuron connections, features learned by various filters, and neuron weight convergence or neuron weight non-convergence. For every n iterations, a periodic analysis of the collected provenance data may be completed, and if it is determined that neuron removal should occur, the network model is reduced.

[0049] For example, consider two neurons N1(l) and N2(l) at layer l, and two corresponding neurons N3(l+1) and N4(l+1) in the final trained model at layer l+1, which have corresponding weights w1 and w2. Therefore, the corresponding weights of N1(l) and N3(l+1) are w1, and the corresponding weights of N2(l) and N4(l+1) are w2. In this example, weight w1 is greater than weight w2. The provenance data may provide information that the weight w1 of the connection of neuron N1(l) and the corresponding N3(l+1) decreases over time, while the weight w2 of the neuron connection N2(l) and the corresponding N4(l+1) increases over time. In this case, and based on the collected provenance data insights, the deep learning programs 110a, 110b may prioritize the weight w2 and the corresponding neuron connections N2(l) and N4(l+1), which may increase over time, over w1 and the corresponding neuron connections N1(l) and N3(l+1). Therefore, the deep learning program 110a, 110b may decide to remove or prune the N2(l) and N4(l+1) neural connections. In addition, over time, the neuron weights may converge, allowing additional reasoning to remove neurons at lower weight layers to reduce the model size while retaining the accuracy of the model.

[0050] At 308, a final trained model is created. For example, the final trained model may be created after n iterations and based on the analysis in the previous steps. The final trained model may be stored in a database for future use. In addition, the final trained model may still undergo more analysis to continuously generate insights for the trained model, such as the analysis provided in steps 304 and 306.

[0051] Reference now Figure 4 , depicts an operational flow chart showing an exemplary inference model building process 400 used by the deep learning programs 110a, 110b according to at least one embodiment. The inference model building process 400 can be created by using the final trained analytical model with query samples. The inference model can provide information about, for example, why certain input test samples fail on the model and how to use the insights created by the analytical model to understand the behavior of the deep learning model.

[0052] At 402, the query sample is classified using the trained model from step 310. The query sample of data may include data from a specific field or industry, or the query sample may include a batch of unrelated data. The query sample data may be classified using the trained model. For example, two different batches of class sample images and documents from two different animal species (class A (e.g., cats) and class B (e.g., dogs)) are input into the trained model for classification. Class A samples and class B samples may include one or more data distributions to be classified.

[0053] At 404, the deep learning program 110a, 110b determines whether the output is correct. The deep learning program 110a, 110b may determine whether the output is correct by analyzing the trained model classification of the input query sample. The determination may include comparing the neurons activated from the beginning iteration of the query sample with the neurons that may be activated during subsequent iterations of the query sample. For example, if samples from the same category activate X neurons in the first few iterations, and then in the next few iterations, samples from the same category activate another X' neurons, then it may indicate that a particular category has multiple distribution samples and incorrect sampling is used. Incorrect sampling may be due to one distribution of samples trained in the first few iterations, and then samples from a different distribution of the same category used in later iterations. Later iterations may include query samples that have been expanded with additional category data samples that may have been distributed.

[0054] Analysis of the output may provide insight into and reasons for correct or incorrect classification or misclassification. For example, applying an inference model to a trained model, a user may want to determine whether the model correctly or incorrectly classified a sample, and then the user may want to understand the reasons for the correct or incorrect output. The user may provide labels for correction purposes, such as ground truth, to create higher accuracy for the model. Classification analysis may include fine-grained classification, underfitting or overfitting of the model, or adversarial sample analysis. Fine-grained categories may include class collection samples that have visual similarities or share similar image structures. Underfitting or overfitting of the model may include using class samples that do not share visual similarities.

[0055] Provenance data can be used to generate graphs showing various iterations of class samples that may be responsible for neuron activation. The generated graphs can show, for example, a specific portion of a class sample image that activates a neuron with a single distribution. A class can have samples from multiple distributions, for example, using animals, such as dog class examples in a single class, but samples from various dog breeds may be in the dog sample. If a variety of dog breeds are included in the samples, multiple distributions can be created within the same class. The generated graphs can visually indicate a single distribution or multiple distributions. Provenance data can also generate graphs to illustrate subsequent iterative neuron hierarchies of underfitting, overfitting, or fine-grained classes.

[0056] If the deep learning program 110a, 110b determines at 404 that the output is correct, the trained model continues to be used at 406. The correct output may be illustrated graphically, and changes or modifications to the trained model may not be necessary. The trained model may continue to be operated by the user.

[0057] If the deep learning program 110a, 110b determines that the output is incorrect at 404, the misclassified query sample is analyzed using the provenance data at 408. For example, a class A sample is classified as a class B sample. Some reasons for misclassified query samples may include: corrupted samples or adversarial samples with different data distributions, underfitting problems, overfitting problems, or sampling problems of the training data 202.

[0058] Continuing with the example at step 402, in the first few iterations of the first few data distributions, class A sampling and class B sampling activate nodes n1, n2 to nk. Class A contains samples from multiple species of class A animals. As class A samples and class B samples expand through further iterations, different sets of nodes, nodes m1, m2 to mj, begin to activate for class A. Different sets of node activations may indicate a classification error in a distribution of class A, or may represent a biased training model with one distribution. For example, if it is assumed that the misclassified test sample belongs to distribution 1 of class A, then the trained model uses an inaccurate sampling strategy. Therefore, the training model may be modified to improve the sampling strategy, for example by adding more samples or determining whether a class needs to be divided into multiple classes to avoid overlapping effects of class distributions.

[0059] At 410, the generated insights are provided to the user. The generated insights may be generated to the user operating the computing device. The generated insights may include visual aids, graphs, charts, images, videos, or typed written messages. The generated insight output may generate a varying range of data analyses for the user based on, for example, the analyses at steps 304 and 408. The output data transmitted to the user may provide insights to the user to make decisions related to model training, usage, and accuracy.

[0060] Understandable Figure 2-4 Only an illustration of one embodiment is provided and does not imply any limitations on how different embodiments may be implemented. Many modifications may be made to the depicted embodiments based on design and implementation requirements.

[0061] Figure 5 According to an illustrative embodiment of the present invention, Figure 1 900 is a block diagram of the internal and external components of a computer depicted in FIG. Figure 5 This is merely an illustration of one implementation and does not imply any limitations on the environments in which different embodiments may be implemented. Many modifications may be made to the depicted environment based on design and implementation requirements.

[0062] Data processing systems 902, 904 represent any electronic device capable of executing machine-readable program instructions. Data processing systems 902, 904 may represent a smart phone, a computer system, a PDA, or other electronic device. Examples of computing systems, environments, and / or configurations that may be represented by data processing systems 902, 904 include, but are not limited to: a personal computer system, a server computer system, a thin client, a thick client, a handheld or laptop device, a multiprocessor system, a microprocessor-based system, a network PC, a small computer system, and a distributed cloud computing environment including any of the above systems or devices.

[0063] The user client computer 102 and the network server 112 may include Figure 5 A corresponding set of internal components 902a, b and external components 904a, b are shown. Each of the sets of internal components 902a, b includes one or more processors 906 on one or more buses 912, one or more computer-readable RAMs 908 and one or more computer-readable ROMs 910, as well as one or more operating systems 914 and one or more computer-readable tangible storage devices 916. One or more operating systems 914, software programs 108 and deep learning programs 110a in the client computer 102, and deep learning programs 110b in the network server 112 may be stored on one or more computer-readable tangible storage devices 916 for execution by one or more processors 906 via one or more RAMs 908 (typically including cache memories). Figure 5 In the illustrated embodiment, each computer-readable tangible storage device 916 is a disk storage device of an internal hard drive. Alternatively, each computer-readable tangible storage device 916 is a semiconductor memory device such as ROM910, EPROM, flash memory or any other computer-readable tangible storage device that can store computer programs and digital information.

[0064] Each set of internal components 902a, b also includes a R / W drive or interface 918 to read from and write to one or more portable computer-readable tangible storage devices 920, such as CD-ROMs, DVDs, memory sticks, tapes, disks, optical disks, or semiconductor storage devices. Software programs such as the software program 108 and the deep learning program 110a, 110b may be stored on one or more of the respective portable computer-readable tangible storage devices 920, read via the respective R / W drive or interface 918, and loaded into the respective hard disk drive 916.

[0065] Each set of internal components 902a, b may also include a network adapter (or switch port card) or interface 922, such as a TCP / IP adapter card, a wireless wi-fi interface card, or a 3G or 4G wireless interface card, or other wired or wireless communication link. The software program 108 and the deep learning program 110a in the client computer 102 and the deep learning program 110b in the network server computer 112 may be downloaded from an external computer (e.g., a server) via a network (e.g., the Internet, a local area network or other wide area network) and a corresponding network adapter or interface 922. The software program 108 and the deep learning program 110a in the client computer 102 and the deep learning program 110b in the network server computer 112 are loaded from the network adapter (or switch port adapter) or interface 922 to the corresponding hard drive 916. The network may include: copper wire, optical fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers.

[0066] Each set of external components 904a, b may include a computer display monitor 924, a keyboard 926, and a computer mouse 928. External components 904a, b may also include touch screens, virtual keyboards, touch pads, pointing devices, and other human-computer interface devices. Each set of internal components 902a, b also includes a device driver 930 to interface with the computer display monitor 924, keyboard 926, and computer mouse 928. The device driver 930, R / W driver or interface 918, and network adapter or interface 922 include hardware and software (stored in storage device 916 and / or ROM 910).

[0067] First, it should be understood that although the present disclosure includes a detailed description about cloud computing, the implementation of the technical solutions recorded therein is not limited to a cloud computing environment, but can be implemented in combination with any other type of computing environment now known or later developed.

[0068] Cloud computing is a service delivery model for convenient, on-demand network access to a shared pool of configurable computing resources. Configurable computing resources are resources that can be quickly deployed and released with minimal management cost or interaction with the service provider, such as networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services. This cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0069] Features include:

[0070] On-demand self-service: Cloud consumers can unilaterally and automatically deploy computing capabilities such as server time and network storage on demand without human interaction with the service provider.

[0071] Broad network access: Computing power can be accessed over the network through standard mechanisms that facilitate the use of the cloud through different types of thin-client or thick-client platforms (e.g., mobile phones, laptops, personal digital assistants (PDAs)).

[0072] Resource pool: The computing resources of the provider are grouped into resource pools and serve multiple consumers through a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated on demand. Generally, consumers cannot control or even know the exact location of the provided resources, but can specify the location (such as country, state or data center) at a higher level of abstraction, so it is location-independent.

[0073] Rapid elasticity: The ability to quickly and elastically (sometimes automatically) deploy computing power to scale up quickly, and quickly release it to scale down quickly. To consumers, the computing power available for deployment often appears to be unlimited, and any amount of computing power can be accessed at any time.

[0074] Measurable services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both service providers and consumers.

[0075] The service model is as follows:

[0076] Software as a Service (SaaS): The capability provided to consumers is to use the provider's applications running on a cloud infrastructure. Applications can be accessed from a variety of client devices through a thin client interface such as a web browser (e.g., web-based email). Aside from limited user-specific application configuration settings, consumers neither manage nor control the underlying cloud infrastructure including networks, servers, operating systems, storage, or even individual application capabilities.

[0077] Platform as a Service (PaaS): The capability provided to consumers is to deploy applications created or acquired by consumers on the cloud infrastructure, which are created using programming languages ​​and tools supported by the provider. Consumers neither manage nor control the underlying cloud infrastructure including networks, servers, operating systems or storage, but have control over the applications they deploy and may also have control over the configuration of the application hosting environment.

[0078] Analytics as a Service (AaaS): The functionality provided to the consumer is to access an analytics platform using a web-based or cloud-based network (i.e., infrastructure). The analytics platform may include access to analytics software resources or may include access to related databases, corpora, servers, operating systems, or storage. The consumer does not manage or control the underlying web-based or cloud-based infrastructure, including databases, corpora, servers, operating systems, or storage, but may control the deployed applications and possibly the configuration of the application hosting environment.

[0079] Infrastructure as a Service (IaaS): The capabilities provided to consumers are processing, storage, network, and other basic computing resources on which consumers can deploy and run arbitrary software including operating systems and applications. Consumers neither manage nor control the underlying cloud infrastructure, but have control over the operating system, storage, and applications they deploy, and may have limited control over selected network components (such as host firewalls).

[0080] The deployment model is as follows:

[0081] Private Cloud: Cloud infrastructure is run solely for an organization. The cloud infrastructure can be managed by the organization or a third party and can exist inside or outside the organization.

[0082] Community Cloud: A cloud infrastructure is shared by several organizations and supports a specific community with common interests (e.g., mission, security requirements, policies, and compliance considerations). A community cloud can be managed by multiple organizations within the community or by a third party and can exist inside or outside the community.

[0083] Public cloud: Cloud infrastructure is provided to the public or to large industry groups and is owned by an organization that sells cloud services.

[0084] Hybrid cloud: A cloud infrastructure consisting of two or more clouds (private, community or public) in a deployment model that remain distinct entities but are bound together by standardized or proprietary technologies (such as cloud burst sharing for load balancing between clouds) that enable data and application portability.

[0085] The cloud computing environment is service-oriented, with characteristics centered on statelessness, low coupling, modularity, and semantic interoperability. The core of cloud computing is the infrastructure that includes a network of interconnected nodes.

[0086] Reference now Figure 6, in which an exemplary cloud computing environment 1000 is shown. As shown, the cloud computing environment 1000 includes one or more cloud computing nodes 100 with which a local computing device used by a cloud computing consumer can communicate. The local computing device may be, for example, a personal digital assistant (PDA) or a mobile phone 1000A, a desktop computer 1000B, a laptop computer 1000C and / or an automobile computer system 1000N. The cloud computing nodes 100 can communicate with each other. The cloud computing nodes 100 can be physically or virtually grouped in one or more networks including but not limited to a private cloud, a community cloud, a public cloud or a hybrid cloud, or a combination thereof as described above (not shown in the figure). In this way, cloud consumers can request Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and / or Software as a Service (SaaS) provided by the cloud computing environment 1000 without having to maintain resources on local computing devices. It should be understood that Figure 6 The various types of computing devices 1000A-N shown are merely illustrative, and the cloud computing node 100 and the cloud computing environment 1000 can communicate with any type of computing device on any type of network and / or network addressable connection (eg, using a web browser).

[0087] Reference now Figure 7 , which shows a set of functional abstraction layers 1100 provided by the cloud computing environment 1000. First, it should be understood that Figure 7 The components, layers, and functions shown are only exemplary and the embodiments of the present invention are not limited thereto. Figure 7 As shown, the following layers and corresponding functions are provided:

[0088] The hardware and software layer 1102 includes hardware and software components. Examples of hardware components include: host 1104; server 1106 based on RISC (Reduced Instruction Set Computer) architecture; server 1108; blade server 1110; storage device 1112; network and network components 1114. Examples of software components include: network application server software 1116 and database software 1118.

[0089] Virtualization layer 1120 provides an abstraction layer that may provide examples of the following virtual entities: virtual servers 1122 , virtual storage 1124 , virtual networks 1126 (including virtual private networks), virtual applications and operating systems 1128 , and virtual clients 1130 .

[0090] In one example, the management layer 1132 may provide the following functions: Resource provisioning function 1134: Provide dynamic acquisition of computing resources and other resources for performing tasks in a cloud computing environment; Metering and pricing function 1136: Track the cost of resource usage within the cloud computing environment and provide bills and invoices for this purpose. In one example, the resource may include an application software license. Security function: Provide identity authentication for cloud consumers and tasks, and provide protection for data and other resources. User portal function 1138: Provide access to the cloud computing environment for consumers and system administrators. Service level management function 1140: Provide allocation and management of cloud computing resources to meet the required service levels. Service level agreement (SLA) planning and fulfillment function 1142: Provide pre-arrangement and provision for future demand for cloud computing resources predicted according to the SLA.

[0091] The workload layer 1144 provides examples of functions that the cloud computing environment can be used for. Examples of workloads and functions that can be provided from this layer include: mapping and navigation 1146; software development and lifecycle management 1148; virtual classroom teaching provision 1150; data analysis processing 1152; transaction processing 1154; and deep model learning 1156. The deep learning program 110a, 110b provides a method for using provenance data to gain insights during the deep learning model training phase.

[0092] At any possible technical detail combination level, the present invention may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0093] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0094] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0095] The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and procedural programming languages, such as "C" language, python programming language, or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

[0096] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and / or block diagram and the combination of each box in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0097] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0098] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0099] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0100] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for improving a deep learning model for classification, comprising: Collect provenance data; Generate insights from models based on the provenance data collected; generating a training model based on the generated insights; Reduce the size of the training model; as well as Create the final trained model; Using the final trained model to classify the query sample; Determining misclassifications of query samples in the final trained model output; analyzing the misclassification using the provenance data; as well as providing the generated insights to a user based on the error classification, The provenance data includes learned weights, learned activation values, learned nodes, and learned gradients based on training data that has been iterated once or multiple times by the deep learning model, the training data includes structured data and / or unstructured data, and the insights include improving the accuracy of the deep learning model in generating correct classifications and the causes of incorrect classifications. Wherein, the structured data includes a spreadsheet, a relational database or data stored in fixed fields, and / or the unstructured data includes a portable document format, an image, a presentation, a web page, video content, audio content, an email, a word processing document or multimedia content.

2. The method according to claim 1, wherein: The provenance data includes data collected during the deep learning training phase.

3. The method according to claim 1, wherein: The provenance data includes learned weights, learned activation values, learned nodes, and learned gradients based on training data passing through one or more layers of the deep learning model.

4. The method according to claim 1, wherein: The model insights are generated by analyzing the provenance data, wherein the model insights include: analysis of neuron weights, neuron learning features, neuron history, neuron lineage, related neurons, and unused neurons.

5. The method according to claim 1, wherein: The reduction in model size is determined based on periodicity analysis.

6. A computer system comprising: One or more processors, one or more computer-readable memories, one or more computer-readable tangible storage media, and program instructions stored on at least one of the one or more computer-readable tangible storage media, to be executed by at least one of the one or more processors via at least one of the one or more computer-readable memories, wherein the computer system is capable of performing the steps in the method according to any one of claims 1 to 5.

7. A computer system comprising means for performing the steps of the method according to any one of claims 1 to 5.

8. A computer program product, comprising a computer-readable storage medium, wherein the computer-readable storage medium embodies program instructions, wherein the program instructions can be read by a processing circuit to enable the processing circuit to execute the steps of the method according to any one of claims 1 to 5.

Images