Using Prototypes to Manipulate Deep Sequence Models

Abstract

A deep sequence model may be manipulated using prototypes. A prototype overview is displayed, the prototype overview including a plurality of prototype sequences learned by the model through back propagation, each of the prototype sequences including a series of events, wherein for each of the prototype sequences, statistical information is presented regarding the use of the prototype sequence by the model. An input is received that adjusts one or more of the prototype sequences to fine-tune the model. The model is updated using the plurality of prototype sequences as adjusted to create an updated model. The updated model is displayed in the prototype overview.

Description

Technical Field

[0001] The present disclosure relates to utilizing prototypes to manipulate deep sequence models. Background Art

[0002] Deep learning models are used in sequence data analysis to help make decisions. Deep sequence models such as recurrent neural networks (RNNs) can be used to predict patient status by modeling electronic health records (EHRs), analyzing the topic or sentiment of text, and understanding audio signals. Such techniques have achieved state-of-the-art results in a variety of applications.

[0003] Despite their impressive performance, RNNs are often viewed as “black boxes”. This is due to their complex architecture and the large size of the model weights. This lack of interpretability limits the adoption of RNNs in many critical decision-making scenarios, where an understanding of the reasons behind a prediction may be required. As an example, the EU’s recent data protection regulations grant individuals a “right to explanation” for decisions made by machine learning systems (see, e.g., Voigt, Paul, and Axel Von dem Bussche, “The EU General Data Protection Regulation (GDPR)”, A Practical Guide, 1st Ed., Cham: Springer International Publishing (2017)).

[0004] In addition to interpretability, another challenge in deploying deep sequence models is manipulability. Many applications can benefit from enabling domain experts to leverage their insights and domain knowledge to directly manipulate the model. For example, a physician with extensive experience with heart disease would gain valuable knowledge that could benefit a heart failure risk prediction model. Although end-to-end training of deep neural networks can alleviate the need to manually manage data features, training alone cannot provide expert users with the functionality to manipulate the model. Summary of the invention

[0005] In one or more illustrative examples, a method for manipulating a deep sequence model using prototypes is described. The method includes: displaying a visual interface, the visual interface including a prototype overview, the prototype overview showing a plurality of prototype sequences learned by the model through back-propagation, each of the prototype sequences including a series of events, wherein for each of the prototype sequences, statistical information is presented about the use of the prototype sequence by the model. The method also includes: receiving input to the visual interface, the input adjusting one or more of the prototype sequences to fine-tune the model; and updating the model using the plurality of prototype sequences as adjusted to create an updated model. The method further includes: displaying in the visual interface updates to the plurality of prototype sequences as identified by the updated model.

[0006] In one or more illustrative examples, a system for manipulating a deep sequence model using prototypes is described. The system includes: a display device; a storage device configured to maintain a prototype sequence and a model; and a processor that communicates with the display device and the storage device. The processor is programmed to display a visual interface that includes a prototype overview showing a plurality of prototype sequences learned by the model through backpropagation, each of the prototype sequences including a series of events, wherein for each of the prototype sequences, statistical information is presented about the use of the prototype sequence by the model. The processor is also programmed to: receive input to the visual interface that adjusts one or more of the prototype sequences to fine-tune the model; update the model using the plurality of prototype sequences as adjusted to create an updated model; and display updates to the plurality of prototype sequences as identified by the updated model in the visual interface. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 illustrates an example architecture of a system for providing a visual interface for manipulating a deep sequence model using a prototype;

[0008] Figure 2A , Figure 2B and Figure 2C collectively illustrate an example of a visual interface for the system;

[0009] Figure 3 An example comparison mode illustrating a prototype overview of the visual interface of the system;

[0010] Figure 4 illustrates example details of an editing diagram of the visual interface of the system;

[0011] Figure 5A and Figure 5B collectively illustrate example details of a sequence query / edit widget of the visual interface of the system;

[0012] Figure 6 illustrates a detailed view of a model for use in the system; and

[0013] Figure 7 An example process for providing a visual interface for manipulating a deep sequence model using a prototype is illustrated. DETAILED DESCRIPTION

[0014] Embodiments of the present disclosure are described herein. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take various forms and alternative forms. The figures are not necessarily to scale; some features may be magnified or minimized to show the details of a particular component. Therefore, the specific structural and functional details disclosed herein will not be interpreted as restrictive, but merely as a representative basis for teaching those skilled in the art to adopt the present embodiment in various ways. As will be understood by those of ordinary skill in the art, the various features illustrated and described with reference to any one of the accompanying drawings may be combined with the features illustrated in one or more other accompanying drawings to produce an embodiment that is not explicitly illustrated or described. The combination of the illustrated features provides a representative embodiment of a typical application. However, for a specific application or implementation, various combinations and modifications of features consistent with the teachings of the present disclosure may be desired.

[0015] Both interpretable and manipulable machine learning methods allow domain experts to gain insights from the model and also use their knowledge to improve the model, thereby creating a mutually beneficial feedback loop. As explained in detail herein, the described method allows users to manipulate sequence learning models without relying on machine learning practitioners as intermediaries. The disclosed concept utilizes a sequence learning model that combines prototype learning and deep neural networks for both interpretability and prediction accuracy. The key component of the model is a small collection of example cases constructed using historical data (referred to as prototypes in this article). Interactive visual interfaces can be designed to help experts in various application fields to examine, criticize and revise sequence prototypes learned by the model. By making these revisions to the model, the functionality of the computer system utilizing the model can be improved.

[0016] The improved visual interface can be configured to support iterative model refinement for end users with limited background in machine learning. The visual interface provides a set of visualizations and interactions to help end users to inspect, critique, and revise prototypes in the model. In particular, the visual interface utilizes a backbone deep sequence model that is inherently interpretable. (Further information about such models is in the publication "Interpretable and Steerable Sequence Learning via Prototypes" by Yao Ming, Panpan Xu, Huamin Qu, and Liu Ren.) Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining), which is incorporated herein by reference in its entirety. ) The model can employ prototype learning, which is a form of case-based reasoning. In doing so, the model exploits a problem-solving process that involves consulting similar cases in the past to develop solutions for new situations. In particular, the model constructs a small set of prototype sequences from the original training data. New inputs can be compared to the prototypes in the latent space of the encoder network for predicting the output. The learned prototypes thus give a concise summary of the sequence because their number is typically much smaller than the size of the original training data. Further aspects of the model and its use are discussed in detail herein.

[0017] Figure 1 An example architecture of a system 100 for providing a visual interface 110 for manipulating a deep sequence model 104 using a prototype is illustrated. As shown, the system 100 includes a storage layer 102, a service layer 120, and a visual interface 110. It should be noted that the illustrated architecture is only one example, and different arrangements of elements or combinations of elements may be used.

[0018] The storage layer 102 may be configured to maintain historical snapshots of the model 104. The storage layer 102 may also maintain a data store 106 containing training / validation and test data to support incremental model 104 training. The storage layer 102 may also maintain a sequence index 108 that may be queried and visualized.

[0019] The visual interface 110 includes a rich set of visualizations to support inspection of the model 104 and a range of user interactions to support efficient user feedback on the model 104. As discussed in detail below, the visual interface 110 may include an archetype overview 112, a sequence detail view 114, a sequence encoder view 116, and an archetype editor 118. The archetype editor 118 may allow a user to create, update, and delete archetypes.

[0020] The service layer 120 may include a model manager 122, an analysis component 124, and a sequence query component 126. The model manager 122 may be configured to communicate between the storage layer 102 and the visual interface 110. The model manager 122 may be configured to fine-tune the model 104 according to user input received via the visual interface 110. The model manager 122 may further support undo or redo functionality.

[0021] The analysis component 124 can also be configured to communicate between the storage layer 102 and the visual interface 110. The analysis component 124 can be configured to collect statistical summaries, which can be displayed on the visual interface 110 to help users better understand the behavior of the model 104. The analysis component 124 can also perform comparisons between different snapshots of the model 104 to help track changes.

[0022] The sequence query component 126 may also be configured to communicate between the storage layer 102 and the visual interface 110. Through the sequence query component 126, a user may search the sequence index 108 in the data storage device 106 to create a new prototype.

[0023] A computing platform may be used to implement the algorithms and / or methods of one or more embodiments described herein. The computing platform may include a memory, a processor, and a non-volatile storage device. The processor may include one or more devices selected from the following: a high-performance computing (HPC) system including a high-performance core, a microprocessor, a microcontroller, a digital signal processor, a microcomputer, a central processing unit, a field programmable gate array, a programmable logic device, a state machine, a logic circuit, an analog circuit, a digital circuit, or any other device that manipulates a signal (analog or digital) based on a computer executable instruction residing in a memory. The memory may include a single memory device or multiple memory devices, including but not limited to: a random access memory (RAM), a volatile memory, a non-volatile memory, a static random access memory (SRAM), a dynamic random access memory (DRAM), a flash memory, a cache memory, or any other device capable of storing information. The non-volatile storage device may include one or more persistent data storage devices, such as a hard drive, an optical drive, a tape drive, a non-volatile solid-state device, a cloud storage device, or any other device capable of persistently storing information.

[0024] The processor may be configured to read into the memory and execute computer executable instructions residing in the software of the non-volatile storage device and embodying the algorithms and / or methods of one or more embodiments as discussed herein. The software may include an operating system and an application. The software may be compiled or interpreted from a computer program created using a variety of programming languages ​​and / or techniques, which may include Java, C, C++, C#, Objective C, Fortran, Pascal, Java Script, Python, Perl, and PL / SQL, without limitation and in combination, alone or in combination. In one embodiment, PyTorch, which is a package for the Python programming language, may be used to implement at least a portion of the code of one or more embodiments.

[0025] The program code embodying the algorithms and / or methods described herein can also be distributed individually or collectively as program products in a variety of different forms. Program code can be distributed using a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to implement aspects of one or more embodiments. Computer-readable storage media that are inherently non-transitory may include volatile and non-volatile, and removable and non-removable tangible media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media may further include: RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state memory technology, portable compact disk read-only memory (CD-ROM) or other optical storage devices, cassettes, tapes, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and can be read by a computer. Computer-readable program instructions can be downloaded from a computer-readable storage medium to a computer, another type of programmable data processing device, or another device, or downloaded to an external computer or external storage device via a network.

[0026] Figure 2A , Figure 2B and Figure 2C An example 200 of a visual interface 110 of the system 100 is collectively illustrated. As shown, the visual interface 110 includes a prototype overview 112, a sequence detail view 114, an edit diagram, and a sequence encoder view. The visual interface 110 may be provided to a display device, which may include an electronic display screen, a touch screen, a projector, or any other suitable device for reproducing a graphical display. It should be noted that the illustrated layout is merely an example, and a different layout of elements or combinations of elements may be used.

[0027] As shown at (A), the prototype overview 112 can be a main component of the visual interface 110. The prototype overview 112 can be configured to display the prototypes in the model 104, accompanied by rich information to help users understand the model 104 behavior. As shown, each prototype can be visualized as a horizontally arranged sequence of rectangular events, whose event names are displayed as text labels. Additionally, in the illustrated example, the border color of each event encodes the type of event. It is worth noting that different color coding schemes can be used for different application scenarios.

[0028] The prototype overview 112 can be further configured to allow visualization of prototype weights. In an example, the prototype overview 112 can include a weight column, where for each prototype, the column displays a value indicating the relative importance of the corresponding prototype in determining different possible model 104 outputs. In a classification task, the column shows the strength of the association between the prototype and different class labels. The displayed value can be a fully connected layer of the model 104. f i The weight matrix W More specifically, W ij It's a prototype p i In determining the j The importance of the model 104 output. The class colors distinguish the labels in the classification task. For example, negative weights are illustrated in a first color or appearance, while positive weights are illustrated in a second color or appearance.

[0029] The prototype overview 112 can also be configured to summarize the prediction performance. For example, for each prototype, the system 100 can calculate all the data instances closest to it, where those instances can be called the neighborhood of the prototype. A summary of the model 104 decision about those nearest neighbors can be illustrated in the prototype overview 112 in the form of stacked horizontal bars displayed in the prediction column. The width of the bar can represent the total number of neighborhood instances, and the category color can indicate the decision made by the model 104 in the classification task (similar to that shown in the weight column). The height of the rectangle can encode the score. In the illustrated example, the solid box is used to indicate the correct classification result, and the patterned box is used to indicate the error (although different methods can be used in other examples). The width of each colored box is shown to indicate the number of instances correctly or incorrectly classified to the corresponding label. This visual encoding helps users quickly identify important prototypes representing a large number of instances, as well as prototypes that are usually associated with inaccurate prediction results. The prototype overview 112 can also illustrate the incremental changes made in response to the user's editing of the prototype and any corresponding updates to the model 104.

[0030] The prototype overview 112 can also be configured to allow the user to reorder the prototype list. The prototype list can be sorted based on several different criteria to solve a variety of analysis tasks. By default, the system 100 sorts the prototypes based on their similarity. To do this, the system 100 can perform hierarchical clustering on the prototypes based on their distance in the latent space, and a linear order can be obtained as a result. Therefore, the sorting can be based on similar groups of prototypes that are similar to each other to help find redundancy. The prototype overview 112 can also be configured to allow the user to sort the prototypes according to the accuracy of the predicted results. This can allow problematic prototypes to be noticed for further analysis.

[0031] The prototype overview 112 can also be configured to allow filtering of prototypes. The number of visual prototypes can be reduced by filtering in various ways. In one example, the user can specify a number of events, and the list will only show prototypes that contain one of these events. In another example, the user selects a prototype, and the list will show the most similar prototypes in the latent space. k If you are interested in a specific prototype, filtering by event can help narrow the search. Filtering by similarity can help identify potential duplicate prototypes. In one example implementation, k Can be set to 10.

[0032] The prototype overview 112 can also be configured to allow visualization of event importance. In an example, the event importance can be displayed as a horizontal bar at the bottom of the rectangle ( Figure 3 ). The length of the bar can be proportional to the importance score to help highlight the most critical events. p Each event in e The importance of can be calculated using the leave-one-out strategy. More specifically, the algorithm can calculate the importance of e Do the following:

[0033] 1. e From the prototype p Remove from the list to get a new sequence p' .

[0034] 2. Computation by Sequence Encoder r The original prototype in the latent space defined p and p' The distance between.

[0035] 3. Convert the distance to a normalized value between 0 and 1, where 0 indicates removal e right p The implicit representation has no effect 。 The normalized value is e The importance of.

[0036] Intuition is an event e The more important it is, the more time to remove e , the prototype will be pushed farther away from its original position in the latent space. Figure 2A and Figure 2B As shown in , the long bars under the words "great" and "horrible" indicate that those words are highly important in prototype #43. The prototype overview 112 can also show how well each event in the prototype aligns with those events in the neighborhood sequence.

[0037] Figure 3 An example comparison mode 300 of the prototype overview of the visual interface of the system is illustrated. The comparison mode 300 can be used to allow a user to visualize differences in the models 104. For example, via use of the comparison mode 300, a user can understand and analyze changes after updating a prototype in the prototype overview.

[0038] Similar to some popular code or document version comparison tools, the system 100 can highlight the prototypes in different colors to indicate different types of edits made in the prototype overview 112. Figure 3 As shown in , added, deleted, and edited prototypes are marked with a green circle containing a green plus sign, a red circle containing a red x, and a blue circle containing a blue exclamation mark, respectively. These visual cues can help users keep track of these changes.

[0039] After the edits are submitted, the system can fine-tune the model 104 using the updated prototype. Therefore, it is useful to visualize these changes in the model 104 after the model 104 has been fine-tuned. Directly comparing the model 104 parameters may not be helpful because they may be extremely difficult to interpret. However, because the system 100 utilizes a deep sequence model that is inherently interpretable, the decisions of the model 104 are made based on the proximity of the input to the prototype. Therefore, the system 100 can visualize the neighborhood of the changes of each prototype to summarize the differences in the model 104 before and after the update.

[0040] To activate the comparison mode 300, the user interface 110 may include a comparison control on the historical interaction record in the edit map, such as Figure 2C In response to selection of the comparison control, an additional column may then be added to the right side of the prototype overview 112 to display the number of instances flowing into and out of the neighborhood of each prototype. Figure 3 This can be illustrated as, for example, two horizontally stacked bar graphs, as shown in . The bars on the right side indicate data instances flowing out of the neighborhood of the prototype, while the bars on the left side indicate data instances flowing into the neighborhood of the prototype. The length of the bar can be proportional to the corresponding number of instances. The colored blocks can indicate the classification results given by the updated model 104. When the user hovers the mouse over the prototype or the bar, the visualization can show curved edges that connect the source and end points of the incoming and outgoing flows. For example, in Figure 3 , these edges are displayed for the top prototype that is hovered over.

[0041] As mentioned above, the user interface 110 may also include an edit map. Figure 2CAs shown at (C) of FIG. 1 , the edit graph may be configured to display a history of edits performed on model 104 using visual interface 110. Thus, the edit graph may help users keep track of their edits to model 104 and undo those edits if necessary.

[0042] Figure 4 Detail of an example 400 of an editing diagram of the visual interface 110 of the system 100 is illustrated. Figure 4 As shown in and continue to refer to Figure 1 and Figure 2A , Figure 2B and Figure 2C , the system 100 can visualize the history of model 104 edits as a directed acyclic graph. Each node in the graph can represent a snapshot of the model 104. Nodes can be revisited and edited again to create alternative branches. The directed links in the graph can indicate editing relationships. Additionally, each node can show a summary of the edits made, such as indicating how many prototypes have been added, deleted, or edited in each snapshot. Thus, the edit graph provides traceability in the system 100.

[0043] Return to reference Figure 2C , as shown at (D), the sequence encoder view can be configured to display hidden state information in the encoder layer for advanced users to analyze sequence alignments and compare different prototypes. Thus, this allows the system 100 to r This further allows experienced users to inspect the interior of the model 104 and edit the prototype based on the insights gained from the hidden state visualization.

[0044] The design of the sequence encoder view is centered around a visual representation of the prototypes similar to the other views. More specifically, the sequence encoder view visualizes the hidden state changes of a prototype as it passes through the sequence encoder. Since the hidden state is a high-dimensional vector, a dimensionality reduction technique is used to project the hidden state onto a 2D plane. The projected hidden states of the same prototype are concatenated to form tracks. These tracks help the user identify significant changes in the hidden state vector, which often indicate the occurrence of key events or substructures in the sequence (see, e.g., Andrej Karpathy, Justin Johnson, and Fei-Fei Li, “Visualizing and understanding recurrent networks”, arXiv preprint arXiv:1506.02078 (2015)). For example, in Figure 2BIn the example sequence encoder view in , the word “horrible” pushes the hidden state significantly to the left, indicating that this word is a key word in the sentence.

[0045] To visualize the distribution of prototype neighbors, the system 100 can also project their hidden state information on the same 2D plane. However, simultaneously projecting such a large amount of data (thousands of sequences x tens of events of hidden states) may consume significant computational resources (especially for algorithms like t-SNE). Therefore, the system 100 can employ a parametric nonlinear projection technique. An example of a parametric nonlinear projection technique is kernel t-SNE (Min, Martin Renqiang, Hongyu Guo, and Dinghan Shen, “Parametric t-distributed stochastic exemplar-centered embedding”, Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, Cham , 2018). The method utilizes a subset of the data called skeleton points (e.g., the hidden states of the prototype sequence in an implementation of system 100) to train a parametric projection function. The remaining points can then be projected as needed without further optimization using the learned function, which can be done in linear time.

[0046] Therefore, the design of the projected visualization combines the trajectory of a prototype with a contour plot that summarizes the hidden state distribution of its neighborhood. This visualization helps identify redundant prototypes whose neighborhood contour plots may have significant overlap with each other. It can be observed that some projected points are too close on the trajectory and the labels may overlap with each other. Therefore, the system 100 can group these points based on their proximity and can display the labels of these points together near the cluster.

[0047] Reference Figure 2A , Figure 2B and Figure 2C As shown at (B), the sequence detail view 114 can be configured to display neighborhood instances and prediction results of the selected prototype in the prototype overview 112. A sequence query / edit widget can be displayed as needed to help the user query the data storage device 106 using a combination of fuzzy and exact phrase matching, as well as create and edit potential prototypes.

[0048] Figure 5A and Figure 5BDetailed examples 500 of a sequence query / edit widget of the visual interface 110 of the system 100 are collectively illustrated. As noted, the system 100 supports a rich set of user interactions to aid in the refinement of the model 104 by adding, deleting, and revising prototypes. The sequence query / edit widget can be used to perform further useful interactions with the model 104.

[0049] In an additional aspect, the visual interface 110 can support direct drag-and-drop interactions. Many operations in the system 100 can be accomplished by direct drag-and-drop. For example, a user can drag prototypes to a recycling bin to remove them, drag prototypes to a sequence detail view 114 to inspect neighborhood instances, and drag prototypes to a projection view (sequence encoder view 116) to analyze hidden state distributions. The interactive design utilizes a physical metaphor, and users can easily perform a variety of tasks through direct manipulation.

[0050] As in Figure 5A and Figure 5B As shown at C2 in , the system 100 may provide a sequence editor. The sequence editor may show a representation of a particular sequence that is selected to be edited by the user (e.g., by selecting a pencil / edit control for the corresponding sequence). Using the sequence editor, the user may add and remove events directly to an existing prototype without having to retype the events in a new form.

[0051] As in Figure 5A and Figure 5B As shown at C1 in FIG. 1 , the system 100 may further provide a search interface to help users create new prototypes using a large amount of training data. Given a search phrase from a user, the search interface may display a ranked list of sequences. In particular, the user may specify subsequences that the user wants to see in the new prototype. s The system 100 can then use the subsequence s to match the sequence in the data storage device 106 .

[0052] System 100 supports two matching schemes: exact matching and soft matching, which can be used together or separately. The sequence in the data storage device 106 s' Rating, where Computes the longest common subsequence between the two inputs. Soft matching does not require the words to be exactly the same. Instead, when performing soft matching, the system 100 uses a sequence encoder r To compute the implicit representation of the most frequent n-gram in the training sequence. During the search, the system 100 queries sThe latent representation of is compared to the n-grams, and the sequence containing the matching n-grams is returned. Thus, comparing in latent space allows the user to find semantically related sequences.

[0053] Figure 6 1 shows a detailed view of a model 104 for use in system 100. Model 104 combines the ability of deep sequence networks to model complex temporal dependencies with the interpretability of prototype-based inference. As shown, model 104 includes three main components: sequence encoder r , Prototype layer p , and a fully connected layer with a softmax output for classification tasks f .

[0054] Sequence Encoder r Can be a deep recurrent neural network. Sequence encoder r Convert variable-length sequences into fixed-length vector representations . Prototype layer p The latent vector obtained by the encoder network is represented e and k Prototype vector By appropriate transformation, the vector of similarity score can be obtained as ,in a i is the input sequence and the prototype p i The similarity score between a i = 1 indicates that the input sequence is consistent with the prototype p i Have the same embedding. Fully connected layer with softmax output f Using the similarity score vector a To calculate the final classification result. For better interpretability, we can f The entries in the weight matrix in are constrained to be non-negative. Similarity score a Together with the entries in the weight matrix, they can be used to interpret the classification results.

[0055] Prototype vectors learned via back-propagation p i are not easily interpretable, since they may not correspond to sequences in real-world applications. Therefore, the optimization algorithm can perform prototype projection every few training epochs, which in effect identifies the sequences from the original training data that are most similar to the prototype vectors in the latent space. The prototype vectors can then be updated with the latent vector representations of these sequences.

[0056] Figure 7An example process 700 for providing a visual interface 110 for manipulating a deep sequence model 104 using a prototype is illustrated. In an example, the process 700 may be performed by Figure 1 The system 100 shown in FIG.

[0057] At operation 702, the system 100 displays the visual interface 110. The visual interface 110 may include a prototype overview 112 showing a plurality of prototype sequences learned by the model 104 through back propagation, each of the prototype sequences including a series of events, wherein for each of the prototype sequences, statistical information is presented about the use of the prototype sequence by the model 104.

[0058] At operation 704, the system 100 receives input to the visual interface 110. The input includes adjustments to one or more of the prototype sequences to fine-tune the model 104. As some examples, the input may include adding or removing events from one or more of the prototype sequences, deleting redundant prototype sequences, and / or adding new prototype sequences.

[0059] At operation 706, the system 100 updates the model 104 using the plurality of prototype sequences as adjusted to create an updated model 104. At operation 708, the system 100 displays the updates to the plurality of prototype sequences as identified by the updated model 104 in the visual interface 110. After operation 708, the process 700 ends.

[0060] Compared to existing systems and methods, the described method adopts a human-in-the-loop approach to build sequence prediction models. In addition, a collection of visualizations is used to help users better understand and inspect sequence prototypes. In addition, novel interaction techniques have been designed to help users add, remove and edit sequence prototypes. The interaction is supported by an intelligent query service. Therefore, the visual interface 110 allows such intelligent interactions to be introduced into such a system.

[0061] Thus, the described system 100 provides a novel human-model interaction scheme that helps refine the learned knowledge of both the user and the model 104. Thus, the novel visual analysis system 100 allows for human-guided refinement of deep sequence networks by interactively editing a set of prototype sequences used in reasoning. Furthermore, a visualization design is proposed that combines projection techniques and contour plots to facilitate high-level analysis of sequence distributions in a latent space.

[0062] It is worth noting that the described techniques can be applied to any type of sequence data. As described above, the described interactive sequence model can be used to classify sentiment in sequence data in the form of textual comments. As another practical application, the described techniques can be applied to traces of web traffic from users. Such data can be used to classify whether a user will perform a purchase based on the user's browsing history sequence.

[0063] As yet another practical application, the described techniques can be applied to vehicle bus traffic to facilitate interactive analysis of models for enumerating future events in vehicles. For example, signals such as CAN bus traffic can be collected from vehicles into sequence data. Based on the events captured in the data, models can be used to predict whether a failure is likely to occur for the vehicle. These models can also be easily checked and refined using the disclosed human-model interaction scheme.

[0064] Computer-readable program instructions stored in a computer-readable medium can be used to instruct a computer, other types of programmable data processing devices, or other devices to operate in a particular manner so that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions, actions, and / or operations specified in the flowchart or diagram. In certain alternative embodiments, consistent with one or more embodiments, the functions, actions, and / or operations specified in the flowchart and diagram can be reordered, processed serially, and / or processed simultaneously. In addition, any flowchart and / or diagram may include more or fewer nodes or boxes than those illustrated in accordance with one or more embodiments.

[0065] Although exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms included in the claims. The words used in the specification are descriptive words, not restrictive words, and it should be understood that various changes can be made without departing from the spirit and scope of the present disclosure. As previously described, the features of the various embodiments can be combined to form other embodiments of the present invention that may not be explicitly described or illustrated. Although various embodiments may have been described as providing advantages over other embodiments or prior art implementations or preferred over other embodiments or prior art implementations relative to one or more desired characteristics, it should be recognized by those of ordinary skill in the art that one or more features or characteristics can be compromised to achieve the desired overall system properties, depending on the specific application and implementation. These properties may include, but are not limited to, cost, strength, durability, life cycle cost, marketability, appearance, packaging, size, maintainability, weight, manufacturability, ease of assembly, etc. As such, to the extent that any embodiment is described as being less desirable than other embodiments or prior art implementations with respect to one or more characteristics, these embodiments do not exceed the scope of the present disclosure and may be desirable for a particular application.

Claims

1. A method for manipulating a deep sequence model using a prototype, comprising: Displaying a visual interface, the visual interface comprising a prototype overview, the prototype overview showing a plurality of prototype sequences learned by the model through back-propagation, each of the prototype sequences comprising a series of events, wherein for each of the prototype sequences, statistical information is presented about the use of the prototype sequence by the model; calculating, for each event in a sequence of events, a corresponding event importance score, the corresponding event importance score being calculated using a leave-one-out strategy, wherein the corresponding event is removed from a prototype sequence, a distance in the latent space of the model between the prototype sequence and the prototype sequence from which the corresponding event is removed is calculated, and the distance is normalized into an event importance score indicating the effect of removing the event from the prototype sequence; For each of the events, displaying a corresponding event importance score in the prototype overview; receiving input to a visual interface, the input adjusting one or more of the prototype sequences to fine-tune the model; updating the model using the plurality of prototype sequences as adjusted to create an updated model; as well as Updates to the plurality of prototype sequences as identified by the updated model are displayed in a visual interface.

2. The method of claim 1, wherein the statistical information includes a weight column that shows the relative importance of each prototype in determining different possible outputs of the model.

3. The method according to claim 1, further comprising: The editing history of the model is displayed in a visual interface as a directed acyclic graph, each node in the directed acyclic graph represents a snapshot of the model, each link between nodes in the directed acyclic graph indicates an edit performed between the snapshots of the model, and each node includes summary information about the edits performed on multiple prototype sequences between the snapshots of the model.

4. The method according to claim 3, further comprising: receiving a selection of a first model and a second model from the directed acyclic graph; as well as Changes between the first model and the second model are visualized in a visual interface as changes in the neighborhood of the updated prototype sequence.

5. The method according to claim 1, further comprising: A sequence detail view is displayed in a visual interface, the sequence detail view including a neighborhood of an instance of a prototype sequence selected from the prototype overview, the neighborhood including prototype sequences from a plurality of prototype sequences that are closest in distance to the prototype sequence in a latent space of the model.

6. The method according to claim 1, further comprising: A sequence encoder view is displayed in a visual interface, wherein the sequence encoder view visualizes hidden state information of an encoder layer of the model, wherein the display includes: projecting a hidden state vector of the model onto a two-dimensional (2D) plane using dimensionality reduction, projecting the hidden state of each event of a prototype sequence of the multiple prototype sequences onto the 2D plane, and connecting the events of the prototype sequences to form a trajectory.

7. The method according to claim 1, further comprising: receiving, from a visual interface, a selection of one of the plurality of prototype sequences; displaying a sequence editor to a visual interface, the sequence editor configured to receive user input to add and remove events from one of the plurality of prototype sequences; as well as In response to the visual interface receiving user input, one or more of the events are added or removed from one of the plurality of prototype sequences.

8. The method according to claim 1, further comprising: receiving a specification of a subsequence to be searched for in a sequence database; searching a sequence database to identify sequences having the longest common subsequence that includes the subsequence; as well as A ranked list of results is displayed in response to the search.

9. The method according to claim 1, further comprising: receiving, via a visual interface, a specification of a subsequence to be searched in a sequence database; Compute the latent representation of the most frequently used n-grams in a sequence database; searching a database of sequences by computing an implicit representation of the subsequence with an implicit representation of a most frequently used n-gram; and A ranked list of results is displayed to a visual interface in response to the search, the ranked list containing matching n-grams.

10. A system for manipulating a deep sequence model using a prototype, comprising: Display devices; a storage device configured to maintain prototype sequences and models; as well as A processor in communication with the display device and the storage device, and programmed to: Displaying a visual interface, the visual interface comprising a prototype overview, the prototype overview showing a plurality of prototype sequences learned by the model through back-propagation, each of the prototype sequences comprising a series of events, wherein for each of the prototype sequences, statistical information is presented about the use of the prototype sequence by the model; calculating, for each event in a sequence of events, a corresponding event importance score, the corresponding event importance score being calculated using a leave-one-out strategy, wherein the corresponding event is removed from a prototype sequence, a distance in the latent space of the model between the prototype sequence and the prototype sequence from which the corresponding event is removed is calculated, and the distance is normalized into an event importance score indicating the effect of removing the event from the prototype sequence; For each of the events, displaying a corresponding event importance score in the prototype overview; receiving input to a visual interface, the input adjusting one or more of the prototype sequences to fine-tune the model; updating the model using the plurality of prototype sequences as adjusted to create an updated model; as well as Updates to the plurality of prototype sequences as identified by the updated model are displayed in a visual interface.

11. The system of claim 10, wherein the statistical information includes a weight column that displays the relative importance of each prototype in determining different possible outputs of the model.

12. The system of claim 10, wherein the processor is further programmed to: The editing history of the model is displayed in a visual interface as a directed acyclic graph, each node in the directed acyclic graph represents a snapshot of the model, each link between nodes in the directed acyclic graph indicates an edit performed between the snapshots of the model, and each node includes summary information about the edits performed on multiple prototype sequences between the snapshots of the model.

13. The system of claim 12, wherein the processor is further programmed to: receiving a selection of a first model and a second model from the directed acyclic graph; and Changes between the first model and the second model are visualized in a visual interface as changes in the neighborhood of the updated prototype sequence.

14. The system of claim 10, wherein the processor is further programmed to: A sequence detail view is displayed in a visual interface, the sequence detail view including a neighborhood of an instance of a prototype sequence selected from the prototype overview, the neighborhood including prototype sequences from a plurality of prototype sequences that are closest in distance to the prototype sequence in a latent space of the model.

15. The system of claim 10, wherein the processor is further programmed to: Displaying a sequence encoder view in a visual interface, wherein the sequence encoder view visualizes hidden state information of an encoder layer of the model, the display comprising: A hidden state vector of the model is projected onto a two-dimensional (2D) plane using dimensionality reduction, a hidden state of each event of a prototype sequence in the plurality of prototype sequences is projected onto the 2D plane, and the events of the prototype sequences are connected to form a trajectory.

16. The system of claim 10, wherein the processor is further programmed to: receiving, from a visual interface, a selection of one of the plurality of prototype sequences; displaying a sequence editor to a visual interface, the sequence editor configured to receive user input to add and remove events from one of the plurality of prototype sequences; and In response to the visual interface receiving user input, one or more of the events are added or removed from one of the plurality of prototype sequences.

17. The system of claim 10, wherein the processor is further programmed to: receiving a specification of a subsequence to be searched for in a sequence database; searching a sequence database to identify sequences having the longest common subsequence that includes the subsequence; and A ranked list of results is displayed in response to the search.

18. The system of claim 10, wherein the processor is further programmed to: receiving, via a visual interface, a specification of a subsequence to be searched in a sequence database; Compute the latent representation of the most frequently used n-grams in a sequence database; searching a database of sequences by computing an implicit representation of the subsequence with an implicit representation of a most frequently used n-gram; and A ranked list of results is displayed to a visual interface in response to the search, the ranked list containing matching n-grams.

Images