Visual analytics system for interpreting open-domain question answering models

Abstract

The present application belongs to the technical field of open domain question answering model analysis, and particularly relates to a visual analysis system for explaining an open domain question answering model.The system comprises an explanation engine module, a process analysis module and a view module; the explanation engine module uses an attribution method to attribute the final output and implicit output of each module of the OpenQA model at global and local levels; the process analysis module visualizes the model information, data and explainability data generated via the explanation engine in the VEQA as various views of a user analysis interface, and the user performs multi-level exploration in the order of data set, subset, single instance and single paragraph according to a linear workflow; the view module comprises a summary view, a context view, an instance view and a tree view, and is used for visual analysis; the system can help understand the decision reasons of the OpenQA model and provide insights for model improvement; and the system also supports fine-grained exploration of the decision process within a single module.

Description

Technical Field

[0001] This invention belongs to the field of open-domain question-answering model analysis technology, specifically relating to a visual analysis system for interpreting open-domain question-answering models. Background Technology

[0002] Question answering (QA) is a field within information retrieval (IR) and natural language processing (NLP) that focuses on building models to automatically answer questions posed by humans in natural language format. Open-domain question answering (OpenQA) allows machines to provide accurate answers to user questions without given context and is considered the ultimate goal of QA research. With this technology, modern search engines, such as Google and Bing, can not only return relevant snippets or lists of hyperlinks as questions based on user queries, but also generate appropriate answers to these questions to harmonize search results. These search engines utilize queries as input to OpenQA models and take the model's output as direct answers, thereby improving user experience and efficiency. For example, asking a search engine "Who was the first person to walk on the moon?" will yield the answer "Neil Armstrong," along with links to further reading.

[0003] Modern OpenQA methods consist of two parts: a retrieval unit and a reader. Given a question, the retrieval unit searches a large unstructured corpus for relevant paragraphs as the top k paragraphs, from which the reader generates the answer. With the development of deep learning techniques, the reader is based on neural machine reading comprehension models, such as BERT, to infer the answer. Meanwhile, the retrieval unit can be considered an IR system, which can be implemented by Transformer-based modules and retrieve paragraphs. Considering the diversity of OpenQA models, this invention discusses a model that uses BERT as the basic architecture for both the retrieval unit and the reader.

[0004] Despite the rapid progress in OpenQA, existing architectures can still be improved. For example, the retrieval efficiency of models—that is, their ability to separate relevant paragraphs from irrelevant ones for a given question—remains limited. Sometimes the model fails to retrieve relevant paragraphs, and sometimes it detects noisy paragraphs containing exact terms from the question but irrelevant to the answer. Some research focuses on enabling modern neural retrieval machines to have greater retrieval capabilities and speeds approaching those of traditional IR systems. However, the behavioral logic of existing techniques focused on optimizing model architecture and training methods has not been fully demonstrated. Therefore, this invention attempts to provide a visual explanation of the model decision-making process, thereby offering experts insights into model improvement.

[0005] There are three main challenges in interpreting OpenQA models. First, because OpenQA is an open-ended task requiring the selection of multiple relevant long segments from a large corpus and involving natural language understanding and processing, interpreting the model is difficult. Second, OpenQA models consist of multiple modules, each a complex black box with numerous connections and parameters. Here, "model" refers to the system or pipeline required to complete the entire OpenQA task, while "module" refers to an independent structure or component within the model that performs a specific part of the task. Understanding the workings of various parallel or serial modules is crucial for identifying bottlenecks in the model and improving it. Third, how to move from interpreting individual model parameters to building a holistic and semantic understanding of the model's decision-making process remains unclear.

[0006] Several researchers have developed visual analytics systems for interpreting machine learning, such as transforming the structure of convolutional neural networks into directed acyclic graphs to aid expert analysis, or proposing a unified structure to interpret deep NLP models for text classification. However, these methods focus on interpreting single models and are limited in exploring the internals of multi-module models. Some work has investigated model interpretability for machine reading comprehension (MRC) tasks, such as using t-SNE dimensionality reduction for layer embeddings and attribution methods to interpret knowledge stored in BERT. Recently, some analytics tools have provided interactive and diagnostic frameworks for MRC models, integrating various analytics methods such as hidden layers, attention matrices, and embedding analysis of adversarial text. However, current research focuses on MRC tasks and fails to analyze the retrieval decision flow, which is a bottleneck limiting the performance of OpenQA models. Furthermore, in scenarios like OpenQA, instances consist of multiple long segments with a given question, and analyzing instances by visualizing layer embeddings using scatter plots and attention matrices using heatmaps can lead to visual clutter and often provides limited insights into global semantic information. This invention aims to address these gaps in the visual analytics literature. Summary of the Invention

[0007] The purpose of this invention is to provide a visual analytics system for interpreting open-domain question-answering models, denoted as the VEQA visual analytics system, to allow NLP experts to semantically understand the decision-making process of OpenQA models and gain insights into how to enhance the models.

[0008] To facilitate the description of visual analytics systems, we will first introduce the Open Domain Question Answering (OpenQA) model. The OpenQA model includes a "retrieval-reader" architecture, such as the DPR model. Both the retrieval and reader are BERT-based modules, used to retrieve relevant paragraphs and extract the answer span from a given paragraph, respectively. Figure 1 As shown. Wherein, given the question q and candidate paragraph p... iThe search engine uses the relevance score between the question and the paragraph (Rel(q, p)) to determine the relevance score. i To determine whether p was found. i To enter the reader. Within the reader, the answer extractor uses the inference layer to extract from p. i The answer is extracted from the selection layer, and the reorderer determines whether to use it as the final prediction based on the results of the selection layer.

[0009] Train a retrieval module that takes the question and each paragraph in the corpus as input and outputs a probability score of the paragraph's relevance to the question. Select the k paragraphs with the highest scores as candidate paragraphs. Train a reader module that takes the question and each candidate paragraph retrieved by the retrieval module as input and outputs the paragraph's reorder score and the start and end positions of the paragraph's predicted answer range. The predicted answer of the paragraph with the highest reorder score is then used as the reader's output.

[0010] The retrieval system comprises two components: a question encoder and a paragraph encoder. The question encoder takes a given question text as input and uses an encoder to predict the embedding representation of the question text. The paragraph encoder takes candidate paragraph texts from the corpus as input and uses an encoder to predict the embedding representation of each paragraph text. Furthermore, the retrieval system obtains the relevance score of a single paragraph to the given question by calculating the inner product of the question's embedding representation and the embedding representation of a single paragraph. The paragraphs are then sorted according to their relevance scores, and the top k paragraphs with the highest scores are selected as candidate paragraphs for the retrieval system's output.

[0011] The reader comprises two components: a reorderer and an answer extractor, both sharing a single encoder. The reorderer takes a pool of k candidate paragraphs retrieved by the retrieval system, concatenates the question with a single candidate paragraph as input, uses the same encoder to obtain the embedding representation of each question-paragraph pair, and then passes this embedding representation through a selection layer to obtain the candidate score for each paragraph. The answer extractor takes another pool of k candidate paragraphs, concatenates the question with a single candidate paragraph as input, uses an encoder to obtain the embedding representation of each question-paragraph pair, and then passes this embedding representation through an inference layer to predict the start and end positions of the answer for each paragraph. The reader sorts the k paragraphs according to the candidate scores obtained from the reorderer, selects the paragraph with the highest score as the final retrieved paragraph, and uses the corresponding output of that paragraph in the answer extractor as the final predicted answer.

[0012] The visual analysis system for interpreting open-domain question-answering models provided by this invention includes an interpretation engine module, a process analysis module, and a view module. Wherein:

[0013] Inspired by previous work on semantic analysis of BERT using saliency methods, the explanation engine module of this invention uses attribution methods to attribute the final output and implicit output of each module of the OpenQA model at both global and local levels, including a feature attribution explainer and a layer attribution explainer; wherein:

[0014] The feature attribution interpreter uses saliency and attribution methods to interpret modules in the OpenQA model, that is, to evaluate the contribution of each input feature to the module output and the contribution of each layer to the module output.

[0015] The layer attribution interpreter employs an attribution method to explain the module, that is, to evaluate the contribution of each layer to the module output; specifically, it uses a tree generation algorithm to capture abstract semantic information and explore the layer information flow; and it uses layer aggregation to calculate the importance of each layer in order to explain the layer function.

[0016] The Explanation Engine module helps experts understand the decisions of the OpenQA model by analyzing module responses and instances.

[0017] In the process analysis module, model information, data, and interpretable data generated by the interpretation engine in VEQA are visualized as various views in the user analysis interface; the user explores in multiple levels according to a linear workflow, in the order of dataset → subset → single instance → single paragraph.

[0018] Dataset and Subset Hierarchy: After selecting a dataset and model in the user panel, the user selects a subset in the summary view based on the statistical information and importance data of each part of the module, and the specific data is presented in the context view;

[0019] Instance hierarchy: In the context view, the user selects a single instance of interest from the subset, and the explanatory data of that instance in the various modules of the model are visualized in the instance view;

[0020] Paragraph hierarchy: Users select candidate paragraphs of interest in the instance view, and detailed information is further displayed in the tree view.

[0021] The view module, here referring to the various views visualized on the user analysis interface, includes: a summary view, a context view, an instance view, and a tree view. These four views are used for visualization analysis and exploration.

[0022] Summary view: Using the first two words of the questions as labels, the dataset is divided into different question types, and a performance metric is calculated for each subset; it provides an overview of the modules and dataset along with the importance of each layer obtained from the layer attribution to guide experts in exploring instances in the subsets;

[0023] Contextual view: Displays all question instances and the prediction results for each instance in the dataset or selected subset, and shows a heatmap of selected paragraphs at a certain stage; it displays the feature importance of all words obtained from the feature attribution interpreter compared to the original text, and complements other views;

[0024] Example View: Using a feature attribution interpreter to summarize the key words of each candidate paragraph in different modules and display their distribution in a novel flow graph with contextual ranking visualization, this is designed to help experts understand the similarities and differences in decision flows between modules;

[0025] Tree View: Based on the attribution tree generated by the layer attribution interpreter, a carefully designed, comparable tree visualization is used to express the semantic changes between layers.

[0026] This invention enables intuitive exploration of complex attribution results in visual design within VEQA, addressing the challenges of analyzing complex data with multiple long texts and complex models with multiple modules. It can help understand the decision-making reasons of OpenQA models and provide insights for model improvement; the system also supports fine-grained exploration of the decision-making process within a single module. Attached Figure Description

[0027] Figure 1 This invention relates to a schematic diagram of an open-domain question-answering model architecture.

[0028] Figure 2 This is a schematic diagram of constructing a tree from the attention matrix in this invention.

[0029] Figure 3 This is a flowchart of the visual analysis system of the present invention.

[0030] Figure 4 These are schematic diagrams illustrating two modes of the tree view of the present invention. In A: The two trees of the two encoders in the retrieval unit are placed vertically as a whole (a1, a2), and placed side-by-side with the other two trees of the reorderer and reader (a3, a4). In B: The evolution of the 12 attribution trees corresponding to the 12 layers of a certain module.

[0031] Figure 5 This is a diagram of the visual analysis system interface of the present invention. In the diagram, A is the user panel, B is the summary view, C is the context view, D is the instance view, and E is the tree view.

[0032] Figure 6 These represent the different layer responses of the four modules. A, B, and D represent the layer responses of the question encoder, paragraph encoder, reorderer, and answer extractor, respectively. Detailed Implementation

[0033] The present invention will now be further described in conjunction with embodiments and accompanying drawings.

[0034] (I) Explanation Engine Module

[0035] This invention provides an interpretation engine for interpreting decision flow-related data in OpenQA models. The interpretation engine includes a feature attribution interpreter and a layer attribution interpreter. Wherein:

[0036] The feature attribution interpreter uses saliency and attribution methods to interpret modules in the OpenQA model, that is, to evaluate the contribution of each input feature to the module output and the contribution of each layer to the module output. Specifically:

[0037] Considering that the OpenQA model uses multiple modules, and a single module can handle multiple tasks, (M, T) is used to denote the module M that handles task T. The complete OpenQA model consists of four (M, T) modules that determine the final prediction: two independent encoders in the retrieval unit, namely the question encoder (Q, E) and the paragraph encoder (P, E), and two modules in the reader, which act as a reorderer (R, R) and an answer extractor (R, S). Furthermore, F... (M，T) This represents the final output of (M, T) as the attribution target.

[0038] For a given module M with a given task T, each word e in the input embedding e of length L... i The significance score Sal is assigned based on the integral gradient. (M，T) (e i ):

[0039]

[0040] In formula (1), b represents the repeated [MASK] vector as the baseline, and m represents the number of steps performed in the Riemann approximation of the integral, which is generally set by default to ensure a balance between accuracy and speed. Specifically, m is set to 50. F represents (M，T) relative word e i gradient,

[0041] The layer attribution interpreter employs an attribution method to explain the module, that is, to evaluate the contribution of each layer to the module output; specifically, it uses a tree generation algorithm to capture abstract semantic information and explore the layer information flow; and it uses layer aggregation to calculate the importance of each layer in order to explain the layer function.

[0042] Using the layer conductance method (a method for obtaining attribution scores), the task-independent output of the l-th layer in module M of task T was obtained. The attribution score, i.e. Its size and same:

[0043]

[0044] In formula (2),, m is the same as in formula (1). Therefore, under the condition of task T, the output of the l-th layer in module M, i.e., the embedding... and the h-th head attention matrix Represented as and Equation (1) is the Riemann approximation of the gradient integral of the input along the path from the given baseline to the input. Similarly, Equation 2 approximates the gradient integral flow of neurons in the layer.

[0045] Based on this, the attribution scores are further processed to explore hierarchical information flow and interpret layer functions.

[0046] (1) Exploring hierarchical information flow

[0047] NLP experts are familiar with hierarchical representations of text. Considering that displaying the raw attribution matrix in the case of long texts would lead to visual clutter, and that most elements of the attribution matrix are very small (close to zero), the invention employs a tree-generating algorithm to display the information flow within the modules. The tree generation is based on the attention attribution derived in the previous section, such as... Figure 2 As shown, the multi-layered original attention (A) is refined into attention attribution (B) using formula (1), and then words and word pairs with high attribution scores are selected from top to bottom as nodes and edges to construct the tree (C). For example, "may" with the highest sum of attribution scores at the top layer is selected as the root node, while "、" and "2018" are selected as child nodes because of their high attribution scores. Due to space constraints, the main part of the tree is shown in the figure. Specifically:

[0048] First, the attribution score of each attention head in layer l is summarized using the L2 norm, denoted as . Secondly, the attribution score of the embedding output of the l-th layer is calculated using the L2 norm, i.e. This is a measure of the saliency of the l-th layer of module (M, T) for the embedding e corresponding to the input sequence t. Again, the word with the highest attribution score is selected. The word i is taken as the root node of the tree, and a heuristic top-down greedy algorithm is used to traverse all nodes that do not exist in the tree from level l all the way to level 1. Finally, word pairs i and j with pairwise attribution scores greater than a certain threshold are selected and added to the tree.

[0049] (2) Interpretation layer function

[0050] To associate module behavior with task type, the importance of the l-th layer for a given module (M, T) is calculated from the importance of all instances in the dataset. The highest attribution score a i，j The average value is calculated. Statistics related to other attention heads are not considered because there are only a few strong interactions between word pairs in each attribution matrix, and the overall distribution within each attention head is similar to that of other attention heads. Given that the OpenQA model consists of multiple modules, this invention does not calculate the importance of each attention head individually, but rather restricts the optimal granularity of exploration to the layer level.

[0051] (II) Process Analysis Module

[0052] The workflow of the Visual Quality Assurance (VEQA) system proposed in this invention is as follows: Figure 3 As shown, this includes storage, an interpretation engine, and a visualization analysis interface. Data generated by a carefully trained and evaluated OpenQA model is stored in the storage to provide users with basic information about the model and dataset. The interpretation engine, as described above, uses an attribution algorithm on the module outputs and layer outputs in the storage and aggregates the layer attribution results into layer responses and dependency trees. This data is provided in each view of the visualization analysis interface for users to explore. Experts typically expect the analysis process to proceed from global to local (i.e., dataset → subset → instance → candidate paragraph). They want to explore the differences in response of individual modules to different task types and the differences in response of different modules to a given task type, which can be obtained by examining the statistics and importance of each part of the module in the summary view. Experts then select an instance from the subset of interest in the context view to analyze and view the decision-making process within the entire model in the instance view. Experts then use the attribution tree in the tree view to continue exploring and comparing the differences in how a given problem-paragraph pair is handled at different stages. The tree view also allows users to further explore the hierarchical changes in the information flow within a single module.

[0053] (III) View Design Module

[0054] The system interface proposed in this invention includes a user panel, a summary view, a context view, an instance view, and a tree view, such as... Figure 5 As shown, this is to support visual exploration of OpenQA. The following describes the last four main views.

[0055] (1) Summary View

[0056] Summary view ( Figure 5 A) Provides an overview of the module responses at each stage under different subset categories, as well as visualizations of evaluation metrics. This helps OpenQA experts understand the distribution of the dataset and the differences in module responses to subsets, thus guiding users to select specific subsets for further exploration.

[0057] This invention follows expert advice, dividing the dataset into multiple subsets based on the first two terms and visualizing them using multiple Sankey graphs with a tree-like layout. The width of each node is proportional to the number of questions it represents. Each subset is connected to two bar graphs by a curve, representing the corresponding performance metrics in the retrieval and reader: top-k accuracy and perfect match probability (EM). To clearly show the performance differences between subsets, the average performance metric of the entire dataset is calculated as a standard value to depict bar graphs of constant height, with hollow portions of the bars indicating below-average performance. Shading is used for portions above the average. To facilitate exploration of module responses related to task type, rectangles are placed below each bar graph, the number of rectangles encoding the layer number for each module, using gradient colors to encode the average layer response within the corresponding subset, which can be obtained using the layer aggregation method described in the interpretation engine.

[0058] (2) Context View

[0059] As a supplement to the summary view, the context view ( Figure 5 .B) List all problem instances and the prediction results for each instance in the dataset or selected subset in a table, and display a heatmap of the selected segment at a certain stage.

[0060] The table in the context view has three columns, displaying information about the instance and the prediction. The first column records the question text; the second column shows the instance's top-k precision during the retrieval phase using a bar chart embedded in the table; the circles in the third column represent the overall evaluation metric, or EM. Hollow circles indicate that the final result is not a perfect match for the gold answer, while solid circles indicate the opposite. The original text of the selected paragraph and its significant score distribution in the selected phase are displayed below the question table, the content of which is interactively controlled with the instance view.

[0061] (3) Instance View

[0062] Instance View ( Figure 5 .C) Display the overall data stream of the top k paragraphs with a given question retrieved from the corpus in the OpenQA model in a ranking visualization that combines text and bar charts.

[0063] The instance view consists of four columns, representing a summary of important words from the three tasks (i.e., the retriever, the reorderer, and the reader) and the final prediction results. The ranking of individual candidate paragraphs in a single stage (M, T) is summarized into a set of words V = t. i The saliency score for each word is Sal. (M，T) (e iIf the candidate word is above a threshold, all candidate words are vertically placed in the current ranking order, with their displayed size proportional to their significance score. Furthermore, preliminary experiments show that this module typically focuses on important words within one or two sentences of long texts. Therefore, color is used to encode word positions, i.e., words belonging to the same sentence are given the same color. To further abstract the model's decision-making process from the top k candidates, bar charts are placed above the first three columns to indicate context-independent words appearing above a certain threshold in that column, using two colors to distinguish their positions. For example, words appearing in the question and paragraph are encoded side-by-side with blue and yellow rectangles on the left, while blue bars in the middle and right represent words appearing multiple times only in the question, and yellow bars represent words appearing multiple times only in the paragraph. Users can hover the mouse over the bar chart to see the corresponding word occurrences in all three columns. Red or green lines are used between the columns to represent the input and output of each stage. Specifically, the line between the first and second columns represents the results before and after re-ranking, with red indicating that the candidate does not contain the true answer, and green indicating the opposite. The line between the second and third columns is horizontal and the same color as before, because the reader does not change the order of the candidates. The horizontal line between the third and fourth columns points to the final prediction for a single candidate, and its color indicates whether the final result is a perfect match for the golden answer.

[0064] (4) Tree view

[0065] Based on the generated attribution tree, the tree view allows users to further explore the processing flow of pairs of questions and paragraphs in the entire model or a specific module through comparable tree visualizations.

[0066] This invention uses color and size to encode information embedded in the attribution tree. To effectively perceive how the tree is constructed, the location of markers in the context is encoded using the same color scheme as in the instance view, and a different gradient color, which will not be confused with the former, is used to encode layer l of edge origin, as shown in the legend in the user panel. Figure 5 A). The link width between two nodes represents the pairwise attribution score of words i and j in level l. The diameter of the node encodes the word attribution score of the word k corresponding to the l-th layer.

[0067] To increase information density and prevent visual clutter, only words contained in the attribution tree are displayed, rather than all tags, as the latter typically exceeds 100 words, while the total number of words appearing in all trees is much smaller when the algorithm's threshold is set normally. Meanwhile, our goal is to ensure comparability between trees. Therefore, we determine the union of the words appearing in the attribution tree of each module and then arrange them at equal intervals. We encode the width of each tree based on its height to facilitate comparison of hierarchical variations.

[0068] The tree view supports attribution tree comparisons at each stage and evolves layer by layer within a single module, each with an adjustable independent threshold. When the view displays the former, the four trees corresponding to the four groups (M, T)s are horizontally placed as three wholes, such as... Figure 4 As shown in Figure A, the two attribution trees corresponding to (Q, E) and (P, E) are placed vertically on the left, aligned with the (R, R) and (R, S) trees. When the user switches the view to hierarchical exploration, the attribution trees corresponding to the module level are arranged horizontally. Figure 4 (B) Users can only change the filtering threshold of this module.

[0069] (iv) Verification and Evaluation

[0070] This invention uses representative OpenQA models to conduct case studies and collaborates extensively with domain experts to demonstrate the effectiveness of describing the OpenQA decision-making process as a visual problem. Based on this, successful cases are categorized, and suggestions for model enhancement are provided. The following example illustrates how to use VEQA to understand the decision flow of an OpenQA model.

[0071] This invention demonstrates how VEQA interprets the data flow in the DPR architecture using the Natural Question (NQ) dataset, which contains 300,000 naturally occurring questions and answers with human annotations from Wikipedia pages. Due to computational resource constraints, this invention uses a model checkpoint trained on the NQ dataset provided by the official website. This checkpoint is trained on negative samples (mined by BM25) that are highly similar to the questions but do not contain answers, as well as random and positive samples from other instances. The same test set split as in previous work is selected, comprising 3610 instances with 100 retrieved paragraphs each. The top 20 retrieved paragraphs are evaluated using an attribution algorithm. In the initial validation process, the user follows the workflow of this invention, freely observing each view and establishing preliminary conclusions about the decision-making process.

[0072] Through initial exploration, the present invention has successfully abstracted and visualized the decision-making process of DPR.

[0073] (1) Overall Overview

[0074] After selecting DPR as the first checkpoint and test dataset in the evaluation architecture and user dashboard, the summary view displays the distribution of the dataset and the performance metrics of a subset, as well as the responses of multiple modules.

[0075] Performance metrics. For example... Figure 5As shown in Figure B, the performance scores corresponding to each subset classified based on the first two words are not equal, especially the top-k accuracy, which is used to measure the retrieval performance. The overall evaluation metric for the entire model, EM, differs between subsets. When the top-k accuracy is significantly below average, the model fails to make correct predictions without accessing paragraphs containing the golden answer, such as the subset classified by "How-". Therefore, future model designs should consider different question types.

[0076] Module response. Then, click the button to visualize the aggregation results of the four (M,T) layers, as shown below. Figure 6 As shown in the diagram, each (M,T) contains specific layers that play a major role in completing the task, especially (Q,E) and (P,E) in the retrieval module, which have a large number of layers but relatively small effects. This suggests that optimizing the model representation space is possible in terms of training methods, dataset selection, and model compression. Furthermore, since (R,R) and (R,S) share the same module, comparisons... Figure 6 .C and Figure 6 D found that the reader functions as a reorderer at the low levels, while focusing on extracting answers at the mid and high levels.

[0077] Potential relationships. We also found inconsistencies in layer responses corresponding to different subsets. That is, the color intensity of lines of different thicknesses on the same line is inconsistent, which may be due to the type of problem, as will be discussed further below.

[0078] (2) Case Study

[0079] By randomly selecting multiple instances and observing the data flow changes of the top 10 candidate segments in the three tasks in the instance view, some preliminary conclusions were drawn about multiple modules.

[0080] The retriever and reorderer. A notable feature is that the first column, representing the reorderer, is filled with words backgroundd in blue and pink, corresponding to sentence 0 (question) and sentence 1 (title), respectively. Therefore, most question words have a significant impact on the results for (Q,E), and the embedded output for (P,E) largely depends on the title. Meanwhile, examining the words presented in the grouped bar chart above the columns reveals that the retriever relies on overlapping related words. Figure 5In the example shown in .C, the words "Dragon" and "Super" appear multiple times in both the question and the selected title, while other (minor) words, such as "episode" and "113," are ignored, which could be the reason for the retrieval failure given that all connecting lines are red. However, the reorderer seems to prioritize overall structure and semantics, with words that separate sentences, such as "[SEP]" and periods, as well as words outside the main entities, appearing frequently (e.g., Figure 5 The first row of the second column in the .C column displays "episode". This confirms the conclusion of previous work that interaction-based retrievers are able to capture more semantic information than representation-based retrievers (such as DPR).

[0081] Reorderer vs. Answer Extractor. Observing the second and third columns, representing the reorderer and answer extractor respectively, we believe the reorderer has begun to capture information about the answer, as some key words summarized from it are similar to those in the answer extractor and are relevant to the final answer. Compared to either the retrieval machine or the reorderer, it can be observed that the reorderer typically pays close attention to the first word, which indicates the type of question, as well as other words important for answering the question, such as... Figure 5 The "when" and "130" in the grouped bar chart above the third column of .C are not named entities and delimiters.

[0082] (3) Candidate paragraph exploration

[0083] Next, by adjusting the thresholds in the tree view, attribution trees at different scales were compared, which verified the conclusions mentioned above and further raised questions related to attribution trees.

[0084] Let's verify the conclusions above. First, as... Figure 5 As shown in .D, the semantic connections in (Q,E) are very close, and (P,E) depends on the title and "[SEP]", which is consistent with the findings in the instance view. Secondly, the edges on the tree of (R,R) originate from the lower and middle levels, which also appears on the tree of (R,S). In contrast, the latter is different because it includes edges related to answers originating from higher levels, which is consistent with the conclusions of the summary view.

[0085] Regarding the attribution tree issue, during our exploration, we found that the overall attribution feature results are roughly similar to those of layered attribution. For example, words filtered out in the instance view do not appear in the attribution tree. Furthermore, the results of hidden embedding attribution differ from those of attention attribution because the node with the largest diameter sometimes does not appear at the top. This result is related to the inherent properties of attribution and tree generation algorithms and is not the focus of this invention, but it represents a direction for future improvements.

[0086] In addition to the exploration process described above, users also follow the general workflow to explore the model and find successful decision cases, and use the system to explore how higher quality training data can help improve the performance of the retrieval engine.

[0087] Although embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above embodiments are merely exemplary and are not intended to limit the scope of the invention. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

Claims

1. A visual analytics system for interpreting open-domain question answering models, characterized in that, The system comprises an explanation engine module, a process analysis module and a view module; wherein: The explanation engine module uses attribution methods to attribute the final output and implicit output of each module of the OpenQA model at global and local levels, including a feature attribution interpreter and a layer attribution interpreter; wherein: The feature attribution interpreter uses a saliency method and an attribution method to explain the modules in the OpenQA model, i.e. to evaluate the contribution of each input feature to the module output and the contribution of each layer to the module output; The layer attribution interpreter uses an attribution method to explain the module, i.e. to evaluate the contribution of each layer to the module output; specifically, a tree generation algorithm is used to capture abstract semantic information and explore layer information flow; layer aggregation is used to calculate the importance of each layer to explain the layer function; The explanation engine module helps experts understand the decision-making of the OpenQA model by analyzing module responses and instances; The process analysis module visualizes the model information, data and explainability data generated by the explanation engine in the VEQA as various views of the user analysis interface, and the user explores in multiple levels according to the linear workflow, in the order of dataset, subset, single instance and single paragraph; wherein: The dataset and subset level: after the user selects a certain dataset and model in the user panel, the user selects a subset in the summary view according to the statistical information and importance of each part of the module, and the specific data is presented in the context view; The instance level: the user selects a single instance of interest in the subset in the context view, and the explainability data of the instance in each module of the model is visualized in the instance view; The paragraph level: the user selects the candidate paragraph of interest in the instance view, and the detailed information is further presented in the tree view; The view module is the view, i.e. the various views visualized in the user analysis interface; the user panel displays the initial selection of the model and the dataset, and the views include: summary view, context view, instance view and tree view; the four views are used for visual analysis and exploration; wherein: The summary view: using the first two words of the question as a label, the dataset is divided into different question types, and the performance indicators of each subset are calculated; together with the importance of each layer obtained in the layer attribution, it provides an overview of the module and the dataset to guide the expert to explore the instances in the subset; The context view: it displays all question instances in the dataset or selected subset and the prediction results of each instance, and displays the heat map of the selected paragraph at a certain stage; it displays the feature importance of all words obtained in the feature attribution interpreter and the original text, which is a supplement to other views; The instance view: using the feature attribution interpreter to summarize the key words of each candidate paragraph in different modules, and displaying their distribution in a novel flowchart containing the ranking visualization of the context, which aims to help experts understand the similarities and differences of the decision-making flow among modules; The tree view: based on the attribution tree generated by the layer attribution interpreter, a comparable tree visualization is designed to express the semantic changes between layers.

2. The visual analytics system interpreting an open-domain question answering model of claim 1, wherein, In the explanation engine module, the feature attribution explainer adopts the saliency method and the attribution method to explain the modules in the OpenQA model, i.e., to evaluate the contribution of each input feature to the module output and the contribution of each layer to the module output, specifically: Considering that the OpenQA model uses multiple modules and a single module carries multiple tasks; therefore, using (M, T) to represent the module M carrying the task T, the complete OpenQA model is composed of four (M, T), which determines the final prediction: two independent encoders in the retriever, namely the question encoder (Q, E) and the passage encoder (P, E), and two modules in the reader, which act as the re-ranker (R, R) and the answer extractor (R, S); using F (M,T) to represent the final output of (M, T) as the attribution target; For a given module M with a given task T, each word e in an input embedding e of length L i The salience score Sal is assigned by integrating the gradient (M,T) (e i ): In Equation (1), b represents the repeated [MASK] vector as the baseline, and m represents the number of steps performed in the Riemann approximation of the integral, which is the default setting to ensure a balance between accuracy and speed. F represents (M,T) relative word e i The gradient of the input, Equation (1) is the Riemann approximation of the gradient integral of the input along the path from the given baseline to the input.

3. The visual analytics system interpreting an open-domain question answering model of claim 2, wherein, In the explanation engine module, the layer attribution explainer adopts the attribution method to explain the modules, i.e., to evaluate the contribution of each layer to the module output; specifically, a tree generation algorithm is used to capture abstract semantic information and explore the layer information flow; layer aggregation is used to calculate the importance of each layer to explain the layer function; specifically: Using the layer conductance method, the task-independent output of the l-th layer in module M of task T is obtained, i.e. which has the same size as ​​ In formula (2), m is the same as in formula (1); the output of the l-th layer in the module M, i.e., the embedding and the h-th head attention matrix is expressed as and Formula 2 approximates the gradient integration flow of neurons in the layer; On this basis, the attribution scores are further processed to explore the hierarchical information flow and explain the layer function.

4. The visual analytics system interpreting an open-domain question answering model of claim 3, wherein, In the explanation engine module, the exploration of the hierarchical information flow is achieved by using a tree generation algorithm to display the information flow inside the module; the generation of the tree is based on the derived attention attribution, i.e., the multi-layer original attention (A) is refined into attention attribution (B) through formula (1), and then the words and word pairs with high attribution scores are selected as nodes and edges from top to bottom to construct a tree (C); specifically: First, the attribution score of each attention head in the l-th layer is summarized using the L2 norm, denoted as Second, the attribution score of the embedding output of the l-th layer is computed using the L2 norm, i.e. This is a measure of the significance of the embedding e corresponding to the input sequence t for the l-th layer of the module (M, T); Next, select the word with the highest attribution score. The word i is taken as the root node of the tree, and a heuristic top-down greedy algorithm is used to traverse all non-existent nodes in the tree from the l-th level all the way to the 1st level. Finally, the word pairs i and j with pair-wise attribution scores greater than a certain threshold are selected to join the tree.

5. The visual analytics system interpreting an open-domain question answering model of claim 4, wherein, In the explanation engine module, the (2) explanation of the layer function is specifically: To associate module behavior with task type, the importance of the l-th layer for a given module (M, T) is calculated from the importance of all instances in the dataset. The highest attribution score a i,j The average value is calculated; considering that the OpenQA model consists of multiple modules, the importance of each attention head is not calculated separately, and the optimal granularity of exploration is limited to the layer level.

6. The visual analytics system interpreting an open-domain question answering model of claim 5, wherein, In the view module: The summary view provides an overview of the module response at each stage under different subset categories and the visualization of the evaluation metrics; it helps OpenQA experts understand the distribution of the dataset and the response difference of the modules to the subsets, thereby guiding the user to select a specific subset for further exploration; According to the first two terms, the dataset is divided into multiple subsets, and is visualized through multiple Sankey diagrams with tree-like layout; the width of each node is proportional to the number of problems it represents; each subset is connected to two bar charts through a curve, and the two bar charts represent the corresponding performance metrics in the retriever and the reader, i.e., top-k precision and exact match probability EM; in order to clearly show the performance difference of the subsets, the average performance indicators of the entire dataset are calculated as standard values to depict the bar charts with constant height, and the hollow part of the bar chart is used to indicate the performance below the average value; at the same time, the shaded part is used for the part above the average value; in order to facilitate the exploration of the module response related to the task type, a rectangle is placed below each bar chart, and the number of rectangles encodes the number of layers of each module, and the average value of the layer response in the corresponding subset is encoded using gradient color, which is obtained by the layer aggregation method described in the explanation engine; The context view lists all problem instances in the dataset or the selected subset and the predicted results of each instance in a table, and displays the heat map of the selected paragraph at a certain stage; The table in the context view has three columns, showing information about instances and predictions; the first column records the question text; the second column shows the top-k precision of instances at the retrieval stage using a bar chart embedded in the table; in the third column, the overall evaluation metric, EM, is represented by a circle; a hollow circle indicates that the final result does not completely match the golden answer, and a solid circle indicates the opposite; the original text of the selected paragraph and its significant score distribution at the selected stage are displayed below the question table, and their content is controlled by interaction with the instance view; The instance view displays the overall data flow of the top-k paragraphs retrieved from the corpus by the OpenQA model for a given question in the form of ranked visualization combining text and bar charts; The instance view consists of four columns, representing three tasks, namely the retriever, the re-ranker, the summary of important words in the reader, and the final prediction result; ordering a single candidate paragraph in a single stage (M, T) into a set of words V = t i where the salience score Sal (M,T) (e i ) of each word is higher than a threshold; placing all candidate words vertically in the ordering sequence of the current stage, with the displayed size being proportional to their salience score; using color to encode the word position, i.e. using the same color for the words belonging to the same sentence; to further abstract the decision process of the model from the top-k candidates, placing a bar chart above the top three columns to indicate the context-agnostic words that appear above a certain threshold in that column, and using two colors to distinguish their position; the user hovers the mouse over the bar chart to see the corresponding word occurrences in all three columns; using a red or green line between two columns to represent the input and output of each stage; specifically, the line between the first and second columns represents the reordering of the top and bottom results, with red indicating that the candidate does not contain the true answer and green indicating the opposite; the line between the second and third columns is horizontal and has the same color as before; the horizontal line between the third and fourth columns points to the final prediction of a single candidate, with the color indicating whether the final result matches the gold answer completely or not. The tree view allows users to further explore the processing flow of pairs of questions and paragraphs in the entire model or a specific module through comparable tree visualizations; Color and size are used to encode information embedded into the attribution tree, respectively; to effectively perceive how the tree is constructed, the same scheme as the one used in the instance view is adopted to encode the position of the label in the context, and another gradient color that does not get confused with the former is used to encode the layer l of the edge origin; the width of the link between two nodes represents the pairwise attribution score of words i and j in the l-th layer The diameter of the node encodes the word attribution score of the k-th word in the l-th layer In order to increase information density and prevent visual clutter, only the words contained in the attribution trees are displayed, not all tokens; at the same time, the union of words appearing in the attribution trees of each module is determined, and then they are arranged equidistantly; The width of each tree is encoded according to its height to facilitate comparison of hierarchical changes; The tree view supports comparison of attribution trees at each stage and layer-by-layer evolution in individual modules, each module having an adjustable independent threshold; when the view displays the former, the four trees corresponding to the four groups (M, T) are placed horizontally as three wholes; Wherein The two attribution trees corresponding to (Q, E) and (P, E) are placed vertically on the left side, aligned with the (R, R) and (R, S) tree pairs; When the user switches the view to hierarchical exploration, the attribution trees corresponding to the number of module layers are arranged horizontally, and the user can only change the filtering threshold of this module.

Images