Using autocompletion as a data discovery scaffolding to support visual analysis

Abstract

A method utilizes data discovery to support visual analysis of a dataset. A user selects a data source, and the method presents a natural language interface for analyzing the data source. The user specifies an incomplete natural language command directed to the data source, and the method correlates words in the incomplete natural language command to data fields in the data source. The method determines a data type of the data fields and a range of data values of the data fields. From the data type and the range of data values, the method presents one or more autocomplete options for the incomplete natural language command. Each option includes respective text and a respective corresponding visual graphic. The user selects one of the autocomplete options, and the method forms a complete natural language command. The method then displays a data visualization in accordance with the complete natural language command.

Description

[0001] Related Applications

[0002] This application is a continuation of U.S. Patent Application 17 / 010,770, filed September 2, 2020, entitled “Utilizing Autocompletion as a Data Discovery Scaffold for Supporting Visual Analysis,” which claims priority to U.S. Provisional Patent Application 63 / 021,066, filed May 6, 2020, entitled “Utilizing Autocompletion as a Data Discovery Scaffold for Supporting Visual Analysis,” each of which is incorporated by reference herein in its entirety.

[0003] This application is related to U.S. Provisional Patent Application 62 / 991,342, filed March 18, 2020, entitled “Incorporating Data Visualizations into Database Conversational Interfaces,” which is incorporated by reference herein in its entirety.

[0004] This application is related to U.S. Patent Application 15 / 804,991, filed November 6, 2017, entitled “Systems and Methods of Using Natural Language Processing for Visual Analysis of a Data Set,” which is incorporated by reference herein in its entirety.

[0005] This application is related to U.S. Patent Application 16 / 234,470, filed December 27, 2018, entitled “Analyzing Underspecified Natural Language Utterances in a Data Visualization User Interface,” which is incorporated by reference herein in its entirety.

[0006] This application is also related to U.S. Patent Application 16 / 221,413, filed December 14, 2018, entitled “Data Preparation User Interface with Coordinated Pivots,” which is incorporated by reference herein in its entirety.

[0007] This application is also related to U.S. Patent Application 16 / 236,611, filed December 30, 2018, entitled “Generating Data Visualizations According to an Object Model of Selected Data Sources,” which is incorporated by reference herein in its entirety.

[0008] This application is also related to U.S. Patent Application 16 / 236,612, filed December 30, 2018, entitled “Generating Data Visualizations According to an Object Model of Selected Data Sources,” which is incorporated by reference herein in its entirety.

[0009] This application is also related to U.S. Patent Application 16 / 679,234, filed November 10, 2019, entitled “Data Preparation Using Semantic Roles,” which is incorporated by reference herein in its entirety. TECHNICAL FIELD

[0010] The disclosed implementations generally relate to natural language processing, and more specifically to systems, methods, and user interfaces for providing relevant autocomplete suggestions based on data values of data fields identified in a command.

[0011] BACKGROUND

[0012] Natural language interaction has evolved as a useful modality to help users explore their data and interact with that data during visual analytics. However, little work has been done to explore how autocomplete can help with data discovery while helping users formulate analytic questions. Traditional interfaces focus on completion of syntactic queries without any guidance about what is in the underlying data. Current available systems that support autocomplete do not help with data discovery. Moreover, traditional systems do not take into account user preferences for autocomplete to support visual analytics in a natural language interface.

[0013] SUMMARY

[0014] To address these issues, some implementations use a design probe (sometimes referred to as the “Sneak Pique system”) to explore how autocomplete can better support users during their analytic workflow.

[0015] A novel system called "Sneak Pique" illustrates the usefulness of autocomplete for visual analytics. Three Mechanical Turk studies evaluated user preferences for various text- and visualization widget-based autocomplete design variants to help complete search queries. Survey results showed that users found data previews useful in suggestions. Widgets were preferred for previewing temporal data, geospatial data, and numeric data, while text autocomplete was preferred for categorical and hierarchical data. Exploratory analyses were conducted on the system to implement specific subsets of the preferred autocomplete variants. Insights about the efficacy of these autocomplete suggestions facilitated the design of natural language interfaces that support visual analytics.

[0016] According to some embodiments, a method utilizes data discovery to support visual analytics of a data set. The method is performed at a computing system having one or more processors and a memory storing one or more programs configured for execution by the one or more processors. A user selects a data source, and the system presents a natural language interface for analyzing data in the selected data source. The user specifies an incomplete natural language command directed to the data source, and the system associates one or more words in the incomplete natural language command with data fields in the selected data source. The system also determines a data type of the data fields and determines a range of data values for the data fields in the data source. In accordance with the data type and the range of data values, the system presents one or more autocomplete options for the incomplete natural language command. Each option includes a respective text and a respective corresponding visual graphic. When the user selects an option of the one or more autocomplete options, the system forms a complete natural language command and displays a data visualization in accordance with the complete natural language command.

[0017] In some embodiments, associating the one or more words in the incomplete natural language command with the data fields in the selected data source includes: (i) generating syntax rules based on attribute values from a data set located under the selected data source; (ii) determining whether the one or more words in the incomplete natural language command include a partially complete query by parsing the incomplete natural language command using the syntax rules and detecting syntax parse tree errors; and (iii) in accordance with a determination that the incomplete natural language command includes a partially complete query: (a) computing a syntactic structure of the partially complete query and related syntax rules that would be satisfied if the partially complete query were completed; and (b) selecting the data fields from the selected data source based on the related syntax rules.

[0018] In some implementations, correlating the one or more words in the incomplete natural language command is based on analyzing user preferences for text auto-completion or widget auto-completion for different data types, including categorical data, numerical data, geospatial data, temporal data, and combinations thereof.

[0019] In some implementations, presenting one or more auto-completion options for the incomplete natural command includes, in accordance with a determination that the data type is a categorical type, displaying a list of suggested text for a categorical data field. The suggestions are numerically ordered according to data frequency, and the list of text includes data previews showing respective data frequencies of the suggestions.

[0020] In some implementations, presenting one or more auto-completion options for the incomplete natural command includes, in accordance with a determination that the data type is a numerical type, displaying a histogram slider widget showing a histogram of data frequencies for a numerical data field from a data source.

[0021] In some implementations, presenting one or more auto-completion options for the incomplete natural command includes, in accordance with a determination that the data type is a geospatial type, displaying a map chart widget showing a data preview of location values for a data field, with numerical ranges shown as visual encodings.

[0022] In some implementations, presenting one or more auto-completion options for the incomplete natural command includes, in accordance with a determination that the data type is a temporal type, displaying (i) a list of text showing time data for a first data field ordered in chronological order, (ii) a calendar widget showing a temporal hierarchy for the first temporal data field, or (iii) a histogram slider widget showing a histogram of data frequencies for a second temporal data field.

[0023] In some implementations, presenting one or more auto-completion options for the incomplete natural command includes, in accordance with a determination that the data type is both a geospatial type and a temporal type, displaying (i) a list of text with semantically grouped data hierarchies, or (ii) a combination of a map and calendar widgets.

[0024] In some implementations, presenting one or more auto-completion options for the incomplete natural command includes displaying one or more widgets showing data previews. In some implementations, the one or more widgets are triggered based on semantics of tokens obtained from parsing the incomplete natural language command.

[0025] In some implementations, the method further includes: (i) receiving a second user input to specify a second incomplete natural language command directed to the data source; and (ii) computing and displaying an update to the data preview based on the second user input and the filter used in the data visualization.

[0026] In some implementations, the method further includes: (i) storing an initial state of the data visualization prior to receiving the second user input; (ii) receiving a third user input to specify a third incomplete natural language command directed to the data source; and (iii) in accordance with a determination that the third incomplete natural language command corresponds to a user request to reset the data visualization, restoring the data visualization based on the initial state of the data visualization.

[0027] In some implementations, the method further includes displaying one or more autocompletion suggestions needed to parse the partially complete query into a complete natural language command based on the relevant grammar rules.

[0028] In some implementations, presenting one or more autocompletion options for the incomplete natural command includes: (i) detecting user input to select respective text for a data field instead of a respective corresponding visual graphic; and (ii) in response to detecting the user input to select the respective text, switching from displaying the respective corresponding visual graphic to displaying the respective text, wherein the respective text provides the one or more text autocompletion options.

[0029] In some implementations, presenting one or more autocompletion options for the incomplete natural command includes: in accordance with a determination that the data type corresponds to hierarchical data, providing one or more text autocompletion options to select a hierarchical value.

[0030] In some implementations, the parsing of the incomplete natural language command is performed using a left-to-right (LL(*)) parser that performs left-most derivation of the incomplete natural language command and backtracks in cases where a suitable parse path cannot be constructed.

[0031] In some implementations, the grammar rules correspond to a context-free grammar that includes pre-defined rules and rules that are dynamically generated based on data values from data fields of an underlying data set.

[0032] In some implementations, the method further includes: (i) detecting a change in user preference in a series of tasks during the visual analysis; and (ii) further associating one or more words in the incomplete natural language command based on the change in adapting the autocompletion workflow to different data types.

[0033] According to some implementations, a system for generating data visualizations includes one or more processors, memory, and one or more programs stored in the memory. The programs are configured to be executed by one or more processors. The programs include instructions for performing any of the methods described herein.

[0034] According to some implementations, a non-transitory computer-readable storage medium stores one or more programs configured to be executed by a computer system having one or more processors and memory. The one or more programs include instructions for performing any of the methods described herein.

[0035] Therefore, methods, systems, and graphical user interfaces that enable users to analyze, prepare, and organize data are disclosed. Brief description of the attached diagram

[0037] For a better understanding of the systems, methods, and graphical user interfaces mentioned above, as well as the additional systems, methods, and graphical user interfaces that provide data visualization analysis and data preparation, reference should be made to the following description of the embodiments in conjunction with the accompanying drawings, wherein similar reference numerals refer to corresponding portions throughout the drawings.

[0038] Figure 1 An example graphical user interface for interactive data analysis is shown according to some implementation methods.

[0039] Figure 2 It is a block diagram of a computing device according to some implementation methods.

[0040] Figure 3A This is a schematic diagram of a system that automatically completes data discovery scaffolding based on some implementation methods.

[0041] Figure 3B An example according to some implementations is shown, in which the parser computes a look-ahead parse tree and predicts the lexical that triggers autocomplete.

[0042] Figures 4A-4L A flowchart is provided for a method of using data discovery to support the visual analysis of datasets according to some implementation methods.

[0043] The embodiments illustrated in the accompanying drawings will now be referred to by way of example. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without these specific details.

[0044] Description of implementation methods

[0045] Figure 1 An example graphical user interface 100 for interactive data analysis is shown in accordance with some embodiments. In accordance with some embodiments, the user interface 100 includes a data tab 114 and an analysis tab 116. When the data tab 114 is selected, the user interface 100 displays a schema information area 110, also referred to as a data pane. The schema information area 110 provides named data elements (e.g., field names) that can be selected and used to build a data visualization. In some embodiments, the list of field names is separated into a set of dimensions (e.g., categorical data) and a set of measures (e.g., numeric quantities). Some embodiments also include a list of parameters. When the analysis tab 116 is selected, the user interface displays a list of analysis functions instead of a list of data elements (not shown).

[0046] The graphical user interface 100 also includes a data visualization area 112. The data visualization area 112 includes a plurality of tool shelf areas, such as a column tool shelf area 120 and a row tool shelf area 122. These are also referred to as column tool shelf 120 and row tool shelf 122. As shown herein, the data visualization area 112 also has a large space for displaying a visual graph (also referred to herein as a data visualization). Because data elements have not yet been selected, this space initially has no visual graph. In some embodiments, the data visualization area 112 has a plurality of layers referred to as sheets. In some embodiments, the data visualization area 112 includes an area 126 for data visualization filters.

[0047] In some implementations, the graphical user interface 100 also includes a natural language input box 124 (also referred to as a command box) for receiving natural language commands. A user can interact with the command box to provide a command. For example, a user can provide a natural language command by typing the command in the box 124. Additionally, a user can indirectly interact with the command box by speaking into a microphone (or audio input device) to provide a command. In some implementations, data elements are initially associated with the column shelf 120 and the row shelf 122 (e.g., using a drag-and-drop operation from the schema information area 110 to the column shelf 120 and / or the row shelf 122). After the initial association, a user can use natural language commands (e.g., in the natural language input box 124) to further explore the displayed data visualization. In some instances, a user uses the natural language input box 124 to create the initial association, which results in one or more data elements being placed on the column shelf 120 and the row shelf 122. For example, a user can provide a command to view a relationship between data element X and data element Y. In response to receiving the command, the column shelf 120 and the row shelf 122 can be populated with the data elements (e.g., the column shelf 120 can be populated with data element X, while the row shelf 122 can be populated with data element Y, or vice versa).

[0048] Figure 2 FIG. 1 is a block diagram illustrating a computing device 200 that can display the graphical user interface 100, according to some implementations. Various examples of the computing device 200 include a desktop computer, a laptop computer, a tablet computer, and other computing devices having a display and a processor capable of running a data visualization application and / or a data preparation application 230. The computing device 200 generally includes one or more processing units / cores (CPUs) 202 for executing modules, programs, and / or instructions stored in memory 206 and thereby executing processing operations; one or more network or other communications interfaces 204; memory 206; and one or more communication buses 208 for interconnecting these components. The communication buses 208 can include circuitry that interconnects and controls communications between system components.

[0049] The computing device 200 includes a user interface 210 that includes a display device 212 and one or more input devices or mechanisms. In some implementations, the input devices / mechanisms include a keyboard 216. In some implementations, the input devices / mechanisms include a "soft" keyboard that is displayed on the display device 212 as needed, enabling the user to "press" the "keys" that appear on the display 212. In some implementations, the display 212 and input devices / mechanisms include a touch screen display or touch surface 214 (also referred to as a touch-sensitive display). Some implementations include an audio input device 220 and / or an audio output device 218.

[0050] In some implementations, the memory 206 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid state memory devices. In some implementations, the memory 206 includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid state storage devices. In some implementations, the memory 206 includes one or more storage devices remotely located from the CPU(s) 202. The memory 206, or alternately the non-volatile memory within the memory 206, comprises a non-transitory computer readable storage medium. In some implementations, the memory 206, or the computer readable storage medium of the memory 206, stores the following programs, modules, and data structures, or a subset thereof:

[0051] an operating system 222 including procedures for handling various basic system services and for performing hardware dependent tasks;

[0052] a communications module 224 for connecting the computing device 200 to other computers and devices via the one or more communication network interfaces 204 (wired or wireless) and one or more communication networks, such as the Internet, other wide area networks, local area networks, metropolitan area networks, and so on;

[0053] a web browser 226 (or other applications capable of displaying and navigating web pages) that facilitates the use of network applications via the web (e.g., the Internet or other wide area networks);

[0054] A data visualization application and / or data preparation application 230 that provides a data visualization user interface 232 (e.g., graphical user interface 100) to a user to construct a visual graph and / or ask natural language queries. For example, a user selects one or more data sources 240 (which can be stored on the computing device 200 or stored remotely), selects data fields from the data sources and / or searches the data sources for data fields that match a natural language query, and uses the selected fields to define a visual graph. In some implementations, the information provided by the user is stored as a visual specification 236. The data visualization application 230 includes a data visualization generation module 234 that takes user input (e.g., visual specification 236) and generates a corresponding visual graph (also referred to as a "data visualization" or "data viz"). The data visualization application 230 then displays the generated visual graph in the user interface 100. In some implementations, the data visualization application 230 is executed as a standalone application (e.g., a desktop application). In some implementations, the data visualization application 230 is executed within a web browser 226 or within another application that uses web pages provided by a web server. In some implementations, the data visualization application 230 includes a language processing module 238 for processing (e.g., interpreting) commands provided by a user of the computing device. In some implementations, the commands are natural language commands (e.g., captured by an audio input device 220). In some implementations, an audio input module 228 processes audio input from one or more audio input devices. In some implementations, the language processing module 238 includes sub-modules (e.g., an autocomplete module, a pragmatics module, and / or an ambiguity module). In some implementations, the memory 206 stores metrics and / or scores determined by the language processing module 238. In addition, the memory 206 can store thresholds and other criteria that are compared to the metrics and / or scores determined by the language processing module 238. For example, the language processing module 238 can determine a relevance metric (discussed in detail below) for an analyzed word / phrase of a received command. The language processing module 238 can then compare the relevance metric to a threshold value stored in the memory 206;

[0055] Zero or more databases or data sources 240 (e.g., first data source 240-1 and second data source 240-2) used by the data visualization application 230. In some implementations, the data sources are stored as spreadsheet files, CSV files, XML files, or flat files, or stored in a relational database; and

[0056] A conversational module 242 that processes user input via a conversational interface and generates textual and / or visual responses based on question type and / or user preferences.

[0057] Each of the above-identified executable modules, applications, or sets of routines, can be stored in one or more of the aforementioned memory devices, and correspond to a set of instructions for performing the above-described function. The above-identified modules or routines (i.e., sets of instructions) need not be implemented as separate software programs, procedures, or modules, and thus various subsets of these modules can be combined or otherwise rearranged in various embodiments. In some embodiments, memory 206 stores a subset of the modules and data structures identified above. Additionally, memory 206 can store additional modules or data structures not described above.

[0058] Although Figure 2 Computing device 200 is shown Figure 2 More specifically, the functions described herein are intended to be implemented by various features of the hardware components described herein, in combination with software components to carry out the processes described herein. It will be understood by those within the art that various hardware elements and software elements can be

[0059] The process of information seeking has moved away from the traditional paradigm that assumes that the information goal is well-formed; even if a user is intentionally seeking information, they do not necessarily know exactly what they want. The transformation of "conceptual knowledge" into a searchable query begins with some vague sense of a need to know something and progresses to the point where a person can describe some attribute of the desired information. This exploratory search is a complex and cognitively demanding activity that relies on recall and sense-making.

[0060] Autocompletion is a useful mechanism to support this complex task, displaying in-situ suggestions as the user types their query in the flow of their search task. Due to its effectiveness as a scaffolding to guide the searcher to improve efficiency, autocompletion is ubiquitous in various search environments. Recently, natural language (NL) interaction in visual analytic tools has gained interest in the way it supports users to interact with their data.

[0061] While information search in visual analytic tasks shares similarities with other forms of search, there are differences. Visual analytics involves understanding the characteristics of the underlying data and the various domains contained in the dataset (e.g., the range and level of detail of data fields). One of the challenges users face in the context of visual analytic tools is the cognitive load of formulating natural language queries based on their analytic queries.

[0062] Query reformulation is often based on data domains being too broad, too narrow, or formulated incorrectly. Users need guidance to understand whether they have found new insights through the returned visualizations. Lack of guidance can interfere with accurate perception of progress toward analytic goals. Previously, autocompletion in these natural language systems was fairly basic and tended to focus on syntactic completion of search queries without any suggestions or useful data previews. Autocompletion is needed in the context of visual analytics to support query formulation with data discovery to keep users "in the loop" to guide them in making relevance judgments.

[0063] To address this problem, some embodiments use a novel interface system called Sneak Pique. The goal is to introduce fluidity of on-the-fly suggestions into the typical analytic expressions of a visual analytics task. Some embodiments use a set of text-based and widget-based autocompletion suggestions that provide data previews of results before they are implemented in the visualization. In some embodiments, the system evaluates the user's query as it is being typed and provides semantically dynamic updates of data previews based on the syntactic structure and tokens of the query.

[0064] Some embodiments use autocompletion as a data discovery scaffolding to help users during their visual analytics workflow. In the simplest form of autocompletion, the interaction surfaces a pre-computed list of text suggestions to the user. Some embodiments extend this basic form to preview data "scent" of what the query will retrieve during visual analytics.

[0065] Some embodiments explore the novel design space of autocompletion variants to better understand user preferences for: (1) display of data previews, (2) ordering of suggestions, and / or (3) navigation of data hierarchies during natural language interaction for visual analytics.

[0066] Some embodiments use the autocompletion system (sometimes called Sneak Pique) as a design probe to implement various autocompletion variants from the design space. In some embodiments, the system employs a look-ahead parser to support basic syntactic completion of partial queries and dynamically suggest missing data values in relevant text-based and widget-based suggestions.

[0067] Some embodiments use data from crowdsourced studies of autocompletion variants to better understand user preferences and reduce the design possibility space of these variants. Some embodiments use survey results from crowdsourced studies to inform a subset of autocompletion variants. Some embodiments use insights from crowdsourced studies and / or historical usage data to design natural language input systems for visual analytics.

[0068] Autocompletion supporting visual analysis

[0069] The primary goal of auto-completion is to suggest a partially completed query with the intent of minimizing the time and effort of the user during the search task. There are multiple ways in which auto-completion can achieve this goal, and it can be categorized into three broad categories: (1) auto-completion that supports syntactic query formulation, (2) auto-completion that supports information recall and preview, and (3) auto-completion that supports visual analytics.

[0070] Query auto-completion (QAC) is prevalent in web search engines, desktop search, and mobile devices where typing is laborious and error-prone. QAC techniques are used to facilitate early typing of searches by providing possible suggestions that contain input characters from the query as prefixes to be matched as keywords. There are also studies that explore the utility of word and phrase level auto-completion. Systems have looked at ways to handle imprecision in search queries by developing fault-tolerant QAC or fuzzy type-ahead. Ranking algorithms for generating suggestions have looked at temporal information, personalization based on search behavior, and diversification to improve recall of users in search. While QAC techniques are useful for precision-oriented fact finding information needs, for exploratory search these QAC techniques tend to be less effective. There is additional complexity as information preview varies with the context of query tokens in play. Some implementations extend techniques developed for QAC to support dynamically guiding users in data exploration while formulating syntactically correct natural language utterances.

[0071] To support information recall and previewed autocomplete, some conventional systems show recall-oriented activities through information previews. Some systems (e.g., DIALOG) track query history and these queries are reused by reference. Some systems (e.g., VOIR) use a histogram of ranked information to show a document's retrieval history. Some systems (e.g., Ariadne) generate a visual representation of search trails to review earlier actions. Some systems (e.g., NRT) implement a history mechanism that records previous run queries so that a searcher can visually scan the result list for new documents. Some systems employ a search interface for data patterns that shows the number of records for each possible suggested attribute. Some systems (e.g., AutoG) show possible graphs when a user draws part of a query graph. Some systems show a query preview widget that provides a visual summary of results before executing a query. Such systems orient the user in the results page by tagging accessed content and highlighting changes in the search documents. Such conventional systems primarily focus on document search goals, which are different from visual analytic tasks.

[0072] Conventional natural language interfaces for data visualization provide limited text autocomplete without any preview of the underlying data. Other interfaces support query reformulation, where input utterances are translated into their corresponding canonical forms that represent the underlying system language. Some conventional systems use graphical user interface controls called "scented widgets" that can support data analytic tasks. Such systems augment conventional visualization widgets (like sliders, combo boxes, and radio buttons) with additional embedded visualizations to facilitate navigation in the information space. On the other hand, the Sneak Pique system described herein can be used as a design probe to examine how text and visual variants of autocomplete with data preview information can guide users in the context of natural language interaction for visual analytic tasks.

[0073] Sneak Pique system

[0074] Some implementations use a system called Sneak Pique that provides autocomplete suggestions with data preview information in a natural language interface during visual analytic exploration. Figure 3A A Sneak Pique system architecture 400 according to some implementations is shown. In some implementations, the system architecture 400 is implemented as the natural language interface 300 described above with reference to FIG. 3. The system architecture 400 includes a query parser 402, a query reformulator 404, a query engine 406, a data previewer 408, a data explorer 410, a query history 412, a query suggestion engine 414, a query suggestion database 416, a query suggestion interface 418, and a query suggestion display 420. Figure 2One or more modules in the memory 206 of the described device 200. According to some embodiments, the system employs a network-based client-server architecture. In some embodiments, an input query 422 (sometimes referred to as an incomplete natural language command, incomplete query, or query) is processed by a parser 402 (e.g., an ANTLR parser) using a grammar 404 (e.g., a context-free grammar) containing predefined rules as well as rules dynamically added based on data values from data fields of an underlying dataset. In some embodiments, the parser 402 accesses the underlying dataset (e.g., data from a database 428) through a data manager 424, which has access functions 426 to handle data requests. In some embodiments, as the user types, an autocomplete detection module 406 polls the query 422 and triggers (408) a parse tree error when the query is partially complete. In some embodiments, the parse tree error is passed to an autocomplete generator 410, which introspects the partial query's syntactic structure and the relevant grammar rules that would be satisfied if the query were complete. The generator 410 determines (412) the type of autocomplete needed to parse the partial query into a complete query. With the help of the data manager 424, in some embodiments, the generator 410 computes (414) the necessary data preview information that will be displayed (416) in the autocomplete suggestions. In some embodiments, the autocomplete suggestions are then rendered and displayed in the user interface of the client (e.g., a visualization 418). Any interaction by the user with these autocomplete suggestions is captured by an event manager 420. In some embodiments, the system updates the visualization results 430 based on the executed autocomplete query through an analysis module 422.

[0075] Autocompletion detection

[0076] In some embodiments, Sneak Pique uses an LL(*) parser that resolves from left to right, which performs the left-most derivation of an input search query. LL(*) parsers are used to generate autocomplete suggestions because such parsers can gracefully throttle up from a traditional fixed k > 1 token look-ahead to arbitrary look-ahead and can backtrack if a suitable parse path cannot be constructed. In some embodiments, the input to the parser is a grammar augmented with predicates that have corresponding look-ahead actions to trigger events sent to the suggestion module. In some embodiments, each grammar rule encapsulates an analysis intent similar to other natural language visual analysis grammar systems. The rules consist of static (i.e., pre-defined in the grammar) or dynamic (i.e., computed on-the-fly from data values of data fields in a database) lexicons. In linguistics, a lexicon is a vocabulary of words and phrases with known meanings. In some embodiments, the parser converts the input grammar to an equivalent augmented transition network (ATN), which is an efficient graph representation of a grammar for parsing relatively complex natural language queries. In some embodiments, a state model is computed using flow analysis, which traces through all nodes reachable from the top-level node of the ATN graph representation. Given a grammar G = (N, T, P, S, Π, μ), an ATN state machine A G = (Q, ∑, Λ, E, F) has five elements: (i) Q is the set of states, (ii) ∑ is the set of tokens NUTUΠUμ, (iii) Λ is the transition relation mapping Q x (∑ U ε) → Q, (iv) E = p A where A ∈ N is the set of entry states, and (v) F = p A where A ∈ N is the set of final states.

[0077] According to some embodiments, A G is computed for each non-terminal lexical element in the grammar. A state model is created for each ∑. Non-terminal symbols form the parsed syntactic structure and are replaced by terminal symbols (i.e., leaf nodes in the parse tree). A non-terminal edge p -+ p' is a function call based on Λ that will return a state p' pushed onto the parse state stack so that it can continue from p' after reaching a stop state in the state flow. In some embodiments, the parser simulates actions in the ATN to predict the next token in Q and compute a look-ahead parse tree. A prediction error occurs in the ATN when a sequence does not parse as a corresponding grammar production rule for the current non-terminal. The LL(*) prediction state model reports an error at a particular token and scans ahead to determine if there is any non-terminal that can parse the error. According to some embodiments, to trigger autocomplete, the shortest look-ahead sequence that will generate a valid parse tree is identified. In some embodiments, the autocomplete detection algorithm is generalized for static and dynamic lexicons.

[0078] In some implementations, the system 400 includes a look ahead parser 402 that includes a grammar 404 and an autocomplete detection module 406 that inspects the query as it is typed or entered by the user (e.g., the incomplete query 422). When the input query is incomplete, the parser 402 parses the query and generates a parse tree error 408 and / or possible parse trees for the input. An autocomplete generator module 410 determines (412) an autocomplete type based on the parse tree error 408 to resolve the partial / incomplete query. With the help of a data manager 424, the generator 410 computes (414) preview data and updates the autocomplete suggestions with data frequency information. The autocomplete suggestions are displayed (416) in the user interface (e.g., the visualization 418). The user can interact with the autocomplete and update the query to create a visualized response 430.

[0079] Figure 3B An example is shown in accordance with some implementations in which the parser 402 computes a look ahead parse tree and predicts the word pieces for triggering autocomplete. The green nodes 430, 432, 434, 436, 438, 440, 442, 444, and 446 depict grammar rules, the orange nodes 448, 450, 452, 454, 456, and 458 depict a lexicon, and the red nodes 460, 462, 464, and 466 indicate parser errors. As shown in the top 468 (labeled (a)), the autocomplete suggestion “correlation” 460 generates a valid parse for “show the cor” 472. For this example, the query “show the cor” 472 generates a parser error at the non-terminal node cor 462 because the word piece does not match any grammar production rule. The parser computes a look ahead to find the static lexicon correlation. The autocomplete generator uses the parse tree error information to render the suggestion. Figure 3B Figure 3B In the bottom 470 (labeled (b)) of the example, the user continues to type based on the underlying data semantics, and the country from the location data is suggested for “show the correlation for” 474. The input “show the correlation for” 474 results in a further down the parse tree error 466 because the query is missing the country name, which is a dynamically generated vocabulary from the data set. The parse tree error information provides input to the autocomplete generator to render the suggestion.

[0080] Autocompletion generation

[0081] ​After detecting when autocomplete is triggered during parsing, some embodiments determine how autocomplete suggestions should be presented in the interface. One key guiding principle for designing the autocomplete interface is recognition over recall: the idea that people are better at recognizing things they have experienced before than they are at recalling them from memory. Autocomplete also helps with information discovery in unfamiliar data, providing guidance when recall is not present.

[0082] Design patterns for autocompletion

[0083] Some embodiments use the following design patterns for autocomplete suggestions in Sneak Pique:

[0084] Providing suggestions in the context of partial search queries: In some embodiments, autocomplete helps users when searching by presenting items that match the user's input as they type. As the user types more text in the search field, the list of matching items shrinks.

[0085] Ordering sequence: In some embodiments, autocomplete orders the most relevant or most likely matches at the top of the list. This allows the user to quickly select a match.

[0086] Semantic grouping: In some embodiments, autocomplete groups similar items into categories to facilitate scanning and finding.

[0087] Reducing visual noise: In some embodiments, autocomplete avoids cognitive overload by only displaying information as a preview, rather than as a final result.

[0088] Manageable list: In some embodiments, autocomplete limits the number of matching items to display, especially when working with large data values.

[0089] Lightweight interactivity: In some embodiments, autocomplete supports lightweight interactivity, allowing the user to select a suggested item, saving time and keystrokes. Autocomplete also remains unobtrusive, so that the user can still type a complete query if they choose to do so.

[0090] Determination of autocompletion type

[0091] Some implementations combine the autocomplete design pattern with best practices for information visualization displays. Some implementations use a variety of text and widget-based autocomplete representations. Using Sneak Pique as a design probe, some systems implement autocomplete variants based on the data type of the missing dynamic term in the input query. To explore the utility of displaying data previews, some implementations generate these variants with or without data frequency numbers that indicate the frequency of the values in the dataset. Some implementations also generate a variety of ordering sequences based on the data type.

[0092] An example design space for autocomplete according to some implementations is shown below. Each row shows a variety of text and widget-based representations by data type. The data preview (DP) variants display data frequency numbers for the values. The example design space describes the following set of variants:

[0093] Categorical: Text list (or bar chart) shows suggestions for categorical data fields with a variety of ordering sequences. If there is a data preview (labeled “DP”), numbers are displayed in the list or encoded as bar lengths. Categorical data is often stored as strings that represent discrete values (e.g., products sold by a company).

[0094] Numerical: Text widget or slider widget shows the range of data values for a data field. Variants of the slider widget show a histogram encoding of data frequencies.

[0095] Geospatial: List widget or map chart widget shows location values. If there is a data preview, numbers are displayed in the list or as visualizations encoded on the map (i.e., color, size). For hierarchical data, nested list views or map widgets are provided to drill down (e.g., from country to city). The numbers displayed (when present) are data values for the data field in the database. Time: List widget or calendar widget shows times and / or time values. If there is a data preview, information is displayed in the list as numbers or in the calendar as color encoding. Based on the time intent, the calendar widget defaults to the appropriate level of detail. For example, “cases on” shows a calendar widget with a date view, while “cases in” shows a month view. For time ranges, the system displays a slider.

[0096] Semantic grouping: In natural language systems, geospatial and temporal intents can be ambiguous. For example, the query “show earthquakes in” can indicate a missing location or time. Some implementations resolve this ambiguity by semantically grouping geospatial values and temporal values in autocomplete suggestions as text lists or combinations of map and calendar widgets.

[0097] Some implementations employ an equal-area projection that tends to favor click interactions in small display areas. Hierarchical data is a tree-structured representation of data records.

[0098] In some implementations, the system also provides partial text matching with dynamic data values and static analytic concepts. For example, when the user types "ma", the system shows matching suggestions for "Massachusetts" and "maximum".

[0099] Computing data previews

[0100] In some implementations, the system computes the data frequency of data values for data fields associated with the predicted dynamic tokens and displays this information. In some implementations, the data preview generates dynamically updated data preview results based on the context of the current query. The system applies appropriate aggregation and filter analysis functions. In some implementations, the system checks for the presence of data fields in the query before generating data previews to prevent the addition of duplicates when interacting with the autocomplete widget.

[0101] Evaluating autocompletion variants

[0102] There are currently no standard guidelines for what the appropriate way to show autocomplete suggestions specifically designed for visual analytic would look like. It is unclear what the user's preferences are for each of these autocomplete variants and how these preferences vary based on data type, ordering sequence, or actual representation. Some implementations explore some of these features for what an appropriate autocomplete suggestion would look like for making reasonable defaults in a visual analytic natural language interface. Some implementations do not use objective measures of speed or accuracy since the relevant content is the user's preference for the most appropriate autocomplete variant. Often, there is a set of reliable design guidelines that most users agree with in certain situations.

[0103] Some implementations use several experiments to identify user preferences for various factors that affect the selection of autocomplete suggestions. The following describes various Mechanical Turk experiments performed using a system to identify a subset of autocomplete variants in accordance with some implementations. A first experiment compares autocomplete variants that show data frequencies and those that do not show data frequencies. A second experiment analyzes ordering sequence preferences. A third experiment analyzes user preferences for navigating hierarchical data. To further illustrate, in some implementations, for example, a first experiment compares autocomplete variants that show data frequency information and those that do not show data frequency information to understand whether such data previews are useful to users. A second experiment examines the type of ordering sequence that would be very useful to apply to items shown in text autocomplete suggestions. And, a third experiment compares autocomplete variants that show data values with hierarchies and those that do not show data values with hierarchies to better understand whether and in what form hierarchical data should be suggested to users. In some cases, participants found data preview information to be useful in all autocomplete variants. In some cases, participants preferred items ordered in descending order of their data frequencies because more prevalent data values should be shown higher in the suggestions. For time items, users preferred ordering in chronological order. In some cases, hierarchical text suggestions were easier to navigate than widgets. However, participants found calendar widgets helpful for navigating time hierarchies.

[0104] Example experimental design

[0105] For each experiment, participants were recruited from Amazon Mechanical Turk, a crowd-sourcing platform that is popular in many contemporary studies. Participants were English-speaking people in the United States with at least 95% acceptance rate and 500 approved tasks. The stimuli did not exclude participants with color deficiencies. Participants could only complete the experiment once to avoid biases that repeated exposure to the task might create. These experiments used a U.S. earthquake dataset that listed magnitudes by location and time because it is likely to be understood by the general public. The experimental steps were:

[0106] Training task: a graph showing scatter plots of life expectancy and income in different countries was presented along with practice questions to ensure understanding of the autocomplete suggestions.

[0107] Overall task description and instructions demonstration.

[0108] Actual task: The new page shows a search box with the content "find earthquakes [in / between]...". The participant is shown two image auto-completion options in random order and asked to select their top preferred option. A free-form text response box is provided to explain the preference.

[0109] Due to space constraints, in some experiments, only a set of sample stimuli for each experiment was shown to the user.

[0110] Experimental results

[0111] Overall, participants had a positive attitude towards the auto-completion interaction and identified many benefits. Sneak Pique allowed participants to introspect the data as they typed ("This is cool... provides me a way to see what I will get while I am typing my question" [P'1]), helped them proactively discover what was in the data ("By typing, I can already filter to a specific country and see what's in there without having to see the result and try again.... I don't have to shoot darts in the dark" [P'4]), and could save time ("I could finish the tasks really fast as the autocompletion guided me to see where to look" [P'7]). In the above descriptions, P'X indicates the participant ID in these studies, to be distinguished from the participant IDs in the Mechanical Turk study.

[0112] Part 1 - Targeted standard tasks

[0113] Six of the eight participants were able to successfully complete all tasks. Tasks were easier to complete when the data frequency information encoded in the autocomplete widget was visually discernible.

[0114] Part 2 - Open-ended tasks

[0115] This open-ended task demonstrated how autocomplete helped data discovery as users typed their queries. Participants demonstrated both syntactic query completion and autocomplete of filter expressions by location, time, and range. The number of individual queries per participant ranged from 8 to 23 (μ = 11.3), of which 46% were reformulations of previously typed queries by editing in place. Overall, a large number of partial queries used autocomplete with data preview to help resolve into complete queries (69%). Usage of widgets was roughly evenly distributed among the calendar (34%) widget, the slider (33%) widget, and the map (29%) widget. The remainder was text autocomplete for accessing hierarchical data. Comments related to this behavior included "That was convenient to type to an extent and rely on the calendar to go to a specific date that was interesting, I hate typing dates" [P'4] and "Getting a range right is a hit or miss for me. Helpful to see where most of the data is and pick with slider" [P'1].

[0116] This experimental evaluation of Sneak Pique confirmed the intuition that users find data previews in autocomplete useful when performing visual analytic tasks. The results showed that participants think more about the search terms when previews are present; they behave more meaningfully during query construction and when examining search results. Observations from this study provide the following design implications for how autocomplete can help visual analytic tasks and open new opportunities for future research:

[0117] Automated completion of various visual analytic workflows: The effect of task intervention during the first part of the study was that there were fewer query reformulations when participants used data previews to complete tasks. The second part of the experiment was more representative of real-world practice: users changed their strategy when formulating their queries. Participants used data previews as scaffolding to build compound queries, incrementally updating multiple filters in the original query. P'2 said, "I find it convenient to pile more filters in my question as the views in the autocompletion get updated... saves me time when I'm playing around." During data exploration, if the data preview was no longer interesting, participants would remove or clear these filter sub-expressions. For direct fact-finding questions, participants would type the question directly without the need for a data preview. However, the text autocompletion helped guide them as they typed the tokens. These observations show that autocompletion is used differently based on the type of query. In some implementations, the autocompletion workflow adapts to a range of tasks during visual analytics.

[0118] Supporting more complex previews

[0119] As described above, Sneak Pique implements some basic analytic functionality (e.g., filters and aggregations) through data previews. In some implementations, this functionality is extended to enable people with limited statistical knowledge to access complex analytics. For queries with geospatial and temporal intent, participants expect the map widget and calendar widget to coordinate consistently, and expect clicking on a country in the map to update its corresponding temporal preview in the calendar widget (and vice versa). Some implementations use data previews to indicate the system's understanding of semantic concepts. Some implementations balance the simplicity of the interaction and the more complex preview to serve a wider range of analytic questions. Some implementations balance adding functionality to autocompletion itself versus letting users explore results in the visualization. In some implementations, autocompletion behavior is designed to be high performance to support real-time interaction.

[0120] Show sources of autocompletion behavior

[0121] While most participants understood the purpose of data preview, they described usability issues regarding the availability of content understanding for autocomplete behavior based on context in the query. As one or more filters are run, data preview is dynamically updated to reflect the data domain in the context. This behavior was not always intuitive, either requiring the experimenter to clarify or the participant to eventually figure out the functionality after experimenting with disabled items in the widget. P'7 stated - "It would be good if I can see a message appearing saying that I am already looking at April and the data in the autocompletion is for that month." While the dynamic generation of autocomplete suggestions helps to provide live guidance to the user, some implementations proactively show this feedback in the interface to set proper user expectations.

[0122] Personalization of autocompletion suggestions

[0123] During the exploratory study, the theme of personalization of autocomplete behavior emerged. P'3, P'4, and P'8 expressed that they wanted autocomplete to keep track of their past interactions and update the default view. Some implementations monitor user interactions with SneakPique, log user queries, and update autocomplete preferences.

[0124] Autocomplete is a useful interaction paradigm for the sensemaking of information construction. As described above, SneakPique is a design probe to explore the space of autocomplete in the context of visual analytics and data discovery. According to some implementations, as described above with reference to Figure 3A and Figure 3B the system uses an efficient look-ahead parser to parse static tokens and dynamic data values for text and widget autocomplete variants. User study results indicate that data preview is indeed useful and inform how this information can be usefully presented in the interface. Some implementations use a preferred subset of autocomplete variants in the final implementation. In this way, autocomplete can be used as a data scaffold to help users make relevance judgments for visual analytics tasks. Some implementations strike a balance between lightweight interactivity and rich analytic previews.

[0125] The following describes design space after the system determines that data frequency pairs are useful (to show to the user) according to some embodiments. Participants preferred sorting data by data frequency pairs, and sorting time (or temporal) data in chronological order. Participants also preferred map and histogram sliders for geospatial data and numerical ranges, respectively. Participants also preferred text auto-completion with semantic grouping for displaying hierarchical data. Participants also preferred a calendar widget for navigating temporal data. The map and calendar widgets were used together to complete queries using time and place values. Some embodiments use the results obtained from these experiments to determine preferred or default variants for different types of data.

[0126] Widgets were often used for time, place, and range selection, while text was used for syntactic auto-completion. Some embodiments also explored (e.g., tradeoffs of) complex previews and personalization, statistical previews, multiple selection of values, and saving custom preferences.

[0127] Figures 4A-4L A flowchart of a method 900 of utilizing (902) data discovery to support visual analysis of a dataset according to some embodiments is provided. The method is performed (904) at a computing system (e.g., system 200) having one or more processors 202 and memory 206 storing (906) one or more programs configured for execution by the one or more processors. The method includes receiving (908) a user selection of a data source. The method also includes presenting (910) a natural language interface for analyzing data from the selected data source. The method also includes receiving (912) user input to specify an incomplete natural language command directed to the data source.

[0128] The method also includes associating (914) one or more words in the incomplete natural language command with data fields in the selected data source.

[0129] Reference is next made to Figure 4CIn some implementations, associating the one or more words in the incomplete natural language command with the data fields in the selected data source includes generating (922) grammar rules based on data values of the data fields in the selected data source. In some implementations, the parsing of the incomplete natural language command is performed (928) using a left-to-right (LL(*)) parser that performs left-most derivation of the incomplete natural language command and backtracks in case a suitable parse path cannot be constructed. In some implementations, the method determines (926) whether the one or more words in the incomplete natural language command include a partially complete query by parsing the incomplete natural language command using the grammar rules and detecting a syntax parse tree error. In some implementations, the grammar rules correspond to (924) a context-free grammar that includes pre-defined rules and rules dynamically generated based on data values of the data fields.

[0130] When the incomplete natural language command includes a partially complete query, the method performs (930): (a) computing (932) a syntactic structure of the partially complete query and related grammar rules that would be satisfied if the partially complete query were completed; and (b) selecting (934) data fields from the selected data source based on the related grammar rules. In some implementations, the method further includes displaying (936) one or more autocomplete suggestions needed to parse the partially complete query into a complete natural language command based on the related grammar rules. In other words, in addition to generating appropriate visualizations by associating incomplete natural language commands with underlying data sets, some implementations also provide text autocompletion that does not involve underlying data fields (e.g., based on common language usage and / or based on historical usage data).

[0131] Referring next to Figure 4D In some implementations, associating the one or more words in the incomplete natural language command is based on (938) analyzing user preferences for text autocompletion or widget autocompletion of different data types, including categorical data, numerical data, geospatial data, temporal data, and combinations thereof. In some implementations, the method further includes: (i) detecting (940) a change in user preference in a series of tasks during the visualization analysis; and (ii) associating (942) the one or more words in the incomplete natural language command further based on adapting the autocompletion workflow to the change in different data types.

[0132] Referring back to Figure 4B The method further includes determining (916) a data type of the data fields and a range of data values in the data source with respect to the data fields.

[0133] The method also includes presenting (918) one or more autocomplete options for the incomplete natural language command, according to the data type and the range of data values. Each option includes respective text and a respective corresponding visual graphic.

[0134] Referring next Figure 4E In some implementations, when the data type is a categorical type, the method displays (944) a list of text that represents suggestions for the categorical data field. The suggestions are numerically ordered (944) according to data frequency, and the list of text includes a data preview that shows respective data frequencies of data values of the data field.

[0135] Referring next Figure 4F In some implementations, when the data type is a numerical type, the method displays (946) a histogram slider widget that shows a histogram of data frequencies of the numerical data field.

[0136] Referring next Figure 4G In some implementations, when the data type is a geospatial type, the method displays (948) a map chart widget that shows a data preview of location values of the data field, showing numerical ranges as visualized encodings.

[0137] Referring next Figure 4H In some implementations, when the data type is a temporal type, the method displays (950) (i) a list of text that shows temporal data ordered by time, (ii) a calendar widget that shows a temporal hierarchy of a first temporal data field, or (iii) a histogram slider widget that shows a histogram of data frequencies of a second temporal data field name. Referring next Figure 4I In some implementations, when the data type is both a geospatial type and a temporal type, the method displays (952) (i) a list of text with semantic grouping for hierarchical data, or (ii) a combination of a map and calendar widgets.

[0138] Referring next Figure 4J In some implementations, when the data type corresponds to hierarchical data, the method provides (954) one or more text autocomplete options for selecting a hierarchical value.

[0139] Referring next Figure 4KIn some implementations, presenting the one or more autocomplete options for the incomplete natural command includes: (i) detecting (956) user input to select respective text for the data field instead of the respective corresponding visual graphic; and (ii) in response to detecting the user input to select the respective text, switching (958) from displaying the respective corresponding visual graphic to displaying the respective text. The respective text provides the one or more text autocomplete options.

[0140] Referring next Figure 4L In some implementations, presenting the one or more autocomplete options for the incomplete natural command includes displaying (960) one or more widgets that illustrate a data preview. In some implementations, the one or more widgets are triggered (962) based on semantics of the tokens obtained from parsing the incomplete natural language command. In some implementations, the method further includes: (i) receiving (964) a second user input to specify a second incomplete natural language command directed to the data source; and (ii) computing and displaying (966) an update to the data preview based on the second user input and the filters used in the data visualization. Some implementations store or retain the context of a previous visualization and reuse the context in future visualizations. For example, when the user types "deaths," some implementations understand that the user refers to the number of deaths related to the location and / or time shown in the current visualization and update the visualization accordingly. To further illustrate, different portions of a world map are populated with data or highlighted in accordance with some implementations.

[0141] In some implementations, the method further includes: (i) prior to receiving the second user input, storing (968) an initial state of the data visualization; (ii) receiving (970) a third user input to specify a third incomplete natural language command directed to the data source; and (iii) when the third incomplete natural language command corresponds to a user request to reset the data visualization, restoring (972) the data visualization based on the initial state of the data visualization.

[0142] Referring back to Figure 4B The method further includes, in response to user selection of an option of the one or more autocomplete options, forming (920) a complete natural language command and displaying the data visualization according to the complete natural language command.

[0143] The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the description of the application and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or," as used herein, refers to any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0144] The foregoing description has been presented for the purpose of illustration. However, the illustrative discussions are not intended to be exhaustive or to limit the application to the precise forms disclosed. Many modifications and variations are possible in light of the above teaching. Embodiments were chosen and described in order to best explain the principles of the application and its practical application to thereby enable others skilled in the art to best utilize the application and various embodiments with various modifications as are suited to the particular use contemplated.

Claims

1. A method for using data discovery to support the visualization analysis of datasets, comprising: In a computing system having one or more processors and memory configured to execute one or more programs by said one or more processors: Receive the user's selection of a data source; Presents a natural language interface for analyzing data from the selected data source; Receive user input to specify an incomplete natural language command pointing to the data source; Associate one or more words of the incomplete natural language command with a data field in the selected data source; Determine the data type of the data field and the range of data values ​​for the data field in the data source; Based on the data type and the range of the data values, one or more autocomplete options are presented for the incomplete natural language command, including displaying one or more widgets showing a preview of the data, wherein each option includes corresponding text and a corresponding visual graphic; as well as In response to the user's selection of one or more autocomplete options, a complete natural language command is generated and a data visualization is displayed based on the complete natural language command.

2. The method according to claim 1, wherein, Associating one or more words in the incomplete natural language command with the data field in the selected data source includes: Generate syntax rules based on data values ​​from data fields in the selected data source; The method involves parsing the incomplete natural language command using the stated syntax rules and detecting parsing tree errors to determine whether one or more words in the incomplete natural language command constitute a partially complete query; and Based on the determination that the incomplete natural language command includes a partially complete query: Calculate the syntactic structure of the partially complete query and the relevant grammatical rules that will be satisfied when the partially complete query is completed; and The data field is selected from the chosen data source based on the relevant syntax rules.

3. The method according to claim 2, further comprising: Based on the relevant syntax rules, one or more autocomplete suggestions are displayed to parse the partially complete query into the complete natural language command.

4. The method according to claim 2, wherein, The incomplete natural language command is parsed using a left-to-right (LL(*)) parser, which performs the leftmost derivation of the incomplete natural language command and backtracks if a suitable parsing path cannot be constructed.

5. The method according to claim 2, wherein, The syntax rules correspond to context-free syntax, which includes predefined rules and rules dynamically generated based on the data values ​​of the data fields in the dataset.

6. The method according to claim 1, wherein, The association of one or more words in the incomplete natural language command is based on the analysis of user preferences for text autocomplete or widget autocomplete of different data types, including categorical data, numerical data, geospatial data, temporal data, and combinations thereof.

7. The method according to claim 6, further comprising: Detect changes in user preferences across a range of tasks during visualization analytics; as well as Furthermore, the incomplete natural language command is associated with one or more words based on adapting the autocomplete workflow to changes in the different data types.

8. The method according to claim 1, wherein, Presenting one or more autocomplete options for the incomplete natural language command includes: Based on the determination that the data type is a categorical type, a text list showing suggestions for the categorical data field is displayed, wherein the suggestions are numerically sorted according to data frequency, and the text list includes a data preview showing the corresponding data frequency of the suggestions.

9. The method according to claim 1, wherein, Presenting one or more autocomplete options for the incomplete natural language command includes: Based on the determination that the data type is numeric, a histogram slider widget is displayed, which shows a histogram of the data frequency of the numeric data field.

10. The method according to claim 1, wherein, Presenting one or more autocomplete options for the incomplete natural language command includes: Based on the determination that the data type is a geospatial type, a map chart widget is displayed, which shows a data preview of the location values ​​of the data fields and displays the numerical range as a visual encoding.

11. The method according to claim 1, wherein, Presenting one or more autocomplete options for the incomplete natural language command includes: Based on the determination that the data type is time type, the display includes (i) a text list of time data sorted in chronological order, (ii) a calendar widget showing the time hierarchy of the first time data field, or (iii) a histogram slider widget showing the histogram of the data frequency of the second time data field.

12. The method according to claim 1, wherein, Presenting one or more autocomplete options for the incomplete natural language command includes: Depending on whether the data type is both a geospatial and temporal type, display (i) a text list with semantic groupings for hierarchical data, or (ii) a combination of a map and a calendar widget.

13. The method according to claim 1, wherein, The one or more widgets are triggered based on the semantics of the lexical units obtained from parsing the incomplete natural language command.

14. The method according to claim 1, further comprising: Receive second user input to specify a second incomplete natural language command pointing to the data source; as well as Updates to the data preview are calculated and displayed based on the second user input and the filters used in the data visualization.

15. The method of claim 14, further comprising: The initial state of the data visualization is stored before receiving the second user input; Receive third user input to specify a third incomplete natural language command pointing to the data source; as well as Based on the determination of the user request to reset the data visualization corresponding to the third incomplete natural language command, the data visualization is restored based on its initial state.

16. The method according to claim 1, wherein, Presenting one or more autocomplete options for the incomplete natural language command includes: Detecting user input for selecting the corresponding text instead of the corresponding visual graphic for the data field; and In response to detecting user input that selects the corresponding text, the display switches from displaying the corresponding visual graphic to displaying the corresponding text, wherein the corresponding text provides one or more text autocomplete options.

17. The method according to claim 1, wherein, Presenting one or more autocomplete options for the incomplete natural language command includes: Based on the determination of the hierarchical data corresponding to the data type, one or more text autocomplete options are provided for selecting hierarchical values.

18. A computer system for supporting visual analysis of datasets using data discovery, comprising: One or more processors; as well as Memory; The memory stores one or more programs configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the methods according to claims 1-17.

19. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by a computer system having a display, one or more processors and a memory, the one or more programs comprising instructions for performing any one of the methods according to claims 1-17.

Images