A method for using natural language for visual analysis of a non-transient computer-readable data set, electronic device, and storage media

Abstract

This is a method that uses natural language processing for visual analysis of a dataset by a computer. The computer displays a data visualization based on a dataset retrieved from a database using an initial set of one or more database queries. The computer receives user input (e.g., keyboard or voice) to specify a natural language command related to the displayed data visualization. Based on the displayed data visualization, the computer extracts one or more independent analytical phrases from the initial natural language command. The computer additionally calculates a set of one or more conversation centers associated with the natural language command based on the set of one or more analytical phrases.The computer then calculates a set of analytical functions associated with the set of one or more conversation centers, thereby creating a set of one or more functional phrases. The device then updates the data visualization based on the set of one or more functional phrases.

Description

1 / 108 “METHOD FOR USING NATURAL LANGUAGE FOR VISUAL ANALYSIS OF A NON-TRANSIENT COMPUTER-READABLE DATA SET, ELECTRONIC DEVICE AND STORAGE MEDIA” TECHNICAL FIELD

[001] The implementations revealed relate, in general, to data visualization, and more specifically, to systems, methods, and interfaces that allow users to interact with and explore datasets using a natural language interface. BACKGROUND OF THE INVENTION

[002] Data visualization applications allow a user to visually understand a dataset, including distribution, trends, discrepancies, and other factors that are important in business decision-making. Some datasets are very large or complex, and include many data fields. Various tools can be used to help understand and analyze the data, including dashboards that can have multiple data visualizations. However, it can be difficult to use or find certain functionalities within a complex user interface. Most systems only return fairly basic interactive visualizations in response to queries, and others require advanced modeling to create effective queries. Still other systems require simple closed-ended questions, and only then are they able to return a single text response or a static visualization. SUMMARY

[003] Consequently, there is a need for tools that allow users to effectively utilize the functionality offered by data visualization applications. One solution to the problem is to offer a natural language interface as part of an application. Petition 870250038578, dated 12 / 05 / 2025, page 7 / 233 2 / 108 Data visualization (e.g., within the user interface for the data visualization application) for an interactive query dialog that provides graphical answers to natural language queries. The natural language interface allows users to access complex functionality using common questions or commands. Questions and insights often arise from previous questions and data patterns that a person sees. By modeling interaction behavior as a conversation, the natural language interface can apply pragmatic principles to enhance interaction with visual analytics. Through various techniques for deducing the grammatical and lexical structure of utterances and their context, the natural language interface supports various pragmatic forms of natural language interaction with visual analytics.These pragmatic approaches include understanding incomplete statements, alluding to entities within statements and visualization properties, supporting long and compound statements, identifying synonyms and related concepts, and "repairing" responses to previous statements. Additionally, natural language interfaces provide appropriate visualization responses both within an existing visualization and by creating new visualizations when necessary, and resolve ambiguity through targeted textual feedback and ambiguity widgets (tools). In this way, natural language interfaces allow users to efficiently explore the displayed data (e.g., in a data visualization) within the data visualization application.

[004] According to some implementations, a method is executed on an electronic device with a display medium. For example, the electronic device could be a smartphone, a tablet, a notebook, a computer, or a desktop computer. The device displays a data visualization based on a dataset retrieved from a database. Petition 870250038578, dated 12 / 05 / 2025, page 8 / 233 3 / 108 data using a first set of one or more database queries. A user specifies a first natural language command related to the displayed data visualization. Based on the displayed data visualization, the device extracts a first set of one or more independent analytical phrases from the first natural language command. The device then calculates a first set of one or more conversation centers associated with the first natural language command based on the first set of one or more analytical phrases. The device then calculates a first set of analytical functions associated with the first set of one or more conversation centers, thus creating a first set of one or more functional phrases. The device then updates the data visualization based on the first set of one or more functional phrases.

[005] In some implementations, the device receives a second natural language command related to the updated data visualization. After receiving the second natural language command, the device extracts a second set of one or more independent analytic phrases from the second natural language command, and calculates a temporary set of one or more conversation centers associated with the second natural language command based on the second set of one or more analytic phrases, according to some implementations. The device then derives a second set of one or more conversation centers from the first set of one or more conversation centers and the temporary set of one or more conversation centers using one or more transition rules.The device calculates a second set of one or more analytical functions associated with the second set of one or more conversation centers, thereby creating a second set of one or more functional phrases. The device then updates the data visualization based on the second set of one or more functional phrases. Petition 870250038578, dated 12 / 05 / 2025, page 9 / 233 4 / 108

[006] In some implementations, each of the conversation hubs in the first set of one or more conversation hubs, the temporary set of one or more conversation hubs, and the second set of one or more conversation hubs comprises a value for a variable (for example, a data attribute or a view property).In such implementations, the device uses transition rules by performing a sequence of operations that includes: determining whether a first variable is included in the first set of one or more conversation centers; determining whether the first variable is included in the temporary set of one or more conversation centers; determining a respective transition rule from one or more transition rules to be applied based on whether the first variable is included in the first set of one or more conversation centers and / or in the temporary set of one or more conversation centers; and applying the respective transition rule.

[007] In some implementations, one or more transition rules used by the device comprise a CONTINUE rule to include each conversation center in the first set of one or more conversation centers in the second set of one or more conversation centers, and to add one or more conversation centers from the temporary set of one or more conversation centers to the second set of one or more conversation centers.

[008] In some of these implementations, the device applies the respective transition rule by performing a sequence of operations which comprises: according to a determination that (i) the first variable is included in the temporary set of one or more conversation centers, and (ii) that the first variable is not included in the first set of one or more conversation centers, the device applies the CONTINUE rule to include the first variable in the second set of one or more conversation centers.

[009] In some implementations, one or more transition rules Petition 870250038578, dated 12 / 05 / 2025, page 10 / 233 5 / 108 used by the device comprise a RETAIN rule to retain each conversation center in the first set of one or more conversation centers in the second set of one or more conversation centers without adding any conversation centers from the temporary set of one or more conversation centers to the second set of one or more conversation centers.

[010] In some of these implementations, the device applies the respective transition rule by performing a sequence of operations which comprises: according to a determination that (i) the first variable is included in the first set of one or more conversation centers, and (ii) that the first variable is not included in the temporary set of one or more conversation centers, apply the RETAIN rule to include each conversation center in the first set of one or more conversation centers in the second set of one or more conversation centers.

[011] In some implementations, one or more transition rules used by the device comprise a MOVE rule to include each conversation center in the first set of one or more conversation centers in the second set of one or more conversation centers, and replace one or more conversation centers in the second set of one or more conversation centers with conversation centers in the temporary set of one or more conversation centers.

[012] In some of these implementations, the device applies the respective transition rule by performing a sequence of operations comprising: according to a determination that (i) the first variable is included in the first set of one or more conversation centers, and (ii) that the first variable is included in the temporary set of one or more conversation centers: determining whether a first value of the first variable in the first set of one or more conversation centers is different from a second value of the Petition 870250038578, dated 12 / 05 / 2025, page 11 / 233 6 / 108 first variable in the temporary set of one or more conversation centers; according to a determination that the first value is different from the second value, apply the SHIFT rule to include each conversation center in the first set of one or more conversation centers in the second set of one or more conversation centers, and replace the value for the first variable in the second set of one or more conversation centers with the second value.

[013] In some of these implementations, the device additionally determines whether a widget corresponding to the first variable has been removed by the user; and, according to a determination that the widget has been removed, applies the SHIFT rule to include each conversation hub in the first set of one or more conversation hubs in the second set of one or more conversation hubs, and replaces the value for the first variable in the second set of one or more conversation hubs with a new value (for example, a maximum value, or a superset value) that includes the first value.

[014] In some implementations, the device creates a first set of one or more queries based on the first set of one or more functional phrases, and repeats the database query using the first set of one or more queries, thus retrieving a second set of data, and then displays an updated data view using the second set of data. In some implementations, the device creates a second set of one or more queries based on the second set of one or more functional phrases, and repeats the database query using the second set of one or more queries, thus retrieving a third set of data, and then displays an updated data view using the third set of data. In some cases, the database query repetition is performed locally on the computing device using data stored or cached on the computing device.For example, repeating the query is usually... Petition 870250038578, dated 12 / 05 / 2025, page 12 / 233 7 / 108 performed locally when the natural language command specifies one or more filters. In some implementations, the device additionally creates and displays a new data visualization using the second dataset or the third dataset.

[015] In some implementations, the device additionally determines whether the user has selected a dataset different from the first dataset, or whether the user has reset the data view, and in this case, resets each of the first set of one or more conversation centers, the temporary set of one or more conversation centers, and the second set of one or more conversation centers to an empty set that does not include any conversation centers.

[016] Typically, an electronic device includes one or more processors, memory, a display medium, and one or more programs stored in memory. The programs are configured for execution by the one or more processors and are configured to perform any of the methods described herein. The one or more programs include instructions to display a data visualization based on a first set of data retrieved from a database using a first set of one or more queries. The one or more programs also include instructions to receive a first user input to specify a first natural language command related to the data visualization. The one or more programs also include instructions to extract a first set of one or more analytical phrases from the first natural language command.One or more programs also include instructions for calculating a first set of one or more conversation centers associated with the first natural language command based on the first set of one or more analytic phrases. One or more programs also include instructions for calculating a first set of analytic functions. Petition 870250038578, dated 12 / 05 / 2025, page 13 / 233 8 / 108 associated with the first set of one or more conversation centers, thus creating a first set of one or more functional phrases, and updating the data visualization based on the first dataset of one or more functional phrases.

[017] In some implementations, one or more programs include instructions to receive a second user input to specify a second natural language command related to the updated data visualization. One or more programs also include instructions to extract a second set of one or more analytic phrases from the second natural language command. One or more programs also include instructions to calculate a temporary set of one or more conversation centers associated with the second natural language command based on the second set of one or more analytic phrases. One or more programs also include instructions to derive a second set of one or more conversation centers from the first set of one or more conversation centers and the temporary set of one or more conversation centers using one or more transition rules.One or more programs may also include instructions for calculating a second set of one or more analytical functions associated with the second set of one or more conversation centers, thereby creating a second set of one or more functional phrases, and updating the data visualization based on the second set of one or more functional phrases.

[018] In some implementations, a non-temporary, computer-readable storage medium stores one or more programs configured for execution by a computing device containing one or more processors, memory, and a display medium. The one or more programs are configured to perform any of the methods described herein. The one or more programs include instructions for displaying a data visualization based on Petition 870250038578, dated 12 / 05 / 2025, page 14 / 233 9 / 108 A first set of data retrieved from a database using a first set of one or more queries. The one or more programs also include instructions to receive a first user input to specify a first natural language command related to data visualization. The one or more programs also include instructions to extract a first set of one or more analytical phrases from the first natural language command. The one or more programs also include instructions to calculate a first set of one or more conversation centers associated with the first natural language command based on the first set of one or more analytical phrases.One or more programs also include instructions for calculating a first set of analytical functions associated with the first set of one or more conversation centers, thus creating a first set of one or more functional phrases, and updating the data visualization based on the first data set of one or more functional phrases.

[019] In some implementations, one or more programs include instructions to receive a second user input to specify a second natural language command related to the updated data visualization. One or more programs also include instructions to extract a second set of one or more analytic phrases from the second natural language command. One or more programs also include instructions to calculate a temporary set of one or more conversation centers associated with the second natural language command based on the second set of one or more analytic phrases. One or more programs also include instructions to derive a second set of one or more conversation centers from the first set of one or more conversation centers and the temporary set of one or more conversation centers using one or more transition rules. One or more programs also include instructions to calculate a second Petition 870250038578, dated 12 / 05 / 2025, page 15 / 233 10 / 108 a set of one or more analytical functions associated with the second set of one or more conversation centers, thereby creating a second set of one or more functional phrases, and updating the data visualization based on the second set of one or more functional phrases.

[020] In another aspect, according to some implementations, the device displays a data visualization based on a dataset retrieved from a database using a first set of one or more database queries. A user specifies a first natural language command related to the displayed data visualization. Based on the displayed data visualization, the device extracts a first set of one or more independent analytic phrases from the first natural language command. The device then calculates a first set of one or more conversation centers associated with the first natural language command based on the first set of one or more analytic phrases. The device then calculates a first set of analytic functions associated with the first set of one or more conversation centers, thus creating a first set of one or more functional phrases.The device then updates the data visualization based on the first set of one or more functional phrases. The user specifies a second natural language command related to the updated data visualization. After receiving the second natural language command, the device extracts a second set of one or more independent analytical phrases from the second natural language command, and calculates a temporary set of one or more conversation centers associated with the second natural language command based on the second set of one or more analytical phrases. The device then calculates the cohesion between the first set of one or more analytical phrases and the second set of one or more analytical phrases. The device then derives a second set of one or more centers of... Petition 870250038578, dated 12 / 05 / 2025, page 16 / 233 11 / 108 conversation from the first set of one or more conversation centers and the temporary set of one or more conversation centers based on cohesion. The device calculates a second set of one or more analytical functions associated with the second set of one or more conversation centers, thus creating a second set of one or more functional phrases. The device then updates the data visualization based on the second set of one or more functional phrases.

[021] In some implementations, the device calculates cohesion and derives the second set of one or more conversation centers by performing a sequence of operations comprising: identifying a phrase structure from the second set of one or more analytic phrases; identifying one or more pragmatic forms based on the phrase structure; and deriving a second set of one or more conversation centers from the first set of one or more conversation centers and the temporary set of one or more conversation centers based on one or more pragmatic forms.

[022] In some of these implementations, the device identifies the sentence structure by performing a sequence of operations comprising: parsing the second natural language command by applying a probabilistic grammar, thereby obtaining parsed output; and resolving the parsed output into corresponding data and categorical attributes. In some such implementations, parsing the second natural language command further comprises deducing the syntactic structure using a morphosyntactic parsing API (“partof-speech”) (e.g., a morphosyntactic tagger (“POS tagger”)) provided by a natural language processing toolkit.

[023] In some implementations, the device identifies one or more forms of pragmatics by performing a sequence of operations comprising Petition 870250038578, dated 12 / 05 / 2025, page 17 / 233 12 / 108 determine whether the second natural language command is an incomplete utterance (sometimes called an ellipsis) by determining whether one or more linguistic elements are missing from the sentence structure. In some of these implementations, the device derives the second set of one or more conversation centers by performing a sequence of operations that includes: according to the determination that the second natural language command is an incomplete utterance: determining a first subset of conversation centers in the first set of one or more conversation centers, the first subset of conversation centers corresponding to the one or more linguistic elements missing from the sentence structure; and calculating the second set of one or more conversation centers by combining the temporary set of one or more conversation centers with the first subset of conversation centers.

[024] In some implementations, the device identifies one or more forms of pragmatics by performing a sequence of operations that includes determining whether the second natural language command is a reference expression by determining whether one or more anaphoric references are present in the sentence structure; and the device derives the second set of one or more conversation centers by performing another sequence of operations that includes: according to the determination that the second natural command is a reference expression: searching the first set of one or more conversation centers to find a first subset of conversation centers that corresponds to a sentence block in the second natural language command that contains a first anaphoric reference from one or more anaphoric references;and calculate the second set of one or more conversation centers based on the temporary set of one or more conversation centers and the first subset of conversation centers.

[025] In some such implementations, the device additionally Petition 870250038578, dated 12 / 05 / 2025, page 18 / 233 13 / 108 determines whether the first anaphoric reference is accompanied by a verb in the second natural language command, and, if so, searches the first set of one or more conversation centers to find a first action conversation center that refers to an action verb (e.g., “filter”); and calculates the second set of one or more conversation centers based on the temporary set of one or more conversation centers, the first subset of conversation centers, and the first action conversation center.

[026] In some of these implementations, the device determines whether the first anaphoric reference is a deictic reference that refers to some object in the environment, typically by indication, and in this case, calculates the second set of one or more conversation centers based on the temporary set of one or more conversation centers, and a characteristic of the object. Deictic references are typically enabled through multimodal interaction (e.g., through the use of a mouse in addition to speech or text).

[027] In some of these implementations, the device additionally determines whether the first anaphoric reference is a reference to a view property in the updated data view, and in this case, calculates the second set of one or more conversation centers based on the temporary set of one or more conversation centers, and data related to the view property.

[028] In some implementations, the device identifies one or more forms of pragmatics by performing a sequence of operations that includes determining whether the second natural language command is a repair utterance, determining whether the sentence structure corresponds to one or more predefined repair utterances (say, to repair a possible ambiguity in the first natural language command or the form that the Petition 870250038578, dated 12 / 05 / 2025, page 19 / 233 14 / 108 results are presented to the user). For example, the user says “get rid of the condominium” or “move from condominium to house”. In such implementations, if the device determines that the second natural language command is a repair utterance, the device calculates the second set of one or more conversation centers based on the temporary set of one or more conversation centers; and updates one or more data attributes in the second set of one or more conversation centers based on the one or more predefined repair utterances and sentence structure.

[029] In some such implementations, the device determines whether the sentence structure matches a repair statement to change a default behavior related to the display of a data visualization (for example, highlighting for selection, such as in response to “no filter, instead”), and, in this case, the device changes the default behavior related to the display.

[030] In some implementations, the device identifies one or more forms of pragmatics by performing a sequence of operations comprising determining whether the second natural language command is a conjunctive expression (i) determining the implicit or explicit presence of conjunctions in the sentence structure, and (ii) determining whether the temporary set of one or more conversation centers includes each conversation center in the first set of one or more conversation centers. In such implementations, the device derives the second set of one or more conversation centers by performing another set of operations comprising: according to the determination that the second natural language command is a conjunctive expression, calculating the second set of one or more conversation centers based on the temporary set of one or more conversation centers.

[031] In some such implementations, the device determines whether the Petition 870250038578, dated 12 / 05 / 2025, p. 20 / 233 15 / 108 The second natural language command has more than one set; and, according to the determination that the second natural language command has more than one set, the device calculates the second set of one or more analytic functions by linearizing the second natural language command.In some of these implementations, the device linearizes the second natural language command by performing a sequence of operations that includes: generating a parse tree for the second natural language command; traversing the parse tree in post-order to extract a first analytic phrase and a second analytic phrase, where the first analytic phrase and the second analytic phrase are adjacent nodes in the parse tree; calculating a first analytic function and a second analytic function corresponding to the first analytic phrase and the second analytic phrase, respectively; and combining the first analytic function with the second analytic function by applying one or more logical operators based on one or more features of the first analytic function and the second analytic function, where one or more features include attribute type, operator type, and a value.

[032] In some such implementations, the first analytic function comprises a first attribute (sometimes called a variable here, and includes a display property), a first operator, and a first value; the second analytic function comprises a second attribute (sometimes called a variable here, and includes a display property), a second operator, and a second value.

[033] In some such implementations, combining the first analytic function with the second analytic function comprises: determining whether the first attribute is a categorical attribute or an ordered attribute, and determining whether the second attribute is a categorical attribute or an ordered attribute; determining whether the first attribute and the second attribute are identical; Petition 870250038578, dated 12 / 05 / 2025, page 21 / 233 16 / 108 and, according to a determination that the first attribute and the second attribute are identical and both are attributes of categorical type, apply a union operator to combine the first analytic function and the second analytic function.

[034] In some such implementations, combining the first analytic function with the second analytic function comprises: determining whether the first attribute is a categorical attribute or an ordered attribute, and determining whether the second attribute is a categorical attribute or an ordered attribute; determining whether the first attribute and the second attribute are identical; and, according to a determination that the first attribute and the second attribute are not identical, applying the intersection operator to combine the first analytic function and the second analytic function.

[035] In some such implementations, combining the first analytic function with the second analytic function comprises: determining whether the first attribute is a categorical attribute or an ordered attribute, and determining whether the second attribute is a categorical attribute or an ordered attribute; determining whether the first attribute and the second attribute are identical; and, according to a determination that the first attribute and the second attribute are identical, and both are ordered attributes: determining the operator types of the first operator and the second operator; and, according to a determination that both the first attribute and the second attribute are equality operators, applying the union operator to combine the first analytic function and the second analytic function.

[036] In some such implementations, combining the first analytic function with the second analytic function comprises: determining whether the first attribute is a categorical attribute or an ordered attribute, and determining whether the second attribute is a categorical attribute or an ordered attribute; determining whether the first attribute and the second attribute are identical; Petition 870250038578, dated 12 / 05 / 2025, p. 22 / 233 17 / 108 and, according to a determination that the first attribute and the second attribute are identical, and both are attributes of the ordered type: determine the operator types of the first operator and the second operator; and, according to a determination that the first operator is a “less than” operator and the second operator is a “greater than” operator: determine if the first value is less than the second value; and, according to a determination that the first value is less than the second value, apply the union operator to combine the first analytic function and the second analytic function.

[037] In some such implementations, combining the first analytic function with the second analytic function comprises: determining whether the first attribute is a categorical attribute or an ordered attribute, and determining whether the second attribute is a categorical attribute or an ordered attribute; determining whether the first attribute and the second attribute are identical; and, according to a determination that the first attribute and the second attribute are identical, and both are ordered attributes: determining the operator types of the first operator and the second operator; and, according to a determination that the first operator is a “greater than” operator and the second operator is a “less than” operator: determining whether the first value is less than the second value;And, based on the determination that the first value is less than the second value, apply the intersection operator to combine the first analytic function and the second analytic function.

[038] In some implementations, the device additionally calculates the semantic relationship between the second set of one or more extracted analytical phrases and one or more data attributes included in the updated data visualization, and calculates analytical functions associated with the second set of one or more analytical phrases, thus creating the second set of one or more functional phrases, based on the semantic relationship of one or more attributes of Petition 870250038578, dated 12 / 05 / 2025, page 23 / 233 18 / 108 data points. Unlike grammatical cohesion or cohesion between contexts, lexical cohesion seeks cohesion within the context.

[039] In some of these implementations, the device calculates semantic relatedness by performing a sequence of operations comprising: training a first neural network model on a large text corpus, thereby learning word embeddings; calculating a first word vector for a first word in a first sentence in the second set of one or more analytical sentences using a second neural network model, the first word vector mapping the first word to the word embeddings; calculating a second word vector for a first data attribute in one or more data attributes using the second neural network model, the second word vector mapping the first data attribute to the word embeddings; and calculating the relatedness between the first word vector and the second word vector using a similarity metric.

[040] In some of these implementations, the first neural network model comprises the Word2vec® model. In some of these implementations, the second neural network model comprises the recurrent neural network model.

[041] In some such implementations, the similarity metric is based at least on (i) the Wu-Palmer distance between the word senses associated with the first word vector and the second word vector, (ii) a weighting factor, and (iii) a pairwise cosine distance between the first word vector and the second word vector.

[042] In some such implementations, the device computes analytical functions by performing a sequence of operations comprising: obtaining word definitions for the second set of one or more analytical sentences from a publicly available dictionary; determining whether the word definitions contain one or more predefined adjectives using an analysis API. Petition 870250038578, dated 12 / 05 / 2025, page 24 / 233 19 / 108 morphosyntactic provided by a library of tools for natural language processing; and, according to the determination that the word definitions contain one or more predefined adjectives, map the one or more predefined adjectives to one or more analytic functions.

[043] Typically, an electronic device includes one or more processors, memory, a display medium, and one or more programs stored in memory. The programs are configured for execution by the one or more processors and are configured to perform any of the methods described herein. The one or more programs include instructions to display a data visualization based on a first set of data retrieved from a database using a first set of one or more queries. The one or more programs also include instructions to receive a first user input to specify a first natural language command related to the data visualization. The one or more programs also include instructions to extract a first set of one or more analytical phrases from the first natural language command.One or more programs also include instructions to calculate a first set of one or more conversation centers associated with the first natural language command based on the first set of one or more analytic phrases. One or more programs also include instructions to calculate a first set of analytic functions associated with the first set of one or more conversation centers, thus creating a first set of one or more functional phrases. One or more programs also include instructions to update the data visualization based on the first set of one or more functional phrases. One or more programs also include instructions to receive a second user input to specify a second natural language command related to the updated data visualization. One or more programs also include... Petition 870250038578, dated 12 / 05 / 2025, page 25 / 233 20 / 108 instructions to extract a second set of one or more independent analytic phrases from the second natural language command. The one or more programs also include instructions to calculate a temporary set of one or more conversation centers associated with the second natural language command based on the second set of one or more analytic phrases. The one or more programs also include instructions to calculate the cohesion between the first set of one or more analytic phrases and the second set of one or more analytic phrases. The one or more programs also include instructions to derive a second set of one or more conversation centers from the first set of one or more conversation centers and the temporary set of one or more conversation centers based on cohesion.One or more programs may also include instructions for calculating a second set of one or more analytical functions associated with the second set of one or more conversation centers, thereby creating a second set of one or more functional phrases, and updating the data visualization based on the second set of one or more functional phrases.

[044] In some implementations, a non-temporary, computer-readable storage medium stores one or more programs configured for execution by a computing device containing one or more processors, memory, and a display medium. The one or more programs are configured to perform any of the methods described herein. The one or more programs include instructions to display a data visualization based on an initial set of data retrieved from a database using an initial set of one or more queries. The one or more programs also include instructions to receive an initial user input to specify an initial natural language command related to the data visualization. The one or more programs also include instructions to extract an initial set Petition 870250038578, dated 12 / 05 / 2025, page 26 / 233 21 / 108 of one or more analytic phrases from the first natural language command. One or more programs also include instructions to calculate a first set of one or more conversation centers associated with the first natural language command based on the first set of one or more analytic phrases. One or more programs also include instructions to calculate a first set of analytic functions associated with the first set of one or more conversation centers, thus creating a first set of one or more functional phrases. One or more programs also include instructions to update the data visualization based on the first set of one or more functional phrases. One or more programs also include instructions to receive a second user input to specify a second natural language command related to the updated data visualization.One or more programs also include instructions to extract a second set of one or more independent analytic phrases from the second natural language command. One or more programs also include instructions to calculate a temporary set of one or more conversation centers associated with the second natural language command based on the second set of one or more analytic phrases. One or more programs also include instructions to calculate the cohesion between the first set of one or more analytic phrases and the second set of one or more analytic phrases. One or more programs also include instructions to derive a second set of one or more conversation centers from the first set of one or more conversation centers and the temporary set of one or more conversation centers based on cohesion.One or more programs also include instructions to calculate a second set of one or more analytical functions associated with the second set of one or more conversation centers, thereby creating a second set of one or more functional phrases, and updating the data visualization accordingly. Petition 870250038578, dated 12 / 05 / 2025, page 27 / 233 22 / 108 second set of one or more functional phrases.

[045] In another aspect, according to some implementations, a method is executed on an electronic device with a display medium. For example, the electronic device could be a smartphone, a tablet, a notebook, a computer, or a desktop computer. The device displays a data visualization based on a dataset retrieved from a database using a first set of one or more database queries. A user specifies a first natural language command related to the displayed data visualization. Based on the displayed data visualization, the device extracts a first set of one or more independent analytical phrases from the first natural language command. Then, the device calculates a first set of one or more conversation centers associated with the first natural language command based on the first set of one or more analytical phrases.The device then calculates a first set of analytical functions associated with the first set of one or more conversation centers, thereby creating a first set of one or more functional phrases. The device then updates the data visualization based on the first set of one or more functional phrases. The user specifies a second natural language command related to the updated data visualization. After receiving the second natural language command, the device extracts a second set of one or more independent analytical phrases from the second natural language command, and calculates a temporary set of one or more conversation centers associated with the second natural language command based on the second set of one or more analytical phrases.The device then derives a second set of one or more conversation centers from the first set of one or more conversation centers and the temporary set of one or more conversation centers using one or more. Petition 870250038578, dated 12 / 05 / 2025, page 28 / 233 23 / 108 transition rules. The device then updates the data view based on the second set from one or more conversation centers.

[046] In some implementations, the device additionally determines one or more data attributes corresponding to the second set of one or more conversation centers; scans displayed data views to identify one or more of the displayed data views that contain data tags whose characteristics match a first data attribute in one or more data attributes; and highlights the data tags whose characteristics match the first data attribute. In some such implementations, the device additionally filters, from the displayed data views, results that contain data tags whose characteristics do not match one or more data attributes. Furthermore, in some such implementations, the device receives user input to determine whether to filter or highlight data tags (for example, via a natural language command such as “delete”, “remove”, and “filter only”).

[047] In some implementations, visualization features include one or more of color, size, and format. In some implementations, visualization features correspond to a visual encoding of data tags. In some implementations, visual encoding is one or more of color, size, and format.

[048] In some implementations, the device determines whether any of the displayed data visualizations contain data tags whose characteristics match the first data attribute, and, in this case, generates a specification for a new data visualization with the first data attribute (e.g., aggregation types) and displays the new data visualization. In some such implementations, displaying the new data visualization additionally comprises determining a chart type based on the specification; and generating and Petition 870250038578, dated 12 / 05 / 2025, page 29 / 233 24 / 108 display the chart. Additionally, in some such implementations, the chart is positioned using a two-dimensional grid-based layout algorithm, automatically coordinating with other data visualizations (sometimes referred to here as "views").

[049] In some implementations, the device additionally performs a sequence of operations comprising: calculating a second set of one or more analytical functions associated with the second set of one or more conversation centers, thereby creating a second set of one or more functional phrases; selecting a first functional phrase from the second set of one or more functional phrases, wherein the first functional phrase comprises a parameterized data selection criterion; selecting an initial range for parameter values ​​of the parameterized data selection criterion; displaying a user-editable interface control (e.g., widgets) corresponding to the parameterized data selection criterion, wherein the user interface control displays the current parameter values;and ordering a displayed set of one or more editable user interface controls based on the query order in the second natural language command, where the query order is inferred while extracting the second set of one or more analytical phrases from the second natural language command. In some such implementations, the user interface control allows adjustment of the first functional phrase. Additionally, in some such implementations, the user interface control displays a slider, which allows a user to adjust the first functional phrase. In some such implementations, ordering the displayed set of one or more editable user interface controls further involves using a library that facilitates the compact placement of small word-scale visualization within the text. In some such implementations, the library is Sparklificator®. Petition 870250038578, dated 12 / 05 / 2025, p. 30 / 233 25 / 108

[050] In some implementations, the device performs a sequence of operations aimed at automatically correcting some user errors. The sequence of operations comprises: determining a first symbol (token) in the second natural language command that does not correspond to any of the analytical phrases in the second set of one or more analytical phrases (for example, due to a parsing failure); searching for a correctly spelled term corresponding to the first symbol using a search library by comparing the first symbol with one or more aspects of the first dataset; and replacing the correctly spelled term for the first symbol in the second natural language command to obtain a third natural language command; and extracting the second set of one or more analytical phrases from the third natural language command.In some such implementations, one or more aspects include data attributes, cell values, and related keywords from the first dataset. In some such implementations, the search library is a fuzzy sequence library, such as Fuse.js®.

[051] In some such implementations, the device performs a sequence of operations comprising: determining whether there is no correctly spelled term corresponding to the first symbol; and, according to a determination that there is no correctly spelled term corresponding to the first symbol: parsing the second natural language command to obtain a parse tree; pruning the parse tree to remove the portion of the tree corresponding to the first symbol; and extracting the second set of one or more analytic phrases based on the pruned parse tree.

[052] In some implementations, the device additionally generates textual feedback indicating that the first symbol was not recognized and therefore removed from the second natural language command - a situation that Petition 870250038578, dated 12 / 05 / 2025, page 31 / 233 26 / 108 typically occurs when the statement has only been partially understood. In some such implementations, the device displays the first symbol.

[053] In some implementations, the device additionally generates textual feedback indicating that the correctly spelled term is used as a substitute for the first symbol in the second natural language command. This is generally the situation when the utterance was not successfully understood, but the device suggested an alternative query. Furthermore, in some such implementations, the device displays and highlights the correctly spelled term.

[054] Typically, an electronic device includes one or more processors, memory, a display medium, and one or more programs stored in memory. The programs are configured for execution by the one or more processors and are configured to perform any of the methods described herein. The one or more programs include instructions to display a data visualization based on a first set of data retrieved from a database using a first set of one or more queries. The one or more programs also include instructions to receive a first user input to specify a first command in natural language related to the data visualization. The one or more programs also include instructions to extract a first set of one or more analytical phrases from the first command in natural language.One or more programs also include instructions to calculate a first set of one or more conversation centers associated with the first natural language command based on the first set of one or more analytic phrases. One or more programs also include instructions to calculate a first set of analytic functions associated with the first set of one or more conversation centers, thus creating a first set of one or more functional phrases. One or more programs also include instructions to update the data visualization with... Petition 870250038578, dated 12 / 05 / 2025, page 32 / 233 27 / 108 based on the first set of one or more functional phrases. The one or more programs also include instructions to receive a second user input to specify a second natural language command related to the updated data visualization. The one or more programs also include instructions to extract a second set of one or more independent analytical phrases from the second natural language command. The one or more programs also include instructions to calculate a temporary set of one or more conversation centers associated with the second natural language command based on the second set of one or more analytical phrases. The one or more programs also include instructions to derive a second set of one or more conversation centers from the first set of one or more conversation centers and the temporary set of one or more conversation centers using one or more transition rules.One or more programs also include instructions to update the data visualization based on the second set of one or more conversation centers, where the update comprises: determining one or more data attributes corresponding to the second set of one or more conversation centers; scanning displayed data visualizations to identify one or more of the displayed data visualizations that contain data tags whose characteristics match a first data attribute in one or more data attributes; and highlighting the data tags whose characteristics match the first data attribute.

[055] In some implementations, a non-temporary, computer-readable storage medium stores one or more programs configured for execution by a computing device containing one or more processors, memory, and a display medium. The one or more programs are configured to perform any of the methods described herein. The one or more programs include instructions for displaying a data visualization based on Petition 870250038578, dated 12 / 05 / 2025, page 33 / 233 28 / 108 A first set of data retrieved from a database using a first set of one or more queries. The one or more programs also include instructions to receive a first user input to specify a first command in natural language related to data visualization. The one or more programs also include instructions to extract a first set of one or more analytic phrases from the first command in natural language. The one or more programs also include instructions to calculate a first set of one or more conversation centers associated with the first command in natural language based on the first set of one or more analytic phrases. The one or more programs also include instructions to calculate a first set of analytic functions associated with the first set of one or more conversation centers, thus creating a first set of one or more functional phrases.One or more programs also include instructions to update the data visualization based on the first set of one or more functional phrases. One or more programs also include instructions to receive a second user input to specify a second natural language command related to the updated data visualization. One or more programs also include instructions to extract a second set of one or more independent analytical phrases from the second natural language command. One or more programs also include instructions to calculate a temporary set of one or more conversation centers associated with the second natural language command based on the second set of one or more analytical phrases.One or more programs also include instructions to derive a second set of one or more conversation centers from the first set of one or more conversation centers and the temporary set of one or more conversation centers using one or more transition rules. One or more programs also include instructions to update the data visualization. Petition 870250038578, dated 12 / 05 / 2025, page 34 / 233 29 / 108 based on the second set of one or more conversation centers, where the update comprises: determining one or more data attributes corresponding to the second set of one or more conversation centers; scanning displayed data views to identify one or more of the displayed data views that contain data tags whose characteristics match a first data attribute in one or more data attributes; and highlighting the data tags whose characteristics match the first data attribute.

[056] In this way, methods, systems and graphical user interfaces are revealed that allow users to efficiently explore the data displayed within a data visualization application.

[057] Both the previous general description and the following detailed description are illustrative and explanatory, and serve to provide a better understanding of the invention as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[058] For a better understanding of the aforementioned systems, methods and graphical user interfaces, as well as the additional systems, methods and graphical user interfaces that offer analytical data visualization tools, reference should be made to the following Implementation Description, together with the following drawings, in which similar reference numerals refer to corresponding parts throughout the figures.

[059] Figure 1 illustrates a graphical user interface used in some implementations.

[060] Figure 2 is a block diagram of a computing device according to some implementations.

[061] Figure 3A is a process flowchart illustrating a process for using natural language applying pragmatic principles for visual analysis of a dataset according to some implementations; and Figure 3B is a Petition 870250038578, dated 12 / 05 / 2025, page 35 / 233 30 / 108 State machine diagram illustrating the states of the conversation center and the transition between states when specific transition rules are triggered, according to some implementations.

[062] Figures 4A to 4B are diagrams illustrating the use of different transition rules in managing analytical conversations according to some implementations; and Figures 4C to 4G illustrate graphical user interfaces (which relate to Figures 4A to 4B) for interactive data analysis using natural language processing in a data visualization application according to some implementations.

[063] Figure 5 is a diagram illustrating a general structure for applying the principles of pragmatics to visual analytics according to some implementations.

[064] Figure 6A is a diagram illustrating the application of pragmatic principles to incomplete statements (sometimes called here “Ellipse”) according to some implementations; and Figures 6B to 6D illustrate graphical user interfaces (which relate to Figure 6A) for interactive data analysis using natural language processing in a data visualization application according to some implementations.

[065] Figure 7A is a diagram illustrating the application of pragmatic principles to utterances with reference expressions (sometimes called anaphoric references here) according to some implementations; and Figures 7B to 7D illustrate graphical user interfaces (which relate to Figure 7A) for interactive data analysis using natural language processing in a data visualization application according to some implementations.

[066] Figure 8A is a diagram illustrating the application of pragmatic principles to utterances with conjunctions according to some Petition 870250038578, dated 12 / 05 / 2025, page 36 / 233 31 / 108 implementations; Figure 8B illustrates a graphical user interface (related to Figure 8A) for interactive data analysis using natural language processing in a data visualization application according to some implementations; and Figure 8C illustrates how a system iteratively connects the analytical functions of adjacent nodes in an analysis tree by linearization, according to some implementations.

[067] Figure 9A is a diagram illustrating the application of pragmatic principles to manage lexical cohesion according to some implementations; and Figures 9B to 9C illustrate graphical user interfaces (which relate to Figure 9A) for interactive data analysis using natural language processing in a data visualization application according to some implementations.

[068] Figure 10A is a diagram illustrating the application of pragmatic principles to repair utterances according to some implementations; and Figures 10B to 10C illustrate graphical user interfaces (which relate to Figure 10A) for interactive data analysis using natural language processing in a data visualization application according to some implementations.

[069] Figure 11A is a diagram illustrating the application of pragmatic principles to manage responses and feedback according to some implementations; Figure 11B is a data visualization that further illustrates the methodology shown in Figure 11A; and Figure 11C shows how a dashboard is progressively built based on input statements, according to some implementations.

[070] Figure 12A illustrates a set of widgets generated to manage ambiguity in a user query according to some implementations; and Figure 12B illustrates examples of feedback for various situations according to Petition 870250038578, dated 12 / 05 / 2025, page 37 / 233 32 / 108 with some implementations.

[071] Figures 13A-13J present a flowchart of a process that uses natural language for visual analysis of a dataset applying pragmatic principles, according to some implementations.

[072] Figures 14A to 14R present a flowchart of a process that uses natural language for visual analysis of a dataset applying pragmatic principles, including managing various forms of pragmatics, according to some implementations.

[073] Figures 15A to 15H present a flowchart of a process that uses natural language for visual analysis of a dataset applying pragmatic principles, including managing ambiguity in a user query, according to some implementations.

[074] Reference will now be made to the implementations, examples of which are illustrated in the accompanying drawings. In the following description, numerous specific details are presented in order to allow a thorough understanding of the present invention. However, it will be clear to those with general knowledge of the art that the present invention can be practiced without needing these specific details. DESCRIPTION OF THE IMPLEMENTATIONS

[075] Figure 1 illustrates a graphical user interface 100 for interactive data analysis. The user interface 100 includes a Data tab 114, and an Analytics tab 116 according to some implementations. When the Data tab 114 is selected, the user interface 100 displays a schema information region 110, which is also called a data panel. The schema information region 110 provides named data elements (e.g., field names) that can be selected and used to construct a data visualization. In some implementations, the list of field names is separated into Petition 870250038578, dated 12 / 05 / 2025, page 38 / 233 33 / 108 a group of dimensions (e.g., categorical data) and a group of measures (e.g., numerical quantities). Some implementations also include a list of parameters. When the Analytics tab 116 is selected, the user interface displays a list of analytical functions instead of the data elements (not shown).

[076] The graphical user interface 100 also includes a data visualization region 112. The data visualization region 112 includes a plurality of shelf regions, such as a column shelf region 120 and a row shelf region 122. These are also called column shelf 120 and row shelf 122. As illustrated here, the data visualization region 112 also has a large space to display a visual chart (also called here a data visualization). Since no data element has been selected so far, the space initially has no visual chart. In some implementations, the data visualization region 112 has multiple layers that are called sheets.

[077] In some implementations, the graphical user interface 100 also includes a natural language processing region 124. The natural language processing region 124 includes an input bar (also called here the command bar) for receiving natural language commands. A user can interact with the input bar to provide commands. For example, the user can type a command into the input bar to provide the command. In addition, the user can interact indirectly with the input bar by using a microphone to speak (e.g., an audio input device 220) to provide commands. In some implementations, data elements are initially associated with the column shelf 120 and the row shelf 122 (e.g., using drag-and-drop operations) from the schema information region 110 to the column shelf 120 and / or to the shelf Petition 870250038578, dated 12 / 05 / 2025, page 39 / 233 34 / 108 of rows 122). After the initial association, the user can use natural language commands (for example, in the natural language processing region 124) to further explore the displayed data visualization. In some cases, a user creates the initial association using the natural language processing region 124, resulting in one or more data elements being placed on the column shelf 120 and the row shelf 122. For example, the user might provide a command to create a relationship between data element X and data element Y. In response to receiving the command, the column shelf 120 and the row shelf 122 might be populated with the data elements (for example, the column shelf 120 might be populated with data element X and the row shelf 122 might be populated with data element Y, or vice versa).

[078] Figure 2 is a block diagram illustrating a computing device 200, which may display the graphical user interface 100 according to some implementations. Various examples of the computing device 200 include a desktop computer, a laptop computer, a tablet computer, and other computing devices that have a display medium and a processor capable of running a data visualization application 230. The computing device 200 typically includes one or more processing units (processors or cores) 202, one or more networks or other communication interfaces 204, memory 206, and one or more communication buses 208 to interconnect these components. The communication buses 208 optionally include sets of circuits (sometimes called a chipset) that interconnect and control communications between the system components. The computing device 200 includes a user interface 210.User interface 210 typically includes a display device 212. In some implementations, computing device 200 is included. Petition 870250038578, dated 12 / 05 / 2025, page 40 / 233 35 / 108 input devices, such as a keyboard, mouse, and / or other input buttons 216. Alternatively or additionally, in some implementations, the display device 212 includes a touch-sensitive surface 214, in which case the display device 212 is a touch-sensitive display medium. In some implementations, the touch-sensitive surface 214 is configured to detect various sliding gestures (e.g., continuous gestures in the vertical and / or horizontal directions) and / or other gestures (e.g., single / double tap). In computing devices that have a touch-sensitive display medium 214, a physical keyboard is optional (e.g., a virtual keyboard) may be displayed when keyboard input is required. The user interface 210 also includes an audio output device 218, such as speakers, or an audio output connection connected to speakers, earphones, or headphones.Additionally, some computing devices 200 use a microphone and voice recognition to supplement or replace the keyboard. Optionally, the computing device 200 includes an audio input device 220 (e.g., a microphone) to capture audio (e.g., a user's speech).

[079] Memory 206 includes high-speed random-access memory, such as DRAM, SRAM, DDR RAM, or other random-access solid-state memory devices; and may include non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. In some implementations, memory 206 includes one or more storage devices located remotely from the processor(s) 202. Memory 206, or alternatively, the non-volatile memory device(s) within memory 206, includes a computer-readable non-temporary storage medium. In some implementations, memory 206, or the computer-readable storage medium Petition 870250038578, dated 12 / 05 / 2025, page 41 / 233 36 / 108 memory computer 206, stores the following programs, modules and data structures, or a subset or superset thereof: • an operating system 222, which includes procedures for handling various basic system services and for performing hardware-dependent tasks; • a communications module 224 that is used to connect the computing device 200 to other computers and devices by means of one or more communication network interfaces 204 (wired or wireless), such as the Internet, other wide area networks, local area networks, metropolitan area networks, and so forth; • a web browser (or other application capable of displaying web pages) that allows a user to communicate over a network with remote computers or devices; • an audio input module 228 (e.g., a microphone module) to process the audio captured by the audio input device 220. The captured audio can be sent to a remote server and / or processed by an application running on the computing device 200 (e.g., the data visualization application 230); • a data visualization application 230 for generating data visualizations and related aspects. The application 230 includes a graphical user interface 232 (for example, the graphical user interface 100 illustrated in Figure 1) for a user to construct visual graphs. For example, a user selects one or more data sources 240 (which may be stored on the computing device 200 or stored remotely), selects data fields from the data source(s), and uses the selected fields to define a visual graph; and • zero or more databases or data sources 240 (for example, a first data source 240-1 and a second data source 240-2), which are used by the data visualization application 230. In some implementations, Petition 870250038578, dated 12 / 05 / 2025, page 42 / 233 37 / 108 Data sources are stored as spreadsheet files, CSV files, text files, JSON files, XML files, or flat files, or stored in a relational database.

[080] In some implementations, the data visualization application 230 includes a data visualization generation module 234, which takes user input (e.g., a visual specification 236) and generates a corresponding visual graph. The data visualization application 230 then displays the generated visual graph in the user interface 232. In some implementations, the data visualization application 230 runs as a standalone application (e.g., a desktop application). In some implementations, the data visualization application 230 runs within the Network browser 226 or in another application using Network pages provided by a Network server (e.g., a server-based application).

[081] In some implementations, the information that the user provides (e.g., user input) is stored as a visual specification 236. In some implementations, the visual specification 236 includes previous natural language commands received from a user or properties specified by the user through natural language commands.

[082] In some implementations, the data visualization application 230 includes a language processing module 238 to process (e.g., interpret) commands provided by a user of the computing device. In some implementations, the commands are natural language commands (e.g., captured by the audio input device 220). In some implementations, the language processing module 238 includes sub-modules, such as an autocomplete module, a pragmatics module, and an ambiguity module, each of which is discussed in more detail below. Petition 870250038578, dated 12 / 05 / 2025, page 43 / 233 38 / 108

[083] In some implementations, memory 206 stores metrics and / or scores determined by language processing module 238. In addition, memory 206 may store thresholds and other criteria, which are compared with the metrics and / or scores determined by language processing module 238. For example, language processing module 238 may terminate a parentage metric (discussed in detail below) for an analytical word / phrase of a received command. Then, language processing module 238 may compare the parentage metric with a threshold stored in memory 206.

[084] Each of the executable modules, applications, or sets of procedures identified above can be stored in one or more of the memory devices mentioned above, and corresponds to a set of instructions to perform a function described above. The modules or programs identified above (i.e., sets of instructions) need not be implemented as separate program software, procedures, or modules, and thus, various subsets of these modules can be combined or otherwise rearranged in various implementations. In some implementations, memory 206 stores a subset of the modules and data structures identified above. Additionally, memory 206 may store additional modules or data structures not described above.

[085] Although Figure 2 illustrates a computing device 200, Figure 2 is conceived more as a functional description of the various aspects that may be present rather than as a structural scheme of the implementations described herein. In practice, and as recognized by those skilled in the art, the items presented separately could be combined and some items could be separated.

[086] Figure 3A is a process flowchart illustrating a process or Petition 870250038578, dated 12 / 05 / 2025, page 44 / 233 39 / 108 A framework for using natural language, applying pragmatic principles to the visual analysis of a dataset according to several implementations. The framework, based on a conversational interaction model, extends the centering approach employed in pragmatic theory to support inter-sentential transition states to continue, retain, and shift the context of the data attributes at play. The framework supports a “visual analysis cycle,” an interface that supports fluid iterative exploration and refinement in visual analytics. Interaction with visual analytics is most effective when users can focus on answering the questions they have about their data, rather than focusing on how to operate the interface for the analysis tool. Pragmatics is particularly important for the visual analysis flow, where questions and insights often arise from previous questions and data patterns that a person observes.

[087] Sequences of utterances that exhibit coherence form a conversation. Coherence is a semantic property of conversation, based on the interpretation of each individual utterance in relation to the interpretation of other utterances. As mentioned earlier, in order to correctly interpret a set of utterances, the process structure uses and extends a model generally used for discourse structure called conversational centralization, according to some implementations. In this model, utterances are divided into constituent discourse segments, incorporating relations that may be valid between two segments. A center refers to the entities serving to link this utterance to other utterances in the discourse. Consider a discourse segment DS with utterances Ui, ..., Um.Each statement Un (1 < n < m) in the DS is assigned to a set of prospective centers, Cf (Un, DS) referring to the current focus of the conversation; each other utterance other than the initial utterance of the segment is assigned to a set of centers. Petition 870250038578, dated 12 / 05 / 2025, p. 45 / 233 40 / 108 regressive, Cb(Un, DS). The set of regressive centers of a new Un+i statement is Cb(Un+i, DS), which is equal to the prospective centers of Un (i.e., Cf(Un, DS)). In the context of visual analytic conversions, prospective and regressive centers include data attributes and values, visual properties, and analytic actions (e.g., filter, highlight).

[088] Each segment of discourse exhibits both global coherence, that is, the overall context of the entire conversation, generally referring to a topic or theme of the conversation, and local coherence, that is, coherence between utterances within that conversation. Local coherence refers to the inference of a sequence of utterances within a local context through the transitional states of continue, retain, and replace between Cf (Un, DS) and Cb (Un, DS). The framework extends this theory of conversational centrality to visual analytic conversation by introducing a set of rules for each of these constructs of local coherence, according to some implementations.

[089] Given a statement Un, a system implementing this structure responds by executing a series of analytical functions derived from prospective centers Cf(Un, DS). An analytical function F(X, op, v) consists of a variable X (which can be an attribute or a view property), an operator op, and a value v (usually a constant), according to some implementations. For example, when the user says “measles in the uk”, the system creates two functions, namely, F_CAT(diseases, ==, measles) and F_CAT(country, ==, uk). When the user provides a new statement Un+i, the system first creates a set of temporary centers Ctemp(Un+i, DS) from Un+i without considering any previous context. The system then applies a set of rules to create a set of prospective centers, Cf(Un+i, DS) based on some operations defined between Cb(Un+i, DS) and Ctemp(Un+i, DS).Prospective centers are then used to respond to the user's statement according to certain implementations. Petition 870250038578, dated 12 / 05 / 2025, page 46 / 233 41 / 108

[090] Figure 3A illustrates this process for two utterances Un (302) and Un+1 (312) according to some implementations. The system creates (304) a set of temporary centers Ctemp(Un) (306) from Un without considering any previous context. At the moment of receiving the utterance Un, the system either pre-calculated a set of regressive conversation centers Cb (Un) (301) based on a previous utterance Un-1, or, if Un is the first utterance, initialized Cb (Un) to a null set. The system then applies (308) one or more transition rules (described below with reference to Figure 3B) to derive a set of prospective centers Cf (Un) (310) from the regressive centers Cb (Un) (301) and temporary conversation centers Ctemp (Un) (306). The prospective conversation centers Cf (Un) (310) are also the regressive conversation centers Cb (Un+1) for the next statement Un+1 (312).The process described so far is repeated for the next utterance Un+1 (312). The system calculates (314) a set of temporary centers Ctemp(Un) (316) from Un without considering any previous context. The system then applies (318) to one or more transition rules to derive a set of prospective centers Cf (Un+1) (319) from the regressive centers Cb (Un+1) (310) and the temporary conversation centers Ctemp (Un+1) (316). The system uses the prospective conversation centers Cf (Un+1) (319) to update one or more data views according to some implementations. The system also uses the set of prospective centers Cf (Un+1) (319) as the regressive conversation centers Cb (Un+2) for the next utterance Un+2, and so on.When the user moves to a different dataset or restarts the view, the system updates the overall consistency of the analytical conversation and removes all previous states (including prospective conversation centers, regressive conversation centers, and temporary conversation centers), according to some implementations.

[091] Figure 3B is a state machine diagram illustrating states Petition 870250038578, dated 12 / 05 / 2025, page 47 / 233 42 / 108 of the conversation center and the transition between states, according to some implementations. State 322 encapsulates regressive conversation centers Cb and temporary conversation centers Ctemp, and each of states 324, 326, and 328 represents different states of the prospective conversation centers Cf. The conversation centers correspond to an utterance Un+1 (not illustrated). What follows is a description of each of the transition rules, that is, when each transition occurs and how the final states are calculated.

[092] The Continue 323 transition continues the context from the regressive Cb centers to the prospective Cf centers, according to some implementations. In other words, each of the conversation centers in the regressive Cb conversation centers is included in the prospective Cf conversation centers (324). Using a defined notation, for a given utterance Un+1, in a speech segment DS, as a result of this transition, Cb(Un+i, DS) and Cf (Un+1, DS), along with other entities

[093] This transition occurs when a variable X is in Ctemp (Un+i) but not in Cb (Un+1, DS), according to some implementations. In this case, the system performs the following union operation: Cf (Un+1, DS) = Cb (Un+1, DS) U Ctemp (Un+1, DS)

[094] The Retain 325 transition retains the context from the regressive centers Cb (322) in the prospective centers Cf (326) without adding additional entities to the prospective centers, according to some implementation; that is, Cf (Un+1, DS) = Cb (Un+1, DS)

[095] The Retain 325 transition is triggered when the variable X is in Cb (Un+1, DS), but not in CtemP(Un+i, DS), according to some implementations.

[096] In some implementations, with the Shift 327 transition, the context moves from the regressive conversation centers 322 to the prospective conversation centers 328; that is, Petition 870250038578, dated 12 / 05 / 2025, p. 48 / 233 43 / 108 Cf(Un+l, DS) * Cb(Un+l,DS)

[097] In some implementations, the Shift 327 transition occurs when the variable X is in both Cb (Un+1, DS) and Ctemp(Un+i, DS), but the corresponding values ​​are different. In this case, the system replaces all regressive centers Cb (Un+1, DS) containing X with Ctemp (Un+1, DS). As illustrated in Figure 3B, this replacement can be plotted using the equation: Cf(Un+1, DS) = Cb (Un+1, DS) — Xcb + Xctemp

[098] In some implementations, the Shift 327 transition also occurs when a filter constraint is removed; for example, removing a widget for “measles” shifts the “disease” variable from measles to all diseases.

[099] Figure 4A is a diagram illustrating the use of different transition rules in handling analytical conversations according to some implementations. Assume that a system implements the structure described above with reference to Figures 3A and 3B. When the system receives an utterance 400 (e.g., “measles in the uk”), the system calculates (402) prospective conversation centers (404) corresponding to the utterance, and initializes regressive centers, according to some implementations. The conversation centers for the illustrative utterance include “measles” and “uk”. Based on the calculated conversation centers, the system applies (406) filters to update the visualization, for example, it applies categorical and spatial filters showing “measles in the UK” in visualization 408. Figure 4D shows an enlarged version of visualization 408.As illustrated in Figure 4D, the statement “measles in the UK” results in the system displaying a “Disease Outbreaks Around the World” visualization focusing on (DISEASES == Measles (442) and COUNTRY == United Kingdom (444)). Referring again to Figure 4A, assuming the user responds to visualization 408 with... Petition 870250038578, dated 12 / 05 / 2025, page 49 / 233 44 / 108 a new utterance 410 (for example, “show me the orange spike”), the system calculates (412) the prospective conversation centers by applying the Continue rule (described above with reference to 323, Figure 3B). This is because the temporary conversation center (not illustrated) for the utterance example includes “orange spike,” which corresponds to the visualization characteristics (a type of variable), namely, variable shape (with the value “spike”) and variable color (with the value “orange”), and these variables are absent in the regressive conversation centers (404). Furthermore, since utterance 410 does not refer to the other variables (DISEASES or COUNTRY) in the regressive conversation centers (404), the system does not apply the Hold rule or the Shift rule.The prospective conversation centers (414) for utterance 410 are thus calculated as {measles, uk, orange spike} (measles, United Kingdom, orange spike). Based on these prospective centers, the system updates the corresponding graphs in the data visualization, for example, highlighting the orange spike in the line for measles in the graph in visualization 418. Figure 4E shows an enlarged version of visualization 418. As the example shows, the system highlights and annotates (446) the spike with the words “4.4 cases of Measles in 2014” in response to the utterance “show me the orange spike”, according to some implementations.

[0100] Figure 4B is another diagram further illustrating the application of different transition rules for example utterances. This example follows the user's response to visualization 328 in Figure 4A. Referring to Figure 4B, when the system receives an utterance 420 (e.g., “mumps over there”), the system calculates (422) prospective conversation centers corresponding to the utterance, according to some implementations. The regressive conversation centers (424) (e.g., {measles, United Kingdom, orange peak}) correspond to the conversation centers Petition 870250038578, dated 12 / 05 / 2025, page 50 / 233 45 / 108 prospective (414) calculated in the previous step (in response to utterance 410) illustrated in Figure 4A. The temporary conversation centers (not illustrated) for utterance 420 (e.g., “mumps over there”) do not include a conversation center (e.g., “orange spike”) derived for the previous utterance. This causes the system to be triggered to apply the Hold rule (described above with reference to 325, Figure 3B). For example, the temporary conversation centers include the DISEASES variable, but the value of this variable is swapped from measles to mumps. This causes the system to be triggered to apply the Shift rule (described above with reference to 327, Figure 3B). As a result, the system calculates prospective conversation centers (424) (for example, {mumps, uk} (mumps, United Kingdom)).Based on these prospective conversation centers, the system responds (426) by applying necessary filters (e.g., holding spatial filter for “UK” (United Kingdom), updating categorical filter for “mumps”), and updating the visualization (e.g., removing the highlighted peak in the line for mumps), as illustrated in visualization 428. Figure 4F shows an enlarged version of visualization 428. As illustrated in Figure 4F, the utterance “mumps over there” results in the system displaying a visualization of “Disease Outbreaks Around the World” focusing on (DISEASES == Mumps (448) and COUNTRY == United Kingdom (450)).

[0101] Referring again to Figure 4B, to continue the example, assuming the user queries with another statement (430) (for example, “measles epidemic in Malawi congo angola” (measles epidemic in Malawi congo angola)), the system calculates (432) the prospective centers corresponding to this statement by applying the Shift rule described above. For the example illustrated, the reason for applying the Shift rule is that the values ​​of the DISEASES variable are now different, namely, “measles” and “epidemic”; Petition 870250038578, dated 12 / 05 / 2025, p. 51 / 233 46 / 108 similarly, the geographic region was changed from “UK” (United Kingdom) to “Malawi, Congo and Angola”. Based on the application of the Shift rule, the system calculates the prospective conversation centers (434) (for this example, the centers are {measles, epidemic, malawi, congo, angola}). Based on the prospective conversation centers (434), the system responds (436) to the utterance by applying appropriate filters (e.g., a categorical filter for measles epidemic, and a new spatial filter on Malawi, Congo and Angola, replacing United Kingdom), thus generating visualization 438 (e.g., showing measles epidemic in Malawi, Congo and Angola). Figure 4G shows an enlarged version of visualization 438.As illustrated in Figure 4G, the statement “measles epidemic in Malawi Congo Angola” results in the system displaying a “Disease Outbreaks Around the World” visualization focusing on (DISEASES == Measles (452), IMPACT = Epidemic (454), COUNTRY == Malawi (456), COUNTRY == Congo (458) and COUNTRY == Angola 460)).

[0102] Figure 4C illustrates the updates to the data visualizations in response to the user statements described above with reference to Figures 4A and 4B, according to some implementations. Visualizations 408, 418, 428, and 438 were described above with reference to the enlarged Figures 4D, 4E, 4F, and 4G, respectively.

[0103] Figure 5 is a diagram illustrating a general framework for applying pragmatic principles to visual analytics according to some implementations. The framework utilizes and extends a commonly used model for discourse structure called conversational centralization, according to some implementations. In this model, utterances are divided into constituent discourse segments, incorporating relations that can be Petition 870250038578, dated 12 / 05 / 2025, page 52 / 233 47 / 108 valid between two segments. A center refers to the entities serving to link this utterance to other utterances in the discourse. Consider a discourse segment DS with utterances Ui, ..., Um. Each utterance Un (1 < n < m) in the DS is assigned to a set of prospective centers, Cf (Un, DS) referring to the current focus of the conversation; each other utterance that is not the initial utterance of the segment is assigned to a set of regressive centers, Cb (Un, DS). The set of regressive centers of a new utterance Un+i is Cb (Un+i, DS), which is equal to the prospective centers of Un (i.e., Cf (Un, DS)). In the context of visual analytic conversions, prospective and regressive centers include data attributes and values, visual properties, and analytic actions (e.g., filter, highlight).

[0104] Given an utterance Un, a system implementing this structure responds by executing a series of analytic functions derived from prospective centers Cf(Un, DS). An analytic function F(X, op, v) consists of a variable X (which can be an attribute or a view property), an operator op, and a value v (usually a constant). For example, when the user says “measles in the uk”, the system creates two functions, namely, F_CAT(diseases, ==, measles) and F_CAT(country, ==, uk). When the user provides a new utterance Un+i, the system first creates a set of temporary centers Ctemp(Un+i, DS) from Un+i without considering any previous context. The system then applies a set of rules to create a set of prospective centers, Cf(Un+i, DS) based on some operations defined between Cb(Un+i, DS) and Ctemp(Un+i, DS). Prospective centers are then used to respond to the user's statement according to certain implementations. [0I05] Figure 5 illustrates this process for two statements Un (500) and Un+i (520) according to some implementations. The system calculates (526) a set of temporary centers Ctemp(Un+i) (528) from Un+i without considering any previous context. At the moment of receiving the statement Un+i, the system or Petition 870250038578, dated 12 / 05 / 2025, p. 53 / 233 48 / 108 precalculated a set of regressive conversation centers Cb (Un+i) (504) based on the previous statement Un, or, if Un was the first statement, initialized Cb (Un+1) (504) to a null set.

[0106] Conversation centralization posits that utterances exhibit connection between them. The way in which these utterances are linked to one another to form a conversation is called cohesion. Cohesion results from the combination of both grammatical and lexical structures in the constituent sentences. Thus, identifying the sentence structure is a logical starting point for resolving this utterance into one or more analytical functions applied to visualization. Sentence structure includes both lexical and grammatical structure. In Figure 5, a system implementing this structure calculates the sentence structure for the utterance Un+1 (520) in step 522. Typically, a parser is used to calculate the sentence structure.A parser receives an input sentence (sometimes called a query or natural language command) and decomposes the input sentence into a sequence of symbols (linguistic elements) by applying a set of grammatical rules specific to a given natural language, such as English. In some implementations, the grammatical rules can be modified to suit the environment. In some implementations, a probabilistic grammar is applied to provide a structural description of the input queries. Probabilistic grammars are useful for resolving ambiguities in sentence parsing. Probability distributions (for grammatical production rules) can be estimated from a corpus of hand-parsed sentences, for example.Some implementations infer additional syntactic structure by employing a Morphosyntactic Tagger (POS Tagger) that assigns sentence components, such as noun, verb, and adjective, to each word (sometimes called a symbol). Some implementations resolve the parsed output into data attributes. Petition 870250038578, dated 12 / 05 / 2025, p. 54 / 233 49 / 108 ordered and categorical corresponding. As shown by the dashed lines connecting blocks 500 and 510, in some implementations, the system also calculates (510) the sentence structure for the utterance Un (500).

[0107] With the phrase structure(s), the system proceeds to determine (530) the type of pragmatic forms (examples of which are described below with references to Figures 6A to 10C) and any other information related to phrase structure (e.g., linguistic elements present in one utterance but absent in another). Based on the pragmatic form and phrase structure information (532), the system then derives (534) prospective conversation centers Cf (536) from the Un+i utterance (520) using the temporary conversation centers (528) and the regressive conversation centers Cb (504), according to some implementations. As shown in Figure 5, even before receiving the Un+i utterance (520), the system derives and / or displays one or more initial views (508) based on the regressive conversation centers Cb (504) by applying (506) a first set of operations (e.g., applying filters) to existing views.In some implementations, based on the prospective conversation centers Cf (536), the system applies (538) a second set of operations (e.g., applying filters) to update existing views or generate new views (540).

[0108] Figure 6A is a diagram illustrating the application of pragmatic principles to incomplete utterances (sometimes called ellipsis) according to some implementations. Ellipses are syntactically incomplete sentence fragments that omit one or more linguistic elements. These utterances can be better understood with the context established previously. Figure 6A illustrates how an incomplete utterance “townhomes” is understood in the context of the previous utterance “houses in less than 1M in Ballard”. When the system receives a Petition 870250038578, dated 12 / 05 / 2025, page 55 / 233 50 / 108 statement Un (600), which, in this example, is the statement “houses in less than 1M in Ballard”, the system calculates prospective centers and initializes regressive centers (for the statement Un+1) in step 602, using the methodology described above with reference to Figure 5 (step 502), according to some implementations. For the example statement, the system calculates the set of conversation centers as {houses, ballard, and 1M}. In some implementations, the system applies (606) filters to the dataset based on the set of conversation centers (604), and displays a data visualization (608). In this example, the system applies numerical and spatial filters showing houses under $1M in Ballard. Figure 6D shows an enlarged version of visualization 608.As illustrated in Figure 6C, the statement “houses less than 1M in Ballard” results in the system displaying a “Past Home Sales - Seattle” view focusing on (LAST_SALE_PRICE less than 1.0M (642) in Ballard (644)). Referring again to Figure 6A, in some implementations, the system also calculates the phrase structure (610) for the statement Un in step 610 using one or more of the techniques described above with reference to step 510 of Figure 5.

[0109] In some implementations, when the system receives an utterance Un+1 (620), which, in this example, is the utterance “townhomes”, the system calculates (626) temporary conversation centers for Un+1 (620). For this example, the system calculates the conversation centers (628) as being the set {townhomes}. Additionally, the system calculates (622) the sentence structure for the utterance Un+1 (620) using the techniques described above with reference to step 522 (Figure 5), according to some implementations.

[0110] As mentioned above, ellipses exclude one or more linguistic elements. With the aid of sentence structures (612 and 624), the system determines Petition 870250038578, dated 12 / 05 / 2025, page 56 / 233 51 / 108 a subset of conversation centers of the utterance Un (600) that corresponds to linguistic elements absent in the utterance Un+1 (620), according to some implementations. In this example, the system calculates the subset as the set {ballard, 1M}, since the linguistic elements, that is, a noun phrase that refers to a location after a prepositional phrase (corresponding to “ballard”) and a noun phrase that refers to a price value after another prepositional phrase (corresponding to “1M” or, more precisely, “less than 1M”), are absent in the utterance Un+1 (620), but were present in the utterance Un (600). On the other hand, the phrase “houses” in the Un (600) utterance and the phrase “townhomes” in the Un+1 (620) utterance correspond to similar linguistic elements (for example, both phrases are noun phrases and refer to types of houses).

[0111] In step 634, the system combines the temporary set of conversation centers, which in this example is the set {townhomes}, with the subset of conversation centers (632) to arrive at a set of prospective conversation centers (638) for the utterance Un+1, according to some implementations. Based on the calculated set of prospective conversation centers (636), the system determines the type of filters to be applied to the dataset and applies the appropriate filters in step 638 to display an appropriate data visualization (640), according to some implementations. In this example, since the conversation centers “ballard” and “1M” were retained from the regressive conversation centers (604), the system retains the numeric filter (corresponding to 1M) and the spatial filter (corresponding to Ballard).Furthermore, since the conversation center value (corresponding to the home_type variable) was changed from “townhomes” to “houses,” the system applies a categorical filter to home_type to display houses. Petition 870250038578, dated 12 / 05 / 2025, page 57 / 233 52 / 108 in the city (instead of just houses). Figure 6D shows an enlarged version of visualization 640. As illustrated in Figure 6D, the enunciation “townhomes” results in the system showing a visualization of “Past Home Sales Seattle” retaining the last widget or filter LAST_SALE_PRICE (642) and the spatial filter 644 from the previous visualization 608, and replacing HOME_TYPE with “townhouses” (646).

[0112] Figure 6B illustrates the updates to the data visualizations in response to the user statements described above with reference to Figure 6A, according to some implementations. Visualizations 608 and 640 were described above with reference to the enlarged Figures 6C and 6D, respectively.

[0113] Figure 7A is a diagram illustrating the application of pragmatic principles to utterances with reference expressions (sometimes called anaphoric references here) according to some implementations. Reference expressions help to unify the text and create economy by avoiding unnecessary repetition. Referencing is a form of conversation that, instead of being interpreted semantically by itself, refers to something else for its interpretation. When the interpretation is within the text, this is known as anaphoric referencing. In interaction with visual analytics, reference pertains to data attributes and analytic functions. Figure 7A illustrates how a statement “previous year” is understood in the context of the preceding statement “prices in 2015” (prices in 2015).When the system receives an utterance Un (700), which, in this example, is the utterance “prices in 2015”, the system calculates prospective centers and initializes regressive centers (for the utterance Un+1) in step 702, using the methodology described above with reference to Figure 5 (step 502), according to some implementations. For the example utterance, the system calculates the set of conversation centers as {prices, 2015}. In some implementations, the system applies (706). Petition 870250038578, dated 12 / 05 / 2025, page 58 / 233 53 / 108 filters the dataset based on the set of conversation centers (704), and displays a data visualization (708). In this example, the system applies temporal filters showing house prices in the year 2015. Figure 7C shows an enlarged version of visualization 708. As illustrated in Figure 7C, the statement “prices in 2015” results in the system showing a visualization of “Past Home Sales - Seattle” in 2015. Although not illustrated, a previous visualization, for example, in response to a previous statement, caused the system to check prices in Seattle. Referring again to Figure 7A, when the system receives a statement Un+1 (720), which, in this example, is the statement “previous year”, the system calculates (722) the sentence structure for the statement Un+1 (720) using techniques described above with reference to step 522 (Figure 5), according to some implementations.

[0114] As mentioned above, reference expressions with anaphoric references refer to something else within the text. Based on the sentence structure (724), the system identifies (726) anaphora in the Un+1 utterance (720), according to some implementations. In this example, the system identifies the anaphora (728) “previous”. Using the identified anaphora, the system then identifies (734) the sentence block (732) containing the reference to identify the entities to which the reference is referring, according to some implementations. For the illustrated example, the system identifies the sentence block “year” which corresponds to the anaphora “previous”. Based on the identified anaphora and the sentence block, in step 730, the system searches through the regressive centers to find such entities and replaces the anaphoric reference with those entities, according to some implementations.Additionally, in some implementations, as is the case in this example, the system also detects and applies functions appropriate to the entity's value. For the illustrated example, the system... Petition 870250038578, dated 12 / 05 / 2025, page 59 / 233 Step 54 / 108 also detects that the user is referring to the “previous” year, and therefore the value for 2015 is decremented by 1 before arriving at the correct value for the “year” variable. The system calculates the date for “previous” using a temporal function (e.g., DATECALC), according to some implementations. The system arrives at a set of prospective conversation centers (736), which, for this example, is the set {prices, 2014}. Based on this set, the system performs the necessary steps to update the visualization in step 738, according to some implementations. For this example, the system retains a year reference and updates the temporal filter to 2014, to show the visualization in 740. Figure 7D shows an enlarged version of visualization 740.As illustrated in Figure 7D, the statement “previous year” results in the system displaying a visualization of “Past Home Sales - Seattle” in 2014, the year prior to 2014 (from the previous visualization).

[0115] Figures 7B to 7D illustrate graphical user interfaces (relating to Figure 7A) for interactive data analysis using natural language processing in a data visualization application according to some implementations. Figure 7B illustrates the updates to the data visualizations in response to the user utterances described above with reference to Figure 7A, according to some implementations. Visualizations 708 and 740 were described above with reference to the enlarged Figures 7C and 7D, respectively.

[0116] Although not illustrated in Figure 7A, in some implementations, a system repeats the steps to recognize multiple anaphoric references in a single expression. Additionally, in some implementations, the system identifies many types of anaphoric references in the given utterance, such as “that”, “those”, “them”, “ones”, “previous”, “next”. Petition 870250038578, dated 12 / 05 / 2025, page 60 / 233 55 / 108 (next). As another illustrative example, consider the statement “Show Fremont, Queen Anne, and Ballard” followed by the statement “condos in those districts.” In this example, “those” refers to some values ​​(i.e., Fremont, Queen Anne, and Ballard) of the neighborhood attribute, as indicated by the word “districts.”

[0117] In some implementations, references refer to values ​​of a data attribute. In some implementations, references refer to actions that need to be performed by the system. For example, consider the statement “filter out ballard” followed by “do that to fremont”. Here, the word “that” is not immediately followed by any noun, but immediately preceded by a verbal word “do”. In such cases, the system determines one or more actions mentioned in the previous statement, which, in this example, is the action “filter out”.

[0118] In some implementations, the system supports references that are located outside the text and within the context of the visualization. In some such implementations, the forward-looking center Cf refers to the context within the visualization, rather than the text in the backward-looking center Cb. In some implementations, this form of indirect referencing includes a deictic reference that refers to some object in the environment, usually by indication. In some such implementations, the system supports deictic references by allowing multimodal interaction (mouse + speech / text). Figure 7E shows an illustrative visualization in response to a deictic reference. In some implementations, this form of indirect referencing includes a visualization property reference that uses properties in the visualization, such as tag properties, text in labels, geometric axes, and titles. Figure 7F shows an example of a visualization in response to a reference to a visualization property. Petition 870250038578, dated 12 / 05 / 2025, p. 61 / 233 56 / 108

[0119] Figure 8A is a diagram illustrating the application of pragmatic principles to utterances with conjunctions according to some implementations. Conjunctions in utterances communicate a range of relationships between sentence fragments called sets. In a conversation, users tend to iteratively construct a compound query by adding multiple sets, generally avoiding the explicit use of conjunctions and connectors such as “and,” “or,” and “furthermore” between sentences. Figure 8A illustrates how an utterance “houses in Ballard under 600k last summer” is understood in the context of the preceding utterance “houses in Ballard.”When the system receives an utterance Un (800), which, in this example, is the utterance “houses in Ballard”, the system calculates prospective centers and initializes regressive centers (for the utterance Un+1) in step 802, using the methodology described above with reference to Figure 5 (step 502), according to some implementations. For the example utterance, the system calculates the set of conversation centers as {houses, ballard}. In some implementations, the system applies (806) filters to the dataset based on the set of conversation centers (804), and displays a data visualization (808). In this example, the system applies categorical and spatial filters showing houses in Ballard.

[0120] When the system receives an utterance Un+1 (810), which, in this example, is the utterance “houses in Ballard under 600k last summer”, the system calculates (812) the sentence structure for the utterance Un+1 (810) using techniques described above with reference to step 522 (Figure 5), according to some implementations. As mentioned above, a compound query consists of multiple (sometimes implicit) sets between the constituent sentences. Based on the sentence structure (814), the system identifies (816) sets in the utterance Un+1 (810), according to some Petition 870250038578, dated 12 / 05 / 2025, page 62 / 233 57 / 108 implementations. In this example, the system identifies multiple queries (818), namely, “houses”, “in Ballard”, “under 600k” and “last summer”. Based on these queries (818) and the set of regressive conversation centers (804), the system calculates (820) a set of prospective conversation centers (822) according to some implementations. For example, the system selects the corresponding context from the Un statement (800), and adds the new conversation centers derived from the sets from the Un+1 statement (810). The set of prospective conversation centers (822), for this example, is the set {houses, ballard, <600k, last summer}. Based on this set of data, the system performs the necessary steps to refine the current visualization in step 824, according to some implementations.For this example, the system applies a numeric filter to the house price and a time filter to show last summer, to display the visualization in 826. Figure 8B shows an enlarged version of visualization 826. As Figure 8B shows, the statement “houses in Ballard under 600k last summer” results in the system displaying a visualization of “Past Home Sales - Seattle” in Ballard (828), with LAST_SALE_PRICE (830), below 600k (832), last summer. For this example, the system additionally resolves the period (for “last summer”) based on the previous visualization, as the time period 2015-6 to 2015-8-31 (834), according to some implementations.

[0121] Figure 8C illustrates how a system iteratively connects the analytical functions of adjacent nodes in a parse tree by linearization, according to some implementations. Finding implicit data coherence between sets is sometimes a challenging task. In the example illustrated in Figure 8B, all sets refer to the same entity “houses in Ballard” (houses in Petition 870250038578, dated 12 / 05 / 2025, p. 63 / 233 58 / 108 Ballard). However, there are cases where sets map to different entities. An example of such a statement is “houses in Ballard under 600k condos in South Lake Union”. The system determines whether the individual sets resolve to the same entity or to different entities, according to some implementations. In some such implementations, the system employs a rule-based technique that takes a potentially long statement with possibly implicit conjunctions, translating the statement into a set of analytic functions chained together by logical operators. The system then executes these analytic functions in response to the user's statement, according to some implementations.

[0122] In some implementations, the system resolves multiple sets within compound statements to invoke one or more corresponding analytic functions through a linearization process. In some such implementations, an analytic function F(X, op, v) consists of a variable X (e.g., an attribute), an operator op, and a value v. Each attribute is either categorical or ordered. The ordered data type is further categorized into ordinal and quantitative. The linearization process considers the attribute and operator types to combine analytic functions using logical operators (i.e., a, v), as described below.

[0123] Applying the v operator: When two or more adjacent sets share an attribute and the data type of that attribute is categorical, then the system connects these sets using v, according to some implementations. Similarly, if this shared attribute is ordered and the function operator is ==, the system applies v, according to some implementations. In such cases, v is logically more appropriate as a choice, since applying it would not match any item in the data table. For example, if the statement Petition 870250038578, dated 12 / 05 / 2025, p. 64 / 233 59 / 108 for “show me condos and townhomes”, then the system generates the following combination of analytical functions: (F_CAT(homeType, ==, condo) v F_CAT(homeType, ==, townhome)), according to some implementations. In this example, both “condo” and “town-home” belong to the same categorical attribute, that is, homeType. Since a given house (item) cannot be both a “condo” and a “townhome” at the same time, applying the v operator is logically more appropriate than applying the a operator. Similarly, if the user outputs “2 3 bedroom houses”, the system generates (F_ORDINAL(bed, ==, 2) v F_ORDINAL(bed, ==, 3)), according to some implementations. The v operator is also appropriate if the attribute type is ordered and involves the X condition.<v1 e X> v2, where v1 <v2.For example, if the statement is "before 2013 and after 2014," then the v operator will be used between the two sets, according to some implementations. Again, in this case, applying the a operator would result in no match for items in the data table.

[0124] Applying the to operator: The to operator is appropriate if the attribute type is ordered and involves the X condition.<v1 e X> v2, where v1 <v2. Por exemplo, “houses over 400k and under 700k” (casas acima de 400 mil e abaixo de 700 mil) resolve para (F NUMERIC(price, > The statement "Beds between 2 to 4" resolves to (F ORDINAL(beds, >=, 2) a F NUMERIC(beds, <=, 4)). Note that applying the v operator would result in a match with all items in the data table. In some implementations, the a operator is also applied when there is no common attribute between two sets. For example, the statement "price below 600k with 2 beds" resolves to (F ORDINAL(beds, ==, 2) a F NUMERIC(price, <=, 600000)).

[0125] In order to generate the analytical function representation of the enunciation Petition 870250038578, dated 12 / 05 / 2025, p. 65 / 233 For the entire 60 / 108 sentence, the system traverses a corresponding parse tree for the utterance generated by a parser (e.g., the parser described above in reference to Figure 5) in post-order and applies the two rules above iteratively to the sentences as illustrated in Figure 8C. For the examples illustrated in Figure 8C, the system takes the utterance “condos under 600K townhomes under 1M” as input, and iteratively applies the rules above to generate the chain of analytic functions.

[0126] Figure 9A is a diagram illustrating the application of pragmatic principles to manage lexical cohesion according to some implementations. The three previous types of pragmatics—ellipsis, referencing, and conjunction—provide grammatical cohesion to conversation. In addition to these grammatical constructs, users often find ways to express concepts through related word meanings, that is, senses in conversation, a term called lexical cohesion. These word senses can be as simple as variations in spelling, root, and plurality (e.g., “profit” and “profits”), synonyms (e.g., “country” and “nation”), to related terms or terms that occur together (e.g., “violence” and “crime”). Generally, word senses are related to each other within a semantic context.

[0127] Figure 9A illustrates how an utterance “the cheapest” is understood in the context of the preceding utterance “most expensive houses in Queen Anne”. When the system receives an utterance Un (900), which, in this example, is the utterance “most expensive houses in Queen Anne”, the system calculates prospective centers and initializes regressive centers (for the utterance Un+i) at step 902, using the methodology described above with reference to Figure 5 (step 502), according to some implementations. For the example utterance, the system calculates the set of conversation centers as being {most expensive, houses, Petition 870250038578, dated 12 / 05 / 2025, page 66 / 233 61 / 108 Queen Anne} (most expensive, houses, Queen Anne). In some implementations, the system maps (906) one or more conversation centers to a corresponding analytical function to generate a data visualization (908), according to some implementations. In the illustrated example, the system maps “most expensive” to the analytical function TOP_N(sale_price) of “houses”. Some implementations also annotate the price range for clarity. In this example, the system applies categorical and spatial filters showing houses in Ballard. Figure 9B shows an enlarged version of visualization 908. As Figure 9B shows, the statement “most expensive houses in Queen Anne” results in the system displaying a visualization of “Past Home Sales - Seattle” comprising the top 10% of LAST_SALE_PRICE (928) in Queen Anne (930), according to some implementations.

[0128] When the system receives an utterance Un+1 (910), which, in this example, is the utterance “the cheapest”, the system calculates (912) the sentence structure for the utterance Un+1 (914) using the techniques described above with reference to step 522 (Figure 5), according to some implementations. As mentioned above, a user utterance sometimes has word meanings that are better understood in the context of previous utterances. Based on the sentence structure (914), the system identifies (916) candidates for lexical cohesion in the utterance Un+1 (910), according to some implementations. In this example, the system identifies a candidate “cheapest” (918) for cohesion analysis. Based on one or more identified cohesion candidates (918) and the set of regressive conversion centers (904), the system calculates (920) a set of prospective conversation centers (922) according to some implementations.For the example illustrated, the system calculates the semantically related data attribute (e.g., sale_price) corresponding to... Petition 870250038578, dated 12 / 05 / 2025, page 67 / 233 62 / 108 lexical cohesion candidates (e.g., “more expensive” or “cheaper”), replacing relevant numerical attributes, while continuing the rest of the context from the Un(904) utterance, according to some implementations.

[0129] In some implementations, the system identifies the meanings of attribute words using the Word2vec® model containing learned vector representations of large bodies of text, calculating word vectors using a recurrent neural network. In some implementations, the semantic relatedness Srei between a word w, in a given utterance, and a data attribute dj, is the maximum value of a score calculated as follows: (whdj)=maxm ηλ cos(ryd) + (1 - λ) ----—- (I)Jdist[Slfíl,S jn) In formula (1), dist(S / ,m, S;,n) is the Wu-Palmer distance between the two directions Si,m, Sj,n.vwi,vdj are the vector representations of wí and esse dj, respectively, λ is a weighting factor applied to a pairwise cosine distance between the vectors.

[0130] The Word2vec® model is used here only as an example. A number of other neural network models can be used to identify word meanings, such as Stanford University's GloVe®. Some libraries, such as GenSim® and Deeplearning4j®, offer a choice of different vector word representation models in a single package.

[0131] In some implementations, the system not only calculates the semantic relatedness between terms and data attributes, but also calculates the type of analytical function associated with each term. For example, the system performs additional steps for the queries “show me the cheapest houses near Ballard” or “where are the mansions in South Lake Union?”. The system considers the corresponding dictionary definitions as additional aspects for these word vectors, and checks if the definitions contain adjectives. Petition 870250038578, dated 12 / 05 / 2025, page 68 / 233 63 / 108 quantitative adjectives such as “less”, “more”, “low”, “high” are mapped using a POS labeler, according to some implementations. The system then maps appropriate analytical functions to these adjectives, according to some implementations. Figure 9B illustrates an example where the expression “more expensive” is mapped to the first N (selling price). Figure 9C illustrates another example where the term “cheapest” is mapped to the last N (selling price). Figure 9D, similarly, illustrates a visualization (940) where the term “deadliest” is mapped to the first N values ​​of the attribute “fatalities”) (942).

[0132] Referring again to Figure 9A, for the example being run, the system calculates the set of prospective conversation centers (922) as {the cheapest, houses, Queen Anne}, according to some implementations. Based on this set, the system performs necessary steps to refine the current visualization in step 924 to generate an update visualization (926), according to some implementations. For this example, the system maps “cheapest” to the last N (sale_price), and refines the current visualization by applying a numerical filter to the house prices. Figure 9C shows an enlarged version of visualization 926.As shown in Figure 9C, the statement “the cheapest” results in the system displaying a view of “Past Home Sales Seattle” (926), with the last 10% of LAST_SALE_PRICE (932) in Queen Anne (930), according to some implementations.

[0133] Figure 10A is a diagram illustrating the application of pragmatic principles to repair utterances according to some implementations. In the course of a conversation, it is common for users to correct or clarify a previous utterance. In some implementations, the system supports the use of accompanying repair utterances to modify or “repair” a Petition 870250038578, dated 12 / 05 / 2025, page 69 / 233 64 / 108 potentially ambiguous utterance or modify the default behavior of how results are presented to the user. For example, to update the system's default behavior, such as highlighting for selection, a user might use utterances like “no, filter instead”. As another example, to update data attributes, a user might use utterances like “get rid of condo” or “change from condo to townhomes”, as shown in Figure 7.

[0134] Figure 10A illustrates how an utterance “remove condos” is understood in the context of the preceding utterance “houses in green lake”. When the system receives an utterance Un (1000), which, in this example, is the utterance “houses in green lake”, the system calculates prospective centers and initializes regressive centers (for the utterance Un+i) in step 1002, using the methodology described above with reference to Figure 5 (step 502), according to some implementations. For the example utterance, the system calculates the set of conversation centers as {houses, green lake}. The system generates or updates (1006) a data visualization (1008) based on the calculated set of conversation centers (1004), according to some implementations.In some implementations, the system applies filters to the dataset based on the set of conversation centers (1004), and displays a data visualization (1008). In this example, the system applies categorical and spatial filters showing houses in Green Lake. Figure 10B shows an enlarged version of visualization 1008. As illustrated in Figure 10B, the utterance “houses in green lake” results in the system displaying a visualization of “Past Home Sales - Seattle” in Green Lake (1028).

[0135] When the system receives an utterance Un+1 (1010), which, in this Petition 870250038578, dated 12 / 05 / 2025, page 70 / 233 65 / 108 example, is the utterance “remove condos” (remove condominiums), the system calculates (1012) the phrase structure for the Un+1 utterance (1010) using the techniques described above with reference to step 522 (Figure 5), according to some implementations. As mentioned above, a repair utterance corrects or clarifies a previous utterance. Based on the phrase structure (1014), the system identifies (1016) the Un+1 utterance (1010) as being a repair utterance and then identifies the relevant repair terms (1018) within the utterance, according to some implementations. In this example, the system identifies a repair term “remove” within the utterance “remove condos” (“remove condominiums”). Based on the set of one or more repair statements and repair terms (1018), the system calculates (1020) a set of prospective conversation centers (1022), according to some implementations.For example, the system identifies conversation centers and / or data attributes in the previous utterance that related to one or more identified repair terms, according to some implementations. In some implementations, as illustrated in Figure 10A, the system repairs or disambiguates between conversation centers based on the repair terms (1018). The set of prospective conversation centers (1022), for this example, is the set {houses not including condos, green lake}. Based on this set, the system performs necessary steps (1024) to update the results from the previous view, according to some implementations. For this example, the system filters condos to show the view in 1026. Figure 10C shows an enlarged version of view 1026.As illustrated in Figure 10C, the statement “remove condos” results in the system displaying a “Past Home Sales - Seattle” view in Green Lake (1028), filtering HOME_TYPE == Condo / Coop (1030). Petition 870250038578, dated 12 / 05 / 2025, page 71 / 233 66 / 108 [0I36] Figure 11A is a diagram illustrating the application of pragmatic principles to manage responses and feedback according to some implementations. A general framework for applying pragmatic principles to visual analytics is first described here to provide context. The framework utilizes and extends a commonly used model for discourse structure called conversational centralization, according to some implementations. In this model, utterances are divided into constituent discourse segments, incorporating relations that may be valid between two segments. A center refers to the entities serving to link this utterance to other utterances in the discourse. Consider a discourse segment DS with utterances Ui, ..., Um.Each statement Un (1 ^ n < m) in the DS is assigned a set of prospective centers, Cf (Un, DS) referring to the current focus of the conversation; each utterance other than the initial utterance of the segment is assigned a set of regressive centers, Cb (Un, DS). The set of regressive centers of a new utterance Un+i is Cb (Un+i, DS), which is equal to the prospective centers of Un (i.e., Cf (Un, DS)). In the context of visual analytic conversions, prospective and regressive centers include data attributes and values, visual properties, and analytic actions (e.g., filter, highlight).

[0137] Given an utterance Un, a system implementing this structure responds by executing a series of analytic functions derived from prospective centers Cf(Un, DS). An analytic function F(X, op, v) consists of a variable X (which can be an attribute or a view property), an operator op, and a value v (usually a constant). For example, when the user says “measles in the uk”, the system creates two functions, namely, F_CAT(diseases, ==, measles) and F_CAT(country, ==, uk). When the user provides a new utterance Un+i, the system first creates a set of temporary centers Ctemp(Un+i, DS) from Un+i without considering any previous context. The system Petition 870250038578, dated 12 / 05 / 2025, page 72 / 233 67 / 108 then applies a set of rules to create a set of prospective centers, Cf (Un+1, DS) based on some operations defined between Cb (Un+1,DS) and Ctemp (Un+1, DS). The prospective centers are then used to respond to the user's statement according to some implementations.

[0138] To support a conversation, the visualizations presented by the system provide cohesive and relevant responses to various utterances. Sometimes, the system responds by altering the visual coding of existing visualizations, while in other cases, the system creates a new graph to support the analytical visual conversation more effectively. In addition to appropriate visualization responses, the system helps the user understand how the system interpreted an utterance by producing appropriate feedback, and allows the user to correct the interpretation through some interface controls as needed. In a traditional dashboard, users interact by selecting items or attributes in a visualization that are highlighted to provide immediate visual feedback. Simultaneously, other graphs are updated by highlighting or filtering items.In a natural language interface, however, instead of making an explicit selection via mouse / keyboard, the user mentions different attributes and values, making it a non-trivial task to decide how each view within an instrument panel should respond to the utterance. Another complication arises when the system needs to support multiple views.

[0139] Figure 11A shows a methodology for generating responses according to some implementations. To decide how the views (V) on an instrument panel should respond to the utterance, a system according to some implementations proceeds as follows. The system calculates a set of prospective conversation centers (1100) Cf (Un+1) corresponding to the utterance Un+1 based on the conversation centers of a previous utterance Un+1 and in Petition 870250038578, dated 12 / 05 / 2025, page 73 / 233 68 / 108 a set of temporary conversation centers calculated using only the current context / utterance. The system creates (1102) a list of all data attributes (1104). Then, the system determines (1106), for example, by invoking a view manager, whether any of the existing views encode a respective attribute in the list of data attributes (1104). Then, the system determines (1118) whether a view (sometimes called a view here) directly encodes (e.g., without using any aggregate functions such as count or average) the respective attribute as its dimensions (e.g., as an aspect of the view). If it is found that the attribute is encoded by an existing view V (i.e., the condition checked in 1118 is true / yes), the system highlights (1122) criteria-related marks corresponding to the respective attribute for an updated dashboard (1124).If, on the other hand, the system determines that a selected visualization (illustrated as visualization V) does not directly encode the respective attribute, the system filters (1120) the results that do not match criteria for an updated dashboard (1124), according to some implementations. This is typically the case when a secondary chart applies additional data transformations to the result set (for example, using a line chart or bar chart). In some such implementations, the system additionally highlights one or more results that match criteria corresponding to the respective attribute for the updated dashboard (1124).

[0140] Figure 11B is an example of visualization (1170) that further illustrates the methodology presented in Figure 11A, according to some implementations. The system highlights the items that correspond to the criteria “measles in the UK” in the map chart (1172). The system also highlights the series (1176) in the line chart (1174), and highlights the bar (1180) in the bar chart (1178) representing “measles”. However, the chart Petition 870250038578, dated 12 / 05 / 2025, page 74 / 233 69 / 108 of bars (1182) in the impact cannot highlight any mark, as it does not encode any attribute in the attribute list (e.g., list {Xi,X2, ...Xm} (1104)). Therefore, the system filters out results that do not match the criteria “measles in the UK” and updates the chart accordingly. Note that users can change the default behavior by explicitly expressing their choice as to whether to filter vs. highlight (e.g., “exclude”, “remove”, “filter only”).

[0141] During the visual analysis workflow, there may be situations where the existing visualization cannot meet the user's growing information needs. This scenario could arise, for example, when a specific data attribute cannot be effectively encoded in the existing visualization (e.g., time values ​​on a map), justifying the need to create a new visualization in response. Drawing inspiration from work that connects visualization with language specification, the system supports the creation of different types of visualizations (e.g., bar chart, line chart, map chart, and scatter plot), according to some implementations.

[0142] Figure 11C shows how an instrument panel is progressively built based on input statements. The system generates visualization 1140 in response to the statement “average price by neighborhood”. When the user provides a subsequent statement “average price over time”, the system responds by generating a line graph (1152) in visualization (1150) that shows the temporal progression of the average price. Now, if the user then provides a statement “by home type”, the system refines the line graph (1152) with lines (1162) corresponding to the different types of houses in visualization (1160).

[0143] Referring again to Figure 11A, the underlying algorithm for Petition 870250038578, dated 12 / 05 / 2025, page 75 / 233 70 / 108 Creating or modifying an existing visualization works as follows. First, the system determines (1106) whether creating a new visualization or modifying an existing one is necessary. The system analyzes the attributes specified in the prospective centers Cf (Un+1) (1100), and searches for any current visualization that encodes these data properties. If there is no match with the specification of existing visualizations, illustrated as the “No” arrow next to decision block 1106, the system generates a new matching specification consisting of attributes and aggregation types. In Figure 11A, this is illustrated by step 1108, which decides the chart type (e.g., a bar chart, a map chart, or a scatter plot) using an algorithm, according to some implementations. In some such implementations, the system employs an automatic presentation algorithm to decide the chart type generated based on this specification.In some such implementations, the system uses a simplified version of the automatic presentation algorithm described in Show me: Automatic presentation for visual analysis, by J. Mackinlay, P. Hanrahan, and C. Stole, which is incorporated herein for reference purposes. Once the chart type (1110) is decided, the system generates (1112) the chart type to obtain a generated chart (1114). The system then positions the new chart (1114), according to some implementations. In some such implementations, the system uses a two-dimensional grid-based layout algorithm, automatically coordinating the presentation of the new chart (1114) with other views of the visualization. The updated instrument panel (1124) responds to subsequent utterances through actions such as highlighting or filtering.

[0144] It is also noted that, although not illustrated in Figure 11A, the system repeats at least steps 1106, 1118, 1120, 1122, 1108, 1112 and 1116 for each data attribute in the list of data attributes (1104), according to some implementations. Petition 870250038578, dated 12 / 05 / 2025, p. 76 / 233 71 / 108

[0145] Figure 12A illustrates a set of widgets generated to manage ambiguity in a user query according to some implementations. A challenge for natural language understanding systems that support interactive dialogue is determining the intent of the utterance. In some implementations, the system automatically resolves various forms of syntactic, lexical, and semantic ambiguities. These resolutions are expressed in the form of widgets and feedback to help the user understand the system's intent and the origin of how the utterance was interpreted. By manipulating these widgets and viewing the feedback on which results are illustrated in the visualization, the user can, for example, instantiate an accompanying repair utterance to override or clarify the system's decisions.

[0146] In some implementations, the system identifies one or more widgets from analytical functions derived from an utterance. In some such implementations, the system organizes and presents the widgets in an intuitive way so that the user can understand how the system interprets their utterance and subsequently modify the interpretation using these widgets. To this end, the system takes the original utterance and orders the widgets in the same sequence as the corresponding query terms. In some such implementations, the system achieves this using a library, such as Sparklificator®, which facilitates the placement and display of small word scale within the text in a compact manner. In addition, some implementations offer a set of user interfaces, including the ability to manipulate and / or remove a widget, modify the query, and resolve ambiguous queries.

[0147] Figure 12A shows how the system presents the widgets for the statement “condo near Ballard under 1.2M” (“condominium near Ballard below 1.2M”, according to some implementations. In this example, the first term Petition 870250038578, dated 12 / 05 / 2025, page 77 / 233 72 / 108 “condo” (condominium) was resolved to the widget representing the criterion “HOME_TYPE equals Condo / coop” (1202). Next, the second widget conveys the fuzzy distance represented by “near Ballard” (1204). Since “under 1.2M” does not explicitly mention any attribute, the system determines if the value 1200000 is within the range of minimum and maximum values ​​of any numeric attribute in the data. If such an attribute exists (LAST_SALE_PRICE in this case), the system communicates this to the user (via widget 1206), and then allows them to change the attribute using the dropdown menu (1208).

[0148] In addition to handling ambiguity, in some implementations, the system also offers feedback and helpful hints for modifying the text when the system fails to fully understand the query. For example, if the system cannot successfully parse a given statement, the system first attempts to automatically correct misspelled terms by comparing the symbols with related attributes, cell values, and keywords in the current dataset using fuzzy sequence correlation. When the user forms a query that is only partially recognized, the system prunes the unrecognized terms from the corresponding parse tree and then displays the results based on the symbols that are understood. Figure 12B shows different examples of situations and the corresponding feedback generated by the system, according to some implementations.

[0149] Figures 13A to 13J present a flowchart illustrating a method 1300 for using (1302) natural language for visual analysis of a dataset applying pragmatic principles. The steps of method 1300 can be performed by a computer (e.g., a computing device 200). In some implementations, the computer includes (1304) a display medium, one or more processors, and memory. Figures 13A to 13J correspond to instructions stored in computer memory or medium. Petition 870250038578, dated 12 / 05 / 2025, page 78 / 233 73 / 108 of computer-readable storage (for example, the memory 206 of the computing device 200). The memory stores (1306) one or more programs configured for execution by one or more processors (for example, the processor(s) 202). For example, the operations of method 1300 are performed, at least in part, by a data visualization generation module 234 and / or by a language processing module 238.

[0150] In some implementations, the device displays (1308) a data visualization based on a dataset retrieved from a database using a first set of one or more queries. For example, referring to Figure 1, a user can associate one or more data fields from a schema information region 110 with one or more shelves (e.g., the column shelf 120 and the row shelf 122, Figure 1) in the data visualization region 112. In response to receiving the user associations, in some implementations, the computer retrieves data for the data fields from the dataset using a set of one or more queries, and then displays a data visualization (e.g., data visualization 408) in the data visualization region 112 that corresponds to the received user inputs. The display of data visualizations is discussed in more detail earlier with reference to Figure 1.

[0151] The computer receives (1310) a first user input to specify a first natural language command related to the displayed data visualization. In some implementations, the user input is received as text input (e.g., via the keyboard 216 or via the touch screen 214) from a user in the data input region in the display medium in proximity to the displayed data visualization. In some implementations, the user input is received as a voice command using a microphone (e.g., an audio input device 220) coupled to Petition 870250038578, dated 12 / 05 / 2025, page 79 / 233 74 / 108 computer. For example, referring to Figure 4A, the data visualization displayed 408 relates to measles in the United Kingdom. Receiving input (e.g., commands / queries) from a user is discussed in more detail above with reference to Figure 1.

[0152] Based on the displayed data visualization, the computer extracts (1312) a first set of one or more independent analytical phrases from the first natural language command. For example, referring to Figure 4A, the first natural language command received by the computer reads “measles in the uk”. The data visualization displayed before receiving the first natural language command concerns disease epidemics worldwide. In some implementations, the computer extracts “measles” and “in the uk” from the first natural language command, as these analytical phrases relate to the displayed data visualization. When the phrases have direct references to the data fields in the displayed data visualization, the extraction (1312) is straightforward: it collects all phrases that are direct references to the data fields.In some implementations, the computer separates the root or removes stopwords (irrelevant words), filler words, or any set of words from the received query, and extracts (1312) all other phrases from the first natural language command, since they may be related to the displayed data visualization. Some implementations use this approach when the phrases in the natural language command have some indirect reference to the data fields in the displayed visualization.

[0153] The language processing module 238 calculates (1314) a first set of one or more conversation centers associated with the first natural language command based on the first set of one or more analytical phrases, according to some implementations. A structure based on a conversational interaction model is described above with reference to Figures 3A, Petition 870250038578, dated 12 / 05 / 2025, page 80 / 233 75 / 108 and 11. A hub refers to the entities that serve to link this utterance (sometimes called here a natural language command) to other utterances in a discourse (a series of utterances). Conversation hubs include data attributes and values, visual properties, and analytical actions. Calculating conversation hubs based on analytical phrases involves mapping analytical phrases to one or more conversation hubs after necessary transformations and analyses. For the example utterance "measles in the UK," the language processing module 238 processes the phrase "measles" and analyzes the phrase to infer that it refers to the data attribute "DISEASE," as illustrated in Figure 4D described above.

[0154] Subsequently, the language processing module 238 computes (1316) a first set of analytic functions associated with the first set of one or more conversation centers, thus creating a first set of one or more functional phrases, according to some implementations. As described above with reference to Figures 3A, 5 and 11, each of the analytic functions consists of a variable, an operator and a value, according to some implementations. In some implementations, for the example of utterance “measles in the uk”, the language processing module 238 creates two functions, namely, F_CAT(diseases, ==, measles) and F_CAT(country, ==, uk). In some implementations, as another example, for a statement "condos under 600K", the language processing module 238 creates two functions: F_CAT(homeType, == condo) and F_NUMERIC(price, <, 600000).In both of these examples, the language processing module 238 searches for one or more attributes related to the displayed data visualization that corresponds to the first set of one or more conversation centers to identify a first. Petition 870250038578, dated 12 / 05 / 2025, page 81 / 233 76 / 108 set of data attributes, according to some implementations. The language processing module 238 also identifies, by examining the first set of one or more conversation centers, a first set of operators (e.g., operator ==, operator <) and a first set of values ​​corresponding to the first set of data attributes, according to some implementations. With the first set of variables (attributes), and the corresponding first set of operators and first set of values, the language processing module 238 constructs the first set of one or more analytical functions, thus creating the first set of one or more functional phrases.

[0155] In some implementations, the computer updates (1318) the data view based on a first set of one or more functional phrases calculated in step 1316. As shown in Figure 13I, in some implementations, the computer queries (1372) the database again using a second set of one or more queries based on the first set of one or more functional phrases, thus retrieving a second set of data. In some cases, the repetition of the database query is performed locally on the computing device using data stored or cached on the computing device. For example, the repetition of the query is typically performed locally when the natural language command specifies one or more filters. In some such implementations, the computer updates (1374) the data view based on the second set of data.In some implementations, the computer additionally creates and displays (1376) a new data view (for example, without updating one or more existing data views) using the second dataset.

[0156] Referring now again to Figure 13B, the computer receives (1320) a second input from the user to specify a second command of Petition 870250038578, dated 12 / 05 / 2025, page 82 / 233 77 / 108 natural language related to the displayed data visualization. In some implementations, user input is received as text input (e.g., via keyboard 216 or via touchscreen 214) from a user in the data input region in the display medium in proximity to the displayed data visualization. In some implementations, user input is received as a voice command using a microphone (e.g., an audio input device 220) attached to the computer. For example, referring to Figure 4A, the displayed data visualization 408 refers to measles in the UK when the computer receives the second user input “show me the orange spike”. Receiving input (e.g., commands / queries) from a user is discussed in more detail above with reference to Figure 1.

[0157] Based on the displayed data visualization, the computer extracts (1322) a second set of one or more independent analytical phrases from the second natural language command. For example, referring to Figure 4A, the second natural language command (410) received by the computer reads “show me the orange spike”. In some implementations, for this example, the computer extracts “the orange spike” from the second natural language command, as these analytical phrases relate to the displayed data visualization (which refers to measles in the UK, and has an orange spike, a visualization property). When the phrases have direct references to the data fields in the displayed data visualization, the extraction (1322) is straightforward: it collects all phrases that are direct references to the data fields.In some implementations, the computer separates the root or removes stopwords (irrelevant words), filler words, or any set of words from the received query, and extracts (1322) all other phrases from the second natural language command, since they may be related to it. Petition 870250038578, dated 12 / 05 / 2025, page 83 / 233 78 / 108 data visualization displayed. Some implementations use this approach when the phrases in the natural language command have some indirect reference to the data fields in the displayed visualization.

[0158] The language processing module (1324) calculates a temporary set of one or more conversation centers associated with the second natural language command based on the second set of one or more analytical phrases, according to some implementations.

[0159] The language processing module (1326) derives a second set of one or more conversation centers from the first set of one or more conversation centers and the temporary set of one or more conversation centers using one or more transition rules, according to some implementations. In some such implementations (1332), each of the conversation centers of the first set of one or more conversation centers, of the temporary set of one or more conversation centers, and of the second set of one or more conversation centers comprises a value for a variable (for example, a data attribute or a view property).In some such implementations, the language processing module uses transition rules by performing a sequence of operations (as shown in Figure 13C) comprising: determining (1334) whether a first variable is included in the first set of one or more conversation centers; determining (1336) whether the first variable is included in the temporary set of one or more conversation centers; determining (1338) a respective transition rule from one or more transition rules to be applied based on whether the first variable is included in the first set of one or more conversation centers and / or in the temporary set of one or more conversation centers; and applying (1339) the respective transition rule.

[0160] In some implementations, as shown in Figure 13D, one or Petition 870250038578, dated 12 / 05 / 2025, page 84 / 233 79 / 108 more transition rules used by the language processing module 238 comprise (1340) comprise a CONTINUE rule to include each conversation center in the first set of one or more conversation centers in the second set of one or more conversation centers, and to add one or more conversation centers from the temporary set of one or more conversation centers to the second set of one or more conversation centers. In some such implementations, applying (1342) the respective transition rule comprises: according to a determination that (i) the first variable is included in the temporary set of one or more conversation centers, and (ii) that the first variable is not included in the first set of one or more conversation centers, applying (1344) the CONTINUE rule to include the first variable in the second set of one or more conversation centers.

[0161] In some implementations, as illustrated in Figure 13E, one or more transition rules used by the language processing module 238 comprise (1346) comprise a RETAIN rule to retain each conversation center in the first set of one or more conversation centers in the second set of one or more conversation centers without adding any conversation center from the temporary set of one or more conversation centers to the second set of one or more conversation centers.In some such implementations, applying (1348) the respective transition rule comprises: according to a determination that (i) the first variable is included in the first set of one or more conversation centers, and (ii) that the first variable is not included in the temporary set of one or more conversation centers, applying (1350) the RETAIN rule to include each conversation center in the first set of one or more conversation centers in the second set of one or more conversation centers.

[0162] In some implementations, as shown in Figure 13F, a or Petition 870250038578, dated 12 / 05 / 2025, page 85 / 233 80 / 108 more transition rules used by the language processing module 238 comprise (1352) a SHIFT rule to include each conversation center in the first set of one or more conversation centers in the second set of one or more conversation centers, and to replace one or more conversation centers in the second set of one or more conversation centers with conversation centers in the temporary set of one or more conversation centers. In some such implementations, applying (1354) the respective transition rule comprises: according to a determination (1356) that (i) the first variable is included in the temporary set of one or more conversation centers, and (ii) that the first variable is not included in the first set of one or more conversation centers,The language processing module 228 performs a sequence of operations to: determine (1358) whether the value of the first variable in the first set of one or more conversation centers is different from the value of the first variable in the temporary set of one or more conversation centers; and, according to a determination that the values ​​of the first variable are different, apply (1360) the SHIFT rule to include each conversation center in the first set of one or more conversation centers in the second set of one or more conversation centers, and replace the value for the first variable in the second set of one or more conversation centers with the value for the first variable in the temporary set of one or more conversation centers. In some such implementations, applying (1354) the respective transition rule additionally comprises, as shown in Figure 13G, determining (1362) whether a widget corresponding to the first variable has been removed by the user,and according to the determination that the widget was removed, apply (1364) the MOVE rule to include each conversation center in the first set of one or more conversation centers in the second set of one or more conversation centers, and, Petition 870250038578, dated 12 / 05 / 2025, page 86 / 233 81 / 108 replace the value for the first variable in the second set of one or more conversation centers with a new value (for example, a maximum value, a superset value) that includes the value of the first variable in the first set of one or more conversation centers.

[0163] Referring now again to Figure 13B, language processing module 238 calculates (1328) a second set of one or more analytical functions associated with the second set of one or more conversation centers, thus creating a second set of one or more functional phrases. Language processing module 238 performs this step, using the second set of one or more conversation centers calculated in step 1326, in a manner similar to step 1316 described above.

[0164] The computer updates (1330) the data view based on the second set of one or more functional phrases, according to some implementations. In some implementations, as shown in Figure 13J, the computer repeats the query (1378) to the database using a second set of one or more queries based on the first set of one or more functional phrases, thus retrieving a third set of data; and updates (1380) the data view based on the third set of data. Additionally, in some such implementations, the computer creates and displays (1382) a new data view (e.g., without updating one or more existing data views) using the third set of data.

[0165] In some implementations, as shown in Figure 13H, the computer additionally determines (1366) whether the user has selected a different dataset from the first dataset, determines (1368) whether the user has restarted the data view; and, according to a determination that (i) the user has selected a different dataset, or (ii) the user has restarted the data view, restarts (1370) each of the first datasets by one or more Petition 870250038578, dated 12 / 05 / 2025, page 87 / 233 82 / 108 conversation centers, the temporary set of one or more conversation centers, and the second set of one or more conversation centers for an empty set that does not include any conversation centers.

[0166] Figures 14A to 14R present a flowchart illustrating a method 1400 for using (1402) natural language for visual analysis of a dataset applying principles of pragmatics, including managing various forms of pragmatics, according to some implementations. The steps of method 1400 can be performed by a computer (e.g., a computing device 200). In some implementations, the computer includes (1404) a display medium, one or more processors, and a memory. Figures 14A to 14R correspond to instructions stored in a computer memory or computer-readable storage medium (e.g., the memory 206 of the computing device 200). The memory stores (1406) one or more programs configured for execution by one or more processors (e.g., the processor(s) 202).For example, the operations of method 1400 are performed, at least in part, by a data visualization generation module 234 and / or by a language processing module 238.

[0167] In some implementations, the device displays (1408) a data visualization based on a dataset retrieved from a database using a first set of one or more queries. For example, referring to Figure 1, a user can associate one or more data fields from a schema information region 110 with one or more shelves (e.g., the column shelf 120 and the row shelf 122, Figure 1) in the data visualization region 112. In response to receiving the user associations, in some implementations, the computer retrieves data for the data fields from the dataset using a set of one or more queries, and then displays a data visualization (e.g., the Petition 870250038578, dated 12 / 05 / 2025, page 88 / 233 83 / 108 data visualization 408) in the data visualization region 112 which corresponds to the received user inputs. The display of data visualizations is discussed in more detail previously with reference to Figure 1.

[0168] The computer receives (1410) a first user input to specify a first natural language command related to the displayed data visualization. In some implementations, the user input is received as text input (e.g., via the keyboard 216 or via the touch screen 214) from a user in the data input region in the display medium in proximity to the displayed data visualization. In some implementations, the user input is received as a voice command using a microphone (e.g., an audio input device 220) attached to the computer. For example, referring to Figure 6A, the displayed data visualization 608 concerns houses less than 1M in Ballard. The receipt of inputs (e.g., commands / queries) from a user is discussed in more detail above with reference to Figure 1.

[0169] Based on the displayed data visualization, the computer extracts (1412) a first set of one or more independent analytical phrases from the first natural language command. For example, referring to Figure 6A, the first natural language command received by the computer reads “houses less than 1M in Ballard”. The data visualization displayed before receiving the first natural language command concerns previous house sales in Seattle. In some implementations, the computer extracts “houses”, “less than 1M”, and “in Ballard” from the first natural language command because these analytical phrases relate to the displayed data visualization. When the phrases have direct references to the data fields in the displayed data visualization, the extraction (1412) is straightforward: it collects all phrases that are direct references to the data fields. In Petition 870250038578, dated 12 / 05 / 2025, page 89 / 233 84 / 108 In some implementations, the computer separates the root or removes “stopwords (irrelevant words), filler words, or any set of words from the received query, and extracts (1412) all other phrases from the first natural language command, since they may be related to the displayed data visualization. Some implementations use this approach when the phrases in the natural language command have some indirect reference to the data fields in the displayed visualization.

[0170] The language processing module 238 calculates (1414) a first set of one or more conversation centers associated with the first natural language command based on the first set of one or more analytic phrases, according to some implementations. A structure based on a conversational interaction model is described above with reference to Figures 3A, 5 and 11. A center refers to the entities that serve to link this utterance (sometimes called here a natural language command) to other utterances in a discourse (a series of utterances). Conversation centers include data attributes and values, visual properties, and analytic actions. Calculating conversation centers based on analytic phrases involves mapping the analytic phrases to one or more conversation centers after necessary transformations and analyses.For the example statement "houses less than 1M in Ballard," the language processing module 238 processes the phrase "less than 1M" and analyzes the phrase to infer that it refers to the data attribute LAST_SALE_PRICE, as illustrated in Figure 6C described above.

[0171] Subsequently, the language processing module 238 calculates (1416) a first set of analytical functions associated with the first set of one or more conversation centers, thus creating a first set of one or more functional phrases, according to some Petition 870250038578, dated 12 / 05 / 2025, page 90 / 233 85 / 108 implementations. As described above with reference to Figures 3A, 5, and 11, each of the analytical functions consists of a variable, an operator, and a value, according to some implementations. In some implementations, for the example statement “houses less than 1M in Ballard”, the language processing module 238 creates two functions, F_CAT(homeType, == condo), F_CAT(homeType, == townhouse), F_CAT(homeType == single), and F_NUMERIC(price, <, 500000). In this example, the language processing module 238 searches for one or more attributes related to the displayed data visualization that corresponds to the first set of one or more conversation centers to identify a first set of data attributes, according to some implementations.The language processing module 238 also identifies, by examining the first set of one or more conversation centers, a first set of operators (e.g., operator ==, operator <) and a first set of values ​​corresponding to the first set of data attributes, according to some implementations. With the first set of variables (attributes), and the corresponding first set of operators and first set of values, the language processing module 238 constructs the first set of one or more analytical functions, thus creating the first set of one or more functional phrases.

[0172] In some implementations, the computer updates (1418) the data view based on a first set of one or more functional phrases calculated in step 1416.

[0173] Referring now to Figure 14B, the computer receives (1420) a second user input to specify a second natural language command related to the displayed data visualization. In some implementations, the user input is received as text input (for example, via the Petition 870250038578, dated 12 / 05 / 2025, page 91 / 233 86 / 108 keyboard 216 or via the touch screen 214) from a user in the data entry region in the display medium in proximity to the displayed data visualization. In some implementations, user input is received as a voice command using a microphone (e.g., an audio input device 220) attached to the computer. For example, referring to Figure 6A, the displayed data visualization 608 relates to houses less than 1M in Ballard, when the computer receives the user's second input “townhomes”. Receiving inputs (e.g., commands / queries) from a user is discussed in more detail above with reference to Figure 1.

[0174] Based on the displayed data visualization, the computer extracts (1422) a second set of one or more independent analytical phrases from the second natural language command. For example, referring to Figure 6A, the second natural language command (620) received by the computer reads “townhomes”. In some implementations, for this example, the computer extracts “townhomes” from the second natural language command, since this analytical phrase relates to the displayed data visualization (which concerns townhomes in Ballard). When the phrases have direct references to the data fields in the displayed data visualization, the extraction (1422) is straightforward: it collects all phrases that are direct references to the data fields.In some implementations, the computer separates the root or removes stopwords (irrelevant words), filler words, or any set of words from the received query, and extracts (1422) all other phrases from the second natural language command, since they may be related to the displayed data visualization. Some implementations use this approach when the phrases in the natural language command have some indirect reference to the data fields in the displayed visualization. Petition 870250038578, dated 12 / 05 / 2025, page 92 / 233 87 / 108

[0175] The language processing module (1424) calculates a temporary set of one or more conversation centers associated with the second natural language command based on the second set of one or more analytical phrases, according to some implementations.

[0176] The language processing module calculates (1426) the cohesion between the first set of one or more analytic sentences and the second set of one or more analytic sentences, and derives a second set of one or more conversation centers from the first set of one or more conversation centers and the temporary set of one or more conversation centers based on cohesion, according to some implementations. As shown in Figure 14C, in some implementations, calculating cohesion involves identifying (1434) a sentence structure from the second set of one or more analytic sentences. The sentence structure calculation is described above with reference to Figure 5 (steps 522 or 510), and through the example in Figure 6A (steps 610 and 622), according to some implementations.In some implementations, identifying the sentence structure involves parsing (1436) the second natural language command by applying a probabilistic grammar (as explained with reference to 522 or 51, Figure 5), thereby obtaining a parsed output. In some implementations, this step further involves deducing (1438) the syntactic structure using a parts of speech parsing API provided by a natural language processing tool library, again as described above with reference to Figure 5. In some implementations, the parsed output is resolved (1440) by the language processing module into corresponding data and categorical attributes. For example, for the utterance “townhomes” (620) in Figure 6A, the language processing module resolves the categorical attribute as being the type of house. In some implementations, although not illustrated, the language processing module. Petition 870250038578, dated 12 / 05 / 2025, page 93 / 233 88 / 108 resolves the parsed output for matching data and categorical attributes after step 1442.

[0177] When the phrase structure is identified in step 1434, the language processing module identifies one or more forms of pragmatic forms based on the phrase structure, according to some implementations. Subsequently, the language processing module derives (1446) the second set of one or more conversation centers from the first set of one or more conversation centers and the temporary set of one or more conversation centers based on the one or more identified pragmatic forms. Figures 14D, 14E, 14H, and 14I described below illustrate how different types of pragmatic forms are identified and how the second set of one or more conversation centers is identified based on the identified pragmatic form.

[0178] In some implementations, language processing module 238 calculates (1430) a second set of one or more analytical functions associated with the second set of one or more conversation centers, thus creating a second set of one or more functional phrases. Language processing module 238 performs this step, using the second set of one or more conversation centers calculated in step 1426, in a manner similar to step 1416 described above.

[0179] In some implementations, as shown in Figure 14P, the language processing module 238 calculates (14.186) the semantic relationship between the second set of one or more extracted analytical phrases and one or more data attributes included in the updated data visualization, and calculates (14.188) analytical functions associated with the second set of one or more analytical phrases, thus creating the second set of one or more functional phrases, based on the semantic relationship of one or more data attributes. Petition 870250038578, dated 12 / 05 / 2025, page 94 / 233 89 / 108 In some implementations, the language processing module 238 calculates semantically related terms for lexically cohesive expressions. The process of calculating semantic relatedness and calculating analytic functions based on semantic relatedness was described above with reference to Figures 9A-9D. In some implementations, although not illustrated in Figure 14B, calculating the second set of one or more analytic functions and the second set of one or more functional phrases based on the semantic relatedness of data attributes is performed in addition to (instead of as an alternative to) step 1430 described above.

[0180] In some implementations, as shown in Figure 14Q, the language processing module 238 calculates semantic relatedness by training (14.190) a first neural network model on a large text corpus, thereby learning word embeddings. In some such implementations (14.192), the first neural network model comprises the Word2vec® model. In some implementations, the language processing module 238 calculates (1494) a first word vector for a first word in a first sentence in the second set of one or more analytical sentences using a second neural network model, the first word vector mapping mapping the first word to the word embeddings learned in step 14.190. In some such implementations (14.196), the second neural network model comprises the recurrent neural network model. In some implementations, the language processing module calculates (14.198) a second word vector for a first data attribute in one or more data attributes using the second neural network model, the second word vector mapping the first data attribute to the word embeddings learned in step 14.190. Although not illustrated in Figure 14Q, the calculation of the first word vector and the second word vector can be performed in. Petition 870250038578, dated 12 / 05 / 2025, page 95 / 233 90 / 108 parallel by language processing module 238. Subsequently, the language processing module calculates (14.200) the relatedness between the first word vector and the second word vector using a similarity metric, according to some implementations. In some of these implementations (14.202), the similarity metric is based at least on (i) the Wu-Palmer distance between the word senses associated with the first word vector and the second word vector, (ii) a weighting factor, and (iii) a pairwise cosine distance between the first word vector and the second word vector.

[0181] In some implementations, as shown in Figure 14R, language processing module 238 obtains (14.204) word definitions for the second set of one or more analytic sentences from a publicly available dictionary, determines (14.206) whether the word definitions contain one or more predefined adjectives using a morphosyntactic parsing API provided by a natural language processing tool library, and, according to the determination that the word definitions contain one or more predefined adjectives, maps the one or more redefined adjectives to one or more analytic functions. These operations have been described above with reference to Figure 9B, according to some implementations. Language processing module 238 calculates the analytic function type for one or more terms in the second set of one or more analytic sentences. For example, the term “cheapest” is mapped to Bottom_N(sale price).

[0182] Referring now again to Figure 14B, the computer updates (1432) the data visualization based on the second set of one or more functional phrases, according to some implementations.

[0183] Each of Figures 14D, 14E, 14H, and 14I shows the steps performed by language processing module 238 to handle Petition 870250038578, dated 12 / 05 / 2025, page 96 / 233 91 / 108 different types of pragmatic forms identified based on sentence structure, and how the second set of one or more conversation centers is derived based on the identified pragmatic form, according to some implementations.

[0184] Figure 14D shows the steps performed by language processing module 238 to derive the second set of one or more conversation centers for incomplete utterances, according to some implementations. Language processing module 238 identifies (1448) the pragmatic form as being an incomplete utterance by determining whether one or more linguistic elements are missing from the sentence structure, according to some implementations.In some implementations, subsequently, the language processing module 238 derives (1450) the second set of one or more conversation centers by performing a sequence of operations (1452) that includes: determining (1454) a first subset of conversation centers in the first set of one or more conversation centers, the first subset of conversation centers corresponding to the one or more missing linguistic elements in the sentence structure, and calculating (1456) the second set of one or more conversation centers by combining the temporary set of one or more conversation centers with the first subset of conversation centers. Figure 6A described above shows an implementation that derives the second set of one or more conversation centers for incomplete utterances.

[0185] Figure 14E shows the steps performed by language processing module 238 to derive the second set of one or more conversation centers for reference expressions, according to some implementations. Language processing module 238 identifies (1458) the pragmatic form as being a reference expression by determining whether one or more anaphoric references are present in the sentence structure, according to Petition 870250038578, dated 12 / 05 / 2025, page 97 / 233 92 / 108 some implementations. In some implementations, subsequently, the language processing module 238 derives (1460) the second set of one or more conversation centers by performing a sequence of operations (1462) that includes: searching (1464) the first set of one or more conversation centers to find a first subset of conversation centers that corresponds to a phrase block in the second natural language command that contains a first anaphoric reference of one or more anaphoric references, and calculating (1466) the second set of one or more conversation centers based on the temporary set of one or more conversation centers and the first subset of conversation centers. Figure 7A described above shows an implementation that derives the second set of one or more conversation centers for reference expressions.

[0186] In some implementations, the language processing module 238 determines (1468) whether the first anaphoric reference is a reference to a view property in the updated data view (sometimes called here a deictic reference), and, according to a determination that the anaphoric reference is a deictic reference, calculates (1470) the second set of one or more conversation centers based on the temporary set of one or more conversation centers, and data related to the view property.

[0187] In some implementations, as illustrated in Figure 14F, the language processing module 238 determines (1472) whether the first anaphoric reference is accompanied by a verb in the second natural language command, and, according to a determination that the anaphoric reference is accompanied by a verb (1474), searches (1476) the first set of one or more conversation centers to find a first action conversation center that refers to an action verb, and calculates (1478) the second set of one or more Petition 870250038578, dated 12 / 05 / 2025, page 98 / 233 93 / 108 conversation centers based on the temporary set of one or more conversation centers and the first subset of conversation centers.

[0188] In some implementations, as shown in Figure 14G, the language processing module 238 determines (1480) whether the first anaphoric reference is a deictic reference that refers to some object in the environment, and, according to a determination that the anaphoric reference is a deictic reference, calculates (1482) the second set of one or more conversation centers based on the temporary set of one or more conversation centers, and a feature of the object.

[0189] Figure 14H shows the steps performed by language processing module 238 to derive the second set of one or more conversation centers for repair utterances, according to some implementations. Language processing module 238 identifies (1484) the pragmatic form as being a repair utterance by determining whether the phrase structure corresponds to one or more predefined repair utterances, according to some implementations.In some implementations, subsequently, the language processing module 238 derives (1486) the second set of one or more conversation centers by performing a sequence of operations (1488) that includes: calculating (1490) the second set of one or more conversation centers based on the temporary set of one or more conversation centers, and updating (1492) one or more data attributes in the second set of one or more conversation centers based on one or more predefined repair utterances and sentence structure. Figure 10A described above shows an implementation that derives the second set of one or more conversation centers for repair utterances.

[0190] In some implementations, the language processing module 238 determines (1494) whether the sentence structure matches a Petition 870250038578, dated 12 / 05 / 2025, page 99 / 233 94 / 108 repair statement to modify a default behavior related to the display of a data visualization, and, according to a determination that the sentence structure corresponds to a repair statement to modify a default behavior, alters (1496) the default behavior related to the display.

[0191] Figure 14I shows the steps performed by language processing module 238 to derive the second set of one or more conversation centers for conjunctive expressions, according to some implementations. Language processing module 238 identifies (1498) the pragmatic form as being a conjunctive expression by determining whether the second natural language command is a conjunctive expression by (i) determining the explicit or implicit presence of conjunctions in the sentence structure, and (ii) determining whether the temporary set of one or more conversation centers includes each conversation center in the first set of one or more conversation centers, according to some implementations. In some implementations, subsequently, language processing module 238 derives (14.100) the second set of one or more conversation centers by calculating (14.104) the second set of one or more conversation centers based on the temporary set of one or more conversation centers, in accordance with the determination (14.102) that the second natural language command is a conjunctive expression. Figure 8A described above shows an implementation that derives the second set of one or more conversation centers for utterances with conjunctions.

[0192] In some implementations, the language processing module 238 determines (14.106) whether the second natural language command has more than one set, and, according to the determination that the second natural language command has more than one set, calculates (14.108) the second set. Petition 870250038578, dated 12 / 05 / 2025, pages 100 / 233 95 / 108 of one or more analytical functions linearizing the second natural language command.

[0193] In some implementations, the language processing module 238 linearizes the second natural language command by performing a sequence of operations illustrated in Figure 14J. The sequence of operations includes generating (14.110) a parse tree for the second natural language command, traversing (14.112) the parse tree in post-order to extract a first analytic phrase and a second analytic phrase, wherein the first analytic phrase and the second analytic phrase are adjacent nodes in the parse tree, and combining (14.114) the first analytic function with the second analytic function by applying one or more logical operators based on one or more features of the first analytic function and the second analytic function, wherein one or more features include attribute type, operator type, and a value.

[0194] Each of Figures 14K-14O illustrates different instances of the last step (14.114) of combining the first analytic function with the second analytic function, according to some implementations. In each case (as illustrated by the labels 14.116, 14.126, 14.36, 14.150, and 14.168 in the respective figures), the first analytic function comprises a first attribute, a first operator, and a first value; the second analytic function comprises a second attribute, a second operator, and a second value.

[0195] In Figure 14K, the language processing module 238 combines (14.118) the first analytic function with the second analytic function performing a sequence of operations, according to some implementations. The sequence of operations includes: determining (14.120) whether the first attribute is a categorical attribute or an ordered attribute, and determining whether the second attribute is a categorical attribute or an ordered attribute, determining (14.122) whether the first attribute and the second attribute are identical, of Petition 870250038578, dated 12 / 05 / 2025, p. 101 / 233 96 / 108 according to a determination that the first attribute and the second attribute are identical and both are attributes of categorical type, apply (14.124) a union operator to combine the first analytic function and the second analytic function.

[0196] In Figure 14L, the language processing module 238 combines (14.128) the first analytic function with the second analytic function performing a sequence of operations, according to some implementations. The sequence of operations includes: determining (14.130) whether the first attribute is a categorical attribute or an ordered attribute, and determining whether the second attribute is a categorical attribute or an ordered attribute, determining (14.132) whether the first attribute and the second attribute are identical, according to a determination that the first attribute and the second attribute are not identical, applying (14.134) the intersection operator to combine the first analytic function and the second analytic function.

[0197] In Figure 14M, the language processing module 238 combines (14.138) the first analytic function with the second analytic function performing a sequence of operations, according to some implementations. The sequence of operations includes: determining (14.140) whether the first attribute is a categorical attribute or an ordered attribute, and determining whether the second attribute is a categorical attribute or an ordered attribute, determining (14.142) whether the first attribute and the second attribute are identical, according to a determination that the first attribute and the second attribute are identical and both are ordered attributes (14.144); determining (14.146) the operator types of the first operator and the second operator, and, according to a determination that both the first operator and the second operator are equality operators, applying (14.148) the union operator to combine the first analytic function and the second analytic function.

[0198] In Figure 14N, the language processing module 238 Petition 870250038578, dated 12 / 05 / 2025, page 102 / 233 97 / 108 combines (14.152) the first analytic function with the second analytic function performing a sequence of operations, according to some implementations. The sequence of operations includes: determining (14.154) whether the first attribute is a categorical attribute or an ordered attribute, and determining whether the second attribute is a categorical attribute or an ordered attribute, determining (14.156) whether the first attribute and the second attribute are identical, and, according to a determination that the first attribute and the second attribute are identical and both are ordered attributes: determining (14.160) the operator types of the first operator and the second operator; according to a determination that the first operator is a “less than” operator and the second operator is a “greater than” operator (14.162): determining (14.164) if the first value is less than the second value, and, according to a determination that the first value is less than the second value, apply (14.166) the union operator to combine the first analytic function and the second analytic function.

[0199] In Figure 14O, the language processing module 238 combines (14.170) the first analytic function with the second analytic function performing a sequence of operations, according to some implementations. The sequence of operations includes: determining (14.172) whether the first attribute is a categorical attribute or an ordered attribute, and determining whether the second attribute is a categorical attribute or an ordered attribute, determining (14.174) whether the first attribute and the second attribute are identical, according to a determination that the first attribute and the second attribute are identical and both are ordered attributes (14.176): determining (14.178) the operator types of the first operator and the second operator; according to a determination that the first operator is a “greater than” operator and the second operator is a “less than” operator (14.180): determining (14.182) if the first value is less than the second value, and according to a determination. Petition 870250038578, dated 12 / 05 / 2025, page 103 / 233 98 / 108 that the first value is less than the second value, apply (14.184) the intersection operator to combine the first analytic function and the second analytic function.

[0200] Figures 15A to 15H present a flowchart of a process (method 1500) that uses (1502) natural language for visual analysis of a dataset applying pragmatic principles, including managing responses and feedback, and handling ambiguity in a user query, according to some implementations. The steps of method 1500 can be performed by a computer (e.g., a computing device 200). In some implementations, the computer includes (1504) a display medium, one or more processors, and memory. Figures 15A to 15H correspond to instructions stored in computer memory or computer-readable storage medium (e.g., the memory 206 of the computing device 200). The memory stores (1506) one or more programs configured for execution by one or more processors (e.g., the processor(s) 202).For example, the operations of method 1500 are performed, at least in part, by a data visualization generation module 234 and / or by a language processing module 238.

[0201] In some implementations, the device displays (1508) a data view based on a dataset retrieved from a database using a first set of one or more queries. For example, referring to Figure 1, a user can associate one or more data fields from a schema information region 110 with one or more shelves (e.g., the column shelf 120 and the row shelf 122, Figure 1) in the data view region 112. In response to receiving the user associations, in some implementations, the computer retrieves data for the data fields from the dataset using a set Petition 870250038578, dated 12 / 05 / 2025, page 104 / 233 99 / 108 of one or more queries, and then displays a data visualization (e.g., data visualization 408) in data visualization region 112 that corresponds to the received user inputs. The display of data visualizations is discussed in more detail earlier with reference to Figure 1.

[0202] The computer receives (1510) a first user input to specify a first natural language command related to the displayed data visualization. In some implementations, the user input is received as text input (e.g., via the keyboard 216 or via the touch screen 214) from a user in the data input region in the display medium in proximity to the displayed data visualization. In some implementations, the user input is received as a voice command using a microphone (e.g., an audio input device 220) attached to the computer. For example, referring to Figure 6A, the displayed data visualization 608 concerns houses less than 1M in Ballard. The receipt of inputs (e.g., commands / queries) from a user is discussed in more detail above with reference to Figure 1.

[0203] Based on the displayed data visualization, the computer extracts (1512) a first set of one or more independent analytical phrases from the first natural language command. For example, referring to Figure 6A, the first natural language command received by the computer reads “houses less than 1M in Ballard”. The data visualization displayed before receiving the first natural language command concerns previous house sales in Seattle. In some implementations, the computer extracts “houses”, “less than 1M”, and “in Ballard” from the first natural language command because these analytical phrases relate to the displayed data visualization. When the phrases have a direct reference to the data fields in the displayed data visualization, the extraction (1512) is Petition 870250038578, dated 12 / 05 / 2025, page 105 / 233 100 / 108 direct: collects all phrases that are direct references to data fields. In some implementations, the computer separates the root or removes stopwords (irrelevant words), filler words, or any set of words from the received query, and extracts (1412) all other phrases from the first natural language command, since they may be related to the displayed data visualization. Some implementations use this approach when the phrases in the natural language command have some indirect reference to the data fields in the displayed visualization.

[0204] The language processing module 238 calculates (1514) a first set of one or more conversation centers associated with the first natural language command based on the first set of one or more analytic phrases, according to some implementations. A structure based on a conversational interaction model is described above with reference to Figures 3A, 5 and 11. A center refers to the entities that serve to link this utterance (sometimes called here a natural language command) to other utterances in a discourse (a series of utterances). Conversation centers include data attributes and values, visual properties, and analytic actions. Calculating conversation centers based on analytic phrases involves mapping the analytic phrases to one or more conversation centers after necessary transformations and analyses.For the example statement "houses less than 1M in Ballard," the language processing module 238 processes the phrase "less than 1M" and analyzes the phrase to infer that it refers to the data attribute LAST_SALE_PRICE, as illustrated in Figure 6C described above.

[0205] Subsequently, the language processing module 238 calculates (1516) a first set of analytical functions associated with the first set of one or more conversation centers, thus creating a first Petition 870250038578, dated 12 / 05 / 2025, page 106 / 233 101 / 108 a set of one or more functional sentences, according to some implementations. As described above with reference to Figures 3A, 5, and 11, each of the analytical functions consists of a variable, an operator, and a value, according to some implementations. In some implementations, for the example statement “houses less than 1M in Ballard”, the language processing module 238 creates two functions, F_CAT(homeType, == condo), F_CAT(homeType, == townhouse), F_CAT(homeType == single), and F_NUMERIC(price, <, 500000). In this example, the language processing module 238 searches for one or more attributes related to the displayed data visualization that corresponds to the first set of one or more conversation centers to identify a first set of data attributes, according to some implementations.The language processing module 238 also identifies, by examining the first set of one or more conversation centers, a first set of operators (e.g., operator ==, operator <) and a first set of values ​​corresponding to the first set of data attributes, according to some implementations. With the first set of variables (attributes), and the corresponding first set of operators and first set of values, the language processing module 238 constructs the first set of one or more analytical functions, thus creating the first set of one or more functional phrases.

[0206] In some implementations, the computer updates (1518) the data view based on a first set of one or more functional phrases calculated in step 1516.

[0207] Referring now to Figure 15B, the computer receives (1520) a second user input to specify a second natural language command related to the displayed data visualization. In some implementations, the Petition 870250038578, dated 12 / 05 / 2025, page 107 / 233 102 / 108 User input is received as text input (e.g., via keyboard 216 or via touchscreen 214) from a user in the data entry region in the display medium in proximity to the displayed data visualization. In some implementations, user input is received as a voice command using a microphone (e.g., an audio input device 220) attached to the computer. For example, referring to Figure 6A, the displayed data visualization 608 relates to houses less than 1M away in Ballard, when the computer receives the second user input “townhomes”. The receipt of inputs (e.g., commands / queries) from a user is discussed in more detail above with reference to Figure 1.

[0208] Based on the displayed data visualization, the computer extracts (1522) a second set of one or more independent analytical phrases from the second natural language command. For example, referring to Figure 6A, the second natural language command (620) received by the computer reads “townhomes”. In some implementations, for this example, the computer extracts “townhomes” from the second natural language command, since this analytical phrase relates to the displayed data visualization (which concerns townhomes in Ballard). When the phrases have direct references to the data fields in the displayed data visualization, the extraction (1522) is straightforward: it collects all phrases that are direct references to the data fields.In some implementations, the computer separates the root or removes stopwords (irrelevant words), filler words, or any set of words from the received query, and extracts (1522) all other phrases from the second natural language command, since they may be related to the displayed data visualization. Some implementations use this approach when the phrases in the natural language command have some indirect reference to the data fields in the visualization. Petition 870250038578, dated 12 / 05 / 2025, page 108 / 233 103 / 108 displayed.

[0209] The language processing module (1524) calculates a temporary set of one or more conversation centers associated with the second natural language command based on the second set of one or more analytical phrases, according to some implementations.

[0210] The language processing module (1526) derives a second set of one or more conversation centers from the first set of one or more conversation centers and the temporary set of one or more conversation centers using one or more transition rules, according to some implementations.

[0211] The computer updates (1528) the data visualization based on the second set of one or more functional phrases, according to some implementations.

[0212] Referring to Figure 15C, in some implementations, language processing module 238 determines (1530) one or more data attributes corresponding to the second set of one or more conversation centers. Language processing module 238 then scans (1532) displayed data views to identify one or more of the displayed data views that contain data tags whose characteristics correspond to a first data attribute in one or more data attributes, according to some implementations. In some such implementations (1534), the view characteristics include one or more of color, size, and format. In some such implementations (1536), the view features correspond to a visual encoding of data tags.

[0213] Subsequently, the computer highlights (1538) the data tags whose characteristics correspond to the first data attribute, according to some implementations. In some of these implementations, the computer filters Petition 870250038578, dated 12 / 05 / 2025, p. 109 / 233 104 / 108 (1540) the results of the displayed data visualizations that contain data tags whose characteristics do not match one or more data attributes. Furthermore, in some such implementations, the computer receives (1542) user input to determine whether to filter or highlight the data tags, and filters or highlights the data tags in the displayed data visualizations based on that determination. Figure 11A described above showed an example of applying pragmatic principles to manage responses and feedback according to some implementations. The descriptions for Figure 11A apply to the steps illustrated in Figure 15C. For example, step 1102 in Figure 11A to create a list of all data attributes corresponds to step 1530 to determine one or more data attributes.Similarly, step 1106, which determines which of the existing views encodes a given attribute, corresponds to step 1532, which scans the displayed data views.

[0214] Referring now to Figure 15D, in some implementations, the computer determines (1544) whether none of the displayed data visualizations contain data tags whose characteristics match the first data attribute. In some implementations, according to the determination that none of the displayed data visualizations contain data tags whose characteristics match the first data attribute (1546): the computer generates (1548) a specification for a new data visualization with the first data attribute, and displays (1550) the new data visualization. In some implementations, displaying (1550) the new data visualization includes: determining (1552) a chart type based on the specification, and generating and displaying (1554) the chart. In some such implementations (1556), the chart is positioned using a two-dimensional grid-based layout algorithm, automatically coordinated with other data visualizations.

[0215] Referring to Figure 15E, in some implementations, the module of Petition 870250038578, dated 12 / 05 / 2025, p. 110 / 233 105 / 108 language processing 238 calculates (1558) a second set of one or more analytical functions associated with the second set of one or more conversation centers, thereby creating a second set of one or more functional phrases. In some implementations, the computer selects (1560) a first functional phrase from the second set of one or more functional phrases, wherein the first functional phrase comprises a parameterized data selection criterion. In some implementations, the computer selects (1562) an initial range for parameter values ​​of the parameterized data selection criterion. In some implementations, the computer displays (1564) an editable user interface control corresponding to the parameterized data selection criterion, wherein the user interface control displays the current parameter values.In some such implementations (1566), the user interface control allows adjustment of the first functional phrase. Additionally, in some such implementations (1568), the user interface control displays a slider, which allows a user to adjust the first functional phrase. In some implementations, the computer orders (1570) a displayed set of one or more editable user interface controls based on the query order in the second natural language command, wherein the query order is inferred while extracting the second set of one or more analytical phrases from the second natural language command. In some such implementations, the computer uses (1572) a library that facilitates the compact placement of small word-scale visualization within the text. In some such implementations (1574), the library is Sparklificator®.Figure 12A described above shows an example of an interface that includes a selectable set of widgets presented to the user, according to some implementations.

[0216] Referring to Figure 15F, in some implementations, the Petition 870250038578, dated 12 / 05 / 2025, p. 111 / 233 106 / 108 The computer determines (1576) a first symbol in the second natural language command that does not correspond to any of the analytic phrases in the second set of one or more analytic phrases. In some implementations, the computer searches (1578) for a correctly spelled term corresponding to the first symbol using a search library by comparing the first symbol with one or more aspects of the first dataset. In some such implementations (1580), the one or more aspects include data attributes, cell values, and related keywords from the first dataset. In some such implementations (1582), the search library is Fuse.js®.

[0217] In some implementations, the language processing module 238 replaces (1584) the correctly spelled term for the first symbol in the second natural language command to obtain a third natural language command, and extracts (1586) the second set of one or more analytic phrases from the third natural language command.

[0218] In some implementations, as illustrated in Figure 15G, the language processing module 238 determines (1588) whether there is no correctly spelled term corresponding to the first symbol. According to a determination that there is no correctly spelled term corresponding to the first symbol (1590), the language processing module parses (1592) the second natural language command to obtain a parse tree, prunes (1594) the parse tree to remove the part of the tree corresponding to the first symbol, and extracts (1596) the parse tree to remove the part of the tree corresponding to the first symbol. In some such implementations, the language processing module 238 replaces (1598) the correctly spelled term for the first symbol in the second natural language command to obtain a third natural language command, and the Petition 870250038578, dated 12 / 05 / 2025, page 112 / 233 107 / 108 computer displays (15,100) the first symbol.

[0219] In some implementations, as illustrated in Figure 15H, the computer generates (15.102) textual feedback indicating that the correctly spelled term is used as a substitute for the first symbol in the second natural language command. Furthermore, in some such implementations, the computer displays (15.104) and highlights the correctly spelled term. Figure 12B described above shows different examples of situations and the corresponding feedback generated by the computer, according to some implementations.

[0220] The terminology used in the description of this invention serves only the purpose of describing specific implementations and is not intended to limit the present invention. As used in the description of the invention and the appended claims, the singular forms "a," "an," "or" are intended to include the plural forms as well, unless otherwise indicated by the context. It will also be understood that the term "and / or," as used herein, refers to and encompasses any and all possible combinations of one or more of the associated listed items. It should also be understood that the terms "comprises" and / or "comprising," when used in this descriptive report, specify the presence of mentioned aspects, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more different aspects, steps, operations, elements, components, and / or groups thereof.

[0221] The preceding description, for explanatory purposes, has been described with reference to specific implementations. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the exact forms disclosed. Many modifications and variations are possible in light of the above teachings. The implementations have been chosen and described in order to best explain the principles of the invention and its practical applications, thereby enabling other individuals skilled in the art to make the best use of the invention. Petition 870250038578, dated 12 / 05 / 2025, page 113 / 233 108 / 108 invention and various implementations with diverse modifications insofar as they are appropriate to the specific use contemplated. Petition 870250038578, dated 12 / 05 / 2025, page 114 / 233

Claims

1 / 4 CLAIMS 1. A method for using natural language for visual analysis of a dataset, CHARACTERIZED by comprising: on a computer having a display medium, one or more processors, and a memory storing one or more programs configured for execution by the one or more processors: displaying, on the display medium, a data visualization application, including displaying a data visualization based on a dataset retrieved from a database using a set of one or more queries; receiving, by the data visualization application, user input to specify a natural language command related to the displayed data visualization; based on the displayed data visualization, extracting one or more independent analytical phrases from the natural language command;for a first analytical sentence among one or more extracted analytical sentences: calculate semantic relatedness between the first analytical sentence and numeric data fields in the dataset, wherein calculating semantic relatedness includes: generating a plurality of word embeddings for the first analytical sentence using a plurality of neural network models trained on a large body of text; and generating statistically similar words for the plurality of word embeddings using a feature reduction algorithm; identifying one or more of the numeric data fields having the highest calculated semantic relatedness to the first analytical sentence; and selecting a numeric function according to the first analytical sentence, wherein the numeric function compares data values ​​in one or more numeric data fields with a threshold value;Retrieve an updated dataset, including querying the database again using an updated set of one or more queries to filter one or more identified numeric data fields according to the numeric function; and display, in the data visualization application, a visualization of the updated data using the updated dataset.

2. Method, according to claim 1, CHARACTERIZED in that the plurality of neural network models comprises Word2vec® models, the Word2vec® models using a continuous bag of words model architecture and a skip-gram model architecture, and in which the plurality of word embeddings are word vectors.

3. Method, according to claim 1, CHARACTERIZED in that training the plurality of neural network models comprises a plurality of training methods, the plurality of training methods including a training method with negative sampling and a training method without negative sampling.

4. Method, according to claim 1, CHARACTERIZED in that calculating semantic relatedness between the first analytic phrase and numerical data fields in the dataset further includes: receiving a large lexical database; and for the first analytic phrase, extracting synonyms for the phrase from the large lexical database received.

5. Method, according to claim 1, CHARACTERIZED in that selecting the numerical function comprises: obtaining word definitions for the first analytic phrase from a publicly available dictionary; computing a set of derived words by deriving the obtained word definitions; computing a bag of words based on the set of derived words; determining whether the bag of words contains a predefined adjective; and according to the determination that the bag of words contains the predefined adjective, mapping the predefined adjective to an analytic function.

6. Method, according to claim 5, CHARACTERIZED in that determining whether the bag of words contains the predefined adjective comprises using a morphosyntactic (part-of-speech) parsing API provided by a natural language tool library to check for the presence of the predefined adjective.

7. Method, according to claim 1, CHARACTERIZED in that a first numerical function comprises a parameterized data selection criterion, and in that querying the database again to filter one or more identified numerical data fields according to the numerical functions includes selecting an initial range for values ​​of the parameters of the parameterized data selection criterion.

8. Method according to claim 7, CHARACTERIZED in that it further comprises displaying an editable user interface control corresponding to the parameterized data selection criteria, wherein the user interface control displays the current values ​​of the parameters.

9. Method according to claim 8, CHARACTERIZED in that the user interface control includes a control for adjusting the numeric function.

10. Method, according to claim 9, CHARACTERIZED by the fact that the user interface control displays a slider to adjust the numeric function.

11. A method according to claim 1, characterized in that, when the natural language command refers to visualization features of one or more data tags in the displayed data visualizations, the method further comprises: scanning the displayed data visualizations to identify one or more of the displayed data visualizations that contain data tags whose features match the natural language command; and highlighting the data tags whose features match the natural language command.

12. Method according to claim 11, CHARACTERIZED in that the display features include one or more of color, size and shape.

13. Electronic device CHARACTERIZED by comprising: a display means; one or more processors; memory; and one or more programs, wherein the one or more programs are stored in memory and configured to be executed by the one or more processors, the one or more programs including instructions to execute the method as defined in any one of claims 1 to 12.

14. Non-transient computer-readable storage medium CHARACTERIZED in that it stores one or more programs configured for execution by an electronic device with a display medium, or one or more programs comprising instructions for executing the method as defined in any of claims 1 to 12. Petition 870250038578, dated 12 / 05 / 2025, pp. 118 / 233

Images