A method and system for constructing visual charts based on natural language

Abstract

This invention relates to a method and system for constructing visualization charts based on natural language. It accepts natural language and data attributes as input through an attribute-level model, predicting attributes and filters for each attribute. A context state machine determines the content to be visualized based on the context and the predicted attributes and filters of the current sentence. Based on the attributes and filters to be visualized, a visualization generation module constructs the visualization chart. Using the method disclosed in this invention, a deep learning-based approach parses fuzzy and diverse natural language into data attributes and filtering conditions, thereby constructing visualization charts based on the extracted data attributes to respond to the user's natural language. This lowers the barrier to data visualization, is highly practical, and has significant potential for widespread adoption.

Description

Technical Field

[0001] This invention belongs to the field of visualization, specifically relating to a method and system for constructing visual charts based on natural language. Background Technology

[0002] Visualization helps to clearly and intuitively represent data characteristics and is an important way to express data content. However, existing visualization creation tools require programming and other methods, demanding a certain level of expertise from users. The high barrier to entry for visualization creation hinders its widespread adoption by the general public.

[0003] Language is a natural way to express needs, and the public can naturally express their needs through language. However, natural language is ambiguous, and the same meaning can be expressed in multiple ways. The challenge in visualizing natural language lies in the ambiguity between the user's language and the visualization, as well as the visualization choices for different scenarios. Traditional natural language visualization interfaces require users to specify specific data attributes and visualization mappings, and users need to follow specific rules to specify the visualization. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for constructing visual charts based on natural language. It proposes a deep learning-based method to parse natural language into data attributes and filtering conditions, and then constructs visual charts based on the extracted data attributes to respond to the user's natural language.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: Firstly, a method for constructing visual charts based on natural language, the method comprising the following steps:

[0006] S1, the attribute-level model accepts natural language and data attributes as input, and predicts attributes and filters for each data attribute;

[0007] S2. The context state machine determines the attributes and filters that need to be visualized based on the context and the predicted attributes and filters of the current sentence.

[0008] S3. Based on the attributes and filters that need to be visualized, construct visualization charts through the visualization generation module.

[0009] Furthermore, in step S1, the input is converted into a unified vector representation through the sentence transfer model in the attribute-level model. These vector representations serve as inputs to the attribute parsing module and the filter parsing module, respectively, and are used to predict the attributes and filters of the input.

[0010] Furthermore, the sentence transfer model is pre-trained using a training dataset, separating the training of attributes and filters.

[0011] Furthermore, the training dataset comprises two parts: the first part is the numerical acquisition dataset WikiSQL borrowed from the database domain, and the second part is a dataset constructed based on users' habits of exploring tabular data.

[0012] Furthermore, the second part of the training dataset collects common expressions of user-explored tabular data and abstracts them into templates. A certain number of table attributes are inserted into the templates to generate candidate sentences. After reverse translation, a large number of diverse sentences are obtained.

[0013] Furthermore, the attribute parsing module accepts the vector w of the sentence transfer model as input, generates a probability from 0 to 1 based on the vector w, determines the aggregation type present in the attributes, determines the number of selected attributes, and selects the n attributes with the highest probability as the selected prediction attributes.

[0014] Furthermore, the filter parsing module accepts the vector w of the sentence transfer model as input, predicts the probability of the presence of a filter in the attribute, the operation type of the filter, the filter value, and the number m of selected filters, and selects the m filter conditions with the highest probability as the selected prediction filters.

[0015] Furthermore, in step S2, a context flag is set to allow users to decide whether to inherit the context filtering conditions. The maximum upper limit of the attributes is set, and the visualization content to be displayed is sorted according to their probability and timeliness.

[0016] Furthermore, in step S3, the appropriate visualization type is determined based on the user's ability to perceive different attributes.

[0017] In a second aspect, a system for constructing visual charts based on natural language employs a method for constructing visual charts based on natural language as described in the first aspect of the present invention and any optional embodiment thereof.

[0018] The beneficial technical effects of this invention are as follows: The method and system for constructing visual charts based on natural language disclosed in this invention use deep learning to parse fuzzy and diversified natural language into data attributes and filtering conditions, and then construct visual charts based on the extracted data attributes to respond to the user's natural language, thereby lowering the threshold for data visualization, having strong practicality, and laying the foundation for the promotion of visual data. Attached Figure Description

[0019] Figure 1 This is a flowchart of a method for constructing visual charts based on natural language, as described in Embodiment 1 of the present invention.

[0020] Figure 2This is a schematic diagram of the attribute parsing process in a method for constructing visual charts based on natural language, as described in Embodiment 1 of the present invention.

[0021] Figure 3 This is a schematic diagram of the filtering and parsing process in a method for constructing visual charts based on natural language, as described in Embodiment 1 of the present invention.

[0022] Figure 4 This is a schematic diagram illustrating the visualization results generated for various sentences using the method for constructing visualization charts based on natural language as described in Embodiment 1 of the present invention.

[0023] Figure 5 This is a system interface diagram of a visualization chart based on natural language, as described in Embodiment 2 of the present invention.

[0024] Figure 6 This is a schematic diagram illustrating the visualization results generated for a sequence of paragraph text using a system for constructing visualization charts based on natural language, as described in Embodiment 2 of the present invention. Detailed Implementation

[0025] The present invention will now be further described with reference to the accompanying drawings and specific embodiments.

[0026] Example 1

[0027] like Figure 1 As shown in the figure, this embodiment of the invention provides a method for constructing visual charts based on natural language, the method comprising the following steps:

[0028] S1, the attribute-level model takes natural language and data attributes as input, such as a sentence and an attribute as input, and generates a prediction for each data attribute.

[0029] In this embodiment of the invention, the input is converted into a unified vector representation through a large-scale pre-trained sentence transfer model in the attribute-level model. These vector representations serve as input to the attribute parsing module and the filter parsing module for predicting attributes and filters.

[0030] This invention provides an example of constructing a training dataset. After training, the sentence transfer model has the ability to accurately extract attributes and filter conditions.

[0031] The training dataset consists of two parts: WikiSQL, a numerical extraction dataset borrowed from the database domain, and a dataset constructed based on users' habits of exploring tabular data. Common expressions used by users exploring tabular data were collected and abstracted into templates. Attributes from a large number of tables (24,000) were then inserted into these templates to generate candidate sentences. After back-translation, a large number of diverse sentences were obtained. Training was performed on the merged dataset of the two datasets. Because the sentence transfer model separates the training of attributes and filters, the trained sentence transfer model possesses the multi-filter extraction performance of WikiSQL and the multi-attribute extraction performance of the constructed dataset without conflicting with each other.

[0032] Large-scale pre-trained sentence transfer models are pre-trained on a large natural language corpus, thus exhibiting strong generalization ability. In this embodiment of the invention, a pre-trained BERT model is used to transform the sequence of input sentences and attributes into an output vector sequence for subsequent deep learning tasks. The first vector w in this vector sequence represents the information of the entire input sequence.

[0033] The attribute parsing module takes a vector w from the pre-trained transfer learning model as input and predicts the attribute probability, attribute aggregation type, and the number of selected attributes n. The attribute probability prediction submodule generates a probability from 0 to 1 based on the vector w. The attribute aggregation type prediction submodule determines the aggregation type among the attributes, selecting the option with the highest probability from "count", "sum", "average", "maximum", "minimum", and "no aggregation". The attribute number prediction submodule determines the number of selected attributes, choosing a number from 0 to the maximum value; this is also a classification problem. The attribute parsing module selects the n attributes with the highest probabilities as the selected attributes.

[0034] like Figure 2 As shown, the sentence and each data attribute are analyzed by the attribute parsing module to predict the probability and number of times each attribute will be selected. The probability of each attribute being selected ranges from 0 to 1, and ultimately, the n attributes with the highest probabilities are selected.

[0035] The filter parsing module takes the vector of the pre-trained transfer learning model as input and predicts the probability of a filter being present in the attributes, the filter operation type, the filter value, and the number of selected filters, n. The prediction of filter probability and number of filters is similar to the prediction of attribute probability and number. The filter condition prediction submodule selects a filter condition from the options of greater than, less than, and equal to, while the filter value is obtained by selecting the start and end positions from the input question.

[0036] like Figure 3As shown, the sentence and each data attribute are analyzed by the filter parsing module to predict the probability of the filter, the number of selected filters, the filter conditions, and the filter values. Finally, the filter conditions with the highest probability are selected.

[0037] S2. The context state machine determines the attributes and filters that need to be visualized based on the context and the attributes and filters selected in the current sentence.

[0038] The context state machine accepts the outputs of the attribute parsing module and the filter parsing module as input, and combines them with context information to determine the attributes and filter conditions to be visualized. In a sequence of sentences, the context of the preceding sentence may affect the following sentence. From the perspective of attributes and filters, there are two main types of subsequent statements: inheritance of filters and inheritance of selected attributes. We set a context flag to allow users to decide whether to inherit context filter conditions. For the context inheritance of attributes, this invention sets a maximum upper limit for attributes and sorts the visualization content to be displayed according to their probability and timeliness.

[0039] The attributes and filters output by S3 and the context state machine are used by the visualization generation module to determine the generated visualization results.

[0040] The visualization generation module selects an appropriate visualization for rendering based on the attributes and filters output by the context state machine. The module determines the appropriate visualization type according to the user's perception of different attributes, allowing the use of multiple visualization rendering engines to create visualization charts.

[0041] In addition to common charts, the method for constructing visual charts based on natural language disclosed in this embodiment of the invention provides a user customization interface. This customization interface consists of two parts: rendering conditions and...

[0042] The rendering function will be used to render data when the rendering conditions are met. The rendering conditions and the rendering function can access properties (including their aggregates and lists of values), filtered data, and the question. Users can manually apply rendering parameters to properties, filtered data, or the question.

[0043] Rendering conditions. Within the rendering function, users can employ various visualization tools for rendering. 0 such as Figure 4 As shown, the input is a movie dataset containing 708 items and 10 attributes.

[0044] The present invention discloses a method for constructing visualizations based on natural language that supports visualization of multiple sentences. (a), (c), and (d) are single questions on the data. (b) is a single declarative sentence. (c), (e), and (g) are sequence questions that inherit filters from preceding sentences, supporting visualization of unfamiliar sentences.

[0045] The dataset was explored. In (d) and (g), the same question in different contexts led to different answers. From (d) to (f), the statements in (f) inherited the context filter from (d).

[0046] Example 2

[0047] This invention provides a system for constructing visual charts based on natural language. The system employs a natural language-based approach as described in Embodiment 1 and any of its optional embodiments.

[0048] A method for constructing visual charts visualizes the user's natural language. The system includes an attribute level 0 model, an attribute parsing module, a filter parsing module, a context state machine, and a visualization generation module.

[0049] The present invention provides a system interface for constructing visual charts based on natural language, as shown in the embodiments of the present invention. Figure 5 As shown, the system allows users to input multiple natural language inputs to obtain visual charts. It includes an interface (a) displaying attribute and filter information, an interface (b) displaying visualization details, and an interface (c) displaying the dialogue history. In the input boxes, users can input questions using voice or keyboard, and the corresponding visualizations will respond promptly (on average, 0.1 seconds).

[0050] like Figure 6 As shown, when visualizing the descriptive sentences in a paragraph that describe Ming Dynasty scholars, the system disclosed in this embodiment of the invention generates corresponding visualization content for the sentences in the paragraph.

[0051] As can be seen from the above embodiments, the method and system for constructing visualization charts based on natural language disclosed in this invention accepts natural language and data attributes as input through an attribute-level model, and predicts attributes and filters for each data attribute; a context state machine determines the attributes and filters to be visualized based on the context and the predicted attributes and filters of the current sentence; and a visualization generation module constructs visualization charts based on the attributes and filters to be visualized. Using the method disclosed in this invention, a deep learning-based approach parses fuzzy and diverse natural language into data attributes and filtering conditions, and then constructs visualization charts based on the extracted data attributes to respond to the user's natural language, lowering the threshold for data visualization, demonstrating strong practicality, and laying the foundation for the promotion of visual data.

[0052] The methods and systems described in this invention are not limited to the embodiments described in the specific implementation. Other implementation methods derived by those skilled in the art based on the technical solutions of this invention also fall within the scope of technical innovation of this invention.

Claims

1. A method for constructing a visualization chart based on natural language, the method comprising the following steps: S1, an attribute-level model accepts natural language and data attributes as input, and predicts attributes and filters for each data attribute; S2, a context state machine determines the attributes and filters that need to be visualized according to the predicted attributes and filters of the context and the current sentence; S3, based on the attributes and filters that need to be visualized, a visualization generation module is constructed; In step S1, the input is converted into a unified vector representation by a sentence migration model in the attribute-level model, and these vector representations are used as inputs for an attribute parsing module and a filter parsing module to predict the attributes and filters of the input, respectively; the attribute parsing module accepts the vector w of the sentence migration model as input, generates a probability from 0 to 1 according to the vector w, judges the aggregation type in the attribute, judges the number of selected attributes, and selects the n attributes with the highest probability as the selected predicted attributes; In step S2, a context flag is set to allow the user to decide whether to inherit the context filter condition, and the maximum upper limit of the attribute is set to determine the visualization content to be displayed according to the probability and timeliness.

2. The method of claim 1, wherein: The sentence migration model is pre-trained using a training data set, and the training of attributes and filters is separated.

3. The method of claim 2, wherein: The training data set includes two parts, the first part is a numerical acquisition data set WikiSQL borrowed from the database field, and the second part is a data set constructed based on the user's habit of exploring table data.

4. The method of claim 3, wherein: The second part of the training data set collects common expression forms of users exploring table data and abstracts them into templates, and inserts a certain number of attributes of the table into the templates to generate candidate sentences, and obtains a large number of sentences with multiple elements after back translation.

5. The method for constructing a visual chart based on natural language of claim 1, wherein: The filter parsing module accepts the vector w of the sentence migration model as input, predicts the probability of the existence of filters in the attribute, the operation type of the filter, the filter value, and the number m of selected filters, and selects the m filter conditions with the highest selection probability as the selected predicted filters.

6. The method of claim 5, wherein: In step S3, the appropriate visualization type is determined according to the user's perception ability of different attributes.

7. A system for constructing visual charts based on natural language, characterized by: A visualization chart is constructed using the method for constructing a visualization chart based on natural language according to any one of claims 1-6.

Images