Data query method and device, electronic equipment and storage medium

Abstract

The invention provides a data query method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the steps of identifying a query intention of to-be-processed data under the condition that the to-be-processed data is acquired; analyzing the query intention to obtain a visual intermediate representation of the query intention; determining a query result of the to-be-processed data according to the visual intermediate representation and a preset database; generating at least one candidate chart based on the query result; and performing visualization processing on the at least one candidate chart, and displaying a visualization result of the at least one candidate chart. According to the embodiment of the invention, the accuracy of the data in the generated candidate chart can be improved, so that a user can intuitively know the data to be queried and the query result, and the user experience is improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a data query method and apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With the development of big data and artificial intelligence, data types in database systems are no longer limited to structured tables, but include multimodal data in various forms such as text, images, graph data, and time-series data. How to automatically generate visual charts that integrate multiple data types through natural language or low-barrier interactive methods has become an important direction for intelligent data analysis and decision support.

[0003] In current practical applications, visualization processing is usually performed on single-modal data, such as structured tables, which makes it difficult to support the visualization of heterogeneous data sources and multimodal data. Summary of the Invention

[0004] To address this issue, the present invention provides a data query method and apparatus, an electronic device, and a computer-readable storage medium to solve the problem that existing technologies struggle to support the visualization of multimodal data.

[0005] To achieve the above objectives, the first aspect of the present invention provides a data query method, the method comprising: upon obtaining data to be processed, identifying the query intent of the data to be processed; parsing the query intent to obtain a visual intermediate representation of the query intent; determining the query result of the data to be processed based on the visual intermediate representation and a preset database; generating at least one candidate chart based on the query result; performing visualization processing on the at least one candidate chart, and displaying the visualization result of the at least one candidate chart.

[0006] Optionally, the data to be processed includes text to be processed; identifying the query intent of the data to be processed includes: performing semantic parsing on the text to be processed to obtain semantic data of the text to be processed; identifying the query type and key fields in the semantic data; and performing structured processing on the query type and the key fields to obtain the query intent.

[0007] Optionally, parsing the query intent to obtain a visual intermediate representation of the query intent includes: parsing to obtain data elements corresponding to each key field in the query intent; determining a visual element corresponding to each data element based on a first mapping relationship; and generating a visual intermediate representation of the query intent based on each data element and each visual element.

[0008] Optionally, the data to be processed further includes multimodal data to be processed; the method further includes: determining the data type of the multimodal data to be processed; determining the visualization type corresponding to the data type based on the second mapping relationship; generating an intermediate visualization representation of the query intent according to each data element and each visualization element, including: generating an intermediate visualization representation of the query intent according to each data element, each visualization element and the visualization type.

[0009] Optionally, the preset database includes at least two different types of databases; determining the query results of the data to be processed based on the visual intermediate representation and the preset database includes: determining at least one target database that the visual intermediate representation can use from the at least two different types of databases; querying the query data corresponding to the visual intermediate representation in the at least one target database; and generating the query results of the data to be processed based on the query data.

[0010] Optionally, generating at least one candidate chart based on the query results includes: performing data analysis on the query results to determine the data distribution type of the query results; generating at least one first initial chart of the query results according to preset chart generation rules and the data distribution type; calling a chart generation model to process the query results according to the data distribution type to generate at least one second initial chart of the query results; evaluating the at least one first initial chart and the at least one second initial chart to obtain an evaluation result for each first initial chart and each second initial chart; and determining at least one candidate chart among the at least one first initial chart and the at least one second initial chart based on the evaluation result for each first initial chart and each second initial chart.

[0011] Optionally, after generating at least one candidate chart based on the query results, the method further includes: generating a text description for each candidate chart; performing visualization processing on the at least one candidate chart and displaying the visualization results of the at least one candidate chart, including: rendering the at least one candidate chart and displaying the visualization results of the at least one candidate chart and the text description of each candidate chart.

[0012] Secondly, the present invention provides a data query device, comprising: an identification module configured to identify the query intent of the data to be processed when the data to be processed is acquired; a parsing module configured to parse the query intent to obtain a visual intermediate representation of the query intent; a determination module configured to determine the query result of the data to be processed based on the visual intermediate representation and a preset database; a generation module configured to generate at least one candidate chart based on the query result; and a display module configured to perform visualization processing on the at least one candidate chart and display the visualization result of the at least one candidate chart.

[0013] Thirdly, the present invention provides an electronic device comprising: one or more processors; a memory storing one or more programs thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-described data query method; and one or more I / O interfaces connected between the processors and the memory, configured to enable information interaction between the processors and the memory.

[0014] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described data query method.

[0015] The present invention has the following advantages:

[0016] Upon acquiring the data to be processed, the system identifies the query intent contained within the data to determine the user's query intent. By parsing the query intent, a visual intermediate representation of it is obtained. This ensures that regardless of the data type, the data can be parsed into a corresponding visual intermediate representation, enhancing its broad applicability. Next, a data query is performed in a pre-defined database based on the visual intermediate representation to determine the query results for the data to be processed. Based on these results, at least one candidate chart is generated, enhancing the richness of the generated charts. The system then visualizes at least one candidate chart and displays its visualization results, enabling the visualization of all query results corresponding to the data to be processed. It supports mixed visualization of query results for different data types, improving the accuracy of the data in the generated candidate charts. This allows users to intuitively understand the data they are querying and the query results, enhancing the user experience.

[0017] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0019] Figure 1 A flowchart of a data query method provided in an embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram illustrating an application scenario of a data query method provided in an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram illustrating the generation of query results according to an embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram of a process for generating candidate charts according to an embodiment of the present invention;

[0023] Figure 5 A schematic diagram illustrating a visualization result provided in an embodiment of the present invention;

[0024] Figure 6 A flowchart illustrating a data query method provided in an embodiment of the present invention;

[0025] Figure 7 A block diagram of a data query device provided in an embodiment of the present invention;

[0026] Figure 8 This is a block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0027] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0028] As used in this invention, the term "and / or" includes any and all combinations of one or more of the associated enumerated entries.

[0029] The terminology used in this invention is for describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms "a" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0030] When the terms “comprising” and / or “made of” are used in this invention, the presence of the said feature, integral, step, operation, element and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or groups thereof is not excluded.

[0031] Unless otherwise specified, all terms used in this invention (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in common dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined by the invention.

[0032] Firstly, embodiments of the present invention provide a data query method.

[0033] See Figure 1 , Figure 1 A flowchart of a data query method provided in an embodiment of the present invention specifically includes the following steps:

[0034] Step 102: If the data to be processed is obtained, identify the query intent of the data to be processed.

[0035] The data query method provided by this invention can be applied to intelligent query systems, intelligent query platforms, or servers. Intelligent query systems and intelligent query platforms can be systems, platforms, or applications that support human-computer interaction for data querying. Servers can be servers that support interaction with user terminals and can support data query functions, or servers that have deployed intelligent query systems or intelligent query platforms.

[0036] In practical applications, users can trigger data query requests through their terminals. These requests contain the query text, or pending text, of the data the user wants to retrieve. For example, the pending text in a data query request might be, "Compare sales figures from the past three months with the same period last year, and provide key trend analysis."

[0037] Furthermore, when a user triggers a data query request, they can also simultaneously upload multimodal data related to the text to be processed. This multimodal data includes at least one of text, images, time-series data, and graph data. This allows the intelligent query system, intelligent query platform, or server to analyze and process the text after obtaining both the text and the multimodal data. Correspondingly, in practical applications, users can also upload only the text to be processed. After obtaining the text, the intelligent query system, intelligent query platform, or server can retrieve relevant multimodal data from a pre-defined multimodal database for use. Therefore, the data to be processed includes both the text and the multimodal data, or it can include only the text. The query intent is used to characterize the user's purpose in performing this data query, as contained in the text.

[0038] Specifically, once the data to be processed is obtained, it is necessary to identify the query intent contained in the data so that data queries and data analysis can be performed on the data to be processed in subsequent processes based on the query intent.

[0039] See Figure 2 , Figure 2 This is a schematic diagram illustrating an application scenario of a data query method provided in an embodiment of the present invention. Figure 2 Taking the application of data query methods in sales comparison analysis as an example, such as Figure 2 As shown, a user triggers a data query request on their terminal. The request includes the data to be processed: "Compare sales figures for the past three months with the same period last year, and provide key trend analysis." At this point, it's necessary to identify the query intent contained in "Compare sales figures for the past three months with the same period last year, and provide key trend analysis." Figure 2 The fields shown are: Sales Revenue; Time Range: Past Three Months vs. Same Period Last Year; Operations: Comparison, Trend Analysis; Convert to Intermediate Visual Representation. The following explains the specific implementation method for identifying the query intent of the data to be processed:

[0040] In one specific embodiment of the present invention, identifying the query intent of the data to be processed includes: performing semantic parsing on the text to be processed to obtain semantic data of the text to be processed; identifying the query type and key fields in the semantic data; and performing structured processing on the query type and the key fields to obtain the query intent.

[0041] Semantic data refers to the data used to express the semantics of the text to be processed after semantic parsing. In practical applications, pre-trained language models can be used to perform semantic parsing on the text to be processed and obtain its semantic data. Query type refers to the type of data query operation performed based on the text to be processed, including but not limited to statistics, comparison, clustering, and trend analysis. Key fields refer to pre-defined specified fields, including but not limited to time range, indicators, dimensions, and conditions.

[0042] Specifically, semantic parsing of the text to be processed is performed based on a pre-trained language model to obtain semantic data. The query types contained within the semantic data are identified; for example, a classifier can be used to identify query types. Key fields in the semantic data are also identified; for example, named entity extraction (NIE) techniques are used to identify and extract predefined categories of key fields from the semantic data. The identified query types and key fields are then structured, for example, by mapping them to operators in the query language. This process continues... Figure 2 In the example, the identified query intent of the text "Compare sales figures for the past three months with the same period last year and provide key trend analysis." is:

[0043] {

[0044] "intent": "trend_analysis"

[0045] "measure" (measurement): "sales" (sales revenue)

[0046] "dimension" (dimension): "region" (region)

[0047] "time_range" (time range): "2024"

[0048] }

[0049] This invention provides an embodiment of the invention that performs semantic parsing on the text to be processed to obtain its semantic data. It then performs intent recognition on the semantic data to determine the query type and key fields within the semantic data, thereby obtaining the query intent of the text to be processed. This approach improves the accuracy of determining the query intent by recognizing the query type and key fields based on the semantic data of the text to be processed.

[0050] Step 104: Parse the query intent to obtain a visual intermediate representation of the query intent.

[0051] In current practical applications, database data types are no longer limited to structured tables, but include multimodal data such as text, images, graph data, and time-series data. Database types are also diverse due to these different data types, including relational databases, graph databases, and time-series databases. Therefore, to improve the accuracy of querying the text being processed and to enable queries across different types of databases, the query intent of the text can be parsed and processed to generate a visual intermediate representation of the query intent. This visual intermediate representation allows for data queries across different types of databases, improving the generality and versatility of data queries. Here, the visual intermediate representation refers to a unified semantic intermediate representation visualization form, namely Visualizable Intermediate Representation (Viz-IR).

[0052] In a specific embodiment of the present invention, parsing the query intent to obtain a visual intermediate representation of the query intent includes: parsing to obtain data elements corresponding to each key field in the query intent; determining a visual element corresponding to each data element based on a first mapping relationship; and generating a visual intermediate representation of the query intent based on each data element and each visual element.

[0053] Here, data elements refer to the field values ​​corresponding to key fields. For example, if the key field is "intent," its corresponding field value is "trend analysis," etc. The first mapping relationship refers to the mapping relationship between data elements and visualization elements. This first mapping relationship can be pre-defined by relevant technical personnel. Visualization elements refer to...

[0054] Specifically, after obtaining the query intent of the text to be processed, the query intent is parsed, and the data element corresponding to each key field in the query intent is determined, that is, the field value corresponding to the key field. Based on the first mapping relationship between the pre-set data elements and visualization elements, the visualization element corresponding to each data element is determined. For example, the first mapping relationship includes "measure" - "y-axis"; "dimension" - "x-axis"; "time" - "sequence"; "category" - "color / shape", etc. According to this first mapping relationship, the visualization element corresponding to the data element "measure" can be determined as "y-axis", the visualization element corresponding to the data element "dimension" is "x-axis", and the visualization element corresponding to the data element "time_range" is "sequence".

[0055] Based on the data elements in the query intent and the corresponding visualization elements for each data element, an intermediate visualization representation corresponding to the query intent is generated. The format of the intermediate visualization representation can be determined according to a pre-defined format, such as JSON format, etc. The format of the intermediate visualization representation can also be other formats, which can be set according to the actual application. This invention does not limit the format of the intermediate visualization representation.

[0056] In this embodiment of the invention, after identifying the query intent in the text to be processed, instead of directly performing a corresponding data query in the database based on the query intent, the visualization elements corresponding to the data elements of the key fields in the query intent are determined according to a pre-set first mapping relationship. A visual intermediate representation of the query intent is then generated based on the data elements and the visualization elements. Thus, through the visual intermediate representation, data queries of different data types can be performed on the text to be processed in multiple different types of databases, improving the accuracy and generalization of data queries on the text to be processed.

[0057] Furthermore, as mentioned above, the data to be processed may also include multimodal data. Therefore, when the data to be processed includes both text and multimodal data, in addition to the aforementioned processing of the text, corresponding processing of the multimodal data is also required. The specific implementation method is as follows:

[0058] In a specific embodiment provided by the present invention, the method further includes: determining the data type of the multimodal data to be processed; determining the visualization type corresponding to the data type based on a second mapping relationship; and generating an intermediate visualization representation of the query intent according to each data element and each visualization element, including: generating an intermediate visualization representation of the query intent according to each data element, each visualization element and the visualization type.

[0059] Here, data type refers to the data variation type of the multimodal data to be processed, including but not limited to continuous variable types and categorical variable types. The second mapping relationship refers to the mapping relationship between data type and visualization type; this second mapping relationship can also be pre-defined by relevant technical personnel. Visualization type refers to the type of chart output to the user, including but not limited to line charts, bar charts, pie charts, and column charts.

[0060] Specifically, when the data to be processed also includes multimodal data, the data type of the multimodal data to be processed is determined, and the visualization type corresponding to different data types is determined according to the pre-set mapping relationship between data types and visualization types, that is, the visualization type corresponding to the multimodal data to be processed is determined. For example, the second mapping relationship includes "continuous variable" - "line chart"; "categorical variable" - "bar chart", etc. According to this second mapping relationship, it can be determined that when the data type of the multimodal data to be processed is a continuous variable, the visualization type corresponding to the multimodal data to be processed is a "line chart", and when the data type of the multimodal data to be processed is a categorical variable, the visualization type corresponding to the multimodal data to be processed is a "bar chart". It should be noted that in practical applications, the same multimodal data to be processed may include multiple different data types. Therefore, the visualization type determined based on the data type and the second mapping relationship may also be multiple different visualization types. That is, when the multimodal data to be processed includes multiple different data types, different visualization types of charts can be generated for the multimodal data to be processed in subsequent processes, such as generating line charts and bar charts for the multimodal data to be processed.

[0061] After determining the visualization type, an intermediate visual representation of the query intent is generated by combining the data elements, visualization elements, and visualization type identified above.

[0062] Following the previous example, the generated JSON format visualization intermediate representation can be:

[0063] {

[0064] "data": "sales_table",

[0065] "encoding": {

[0066] "x": "region",

[0067] "y": "sales",

[0068] "color": "year"

[0069] },

[0070] "mark": "line

[0071] }

[0072] The embodiments of the present invention can process multimodal data to be processed and determine the data type of the multimodal data to be processed. Based on the data type, the visualization type used for subsequent visualization processing of the multimodal data to be processed is determined. By combining the data elements in the query intent, the visualization elements corresponding to the data elements, and the visualization type, an intermediate visualization representation of the query intent is generated, thereby improving the accuracy and richness of the charts generated based on the intermediate visualization representation of the data to be processed.

[0073] Furthermore, in practical applications, the multimodal data to be processed may contain multiple modalities, such as at least two of text, images, graph data, and time series data, and the data sources of these multimodal data to be processed may be different, which may lead to semantic barriers between these multimodal data to be processed. In order to improve the quality of the intermediate visualization representation, feature fusion can be performed on the multimodal data to be processed after obtaining it, and the fused feature representation can be generated.

[0074] Based on this, in this embodiment of the invention, a Hierarchical Cross-Attention Network (HCAN) is used to fuse multimodal features of the multimodal data to be processed. The fusion process can be seen in the following formula 1:

[0075] Formula 1

[0076] in, The fused feature is the output of the l-th layer, where l is the fusion level. For the modal features to be fused in layer l, the calculation of cross-attention can be found in the following formula 2:

[0077] Formula 2

[0078] in, The query feature is the one described in Formula 1 above. , The modal features to be fused are those in Formula 1 above. , , , For the projection matrix, The feature dimension is defined by formulas 1 and 2 above. The fused features after feature fusion of the multimodal data to be processed can be calculated. After L layers of feature fusion, the final output fused features can be seen in formula 3 below:

[0079] Formula 3

[0080] After performing the aforementioned feature fusion process on the multimodal data to be processed, subsequent processing can be carried out based on the final output fused features to improve the accuracy of the intermediate visualization representation of the multimodal data to be processed.

[0081] Step 106: Determine the query results of the data to be processed based on the visual intermediate representation and the preset database.

[0082] After generating the intermediate visualization representation, a query operation on the data to be processed is performed in a preset database based on the intermediate visualization representation to determine the query results for the data to be processed. The preset database refers to a pre-configured database, including but not limited to relational databases (e.g., Hive, ClickHouse, Presto), graph databases (e.g., Neo4j, NebulaGraph), time-series databases (e.g., InfluxDB, Prometheus), as well as third-party data interfaces (e.g., third-party statistical interfaces, log service APIs) and search engines (e.g., Elasticsearch, Solr). In practical applications, relational databases are suitable for large-scale data aggregation analysis, graph databases are suitable for association analysis, and time-series databases are suitable for time-series data queries. For non-database data sources, the corresponding query data can be obtained based on their respective third-party data interfaces; search engines are suitable for full-text search and fuzzy matching data queries. The query results refer to the data results in a preset format generated based on the query data of the data to be processed. For example, the query results can be a key-value pair array structure. This embodiment of the invention does not limit the preset format of the query results; the specific format can be determined according to the actual application.

[0083] Based on this, in a specific embodiment provided by the present invention, the preset database includes at least two databases of different types; determining the query result of the data to be processed according to the visual intermediate representation and the preset database includes: determining at least one target database that the visual intermediate representation can use in the at least two databases of different types; querying the query data corresponding to the visual intermediate representation in the at least one target database; and generating the query result of the data to be processed based on the query data.

[0084] The target database refers to the database determined from the preset database for data querying in the visualization intermediate representation; the query data refers to the data of the visualization intermediate representation obtained from at least one target database.

[0085] See also Figure 2 ,like Figure 2 As shown, after generating a visual intermediate representation of the data to be processed, it can be displayed in a preset database ( Figure 2Data querying is performed using relational databases, graph databases, and time-series databases as examples. Since the data to be processed includes different types of data, such as text, images, graph data, and time-series data, the query operations required for different types of data also differ. Therefore, a suitable target database can be determined from the preset databases based on the different types of data and the different query operations. Specifically, for large-scale data aggregation analysis, a relational database can be selected; for time-series data querying, a time-series database can be selected; for graph data association analysis, a graph database can be selected, and so on. After determining at least one target database, queries can be performed in at least one target database based on the visualization intermediate representation to obtain the corresponding query data, and query results in a preset format can be generated based on the query data.

[0086] In practical applications, different types of databases use different query languages. Therefore, after identifying the target database for data querying using the visualization intermediate representation, it is also necessary to convert the visualization intermediate representation into a query language that matches the target database.

[0087] Further, see Figure 3 , Figure 3 This is a schematic diagram illustrating the generation of query results provided in an embodiment of the present invention, such as... Figure 3 As shown, after obtaining the visual intermediate representation corresponding to the query intent, the target database for subsequent data queries is determined based on the different types of data and different query operations in the visual intermediate representation. According to the query language supported by the target database, the visual intermediate representation is converted into a language. For example, for data that needs to be queried in a relational database, its corresponding visual intermediate representation is converted into SQL; for data that needs to be queried in a graph database, its corresponding visual intermediate representation is converted into Cypher; for non-database data sources, its corresponding visual intermediate representation is converted into interface request parameters conforming to third-party interface specifications, and so on. After the language conversion is completed, data queries are performed in the corresponding target database based on the converted visual intermediate representation to obtain the corresponding query data. The query results are obtained by converting the query data into a preset format. See the following example for details:

[0088] {"Province":"A","Sales Amount":5000,"Year":2024};

[0089] {"Province":"B","Sales Amount":6200,"Year":2024}.

[0090] In this embodiment of the invention, after generating a visual intermediate representation of the query intent, the visual intermediate representation is further analyzed to identify different query operations for different types of data in the visual intermediate representation. This allows for the selection of target databases from a preset database. Based on the different types of target databases, the visual intermediate representation is converted into a query language supported by the target database to retrieve the corresponding query data. Finally, the query data is converted according to a preset format to generate the query results for the data to be processed. This achieves adaptation to queries of different types of data and improves the versatility of data queries.

[0091] Step 108: Generate at least one candidate chart based on the query results.

[0092] After obtaining the query results of the data to be processed, at least one candidate chart corresponding to the query results is generated, wherein the candidate chart refers to the data chart displayed on the user terminal.

[0093] In a specific embodiment of the present invention, generating at least one candidate chart based on the query result includes: performing data analysis on the query result to determine the data distribution type of the query result; generating at least one first initial chart of the query result according to a preset chart generation rule and the data distribution type; calling a chart generation model to process the query result according to the data distribution type to generate at least one second initial chart of the query result; evaluating the at least one first initial chart and the at least one second initial chart to obtain an evaluation result for each first initial chart and each second initial chart; and determining at least one candidate chart among the at least one first initial chart and the at least one second initial chart based on the evaluation result for each first initial chart and each second initial chart.

[0094] The data distribution type refers to the distribution type of the data in the query results, including discrete distribution, correlated distribution, etc. Chart generation rules refer to the pre-defined rules used to generate charts, including the mapping relationship between data distribution types and chart types. The first initial chart is the chart directly generated according to the chart generation rules. The chart generation model is the model used to generate charts based on data distribution types. The second initial chart is the chart directly generated by calling the chart generation model. Candidate charts are those selected from the first and second initial charts.

[0095] Specifically, see Figure 4 , Figure 4 This is a schematic diagram of a process for generating candidate charts provided in an embodiment of the present invention, such as... Figure 4As shown, data analysis is performed on the query results to determine the data distribution type. In practical applications, this can be achieved by constructing prompts and calling a large language model. The data distribution type of the query results is searched within preset chart generation rules, and the corresponding chart type is determined within those rules. The query results are then used to generate a chart of the determined chart type, i.e., the first initial chart. For example, if the chart generation rules include "part and whole - pie chart / stacked bar chart," after finding the "part and whole" data distribution type, the corresponding chart type can be determined to be either a pie chart or a stacked bar chart. In this case, the query results can be generated as a pie chart and / or a stacked bar chart. Alternatively, prompts can be constructed based on the data distribution type of the query results, and these prompts can be input into the chart generation model to obtain the chart output by the model, i.e., the second initial chart.

[0096] Furthermore, the generated first and second initial charts can be evaluated to determine whether the accuracy, clarity, layout rationality, and color usage of each first and second initial chart meet the standards.

[0097] In practical applications, the first and second initial charts can be evaluated separately based on factors such as whether the chart type aligns with the data facts (accuracy), the appropriateness of the use of visual elements (readability), and the effectiveness of the data conveyed by the chart (validity). A comprehensive evaluation is then performed, combining these aspects to improve the accuracy of the evaluation of the first and second initial charts. This process can also be implemented using models, such as constructing prompts. The prompt model evaluates the charts from the aforementioned aspects and outputs evaluation scores. Finally, the evaluation scores for each aspect are combined to obtain the evaluation result for each chart. The determination of the evaluation results can be found in Formula 4 below:

[0098] Formula 4

[0099] in, For the evaluation results, The chart is evaluated based on its accuracy. The chart is evaluated as a score in terms of its validity. The chart is scored in terms of readability. , , This is a weighting parameter used to balance the importance of accuracy, effectiveness, and readability.

[0100] After determining the evaluation result of each chart according to Formula 4 above, the multiple first initial charts and multiple second initial charts are sorted according to the evaluation results, and one or more charts with higher evaluation results are selected as candidate charts.

[0101] It should be noted that in practical applications, either the chart generation rule or the chart generation model can be selected to generate the initial chart. This invention does not limit the method of generating the initial chart, and the specific method can be determined according to the actual application.

[0102] In this embodiment of the invention, query results are processed by chart generation rules and chart generation models, and corresponding first initial charts and second initial charts are generated. After evaluating the first initial charts and second initial charts, candidate charts that can be provided to users are determined from the first initial charts and second initial charts based on the evaluation results, thereby improving the accuracy, readability and effectiveness of the finally generated candidate charts.

[0103] Step 110: Perform visualization processing on the at least one candidate chart and display the visualization results of the at least one candidate chart.

[0104] After identifying at least one candidate chart, the candidate chart is visualized and the visualization results are displayed on the user terminal so that the user can intuitively understand the query results of the data to be processed.

[0105] In a specific embodiment of the present invention, after generating at least one candidate chart based on the query result, the method further includes: generating a text description for each candidate chart; performing visualization processing on the at least one candidate chart and displaying the visualization result of the at least one candidate chart, including: rendering the at least one candidate chart and displaying the visualization result of the at least one candidate chart and the text description of each candidate chart.

[0106] The text description refers to the description of the data in the candidate chart using natural language and presented in text form. It can be understood as the parsed text of the candidate chart. In practical applications, prompts are constructed based on the underlying data and chart structure description in the candidate chart. A large language model is then used to generate a text description of the candidate chart based on these prompts. The generated candidate chart is then rendered to the user's terminal. For example, a visualization framework (Vega-Lite, Plotly, etc.) can be used to render the candidate chart, and the visualization results and text descriptions of the candidate chart are displayed on the user's terminal.

[0107] See Figure 5 , Figure 5 A schematic diagram illustrating a visualization result provided in an embodiment of the present invention, such as... Figure 5 As shown, the user-input text to be processed is "to display a comparison of the company's sales and profits for each quarter of 2023 and analyze the trend." The candidate charts generated are bar charts, line charts, and pie charts, with corresponding text descriptions. Figure 5 The text content shown in the "Intelligent Analysis and Insights" section.

[0108] In this embodiment of the invention, after generating candidate charts and text descriptions, the candidate charts and text descriptions are displayed together on the user terminal, allowing users to intuitively understand and analyze the data in the candidate charts, thereby improving the user experience.

[0109] This invention implements the following steps: upon acquiring data to be processed, the system identifies the query intent contained within the data to determine the user's query intent. By parsing the query intent, a visual intermediate representation of the query intent is obtained. Thus, regardless of the data type, the data to be processed can be parsed into a corresponding visual intermediate representation, enhancing its broad applicability. Next, a data query is performed in a preset database based on the visual intermediate representation to determine the query results for the data to be processed. Then, at least one candidate chart is generated based on the query results, enhancing the richness of the generated candidate charts. The system performs visualization processing on at least one candidate chart and displays the visualization results of at least one candidate chart. This achieves the visualization of all query results corresponding to the data to be processed, supports mixed visualization of query results for different data types, improves the accuracy of the data in the generated candidate charts, and allows users to intuitively understand the data they want to query and the query results, improving the user experience.

[0110] The following is in conjunction with the appendix Figure 6 The data query method provided in the embodiments of the present invention will be further explained and described below. Figure 6 This is a flowchart illustrating a data query method provided in an embodiment of the present invention, as shown below. Figure 6As shown, the process involves acquiring the text to be processed and the multimodal data to be processed. Semantic parsing is performed on the text to obtain semantic data. Entity recognition is then performed on the semantic data to determine the query type and key fields. The query type and key fields are then structured to obtain the query intent of the text to be processed. Based on the query intent and the structured data, text data, image data, and time-series data in the multimodal data, a corresponding intermediate visualization representation is generated. Multimodal feature fusion is performed based on the intermediate visualization representation to obtain the fused modal features. A target database for subsequent data querying is determined from a preset database, a third-party data interface, and a search engine. The fused modal features are converted into a query language supported by the target database, and a query is performed in the corresponding target database based on the converted query language to obtain query data. Finally, the query data is converted into query results in a preset format. First initial charts of the query results are generated based on chart generation rules, and second initial charts are generated based on chart generation models. The first and second initial charts are evaluated in terms of accuracy, effectiveness, and readability, resulting in an evaluation score for each initial chart across each evaluation dimension. These scores are then weighted using preset weighting parameters to obtain the final evaluation result for each initial chart. Based on the evaluation results of each initial chart, one or more charts with higher evaluation scores are selected as candidate charts. A visualization framework is used to render the candidate charts, generating visualizations of their visual results. Simultaneously, a large language model is used to generate text descriptions for the candidate charts. Both the visualizations and text descriptions of the candidate charts are then displayed on the user's terminal for presentation.

[0111] This invention implements the following: semantic parsing of the text to be processed to obtain semantic data; intent recognition of the semantic data to determine the query intent of the text to be processed, thereby improving the accuracy of determining the query intent; by generating a visual intermediate representation, data queries of different data types can be performed on the text to be processed in multiple different types of databases, improving the accuracy and generalization of data queries on the text to be processed; according to different types of target databases, the visual intermediate representation is converted into the query language supported by the target database to obtain the corresponding query data in the target database, thereby achieving adaptation to queries of different types of data and improving the universality of data queries.

[0112] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0113] Secondly, embodiments of the present invention provide a data query device.

[0114] See Figure 7 , Figure 7 This is a block diagram of a data query device provided in an embodiment of the present invention. The device includes:

[0115] The identification module 702 is configured to identify the query intent of the data to be processed when the data to be processed is acquired;

[0116] The parsing module 704 is configured to parse the query intent to obtain a visual intermediate representation of the query intent;

[0117] The determination module 706 is configured to determine the query result of the data to be processed based on the visual intermediate representation and the preset database;

[0118] The generation module 708 is configured to generate at least one candidate chart based on the query results;

[0119] Display module 710 is configured to perform visualization processing on the at least one candidate chart and display the visualization results of the at least one candidate chart.

[0120] Optionally, the data to be processed includes text to be processed;

[0121] The identification module 702 is further configured as follows:

[0122] Semantic parsing is performed on the text to be processed to obtain the semantic data of the text to be processed;

[0123] Identify the query type and key fields in the semantic data;

[0124] The query type and the key fields are structured to obtain the query intent.

[0125] Optionally, the parsing module 704 is further configured to:

[0126] Parse to obtain the data elements corresponding to each of the key fields in the query intent;

[0127] Based on the first mapping relationship, determine the visualization element corresponding to each data element;

[0128] Based on each of the data elements and each of the visualization elements, a visual intermediate representation of the query intent is generated.

[0129] Optionally, the data to be processed may further include multimodal data to be processed;

[0130] The device further includes a visualization type determination module, configured as follows:

[0131] Determine the data type of the multimodal data to be processed;

[0132] Based on the second mapping relationship, determine the visualization type corresponding to the data type;

[0133] The parsing module 704 is further configured as follows:

[0134] Based on each of the data elements, each of the visualization elements, and the visualization type, an intermediate visual representation of the query intent is generated.

[0135] Optionally, the preset database includes at least two databases of different types;

[0136] The determining module 706 is further configured to:

[0137] Among the at least two different types of databases, determine at least one target database from which the visualization intermediate representation can be used;

[0138] Query the data corresponding to the visual intermediate representation in the at least one target database;

[0139] The query results for the data to be processed are generated based on the query data.

[0140] Optionally, the generation module 708 is further configured to:

[0141] Perform data analysis on the query results to determine the data distribution type of the query results;

[0142] Based on the preset chart generation rules and the data distribution type, at least one first initial chart of the query results is generated;

[0143] The chart generation model is invoked to process the query results according to the data distribution type, and at least one second initial chart of the query results is generated;

[0144] The at least one first initial chart and the at least one second initial chart are evaluated to obtain an evaluation result for each first initial chart and each second initial chart;

[0145] Based on the evaluation results of each of the first initial charts and each of the second initial charts, at least one candidate chart is determined from the at least one first initial chart and the at least one second initial chart.

[0146] Optionally, the apparatus further includes a description generation module configured to:

[0147] Generate a text description for each of the candidate charts;

[0148] The display module 710 is further configured as follows:

[0149] The at least one candidate chart is rendered, and the visualization results of the at least one candidate chart and the text description of each candidate chart are displayed.

[0150] The data query apparatus provided in this embodiment of the invention includes: an identification module configured to identify the query intent of the data to be processed when the data to be processed is acquired; a parsing module configured to parse the query intent to obtain a visual intermediate representation of the query intent; a determination module configured to determine the query result of the data to be processed based on the visual intermediate representation and a preset database; a generation module configured to generate at least one candidate chart based on the query result; and a display module configured to perform visualization processing on the at least one candidate chart and display the visualization result of the at least one candidate chart.

[0151] This invention implements the following steps: upon acquiring data to be processed, the system identifies the query intent contained within the data to determine the user's query intent. By parsing the query intent, a visual intermediate representation of the query intent is obtained. Thus, regardless of the data type, the data to be processed can be parsed into a corresponding visual intermediate representation, enhancing its broad applicability. Next, a data query is performed in a preset database based on the visual intermediate representation to determine the query results for the data to be processed. Then, at least one candidate chart is generated based on the query results, enhancing the richness of the generated candidate charts. The system performs visualization processing on at least one candidate chart and displays the visualization results of at least one candidate chart. This achieves the visualization of all query results corresponding to the data to be processed, supports mixed visualization of query results for different data types, improves the accuracy of the data in the generated candidate charts, and allows users to intuitively understand the data they want to query and the query results, improving the user experience.

[0152] The apparatus provided in the embodiments of the present invention has functions or includes modules that can be used to perform the methods described in the first aspect of the method embodiments above. The specific implementation and technical effects can be referred to the description of the method embodiments above. For the sake of brevity, they will not be repeated here.

[0153] It should be noted that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by this invention; however, this does not mean that other units are absent from this embodiment.

[0154] See Figure 8 , Figure 8 This is a block diagram of an electronic device provided in an embodiment of the present invention.

[0155] This invention provides an electronic device comprising:

[0156] One or more processors 801;

[0157] The memory 802 stores one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the data query method described above; one or more I / O interfaces 803 are connected between the processor and the memory and configured to enable information interaction between the processor and the memory.

[0158] Among them, processor 801 is a device with data processing capabilities, including but not limited to central processing unit (CPU); memory 802 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); I / O interface (read-write interface) 803 is connected between processor 801 and memory 802, and can realize information interaction between processor 801 and memory 802, including but not limited to data bus (Bus).

[0159] In some embodiments, the processor 801, memory 802, and I / O interface 803 are interconnected via a bus, and thus connected to other components of the computing device.

[0160] This embodiment also provides a computer-readable medium having a computer program stored thereon. When the program is executed by a processor, it implements the data query method provided in this embodiment. To avoid repetition, the specific steps of the data query method will not be repeated here.

[0161] This invention also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the data query method described above.

[0162] Those skilled in the art will understand that all or some of the steps, systems, or apparatuses in the methods, systems, and apparatuses described above can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0163] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0164] Those skilled in the art will understand that although some embodiments described herein include certain features that are included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of this embodiment and form different embodiments.

[0165] It is understood that the above embodiments are merely exemplary implementations used to illustrate the principles of the present invention, and the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A data query method, characterized in that, include: Upon acquiring the data to be processed, identify the query intent behind the data. The query intent is parsed to obtain a visual intermediate representation of the query intent; Based on the visual intermediate representation and the preset database, determine the query results of the data to be processed; At least one candidate chart is generated based on the query results; The at least one candidate chart is visualized, and the visualization results of the at least one candidate chart are displayed.

2. The method as described in claim 1, characterized in that, The data to be processed includes text to be processed; Identifying the query intent of the data to be processed includes: Semantic parsing is performed on the text to be processed to obtain the semantic data of the text to be processed; Identify the query type and key fields in the semantic data; The query type and the key fields are structured to obtain the query intent.

3. The method as described in claim 2, characterized in that, The query intent is parsed to obtain a visual intermediate representation of the query intent, including: Parse to obtain the data elements corresponding to each of the key fields in the query intent; Based on the first mapping relationship, determine the visualization element corresponding to each data element; Based on each of the data elements and each of the visualization elements, a visual intermediate representation of the query intent is generated.

4. The method as described in claim 3, characterized in that, The data to be processed also includes multimodal data to be processed; The method further includes: Determine the data type of the multimodal data to be processed; Based on the second mapping relationship, determine the visualization type corresponding to the data type; Based on each of the data elements and each of the visualization elements, a visual intermediate representation of the query intent is generated, including: Based on each of the data elements, each of the visualization elements, and the visualization type, an intermediate visual representation of the query intent is generated.

5. The method as described in claim 1, characterized in that, The preset database includes at least two different types of databases; Based on the visual intermediate representation and the preset database, the query results for the data to be processed are determined, including: Among the at least two different types of databases, determine at least one target database from which the visualization intermediate representation can be used; Query the data corresponding to the visual intermediate representation in the at least one target database; The query results for the data to be processed are generated based on the query data.

6. The method as described in claim 1, characterized in that, At least one candidate chart is generated based on the query results, including: Perform data analysis on the query results to determine the data distribution type of the query results; Based on the preset chart generation rules and the data distribution type, at least one first initial chart of the query results is generated; The chart generation model is invoked to process the query results according to the data distribution type, and at least one second initial chart of the query results is generated; The at least one first initial chart and the at least one second initial chart are evaluated to obtain an evaluation result for each first initial chart and each second initial chart; Based on the evaluation results of each of the first initial charts and each of the second initial charts, at least one candidate chart is determined from the at least one first initial chart and the at least one second initial chart.

7. The method as described in claim 1 or 6, characterized in that, After generating at least one candidate chart based on the query results, the method further includes: Generate a text description for each of the candidate charts; Visualizing the at least one candidate chart and displaying the visualization results of the at least one candidate chart includes: The at least one candidate chart is rendered, and the visualization results of the at least one candidate chart and the text description of each candidate chart are displayed.

8. A data query device, characterized in that, include: The identification module is configured to identify the query intent of the data to be processed when the data to be processed is acquired; The parsing module is configured to parse the query intent to obtain a visual intermediate representation of the query intent; The determination module is configured to determine the query result of the data to be processed based on the visual intermediate representation and the preset database; The generation module is configured to generate at least one candidate chart based on the query results; The display module is configured to perform visualization processing on the at least one candidate chart and display the visualization results of the at least one candidate chart.

9. An electronic device, characterized in that, include: One or more processors; A memory having stored one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method according to any one of claims 1-7; One or more I / O interfaces are connected between the processor and the memory and configured to enable information exchange between the processor and the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images