Visual data query method based on natural language and configured vector library

Abstract

The embodiment of the invention discloses a visual data query method based on a natural language and a configured vector library. The method comprises the steps that DDL structure information and description documents of all databases, a plurality of historical questions and effective SQL query statements corresponding to the historical questions are trained into the vector library; searching DDL structure information and a description document which are most related to the new questions from a vector library, and effective SQL query statements corresponding to historical questions which are most similar to the new questions, packaging the effective SQL query statements into first prompt words, and prompting a large language model to generate new SQL query statements used for answering the new questions; and running the new SQL query statement to obtain an answer of the new question, packaging the new question, the new SQL query statement and a header of the database into a second prompt, prompting the large language model to generate a visual code, and visually displaying data in a table used when the new question is answered. According to the embodiment, the question and answer accuracy and the visual effect are improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of data query, and in particular, to a visual data query method based on natural language and a configured vector library. Background Art

[0002] With the rapid growth of the enterprise data scale, the ability requirements for data usage are continuously increasing. How to provide efficient and user-friendly data query services to non-professional personnel has become a key problem to be solved. Natural language is the main form of people's communication and expression. If data can be queried through natural language and the natural language query is converted into a data query statement, the ability requirements for users can be greatly reduced, and the daily query needs can be met.

[0003] The existing natural language-based data query technologies mainly perform data query by establishing a mapping relationship between natural language and SQL (Structured Query Language) statements and directly mapping natural language to the corresponding SQL statements. This method does not have a deep understanding of semantics and can only return the mapped data query results through surface literal words, which is likely to result in inaccurate data query; at the same time, the returned results are usually the SQL statements for data query, and for some non-professional users, they may hope to see more intuitive data or charts of the corresponding results. For example, Patent CN117667992A provides a method for converting natural language questions into SQL statements, and Patent CN117725078A provides a method for multi-table data query and prediction based on natural language, both of which cannot well solve these problems. Summary of the Invention

[0004] The embodiments of the present invention provide a visual data query method based on natural language and a configured vector library to solve at least one of the above technical problems.

[0005] In a first aspect, the embodiments of the present invention provide a visual data query method based on natural language and a configured vector library, including:

[0006] Obtain multiple question scenarios and multiple databases to be queried;

[0007] According to each question scenario, generate a description document for each database that can match each question scenario; and train the DDL structure information and description document of each database, as well as multiple historical questions and their corresponding valid SQL query statements, into the vector library;

[0008] In response to a new question from the user, search the vector library for the most relevant DDL structure information and description documents, as well as the valid SQL query statements corresponding to the historical question most similar to the new question;

[0009] Encapsulate the new question, the most relevant DDL structure information and description documents and the name of their corresponding database, as well as the most similar historical question and its corresponding valid SQL query statements together as a first prompt to prompt the large language model to generate a new SQL query statement for answering the new question;

[0010] Run the new SQL query statement to obtain the answer to the new question, and encapsulate the new question, the new SQL query statement and the table header of the database together as a second prompt to prompt the large language model to generate visualization code to visually display the data in the table used when answering the new question.

[0011] In a second aspect, an embodiment of the present invention provides an electronic device, the electronic device includes:

[0012] One or more processors;

[0013] A memory for storing one or more programs,

[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement the visualization data query method based on natural language and a configured vector library according to any embodiment.

[0015] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the visualization data query method based on natural language and a configured vector library according to any embodiment.

[0016] An embodiment of the present invention provides a visual data query method based on natural language and a configured vector library. First, corresponding document information is constructed for each database according to the question scenario, and the DDL structure, document information of each database, as well as historical questions and their corresponding valid SQL query statements are trained into the vector library to implement a configured vector library. Then, based on this vector library, more accurate database information and reference SQL statements are retrieved for specific questions, deepening the semantic understanding of natural language questions and database data information by the large language model, guiding the large model to select the correct database and database fields, generating more accurate result SQL statements, and improving the accuracy of answers. At the same time, this embodiment uses the query results, questions, and result SQL statements to guide the large language model to extract the database data used when answering questions, and selects a suitable visualization model according to the data characteristics to visually display the relevant data for generating answers, providing more intuitive data charts for users and improving the intelligence of data queries. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 is a flowchart of a visual data query method based on natural language and a configured vector library provided by an embodiment of the present invention;

[0019] Figure 2 is a flowchart of another visual data query method based on natural language and a configured vector library provided by an embodiment of the present invention;

[0020] Figure 3 is a schematic diagram of the layout of function modules provided by an embodiment of the present invention;

[0021] Figure 4 is another schematic diagram of the layout of function modules provided by an embodiment of the present invention;

[0022] Figure 5 is still another schematic diagram of the layout of function modules provided by an embodiment of the present invention;

[0023] Figure 6 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be described clearly and completely below. Apparently, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0025] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0026] In the description of the present invention, it should also be noted that unless otherwise clearly specified and defined, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0027] Figure 1 and Figure 2 are respectively the flowcharts of a visualization data query method based on natural language and a configured vector library provided by the embodiments of the present invention, showing the key steps in the method in different forms. This method is executed by an electronic device, in combination with Figure 1 and Figure 2 , and specifically includes the following steps:

[0028] S110. Obtain a plurality of question scenarios and a plurality of databases to be queried.

[0029] Here, the question scenario refers to the type of question raised in natural language. Multiple similar questions can be included under each question scenario; the database refers to the data resource that can answer these questions. For example, in the question scenario "How many students are enrolled in the faculty", multiple similar questions can be included: "How many students are enrolled in the Henan branch in the spring semester of 2024", "How many students are enrolled in the domestic branches in 2024", etc.; and the name of the database that can answer these questions is "Organizational Inspection Statistics", and its specific data structure is as follows:

[0030] table org inspect statistics / / Data table "Organization Inspection Statistics"

[0031] {semester code varchar(255) null comment 'Academic year and semester' (formatted as YYYYmm, where YYYY represents the year and 03 in mm represents the spring semester, 09 represents the autumn semester),

[0032] dt varchar(255) null comment 'Statistics date',

[0033] branch_code varchar(255) null comment 'Branch code,

[0034] branch_name varchar(255) null comment 'Branch name',

[0035] ……

[0036] register_number int null comment 'Number of students on the roll',

[0037] login_number int null comment 'Number of logins',

[0038] teacher_count int null comment 'Number of teachers',

[0039] student_count int null comment 'Number of students',

[0040] ……}

[0041] Fields such as "Academic year and semester", "Branch name", "Branch code", and "Number of students on the roll" in this database can be used to answer the question scenario.

[0042] Based on the above concepts, in this embodiment, the common question scenarios in the user's question and all databases allowed to be queried when answering questions are first obtained as the data sources of the entire method.

[0043] S120. According to each question scenario, generate a description document that can match each question scenario for each database; and train the DDL structure information and description document of each database, as well as multiple historical questions and their corresponding valid SQL query statements, into the vector library.

[0044] To improve the accuracy of data query, in this embodiment, description documents are generated for each database for the question scenarios. The description documents are used to explain which question scenarios the current database can provide effective data query for, including detailed description texts, annotations, and other contents. Optionally, the description documents of each database can be generated in the following ways:

[0045] Step 1: For question scenarios not covered in the description documents of all databases, determine the databases that can provide effective query data for the question scenarios, and add content in the description documents of these databases about which fields in the data can provide effective data for the question scenarios. This can ensure the full coverage of the description documents for the question scenarios and improve the accuracy of subsequent database queries.

[0046] Step 2: For question scenarios with field confusion in historical questions, add content for differentiating the confusion points in the description documents of the databases where the confused fields are located. Exemplarily, for the question scenario "How many students are enrolled in the department", in historical questions, sometimes the register_number field in the above database is queried, and sometimes the login_number field in the above database is queried. Then, the following content can be added to the description document of this database: The register_number in this database refers to the number of registered users, and the login_nuber refers to the number of logged-in users, and the two should be distinguished.

[0047] The DDL (Data Definition Language) structure information in this embodiment refers to the database structure described in DDL language. For example, the table org inspect statistics{…} (the specific content in the brackets is omitted) exemplified in S110 is a simple DDL structure information, including the field names, types, meanings, etc. in the database, and describes the data content of the database in the form of a data structure.

[0048] The valid SQL query statements corresponding to historical questions refer to the SQL query statements that have been used and verified as valid when answering historical questions. The "valid" here means that the syntax is valid and can be run, and can find the correct database and query fields for the question to give a correct answer. Since the entire method in this embodiment generates SQL query statements through a large language model, the valid query statements can be the query statements generated by the large language model, or the query statements that have been corrected manually when the query statements generated by the large language model are invalid. Among them, the specific details of manual correction will be described in detail in the subsequent steps and will not be elaborated here.

[0049] Before answering the user's specific question, a description document is constructed for the question scenario, and the description documents, DDL structure information of each database, as well as historical questions and their corresponding valid SQL query statements are trained into the vector library to obtain Figure 2 the intelligent base vector library shown.

[0050] In a specific embodiment, multiple vector libraries can be used to assist in implementing natural language queries. Users can select a familiar vector library according to the actual situation and the existing system, or select multiple for testing and determine the selected vector library according to the results. Optionally, the multiple vector libraries include ChromaDB, Qdrant, Marqo, etc. If there are other databases, the unified standard can also be met through the interface. After selecting the vector library, the user can configure the use of the vector library as needed, and train and store the description documents, DDL structure information of the databases used in the natural language query, and the valid SQL query statements corresponding to the historical questions into the vector library. This is the process of implementing the configured vector library in this embodiment.

[0051] By adding description documents, DDL structures, historical questions and their corresponding valid SQL query statements to the vector library, the semantic understanding ability of the large language model for natural language questions and database data information can be improved in subsequent steps, so as to select the correct database and database fields for query and improve the accuracy of the answer.

[0052] S130. In response to the user's new question, find the most relevant DDL structure information and description document in the vector library, and the valid SQL query statement corresponding to the historical question most similar to the new question.

[0053] Based on the above vector library, this embodiment can deploy a service, receive the user's specific data query question through the API interface, and record the corresponding log. After receiving a specific natural language question, this embodiment does not directly map it to the corresponding SQL statement, but finds the DDL information and document information associated with the question in the vector library, and at the same time queries the historical questions similar to the question and their corresponding valid SQL query statements in the vector library, and uses this information as the context information for guiding the large language model to generate valid SQL query statements subsequently.

[0054] S140. Package the new question, the most relevant DDL structure information and description document information and their corresponding database names, and the valid SQL query statement corresponding to the historical question most similar to the new question together as a prompt to prompt the large language model to generate a new SQL query statement for answering the new question.

[0055] After obtaining the context information, in this embodiment, these information, specific questions, and the fixed part of the prompt are assembled together into a question prompt (prompt) for the large language model to prompt the large model to generate an SQL query statement that can answer the question.

[0056] Optionally, the role of the large language model can be specified as an SQL expert; the task of the large language model can be specified as generating an SQL query statement for answering the new question based on the context information; the most relevant DDL structure information or description document information and its corresponding database name, as well as the valid SQL query statement corresponding to the most similar historical question, are jointly specified as the context information; at the same time, a remedial strategy when the context information is insufficient is specified; and the role, task, context information, and remedial strategy are jointly encapsulated into a prompt to prompt the large language model to generate a new SQL query statement for answering the new question.

[0057] Exemplarily, taking the question "How many students are enrolled in the Henan branch during the spring semester of 2024" as an example, the prompt finally generated according to the above steps is as follows (the content guided by " / / " is the explanation of each part of the prompt and is not the text in the prompt):

[0058] You are an SQL expert. Please help generate an SQL query statement to answer the question {How many students are enrolled in the Henan branch during the spring semester of 2024}.

[0059] Please use the following relevant tables:

[0060] table org inspect statistics / / This part corresponds to the DDL structure information

[0061] {semester code varchar(255)null comment 'Academic year and semester' (in the format of YYYYmm, where YYYY represents the year and 03 in mm represents the spring semester, 09 represents the fall semester)',

[0062] dt varchar(255)null comment 'Statistical date',

[0063] branch_code varchar(255)null comment 'Branch code,

[0064] branch_name varchar(255)null comment 'Branch name',

[0065] ……

[0066] register_number int null comment 'Number of registered students',

[0067] login_number int null comment 'Number of logged-in users',

[0068] teacher_count int null comment 'Number of teachers',

[0069] student_count int null comment 'Number of students',

[0070] ……}

[0071] This data table is used to describe the content of the statistics of the academic department organization. The register_number in the table refers to the number of registered users, and the login_nuber refers to the number of logged-in users. The two should be distinguished. / / This part corresponds to the content of the description document

[0072] If the question has been asked and answered before, please accurately repeat the previous answer. The most similar historical question is …, and the corresponding valid SQL query statement for this question is …. / / This part corresponds to the most similar historical question and its valid SQL query statement

[0073] Your response can only be based on the given context and follow the response guidelines and format instructions. Ensure that the output SQL complies with SQL standards and is executable without syntax errors.

[0074] Response guidelines: / / This part corresponds to the remedial measures

[0075] 1. If the provided context is sufficient, generate a valid SQL query without any explanation of the question.

[0076] 2. If the provided context is basically sufficient but you need to know the specific strings in a specific column, generate an intermediate SQL query to find the different strings in that column and add a comment before the query.

[0077] 3. If the provided context is insufficient, explain why it is not possible to generate.

[0078] Providing the above prompt to the large language model will result in an SQL query statement generated by the large language model (referred to as the result SQL), such as:

[0079] SELECT register_number

[0080] FROM org_inspect_statistics

[0081] WHERE semester_code = '202403' AND branch_name LIKE '%Henan Branch'.

[0082] S150. Run the new SQL query statement to obtain the answer to the new question, and encapsulate the new question, the new SQL query statement, and the header of the database together as another prompt to prompt the large language model to generate visualization code for visualizing the data in the table used to answer the new question.

[0083] After obtaining the above result SQL, in this embodiment, the SQL can be directly returned according to the user configuration or used to query the database. Specifically, it is configured in the user configuration whether the current user has the permission to access the actual data in the database to be queried by the result SQL. If there is no permission, the result SQL is directly returned and the process ends. If there is permission, this SQL statement is executed and the data query result is obtained.

[0084] Furthermore, for management users, if the answer is incorrect and it is caused by an incorrect result SQL, the above result SQL can also be manually corrected. After the correction, the natural language question and the correct SQL query statement of this time are saved and updated to the vector library as the historical question and its corresponding valid SQL query statement in S120, which are used to optimize subsequent similar questions.

[0085] In addition, if the chart generation function is enabled in the system, a new prompt can also be generated according to this question, the valid SQL query statement corresponding to this question (the SQL statement generated by the large language model or the corrected SQL statement), and the header of the database used to generate the query result (such as some fields in the data table used), and submitted to the large language model to instruct the large language model to generate the corresponding chart drawing code. If the returned generated code is unavailable, a preset dot chart / bar chart / pie chart / line chart can be drawn according to the number of rows and columns of the data query result. For the convenience of distinction and description, in this embodiment, the prompt used to generate the result SQL statement in S140 is called the first prompt, and the prompt used to generate the icon drawing code in S150 is called the second prompt.

[0086] Optionally, it is possible to record which headers in the data table were queried when running the valid SQL query statement; and encapsulate the new question, the DDL structure information of the data table, the valid SQL query statement, the headers and the relationships between them into context information; then, specify the task of the large language model as generating chart drawing code that can parse the answer to the question based on the context information, and specify the chart display format; finally, encapsulate the context information, the task and the chart display format into a second prompt.

[0087] Exemplarily, taking the question "How many students are enrolled in the Henan branch during the spring semester of 2024" as an example, the second prompt finally generated according to the above steps is as follows:

[0088] The valid SQL query statement executed when answering the question "How many students are enrolled in the Henan branch during the spring semester of 2024" is:

[0089] SELECT register_number

[0090] FROM org_inspect_statistics

[0091] WHERE semester_code='202403' AND branch_name LIKE'%Henan branch'

[0092] The headers queried when running this statement are:

[0093] branch name object

[0094] register_number int64

[0095] dtype:object

[0096] Please generate chart drawing code for the answer to this question based on the above information to show the data in the table used to generate the answer and the relationship between the answer and the data in these tables. If only one table's data is used, display it as an indicator chart; if more than one table's data is used, please select an appropriate chart form.

[0097] Under the above second prompt, the large language model will select the appropriate chart result according to the data in the table used to answer the question. Exemplarily, if the data "the number of students enrolled in Henan in spring" in the table is used to answer the question, it will be displayed in the form of an indicator chart; if three items of data in the table (the number of students enrolled in the Henan branch in the first 3 months) are used to answer the question, a pie chart may be generated, which shows the distribution of the number of students enrolled each month in spring on the basis of showing the total number of students enrolled in the spring semester.

[0098] Further, in a specific embodiment, after visualizing the data in the table used to answer the new question, the user may not be completely satisfied with the chart form given by the system and hopes to perform secondary editing. At this time, this embodiment provides a corresponding image editing component to support the user's secondary editing operation. It is worth noting that when providing this image editing component, this embodiment does not directly display the image editing interface according to the system's default layout (or called the original layout), but arranges the image editing interface according to the user's habits at the initial stage of system use, and gradually restores the interface layout to the system's default interface layout during the use process. Here, the system can be software or an APP that runs the method of this embodiment, such as an online learning software on a computer or a mobile phone. Correspondingly, the above process may include the following steps:

[0099] Step 1: In response to the user's editing operation on the visual image, select the visualization software with the highest usage frequency from the software installed on the same machine as the online learning software. Optionally, this embodiment provides an editing button for the chart form (i.e., the visual image) given by the system, such as providing an editing button through the right-click menu; when the user clicks this button, this embodiment automatically searches for all the visualization software installed on the computer or mobile phone where the online learning software runs, and selects the one with the highest usage rate by the user as the reference for each function module in the subsequent displayed image editing component.

[0100] Step 2: Establish a mapping relationship between each function module in the visualization software and each function module with the same function in the image editing component of the online learning software. Optionally, the mapping relationship can be established between the function modules of the visualization software with the same or similar functions and the function modules of the online learning software through the function name or function description. Exemplarily, assume that the visualization software most frequently used by the user can be the chart editing tool in Excel, and the image editing interface in Excel is as Figure 3 shown. The figure shows a visual image of a piece of data in the form of a bubble chart. The several function options enclosed by the red frame on the right side of the image are the top-level function modules under this visualization software, and their display layout is as Figure 3As shown; similarly, the online learning software also has functional modules such as "background color", "border", and "delete", which are equivalent to "fill", "border", and "delete" in Excel respectively. Then, mapping relationships can be established between "background color" and "fill", "border" and "border", and "delete" and "delete". For the convenience of distinction and description, in this embodiment, each functional module in the visualization software with the highest user usage rate is referred to as the first functional module, and each functional module in the online learning software is referred to as the second functional module.

[0101] Step 3: Display each second functional module according to the layout of each first functional module with a mapping relationship in the visualization software. Combining Figure 3 With, the positions of the three second functional modules of "background color", "border", and "delete" in the default interface layout of the online learning software may not be exactly the same as the positions of the three first functional modules of "fill", "border", and "delete" in Figure 3 . For example, Figure 3 The "fill" in is displayed in the first position of the middle functional area, then the "background color" is defaultly displayed in the second position of the top functional area. At this time, in order to accommodate user habits, in the initial stage of the user using the online learning software, this embodiment preferentially displays each module in the online learning software according to user habits, that is, in the way of Figure 3 , in the first-layer interface, display the second functional modules with the same functions as each first functional module at the same positions and in the same quantities. For example, display the "background color" also in the first position of the middle functional area. For the second functional module A for which no mapping relationship can be established, it can be displayed at the next lower position of the upper second functional module B in the current layout according to its hierarchical relationship in the default layout of the online learning software. For example, module A has no first functional module with the same function in Excel, and in the default layout of the online learning software, module A belongs to the next lower module of module B, and module B has been displayed at the same position according to the layout of the first functional module with the same function in Excel at this time. Then, display module A as the last module at the next lower position of this position.

[0102] Step 4: During the use of the online learning software, according to the user's click operations on each second function module, gradually restore the second function module with the lowest click-through rate to its position in the default layout of the image editing component. In this embodiment, during the user's use of the system, the cumulative click counts of each second function module are regularly counted, and the click-through rate of each second function module is calculated based on this count (for example, how many times are clicked per day). The higher the click-through rate, the more the user uses the function module, and the deeper the impression of the position of the module in the user's habit. If the position of the module is suddenly changed, it may cause the user to feel uncomfortable and disgusted, which is not conducive to improving user stickiness. Therefore, after calculating the click-through rate for a period of time each time, this embodiment preferentially selects one or more second function modules with the lowest click-through rate and restores them to the default position of the online learning software. For example, assume that the "Delete" module has the lowest click-through rate. In the current interface layout (i.e., Figure 3 the interface layout of Excel), it is located at the first position in the bottom function area, while in the default function module layout of the online learning software, the "Delete" module is located at the last item in the first layer interface or in the next layer interface (i.e., the next layer interface entered after clicking a function module in the first layer interface). Then, the "Delete" module is adjusted to this position to reduce the probability of the user accidentally deleting data by misclicking the "Delete" in the first layer interface. This is also the reason why the online learning software places it at the last item in the first layer interface or the second layer interface in the default function module layout. It can be seen that in this embodiment, starting from the modules that are not easily noticed by the user during the system use, it gradually guides and infiltrates the user's habits, retains some advantages of the online learning software itself in the function module layout, guides the user to gradually adapt to the current software, and finally replaces the previously habitual visualization software.

[0103] Further, when calculating the cumulative click counts of each second function module, considering that sometimes the user clicks on a certain module not to use the module, but to check whether there is a desired function module under the module (i.e., the user is looking for a certain desired function module), therefore, this embodiment differentiates each second function module. The module that directly performs the image editing operation is called the execution module, and the function modules above the execution module that are used to guide the user to drill down layer by layer to find the execution module are all called the guiding modules. Exemplarily, assume Figure 3 is the current module layout of the online learning software. Taking the last item "Set Drawing Area Format" in the bottom function area of the figure as an example, when the user clicks on this function module, it will display as Figure 4The next-level function interface shown, where function modules such as "no stroke" and "no fill" are execution modules. After the user clicks on these function modules, the corresponding graphics will directly remove the fill color and stroke; while the two function modules "fill" and "border" are guiding modules, which are set to guide the user to find operation modules such as "no fill" and "no stroke". After the user clicks on "fill" and "border", the fill color and stroke of the chart will not be directly changed.

[0104] Based on the above two types of modules, the cumulative click count of each second function module can be calculated in the following way: In response to the user's top-down click operation on a certain guiding module C, increment the cumulative click count of the guiding module by 1. Optionally, if the user starts from the upper-level guiding module and clicks on the lower-level modules layer by layer, a top-down click sequence can be formed, and the cumulative click count of each module in this sequence can be incremented by 1.

[0105] To distinguish it from subsequent click operations, the above-mentioned "click operation on a certain guiding module C" is referred to as the first click operation. If after the first click operation, no click operation on any execution module under the guiding module C (referred to as the second click operation) is detected, but instead a click operation on other modules outside the drill-down module branch starting from the guiding module C (referred to as the third click operation) is detected, then decrement the cumulative click count of the guiding module C by 1. Exemplarily, taking Figure 5 the guiding modules "fill" and "border" in as an example, there are at least one layer of subordinate modules guided by the two guiding modules respectively, forming two top-down drill-down module branches. If after the user performs the first click operation on the guiding module "fill", they do not reach an execution module along the top-down drill-down path to perform an image editing operation, but instead exit the large function module "fill" midway and click on the large function module "border" (i.e., exit the drill-down module branch guided by "fill" and enter the drill-down module branch guided by "border"), this is very likely that the user is looking for a desired image editing function. Clicking on "fill" is to check if the function is in the "fill" function branch. When not found, they click on "border" to continue searching. In this case, the click on "fill" does not reflect the user's usage of this function module, so the usage count that has been incremented during the drill-down process should be removed.

[0106] Meanwhile, if after the first click operation, no second click operation on any execution module under the guiding module is detected, and a backtracking click sequence with the guiding module as the end point is detected, the cumulative click count of the guiding module is also decremented by 1. Still taking Figure 5Taking the "Fill" and "Border" in the guidance module as an example, when the user clicks on "Fill" and then clicks on the module sequence of a - b - c layer by layer and fails to find the desired image editing function, so the user clicks on the module sequence of "c - a - b - Fill" layer by layer to return to the current interface. At this time, the click process of drill - down + rollback should not be regarded as the user's usage of this function module, but the click times accumulated during the drill - down process should be deleted.

[0107] Furthermore, in another specific implementation, since the function module layout changes every once in a while, the explanatory documents for each function module in the help module should also be changed accordingly. Optionally, if there is text about the hierarchical path of a certain function module in the document, then when the user clicks on the help module, the text about the module hierarchical path in the document can be extracted first. If this text is inconsistent with the hierarchical path where the module is currently located in the function layout interface, then this text should be replaced with the current hierarchical path. Exemplarily, in the original help text, there is text like "The access path of module D is 'background color -> solid color -> color'", while the hierarchical path of module D in the current function layout is 'background color -> drawing area -> solid color -> color', then this text needs to be modified to "The access path of module D is 'background color -> drawing area -> solid color -> color'" so that the user can access the correct module in the current module layout.

[0108] In summary, this embodiment provides a visualization data query method based on natural language and a configured vector library. First, corresponding document information is constructed for each database for the question - asking scenario, and the DDL structure, document information of each database, as well as historical questions and their corresponding valid SQL query statements are trained into the vector library to implement a configured vector library. Then, based on this vector library, more accurate database information and reference SQL statements are provided for specific question retrieval, deepening the semantic understanding of the large - language model for natural - language questions and database data information, guiding the large model to select the correct database and database fields, generating more accurate result SQL statements, and improving the accuracy of the answer. At the same time, this embodiment uses the query results, questions, and result SQL statements to guide the large - language model to extract the database data used when answering questions, and selects a suitable visualization model according to the data characteristics to visually display the relevant data for generating answers, providing more intuitive data charts for users and improving the intelligence of data query.

[0109] Specifically, when the user needs to perform secondary editing on the data image, this embodiment takes other visualization software installed on the same machine as a reference, learns the interface layout familiar to the user, and preferentially restricts the function layout according to the user's habits at the initial stage of system use to avoid losing users due to unfamiliar use. During the system use process, the system's built-in function layout is gradually restored starting from the module that the user is least likely to notice, realizing the gradual guidance and penetration of the user's habits, and finally promoting the built-in function layout to the user, so as to retain the layout form specifically designed by the system for specific services, which is convenient for users to use and also improves user stickiness. Among them, the degree of attention of the user to each module is measured by the click-through rate. When calculating the cumulative click count of the user, this embodiment specifically considers the click process of the user on each function module when looking for a certain image editing function and excludes this process from the user's usage of the module, improving the accuracy of calculating the cumulative click count and click-through rate, thereby improving the performance of the entire function module layout method.

[0110] It should be noted that all user data involved in this application are information and data that have been authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for the user to choose to authorize or refuse.

[0111] Figure 6 The following is a schematic structural diagram of an electronic device provided by an embodiment of the present invention, as Figure 6 shown, the device includes a processor 60, a memory 61, an input device 62, and an output device 63; the number of processors 60 in the device can be one or more, Figure 6 taking one processor 60 as an example; the processor 60, memory 61, input device 62, and output device 63 in the device can be connected through a bus or other means, Figure 6 taking the connection through a bus as an example.

[0112] The memory 61, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the visualization data query method based on natural language and a configured vector library in the embodiment of the present invention. The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 61, that is, implements the above-mentioned visualization data query method based on natural language and a configured vector library.

[0113] The memory 61 may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 61 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 61 may further include a memory remotely provided with respect to the processor 60, and these remote memories may be connected to the device through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0114] The input device 62 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the device. The output device 63 may include a display device such as a display screen.

[0115] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the visualization data query method based on natural language and a configured vector library in any embodiment.

[0116] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.

[0117] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program code is carried. Such a propagated data signal may take various forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0118] The program code contained on a computer-readable medium can be transmitted with any suitable medium, including but not limited to wireless, wire, optical fiber cable, RF, etc., or any suitable combination of the above.

[0119] The computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the C language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A visual data query method based on natural language and configured vector library, characterized in that: include: Obtain multiple questioning scenarios and multiple databases to be queried; According to each questioning scenario, a description document that can match each questioning scenario is generated for each database; The DDL structure information and description documents of each database, as well as multiple historical questions and their corresponding valid SQL query statements, are trained into the vector library; In response to a new question from a user, searching the vector library for DDL structure information and description documents most relevant to the new question, as well as valid SQL query statements corresponding to historical questions most similar to the new question; The new question, the most relevant DDL structure information and description document and the name of the corresponding database, and the most similar historical question and the corresponding valid SQL query statement are encapsulated together as a first prompt to prompt the large language model to generate a new SQL query statement for answering the new question; Run the new SQL query statement to obtain the answer to the new question, and encapsulate the new question, the new SQL query statement and the table header of the database together as a second prompt, prompting the large language model to generate visualization code to visualize the data in the table used to answer the new question.

2. The method according to claim 1, characterized in that The method is applied to online learning software; After prompting the large language model to generate a visualization code and visually displaying the data in the table used to answer the new question, the method further includes: In response to the user's editing operation on the visualization image, selecting the visualization software most frequently used by the user from the software installed on the same machine as the online learning software; Establishing a mapping relationship between each first functional module in the visualization software and each second functional module having the same function in the image editing component of the online learning software; Displaying each second functional module according to the layout of each first functional module having a mapping relationship in the visualization software; According to the click rate of each displayed second functional module by the user, the second functional module with the lowest click rate is gradually restored to the position in the original layout of the image editing component.

3. The method according to claim 2, characterized in that The second functional module includes an execution module for executing the image editing operation, and a guidance module for guiding the user to drill down layer by layer to find the execution module; The method of gradually restoring the second functional module with the lowest click rate to the position in the original layout of the image editing component according to the click rate of each displayed second functional module by the user includes: In response to a user's first click operation on a certain guide module from top to bottom, the cumulative number of clicks on the guide module is increased by 1; If after the first click operation, a second click operation on any execution module under the guiding module is not detected, but a third click operation on a module other than the drill-down module branch starting from the guiding module is detected, the cumulative number of clicks on the guiding module is reduced by 1; If after the first click operation, no second click operation on any execution module under the guiding module is detected, but a fallback click sequence ending at the guiding module is detected, the cumulative number of clicks on the guiding module is reduced by 1; At regular intervals, the click rate of each second functional module is calculated based on the accumulated number of clicks on each second functional module.

4. The method according to claim 2, characterized in that: Also includes: In response to a user clicking operation on a help module in the image editing component, identifying text about a hierarchical path of each second functional module in the help document; When each text is inconsistent with the current hierarchical path of each second functional module in the current functional module layout, each text is replaced with the current hierarchical path.

5. The method according to claim 1, characterized in that The step of generating a description document for each database that can match each questioning scenario according to each questioning scenario includes: For questioning scenarios that are not covered in the description documents of all databases, determine the database that can provide valid query data for the questioning scenario, and add content in the description documents of the database about which fields in the database can provide valid data for the questioning scenario; For question scenarios where field confusion occurs in historical questions, content used to distinguish confusion points is added to the description document of the database where the confused fields are located.

6. The method according to claim 1, characterized in that The step of encapsulating the new question, the most relevant DDL structure information and description document and the name of its corresponding database, and the most similar historical question and its corresponding valid SQL query statement into a first prompt, prompting the large language model to generate a new SQL query statement for answering the new question, includes: Designate the role of the large language model as a SQL expert; Assigning the task of the large language model to generate an SQL query statement for answering the new question based on the context information; The most relevant DDL structure information and description document information and the name of the corresponding database, as well as the most similar historical question and the corresponding valid SQL query statement are collectively designated as context information; Specify remediation strategies when contextual information is insufficient; The role, task, context information and remediation strategy are encapsulated together as a first prompt, prompting the large language model to generate a new SQL query statement for answering the new question.

7. The method according to claim 1, characterized in that The step of encapsulating the new question, the new SQL query statement, and the table header of the database into a second prompt, prompting the large language model to generate a visualization code, and visually displaying the data in the table used to answer the new question, includes: Record the header of the data table queried when running the new SQL query statement; Encapsulate the new question, the DDL structure information of the data table, the valid SQL query statement, the table header and the relationship between them as context information; Assigning the task of the large language model to generate a chart drawing code capable of parsing the answer to the new question according to the context information, and specifying a chart display format; The context information, the task and the chart display format are packaged together as a second prompt.

8. The method according to claim 1, characterized in that The visual display of the data in the table used to answer the new question includes: If the visualization code is not available, draw a preset dot chart / bar chart / pie chart / line chart by yourself according to the number of rows and columns of the data query results.

9. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the visual data query method based on natural language and configured vector library as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the visual data query method based on natural language and a configured vector library as described in any one of claims 1-8 is implemented.

Images