Data analysis method and device, server and readable storage medium
By rewriting natural language problems and generating SQL prompt words, automatic data screening is realized, solving the problems of low efficiency and poor accuracy of manual data screening, and improving the efficiency and accuracy of data analysis.
Patent Information
- Application Number
- CN202510387026.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-28
- Publication Date
- 2025-04-29
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing data analysis methods require manual screening of data, which are inefficient and have poor accuracy.
By rewriting natural language problems, a structured query statement SQL prompt words are generated, and SQL is automatically generated using SQL prompt words, so as to automatically filter data based on natural language problems.
Improves the efficiency and accuracy of data analysis, saves a lot of manpower, and ensures that the generated SQL is more accurate, thus obtaining more accurate data analysis results.
Smart Images

Figure CN119903080A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data analysis method, device, server, and readable storage medium. Background Art
[0002] Data analysis refers to the means of screening out valid data from a large amount of data and extracting useful information such as trends and data correlations from the valid data in order to make corresponding decisions. This is crucial in all walks of life.
[0003] At present, when performing data analysis, it is necessary to manually screen the data according to different needs, and manually draw tables or charts based on the screened data to visualize the analysis results. However, this data analysis method that requires manual data screening has the problems of low efficiency and poor accuracy. Summary of the invention
[0004] In view of this, embodiments of the present application provide a data analysis method, device, server, and readable storage medium to improve the efficiency and accuracy of data analysis.
[0005] The present application provides a data analysis method, which includes: Rewrite the natural language question so that the key words in the natural language question are replaced accordingly; Search the existing database for a data table that matches the rewritten natural language question; Generate structured query statement SQL prompt words based on the rewritten natural language question, data information in the data table matching the rewritten natural language question, cases matching the rewritten natural language question in the existing case library, and set constraints; the cases matching the rewritten natural language question include the question and the SQL corresponding to the question that meets the requirements; Using SQL prompt words, generate SQL that matches the rewritten natural language question.
[0006] The present application also provides a data analysis device, which includes: A rewriting module is used to rewrite natural language questions so that keywords in the natural language questions are replaced accordingly; A screening module is used to search for a data table matching the rewritten natural language question from an existing database; A prompt word generation module is used to generate structured query statement SQL prompt words based on the rewritten natural language question, data information in a data table matching the rewritten natural language question, cases matching the rewritten natural language question in an existing case library, and set constraints; the cases matching the rewritten natural language question include the question and the SQL corresponding to the question that meets the requirements; The SQL generation module is used to generate SQL matching the rewritten natural language question by using SQL prompt words.
[0007] An embodiment of the present application also provides an electronic device, comprising: a processor and a computer-readable storage medium for storing computer program instructions, wherein the computer program instructions, when executed by the computer-readable storage medium, enable the processor to execute the steps of the above method.
[0008] An embodiment of the present application also provides a machine-readable storage medium, which stores computer program instructions. When the computer program instructions are executed, the steps of the above method can be implemented.
[0009] It can be seen from the above technical scheme that in this embodiment, the natural language question is first rewritten so that the keywords in the natural language question are replaced accordingly, and then the data table matching the rewritten natural language question is searched from the existing database. Afterwards, according to the rewritten natural language question, the data information in the data table matching the rewritten natural language question, the case matching the rewritten natural language question in the existing case library, and the set constraints, the structured query statement SQL prompt is generated, and the SQL matching the rewritten natural language question is generated by using the SQL prompt. In this way, the generated SQL can be directly run in the database, so as to realize the screening of data matching the natural language question from the database. In other words, this method of rewriting the natural language question, generating SQL prompts, and automatically generating SQL using SQL prompts can realize automatic screening of data according to the natural language question, which can effectively improve the efficiency of data analysis and save a lot of manpower compared to manual screening of data.
[0010] Furthermore, since the natural language question is rewritten first, the subsequently generated SQL prompt words are generated based on the more standardized rewritten natural language question, which makes the SQL prompt words generated based on the rewritten natural language question more accurate, thereby ensuring that the SQL generated based on the SQL prompt words is more accurate, and achieving the most accurate data from the database based on the most accurate SQL, as well as the most accurate data analysis results, which effectively improves the accuracy of data analysis. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 A schematic diagram of a process for a method provided in an embodiment of the present application; Figure 2 A schematic diagram of the SQL verification process provided in the embodiment of the present application; Figure 3 A schematic diagram of a process for obtaining positive and negative cases provided in an embodiment of the present application; Figure 4A schematic diagram of the structure of the device provided in the embodiment of the present application; Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0012] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and understandable, the technical solutions in the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings.
[0013] It should be noted that the data analysis method provided in the embodiment of the present application is applicable to data analysis of data in a database in the medical field, and is also applicable to data analysis of data in a database in other fields such as the financial field. The embodiment of the present application is not specifically limited.
[0014] See also Figure 1 , Figure 1 The flowchart of the method provided in the embodiment of the present application is as follows. As an embodiment, the method is applied to an electronic device, which may be a server, and the present application does not specifically limit it.
[0015] like Figure 1 As shown, the process may include the following steps: S101, rewriting the natural language question so that the key words in the natural language question are replaced accordingly.
[0016] In this embodiment, the natural language question may be input by the user.
[0017] In this embodiment, the keywords include at least one of a technical term and a descriptive word with context. Here, the technical term can be an abbreviation of a professional term or a professional term in a dialect. The descriptive word with context can be a pronoun such as "this", "his", "other", etc.
[0018] In the specific implementation of step S101, a search engine (such as ES search engine) or a noun matching method is used to determine the keywords in the natural language question. If the keywords are technical nouns, the technical nouns in the natural language question are replaced with professional terms. For example, PDA in the natural language question is replaced with coronary heart disease.
[0019] If the keyword is a descriptive word with context, the descriptive word in the natural language question is replaced with a noun. For example, the Large Language Model (LLM) analyzes "this" in the natural language question based on the contextual semantics of "this" and determines that "this" refers to "age", so "this" in the natural language question is replaced with "age".
[0020] Compared with natural language questions, the rewritten natural language questions are more standardized, easier to be recognized by large language models, and easier to be converted into structured query statements (SQL) that can be executed by the database.
[0021] S102, searching an existing database for a data table that matches the rewritten natural language question.
[0022] In this embodiment, each data table in the database has a corresponding description. The description of any data table includes: table description information and field description information of each field in the data table. Table description information is pre-marked information used to summarize the content of the table, such as a department personnel status table. For example, the table description information "contains each person's name, age, height, education and other information" is used to obtain user information, etc. The field description information is used to summarize the content of the field, such as the doctor's name. It should be noted that the field description information can better play a distinguishing role when the fields cannot be distinguished. For example, the field is a user, and the field description information indicates a doctor user, a nurse user or a patient user, etc. The specific implementation method of the above step S102 can be: based on the description of each data table, find a data table that matches the rewritten natural language question from the database.
[0023] Optionally, the above-mentioned searching for a data table matching the rewritten natural language question from the database based on the description of each data table can be implemented using a large language model. When the large language model is implemented, for each data table, the first matching degree between the table description information of the data table and the rewritten natural language question is calculated. For example, semantic matching is performed between the table description information of the data table and the rewritten natural language question to obtain the first matching degree.
[0024] The second matching degree between the field description information of each field in the data table and the rewritten natural language question is calculated. For example, a large language model is used to perform keyword matching between each field description information of the data table and the rewritten natural language question to obtain the second matching degree.
[0025] Based on the first matching degree and each second matching degree, the matching degree of the data table is determined. For example, a specified operation, such as a weighted operation, is performed on the first matching degree and each second matching degree to obtain the matching degree of the data table.
[0026] Find a data table from the database whose matching degree meets the set matching degree requirement, such as selecting the data table with the highest matching degree, or selecting the data table ranked in the top K matching degrees. Determine the found data table as the data table that matches the rewritten natural language question.
[0027] This method of obtaining a data table that matches the rewritten natural language question can screen out the data table with the highest matching degree from many data tables. This can not only effectively reduce the amount of data in subsequent SQL queries and improve query efficiency, but also provide a basis for solving the problem that when there are a large number of data tables in the database, the information of many tables interferes with each other and SQL cannot be generated correctly.
[0028] S103, generating a structured query statement SQL prompt word according to the rewritten natural language question, the data information in the data table matching the rewritten natural language question, the cases matching the rewritten natural language question in the existing case library, and the set constraints.
[0029] In this embodiment, the data information of any data table may include the table data definition language (DDL) of the table. Optionally, the DDL of any data table is determined based on the table name of the data table, the table description information of the table, the fields in the table, and the field description information of each field. For example, the table name of the data table, the table description information of the table, the fields in the table, and the field description information of each field are smoothly spliced according to the setting to obtain the DDL of the data table.
[0030] The case library includes positive cases and negative cases. The positive case includes the question and the SQL corresponding to the question without abnormal information, and the negative case includes the question and the SQL corresponding to the question with abnormal information.
[0031] Obtaining a case that matches the rewritten natural language question from the case library is achieved through the following steps: Find a question that matches the voice of the rewritten natural language question from the case library, and the positive case to which the question belongs is the case that matches the rewritten natural language question.
[0032] The set constraints include at least one of the following conditions: The first constraint condition is used to prohibit searching questions containing set sensitive words.
[0033] The second constraint condition is used to limit the fields in the generated SQL to be selected from the fields in the data table that match the rewritten natural language question.
[0034] The third constraint condition is used to limit the amount of query data.
[0035] The specific implementation of step S103 is as follows: the rewritten natural language question, the table data definition language DDL of the data table matching the rewritten natural language question, and the SQL that meets the requirements and the set constraints in the case matching the rewritten natural language question are spliced in a specified order, and the spliced result is determined as the SQL prompt word. The SQL prompt word is used to provide prompts and rules to be followed when converting the rewritten natural language question into SQL.
[0036] For example, the rewritten natural language question is: I want to find the 10 oldest people.
[0037] The table data definition language DDL that matches the rewritten natural language question is as follows: CREATE TABLE `user` ( `id` int(11) NOT NULL AUTO_INCREMENT, `name`varchar(255) DEFAULT NULL COMMENT 'name', `age` int(11) DEFAULT '0' COMMENT 'age', PRIMARY KEY (`id`)) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4.
[0038] The examples that match the rewritten natural language question are as follows: Question: Please help me find the five oldest people. The matching SQL for this question is: SELECT u.id,u.name, u.age, s.math_score FROM `user` u JOIN `scores` s ON u.id = s.user_id ORDER BY u.age DESC LIMIT 5.
[0039] The constraints are: 1. Refuse to answer if there are sensitive words; 2. The fields in the SQL generated statement are only obtained from the data table that matches the rewritten natural language question, and the use of additional fields is prohibited; 3. If the amount of query data is too large, output 100 records.
[0040] The rewritten natural language question, DDL, the above-mentioned matching SQL, and the above-mentioned three constraints are concatenated at once to obtain the SQL prompt word.
[0041] Through the above manner, an SQL prompt word for limiting details in the SQL generation process is obtained, which can make the obtained SQL more accurate.
[0042] S104, using SQL prompt words, generating SQL matching the rewritten natural language question.
[0043] In the specific implementation of the above step S104, the rewritten natural language question and the SQL prompt word may be input into the obtained large language model to obtain the SQL matching the rewritten natural language question.
[0044] The large language model is pre-trained, optionally with a manually written training set, and the training samples in the training set include at least: question samples, SQL prompt word samples, and SQL samples corresponding to the question samples, where the SQL samples are labels for the question samples. The initial large language model is supervisedly trained using the above training set, and the specific training process is similar to the supervised training model in the related technology, which will not be repeated here. The large language model has the ability to convert questions into SQL (that is, the Text2Sql function). After the rewritten natural language question and SQL prompt word are input into the large language model, the large language model generates the SQL corresponding to the rewritten natural language question according to the prompt of the SQL prompt word.
[0045] After obtaining the SQL, you can directly run the SQL in the database, and the database will execute the corresponding query program to obtain the target data. The target data can be used for data analysis.
[0046] So far, completed Figure 1 The process shown.
[0047] pass Figure 1 As can be seen from the process shown, in this embodiment, the natural language question is first rewritten so that the keywords in the natural language question are replaced accordingly, and then a data table matching the rewritten natural language question is searched from the existing database. Afterwards, based on the rewritten natural language question, the data information in the data table matching the rewritten natural language question, the cases matching the rewritten natural language question in the existing case library, and the set constraints, a structured query statement SQL prompt is generated, and the SQL prompt is used to generate the SQL matching the rewritten natural language question. In this way, the generated SQL can be directly run in the database, so as to filter out the data matching the natural language question from the database. In other words, this method of rewriting the natural language question, generating SQL prompts, and automatically generating SQL using SQL prompts can automatically filter data according to the natural language question, which can effectively improve the efficiency of data analysis and save a lot of manpower compared to manual data filtering.
[0048] Furthermore, since the natural language question is rewritten first, the subsequently generated SQL prompt words are generated based on the more standardized rewritten natural language question, which makes the SQL prompt words generated based on the rewritten natural language question more accurate, thereby ensuring that the SQL generated based on the SQL prompt words is more accurate, and achieving the most accurate data from the database based on the most accurate SQL, as well as the most accurate data analysis results, which effectively improves the accuracy of data analysis.
[0049] Furthermore, considering that the SQL generated by the above step S104 may contain abnormal information, it is necessary to verify the SQL. Figure 2 The SQL verification process is elaborated in detail: See also Figure 2 , Figure 2 The following is a flow chart of the SQL verification process provided by the embodiment of the present application. Figure 2 As shown, the process includes the following steps: S201, performing abnormal information detection on the generated SQL based on the obtained large language model.
[0050] Here, the abnormal information includes at least one of the following information: illegal characters, redundant conditions, and logical errors. Illegal characters include banned words. Redundant conditions indicate that there are similar constraints in SQL. Logical errors indicate that the SQL is not the SQL language for executing "Select", for example, the SQL statement of "Delect" is a logical error.
[0051] In this embodiment, the specific implementation of the above step S201 can be: inputting the above SQL and the prompt words used to detect abnormal information on the SQL into the large language model to obtain the detection result of the SQL.
[0052] In this embodiment, the prompt word is an external input, and its semantics is to detect abnormal information on the SQL. For example, the prompt word may be "Please help me analyze whether there is abnormal information in the obtained SQL."
[0053] It should be noted that the large language model can be trained using a manually written training set, and each training sample in the training set includes at least: a problem sample, a SQL sample with abnormal information corresponding to the problem sample, and a sample label corresponding to the SQL sample. Here, the abnormal information is used as the sample label of the SQL sample. The initial large language model is supervised by using the above training set. The specific training process is similar to the supervised training model in the related technology, and will not be repeated here.
[0054] S202, based on the anomaly detection result output by the large language model, determine whether the SQL contains abnormal information. If yes, execute the following step S203, and then return to S201. If no, execute the following step S204.
[0055] S203, when it is determined that the set stop iteration condition is not met, SQL prompt words, detected abnormal information, and SQL with abnormal information are used to regenerate SQL matching the rewritten natural language question.
[0056] In this embodiment, the condition for stopping iteration may be set as: the number of times that the SQL generated for the rewritten natural language question is determined to contain abnormal information reaches an upper limit, such as 3 times.
[0057] When regenerating SQL, the rewritten natural language question, SQL prompt, detected abnormal information, and SQL with abnormal information are re-entered into the large language model to obtain the regenerated SQL. The reason why the detected abnormal information and SQL with abnormal information are input together with the rewritten natural language question and SQL prompt is to make the large language model learn that the SQL with abnormal information is wrong, and when continuing to regenerate the reference SQL, it will not generate the same SQL with the same abnormal information.
[0058] S204, performing data query in the database based on SQL to obtain target data matching the rewritten natural language question.
[0059] So far, completed Figure 2 The process shown.
[0060] pass Figure 2 It can be seen from the process shown that this method of verifying SQL can ensure that the final SQL is optimal, and then ensure that the target data obtained by using the final SQL for data query is optimal, so as to achieve the optimal data analysis results using the optimal target data, which effectively improves the accuracy of data analysis.
[0061] The above describes the SQL verification process in detail.
[0062] See also Figure 3 , Figure 3 A schematic diagram of the process of obtaining positive and negative cases provided in the embodiment of the present application. Figure 3 As shown, the process includes the following steps: S301, after obtaining the SQL that does not contain abnormal information, obtain a positive case based on the corrected natural language question and the SQL, and store the positive case in the case library.
[0063] Here, the positive cases in the case library can be used to regularly update the large language model to improve the accuracy of SQL generated by the large language model. It can also provide rich samples for the cases used in the SQL prompt words that match the corrected natural language questions.
[0064] S301, for the SQL that fails the verification, based on the corrected natural language question, the SQL and the abnormal information indicated by the detection result, a negative case is obtained, and the negative case is stored in the case library.
[0065] Here, the negative cases in the case library can be used to regularly update the large language model to improve the accuracy of the large language model in detecting abnormal information.
[0066] Positive and negative cases are obtained in the above manner, which provides sufficient samples for the subsequent optimization of the abnormal information detection capability of the large language model, and also provides sufficient samples for obtaining corrected natural language problem matching cases.
[0067] After obtaining the target sample by querying the database based on SQL, the method further includes: using the obtained large language model to generate a visual analysis result based on the target data.
[0068] Specifically, the target data and the analysis statement used to indicate the data analysis intention are input into the large language model, and the large language model analyzes the target data according to the data analysis intention to obtain the analysis result. Here, an example of the analysis statement indicating the data analysis intention is given. For example, if the target statement is "help me find the 10 oldest orthopedic patients in the database", the target data obtained is the data of these 10 people, and the analysis statement is "help me analyze the geographical distribution of these 10 people", then the large language model will output the geographical distribution of these 10 people.
[0069] Furthermore, if the analysis statement also includes "representing the geographical distribution of the 10 people analyzed using a chart", the large language model will generate the code for the ECharts chart, and then call the code to achieve the final effect of visualizing the analysis data.
[0070] The method provided in the embodiment of the present application is described above. The device provided in the embodiment of the present application is described below: See also Figure 4 , Figure 4 This is a diagram of the device structure provided in the embodiment of the present application. Figure 4 As shown, the device includes: a rewriting module 401, a screening module 402, a prompt word generating module 403 and an SQL generating module 404.
[0071] A rewriting module is used to rewrite natural language questions so that keywords in the natural language questions are replaced accordingly; A screening module is used to search for a data table matching the rewritten natural language question from an existing database; A prompt word generation module is used to generate structured query statement SQL prompt words based on the rewritten natural language question, data information in a data table matching the rewritten natural language question, cases matching the rewritten natural language question in an existing case library, and set constraints; the cases matching the rewritten natural language question include the question and the SQL corresponding to the question that meets the requirements; The SQL generation module is used to generate SQL matching the rewritten natural language question by using SQL prompt words.
[0072] As an embodiment, rewriting the natural language question includes: Keywords include technical terms, which replace technical terms in natural language questions with professional terms; and / or, Keywords include descriptive words with context, and descriptive words in natural language questions are replaced with nouns.
[0073] As an embodiment, each data table in the database has a corresponding description; the description includes: table description information and field description information of each field in the data table; Among them, searching for data tables matching the rewritten natural language question from the existing database includes: Based on the description of each data table, a data table matching the rewritten natural language question is found from the database.
[0074] As an embodiment, based on the description of each data table, the data table matching the rewritten natural language question is found from the database, including: For each data table, calculating a first matching degree between the table description information of the data table and the rewritten natural language question; Calculating a second matching degree between the field description information of each field in the data table and the rewritten natural language question; Determining a matching degree of the data table based on the first matching degree and each second matching degree; A data table whose matching degree meets the set matching degree requirement is found from the database, and the found data table is determined as a data table that matches the rewritten natural language question.
[0075] As an embodiment, according to the rewritten natural language question, the data information in the data table matching the rewritten natural language question, the cases matching the rewritten natural language question in the existing case library, and the set constraints, the structured query statement SQL prompt words are generated, including: The rewritten natural language question, the table data definition language DDL in the data information in the data table matching the rewritten natural language question, and the SQL that meets the requirements and the set constraints in the case matching the rewritten natural language question are spliced in a specified order to obtain SQL prompt words.
[0076] As an embodiment, using SQL prompt words to generate SQL matching the rewritten natural language question includes: The rewritten natural language question and the SQL prompt words are input into the obtained large language model to obtain SQL matching the rewritten natural language question.
[0077] As an embodiment, the device further includes a verification module, and the verification module is used to: After generating SQL matching the rewritten natural language question, abnormal information detection is performed on the generated SQL based on the obtained large language model; If the abnormal information detection indicates that the SQL does not contain abnormal information, a data query is performed in the database based on the SQL to obtain target data that matches the rewritten natural language question; If the abnormal information detection indicates that the SQL contains abnormal information, then when it is determined that the set stop iteration condition is not met, the SQL prompt word, the detected abnormal information, and the SQL with abnormal information are used to regenerate the SQL that matches the rewritten natural language question, and return to the step of performing abnormal information detection on the generated SQL based on the obtained large language model; wherein the regenerated SQL is different from the SQL with abnormal information.
[0078] As an embodiment, the device further includes a visualization module, and the visualization module is used to: After performing data query in the database based on the SQL matching the rewritten natural language question and obtaining the target data matching the corrected natural language question, the obtained large language model is used to generate visual analysis results based on the target data.
[0079] So far, completed Figure 4 Structural description of the device shown.
[0080] See also Figure 5 , Figure 5 This is a structural diagram of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.
[0081] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.
[0082] Exemplarily, the above-mentioned machine-readable storage medium can be any electronic, magnetic, optical or other physical storage device, which can contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium can be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as optical disk, DVD, etc.), or similar storage medium, or a combination thereof.
[0083] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A data analysis method, characterized in that: The method comprises: Rewriting the natural language question so that keywords in the natural language question are replaced accordingly; Search the existing database for a data table that matches the rewritten natural language question; Generate a structured query statement SQL prompt word based on the rewritten natural language question, data information in a data table matching the rewritten natural language question, cases matching the rewritten natural language question in an existing case library, and set constraints; the cases matching the rewritten natural language question include the question and the SQL corresponding to the question that meets the requirements; The SQL prompt word is used to generate SQL matching the rewritten natural language question.
2. The method according to claim 1, characterized in that The rewriting of the natural language question includes: The keywords include technical terms, and the technical terms in the natural language questions are replaced with professional terms; and / or, The keywords include descriptive words with context, and the descriptive words in the natural language question are replaced with nouns.
3. The method according to claim 1, characterized in that Each data table in the database has a corresponding description; the description includes: table description information and field description information of each field in the data table; The step of searching an existing database for a data table that matches the rewritten natural language question includes: Based on the description of each data table, a data table matching the rewritten natural language question is searched from the database.
4. The method according to claim 3, characterized in that The step of searching the database for a data table matching the rewritten natural language question based on the description of each data table includes: For each data table, calculating a first matching degree between the table description information of the data table and the rewritten natural language question; Calculating a second matching degree between the field description information of each field in the data table and the rewritten natural language question; Determining a matching degree of the data table based on the first matching degree and each of the second matching degrees; A data table whose matching degree meets the set matching degree requirement is found from the database, and the found data table is determined as a data table matching the rewritten natural language question.
5. The method according to claim 1, characterized in that The SQL prompt words for generating a structured query statement based on the rewritten natural language question, the data information in the data table matching the rewritten natural language question, the cases matching the rewritten natural language question in the existing case library, and the set constraints include: The rewritten natural language question, the table data definition language DDL in the data information in the data table matching the rewritten natural language question, and the SQL that meets the requirements and the set constraints in the case matching the rewritten natural language question are spliced in a specified order to obtain the SQL prompt word.
6. The method according to claim 1, characterized in that The step of using the SQL prompt word to generate SQL matching the rewritten natural language question includes: The rewritten natural language question and the SQL prompt word are input into the obtained large language model to obtain SQL matching the rewritten natural language question.
7. The method according to claim 1, characterized in that The method further comprises: After generating the SQL matching the rewritten natural language question, performing abnormal information detection on the generated SQL based on the obtained large language model; If the abnormal information detection indicates that the SQL does not have abnormal information, performing a data query in the database based on the SQL to obtain target data matching the rewritten natural language question; If the abnormal information detection indicates that the SQL contains abnormal information, then when it is determined that the set stop iteration condition is not met, the SQL prompt word, the detected abnormal information, and the SQL with abnormal information are used to regenerate SQL that matches the rewritten natural language question, and return to the step of performing abnormal information detection on the generated SQL based on the obtained large language model; wherein the regenerated SQL is different from the SQL with abnormal information.
8. The method according to claim 1, characterized in that The method further comprises: After performing a data query in the database based on the SQL matching the rewritten natural language question to obtain target data matching the corrected natural language question, a visual analysis result is generated based on the target data using the obtained large language model.
9. A data analysis device, characterized in that: The device comprises: A rewriting module, used to rewrite the natural language question so that the key words in the natural language question are replaced accordingly; A screening module is used to search for a data table matching the rewritten natural language question from an existing database; A prompt word generation module is used to generate a structured query statement SQL prompt word based on the rewritten natural language question, data information in a data table matching the rewritten natural language question, cases matching the rewritten natural language question in an existing case library, and set constraints; the cases matching the rewritten natural language question include the question and the SQL corresponding to the question that meets the requirements; The SQL generation module is used to use the SQL prompt words to generate SQL matching the rewritten natural language question.
10. The device according to claim 9, characterized in that The rewriting of the natural language question includes: The keywords include technical terms, and the technical terms in the natural language question are replaced with professional terms; and / or, the keywords include descriptive words with context, and the descriptive words in the natural language question are replaced with nouns; and / or, Each data table in the database has a corresponding description; the description includes: table description information and field description information of each field in the data table; The step of searching an existing database for a data table that matches the rewritten natural language question includes: Based on the description of each data table, searching the database for a data table that matches the rewritten natural language question; and / or, The step of searching the database for a data table matching the rewritten natural language question based on the description of each data table includes: For each data table, calculating a first matching degree between the table description information of the data table and the rewritten natural language question; Calculating a second matching degree between the field description information of each field in the data table and the rewritten natural language question; Determining a matching degree of the data table based on the first matching degree and each of the second matching degrees; Finding a data table from the database whose matching degree meets the set matching degree requirement, and determining the found data table as a data table matching the rewritten natural language question; and / or, The SQL prompt words for generating a structured query statement based on the rewritten natural language question, the data information in the data table matching the rewritten natural language question, the cases matching the rewritten natural language question in the existing case library, and the set constraints include: The rewritten natural language question, the table data definition language DDL in the data information in the data table matching the rewritten natural language question, and the SQL and the set constraints that meet the requirements in the case matching the rewritten natural language question are spliced in a specified order to obtain the SQL prompt word; and / or, The step of using the SQL prompt word to generate SQL matching the rewritten natural language question includes: Inputting the rewritten natural language question and the SQL prompt word into the obtained large language model to obtain SQL matching the rewritten natural language question; and / or, The device further comprises a verification module, wherein the verification module is used to: After generating the SQL matching the rewritten natural language question, performing abnormal information detection on the generated SQL based on the obtained large language model; If the abnormal information detection indicates that the SQL does not have abnormal information, performing a data query in the database based on the SQL to obtain target data matching the rewritten natural language question; If the abnormal information detection indicates that the SQL has abnormal information, then when it is determined that the set stop iteration condition is not met, the SQL prompt word, the detected abnormal information, and the SQL with abnormal information are used to regenerate SQL that matches the rewritten natural language question, and return to the step of performing abnormal information detection on the generated SQL based on the obtained large language model; wherein the regenerated SQL is different from the SQL with abnormal information; and / or, The device further comprises a visualization module, wherein the visualization module is used to: After performing a data query in the database based on the SQL matching the rewritten natural language question to obtain target data matching the corrected natural language question, a visual analysis result is generated based on the target data using the obtained large language model.
11. An electronic device, characterized in that: The electronic device includes: Processor; and A computer-readable storage medium, wherein computer program instructions are stored in the computer-readable storage medium, and when the computer program instructions are executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 8.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, enable the processor to perform the steps of the method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Database mode matching method and device and SQL query statement generation method
CN113868252A
Method, device, equipment and medium for converting natural language problem into SQL (Structured Query Language) statement
CN117667992A
Database question and answer method, system and equipment based on large language model and medium
CN118733747A
Data query method and device, electronic equipment and storage medium
CN118861077A