Data analysis method and device, electronic equipment and storage medium

Through large language model and data mining algorithm, the field name alignment and dynamic modeling of multiple data tables is automatically processed, and the problem of inconsistency of data tables is solved, efficient and automated data analysis and query conversion are realized, and data analysis efficiency and accuracy are improved.

CN120407592APending Publication Date: 2025-08-01CHINA UNICOM (GUANGDONG) IND INTERNET CO LTD

Patent Information

Application Number
CN202510468738.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-04-15
Publication Date
2025-08-01

AI Technical Summary

Technical Problem

In large-scale data analysis, the data analysis is time-consuming and laborious and error-prone due to the inconsistent field naming and field value formats of different data tables. The existing solutions rely on manual processing and are inefficient.

Method used

The large language model and data mining algorithm are used to automatically process field name alignment and dynamic modeling in multiple historical data tables, generate target field alignment tables, and generate a unified dynamic schema through semantic equivalence mapping and format conversion.

Benefits of technology

It significantly improves data analysis efficiency, reduces manual intervention, ensures data consistency and accuracy, supports automatic conversion from natural language query to SQL queries, and improves query accuracy and efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120407592A_ABST
    Figure CN120407592A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a data analysis method and device, electronic equipment and a storage medium, which are used for analyzing and aligning different field names in a plurality of historical data tables by using at least one of a large language model and a data mining algorithm, so that the data analysis efficiency is improved. The method comprises the steps that historical data is obtained, the historical data comprises a plurality of historical data tables, historical natural language query statements and historical structured query language SQL query statements corresponding to the historical natural language query statements, and each historical data table comprises at least one field name; the historical data are input into at least one of a large language model and a data mining algorithm, a target field alignment table is generated, and the target field alignment table comprises the mapping relation of different field names with semantic equivalence in multiple historical data tables; and according to the target field alignment table, updating the structured description information of the plurality of historical data tables, and generating updated structured description information of the plurality of historical data tables.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and data analysis, and particularly to a data analysis method, apparatus, electronic device, and storage medium. Background Art

[0002] In large-scale data analysis work, it is often necessary to extract required information from multiple data tables. However, there may be serious inconsistencies in field naming, field value formats, field types, etc. among different data tables, which poses a huge challenge to data analysis. Existing technologies generally rely on manual methods to handle these field and data problems (such as field naming correction, value format conversion, etc.), which is not only time-consuming and laborious but also prone to errors. Summary of the Invention

[0003] Embodiments of this application provide a data analysis method, apparatus, electronic device, and storage medium, which are used to analyze and align different field names in multiple historical data tables by using at least one of large language models and data mining algorithms, so as to improve data analysis efficiency.

[0004] In a first aspect of this application, a data analysis method is provided, which may include: obtaining historical data, where the historical data includes multiple historical data tables and historical natural language query statements, as well as historical structured query language (SQL) query statements corresponding to the historical natural language query statements, and each historical data table includes at least one field name; inputting the historical data into at least one of a large language model and a data mining algorithm to generate a target field alignment table, where the target field alignment table includes mapping relationships of different field names that are semantically equivalent in the multiple historical data tables; updating the structured description information of the multiple historical data tables according to the target field alignment table to generate updated structured description information of the multiple historical data tables; where the target field alignment table and the updated structured description information of the multiple historical data tables are used to query target field names.

[0005] In a second aspect of this application, a data analysis apparatus is provided, which may include:

[0006] An obtaining module, configured to obtain historical data, where the historical data includes multiple historical data tables and historical natural language query statements, as well as historical structured query language (SQL) query statements corresponding to the historical natural language query statements, and each historical data table includes at least one field name;

[0007] A processing module, configured to input the historical data into at least one of a large language model and a data mining algorithm to generate a target field alignment table, where the target field alignment table includes mapping relationships of different field names with equivalent semantics in the multiple historical data tables; update the structured description information of the multiple historical data tables according to the target field alignment table to generate updated structured description information of the multiple historical data tables; wherein, the target field alignment table and the updated structured description information of the multiple historical data tables are used to query a target field name.

[0008] In a third aspect of the present application, there is provided an electronic device, including a memory, one or more processors, and a display. The memory is coupled to the processor. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the method according to any one of the first aspect.

[0009] In a fourth aspect of the present application, there is provided a chip, which includes a processor and a memory, and a display. The processor may be a logic circuit, an integrated circuit, or a general-purpose processor, etc. Instructions are stored in the memory, and the processor may implement the method in the first aspect above by reading the software code stored in the memory. The memory may be integrated in the processor or may exist independently outside the processor.

[0010] In a fifth aspect of the present application, there is provided a chip system, which is applied to an electronic device including the chip system. The chip system includes one or more interface circuits and one or more processors, and a display. The interface circuits and the processors are interconnected by lines. The interface circuits are configured to receive signals from the memory of the electronic device and send the signals to the processors. The signals include computer instructions stored in the memory. When the processors execute the computer instructions, the electronic device executes the method according to any one of the first aspect.

[0011] In a sixth aspect of the present application, there is provided a computer program product, which includes: a computer program (which may also be referred to as code or instructions). When the computer program is run, the computer is caused to execute the method in the first aspect above.

[0012] In a seventh aspect of the present application, there is provided a computer-readable storage medium, which stores a computer program (which may also be referred to as code or instructions). When the computer program runs on the computer, the computer is caused to execute the method in the first aspect above.

[0013] From the above technical solutions, it can be seen that the embodiments of the present application have the following advantages:

[0014] In an embodiment of the present application, historical data is obtained. The historical data includes multiple historical data tables, historical natural language query statements, and historical structured query language (SQL) query statements corresponding to the historical natural language query statements. Each historical data table includes at least one field name. The historical data is input into at least one of a large language model and a data mining algorithm to generate a target field alignment table. The target field alignment table includes mapping relationships of different field names that are semantically equivalent in the multiple historical data tables. According to the target field alignment table, the structured description information of the multiple historical data tables is updated to generate updated structured description information of the multiple historical data tables. Among them, the target field alignment table and the updated structured description information of the multiple historical data tables are used to query target field names. By using at least one of a large language model and a data mining algorithm to analyze and align different field names in multiple historical data tables, the data analysis efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments and the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained according to these drawings.

[0016] Figure 1 Schematic diagram of an embodiment of the data analysis method in an embodiment of the present application;

[0017] Figure 2 Schematic diagram of another embodiment of the data analysis method in an embodiment of the present application;

[0018] Figure 3 Schematic diagram of an embodiment of the data analysis device in an embodiment of the present application;

[0019] Figure 4 Schematic diagram of an embodiment of the electronic device in an embodiment of the present application;

[0020] Figure 5 Schematic diagram of another embodiment of the terminal device in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] The embodiments of the present application provide a data analysis method, device, electronic device, and storage medium for using at least one of a large language model and a data mining algorithm to analyze and align different field names in multiple historical data tables, thereby improving the data analysis efficiency.

[0022] To enable those skilled in the art to better understand the solution of this application, the following will describe the technical solutions in the embodiments of this application in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. Based on the embodiments in this application, they should all fall within the scope of protection of this application.

[0023] In large-scale data analysis work, it is often necessary to extract the required information from multiple data tables. However, there may be serious inconsistencies in the field naming, field value formats, field types, etc. of different data tables, which poses a huge challenge to data analysis.

[0024] The field names in the data table may be non-standard. For example, some tables name the time field as "Time", while others name it as "Timestamp". The value formats of fields in different data tables may not be unified. For example, time may be stored in character format (such as "October 10, 2023"), or it may be stored in the form of a timestamp (such as "1696886400"). There may also be differences in the value distributions of fields between different tables, and these fields may be equal or similar in semantics.

[0025] To solve the above problems, the existing technologies generally rely on manual methods to handle these field and data problems (such as field naming correction, field value format conversion, etc.). When there are many tables, many fields, and many people maintaining them, it is very easy to make mistakes and omissions; new data tables are generated every day, and the maintenance volume is very large; if the correlation of fields with different names but the same attribute meaning is not maintained, the ChatBI system will not be able to understand the Schema and cannot generate effective SQL statements, resulting in the inability to improve the accuracy. Therefore, in the existing technologies, the maintenance of data tables is not only time-consuming and laborious but also prone to errors. In addition, data analysts need to repeatedly involve query statements (such as SQL (Structured Query Language). It allows users to perform operations such as querying, inserting, updating, and deleting data), and manually correct the generated results, further increasing the workload. Among them, SQL is a standard language for managing and operating relational databases.

[0026] Therefore, this application proposes a data analysis method, which relates to the fields of artificial intelligence and data analysis, and particularly relates to a data analysis method that combines large language models and data mining algorithms to achieve automated cleaning, field name alignment, and dynamic modeling of multiple data tables. With the development of generative large language models and data mining algorithms, combining the natural language understanding ability of large language models with the analysis ability of data mining can help solve the problem of inconsistent field names in multiple data tables and provide an efficient and automated solution for data cleaning, dynamic modeling, and data analysis.

[0027] In the embodiments of the present application, the electronic device may include a server, or a terminal device. The terminal device may be a mobile phone, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal device in industrial control, a wireless terminal device in self-driving, a wireless terminal device in remote medical, a wireless terminal device in smart grid, a wireless terminal device in transportation safety, a wireless terminal device in smart city, a wireless terminal device in smart home, an in-vehicle infotainment system (IVI), etc. The in-vehicle infotainment system (IVI) refers to an intelligent in-vehicle system installed in an automobile, integrating functions such as information entertainment, navigation, communication, and vehicle control.

[0028] By way of example and not limitation, in the embodiments of the present application, the terminal device may also be a wearable device with a display interface. The wearable device may also be referred to as a wearable intelligent device, which is a general term for devices developed by applying wearable technology to the intelligent design of daily wear, such as glasses, gloves, watches, clothing, and shoes. The wearable device is a portable device that can be directly worn on the body or integrated into the user's clothes or accessories. The wearable device is not only a hardware device, but also realizes powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable intelligent devices include those with complete functions and large sizes that can realize complete or partial functions without relying on a smart phone, such as smart watches or smart glasses, etc., and those that only focus on a certain type of application function and need to cooperate with other devices such as smart phones, such as various smart bracelets and smart jewelry for physical sign monitoring.

[0029] Next, the technical solution of the present application will be further described by way of embodiments, as Figure 1 shown, which is a schematic diagram of an embodiment of the data analysis method in the embodiments of the present application, and may include:

[0030] 101. Obtain historical data, where the historical data includes multiple historical data tables, historical natural language query statements, and historical SQL query statements corresponding to the historical natural language query statements. Each historical data table includes at least one field name.

[0031] Exemplarily, an electronic device obtains historical data; the historical data includes multiple historical data tables and historical natural language query statements, as well as historical SQL query statements corresponding to the historical natural language query statements. The historical data may be all the data in the local database of the electronic device, or part of the data in the local database of the electronic device, or the data in the local database obtained by the electronic device and the databases of other electronic devices. The embodiments of the present application do not make specific limitations.

[0032] Wherein, each historical data table includes at least one field name.

[0033] In some possible implementation manners, each historical data table further includes field values corresponding to at least one field name. It should be noted that the field values corresponding to at least one field name include the field values corresponding to all field names, or the field values corresponding to at least some field names. The embodiments of the present application do not make specific limitations in particular.

[0034] SQL (Structured Query Language) is a standard programming language for managing and operating relational databases. It is widely used in various database management systems (such as MySQL, PostgreSQL, Oracle, SQL Server, etc.) to create, query, update, and manage data.

[0035] Converting natural language query statements into SQL query statements is a common requirement, especially in data analysis and database development. This requires understanding the intention of natural language and accurately converting it into an SQL statement.

[0036] Historical natural language query statements, such as: "Query Shanghai customers with sales exceeding 1 million in 2023", "Product categories with the most complaints in February".

[0037] Historical SQL query statements corresponding to historical natural language query statements, such as:

[0038] Example 1:

[0039] User input: "Query Shanghai customers with sales exceeding 1 million in 2023";

[0040] Generated SQL: "SELECT customer_name FROM orders

[0041] WHERE region = 'Shanghai'

[0042] AND total_amount > 1000000

[0043] AND order_date BETWEEN '2023-01-01' AND '2023-12-31'"。

[0044] Example 2:

[0045] User input: "The product category with the most complaints last month";

[0046] Generate SQL: "SELECT category, COUNT(*)

[0047] FROM complaints

[0048] WHERE date BETWEEN '2025-02-01' AND '2025-02-28'

[0049] GROUP BY category ORDER BY COUNT(*) DESC LIMIT 5".

[0050] 102. Input the historical data into at least one of the large language model and the data mining algorithm to generate a target field alignment table, where the target field alignment table includes the mapping relationships of different field names with semantic equivalence in the multiple historical data tables.

[0051] Exemplarily, input multiple historical data tables, historical natural language query statements, and historical SQL query statements corresponding to the historical natural language query statements into at least one of the large language model and the data mining algorithm, that is, use at least one of the large language model and the data mining algorithm to analyze the semantic relationships and context information of different field names in the multiple historical data tables, and mine the equivalence of field names from them. For example, parse "time" and "timestamp" as different field names with semantic equivalence.

[0052] Here, different field names with semantic equivalence can represent field names with the same data semantics, or heterogeneous field names of the same entity attribute, etc.

[0053] In some possible implementation manners, the target field alignment table further includes the field values corresponding to the different field names with semantic equivalence.

[0054] After using at least one of the large language model and the data mining algorithm to analyze the semantic relationships and context of different field names in the multiple historical data tables and mine the equivalence of field names from them, it is also necessary to regularize the field values corresponding to the different field names with semantic equivalence.

[0055] The field values corresponding to different field names with semantic equivalence, which can also be referred to as reference field values, are the most relevant and closest field values corresponding to each different field name with semantic equivalence.

[0056] In this technical solution, it can be processed efficiently and automatically. By using large language models and data mining techniques, the manual cleaning and alignment operations on different field names and corresponding field values in multiple historical data tables can be significantly reduced, improving the data analysis efficiency.

[0057] In some possible implementation manners, inputting the historical data into at least one of a large language model and a data mining algorithm, analyzing the semantic relationships and context information of different field names in the multiple historical data tables, and generating a target field alignment table may include: inputting the historical data into at least one of a large language model and a data mining algorithm, analyzing the semantic relationships and context information of different field names in the multiple historical data tables, performing semantic clustering and semantic mapping, and generating the target field alignment table.

[0058] Exemplarily, inputting the historical data into at least one of a large language model and a data mining algorithm, analyzing the semantic relationships and context information in the multiple historical data tables, performing semantic clustering and semantic mapping, that is, mapping different field names with semantic equivalence in the multiple historical data tables, and finally determining the alignment relationships of different field names within the same table or across tables to obtain a target field alignment table. The target field alignment table records the mapping relationships of different field names with semantic equivalence in the same historical data table or different historical data tables.

[0059] In some possible implementation manners, the method may further include: using at least one of the large language model and the data mining algorithm to perform at least one of statistical distribution, format analysis, and equivalence analysis on the field values of different field names in the target field alignment table to obtain the field values corresponding to different field names with semantic equivalence in the target field alignment table.

[0060] Exemplarily, after obtaining the target field alignment table, at least one of the large language model and the data mining algorithm can be used to perform statistical distribution, format analysis, and semantic understanding, etc. on the field values of different field names with semantic equivalence in the target field alignment table. For example, it is found through analysis that "2023-01-01" and its corresponding timestamp (1672444800) have semantic equivalence.

[0061] In this technical solution, an automated data analysis method based on at least one of large language models and data mining algorithms is provided. Multiple historical data tables, historical natural language query statements, and historical SQL query statements corresponding to the historical natural language query statements can be analyzed for the semantic relationships and context information of field names, and the equivalence of different field names can be mined through at least one of large language models and data mining algorithms, and a target field alignment table can be generated. Optionally, at least one of statistical distribution, format analysis, and equivalence analysis can also be performed on the field values of different field names in the target field alignment table to obtain the field values corresponding to different field names that are semantically equivalent in the target field alignment table. This method can automatically identify and align field names that are semantically similar or equal in different historical data tables, as well as the field values corresponding to the field names, which can significantly reduce manual intervention and improve the recognition efficiency.

[0062] In some possible implementation manners, using at least one of the large language model and the data mining algorithm to perform at least one of statistical distribution, format analysis, and semantic understanding on the field values of different field names in the field alignment table to obtain the target field values corresponding to different field names in the field alignment table may include:

[0063] 1) Using at least one of the large language model and the data mining algorithm to perform statistical distribution processing on the field values of different field names in the target field alignment table to determine the distribution probability of the field values of different field names in the target field alignment table. That is, the target field alignment table also includes the distribution probability of the field values corresponding to different field names that are semantically equivalent.

[0064] Exemplarily, using at least one of the large language model and the data mining algorithm to perform statistical distribution processing on the field values of different field names in the target field alignment table to determine the distribution probability of the field values of different field names in the target field alignment table, the main applications include but are not limited to the following directions:

[0065] (1) Data quality assessment and cleaning

[0066] Outlier detection: Identify outliers with extremely low distribution probabilities;

[0067] Data consistency verification: Compare the distribution differences of field values from different sources to discover potential data quality problems;

[0068] Missing value handling: Determine the filling strategy for missing values based on the distribution probability.

[0069] (2) Field alignment verification

[0070] Semantic equivalence verification: Fields with similar distributions are more likely to be semantically equivalent;

[0071] Alignment confidence score: Provides a quantitative confidence metric for field mapping relationships;

[0072] Automatic alignment suggestions: Recommends potential field mapping relationships based on distribution similarity.

[0073] (3) Data integration optimization

[0074] Source system selection: Selects the source system with a field value distribution that better meets business requirements as the primary data source;

[0075] Transformation rule optimization: Designs more reasonable data transformation rules based on distribution characteristics;

[0076] Data fusion strategy: Determines how to merge field values from different sources.

[0077] (4) Analysis and decision support

[0078] Business insight discovery: Identifies differences in business practices between different systems through distribution differences;

[0079] Risk prediction: Specific value distributions may indicate business risks;

[0080] Resource allocation optimization: Optimizes resource allocation based on distribution hotspots.

[0081] 2) Use at least one of the large language model and data mining algorithms to convert the formats of the first field value and the second field value to the same format when the formats of the first field value and the second field value in the target field alignment table are inconsistent; wherein, the first field value is the field value corresponding to the first field name in the target field alignment, the second field value is the field value corresponding to the second field name in the target field alignment table, and the first field name and the second field name are semantically equivalent.

[0082] That is, the different field names with semantic equivalence include the first field name and the second field name, and the format of the first field value corresponding to the first field name is the same as the format of the second field value corresponding to the second field name.

[0083] Exemplarily, at least one of the large language model and the data mining algorithm is used to perform format analysis processing on the field values corresponding to each field name in the field alignment table. In the case where the formats of the field values corresponding to field names with the same semantics are inconsistent, the formats are converted to the same target format. For example, field names with the same semantics include a first field name and a second field name. The format of the first field value corresponding to the first field name is a first format, and the format of the second field value corresponding to the second field name is a second format. The format of the second field value can be converted to the first format, or the format of the first field value can be converted to the second format, or the first format of the first field value and the format of the second field value can be converted to a third format, where the third format is different from both the first format and the second format.

[0084] That is, for the problem of inconsistent formats of the field values corresponding to each field name in the target field alignment table (such as the character time format and the timestamp format), conversion rules are automatically generated to unify the field values into a standard format.

[0085] 3) Use at least one of the large language model and the data mining algorithm to perform correction processing on the third field value of the third field name in the field alignment table to obtain the corrected third field value.

[0086] That is, the different field names with semantic equivalence include the third field name, and the third field value corresponding to the third field name is the corrected third field value.

[0087] Exemplarily, after performing semantic understanding processing on the field values of each field name in the target field alignment table, correction processing can be performed to obtain the corrected field values corresponding to each field name. For example, noises and error values that may exist in the field values of each field name in the target field alignment table are corrected or rectified. For example, by analyzing semantic relationships, the most relevant and closest field values corresponding to each field name in the same data table or different data tables are provided.

[0088] In this technical solution, a method for processing the normalization and unification of field values corresponding to different field names in multiple historical data tables is provided, including processing methods such as statistical distribution of field values, automatic generation of field value format conversion rules, and correction of noises and error values; this mechanism can ensure that field values in different formats (such as the character time format and the timestamp format) are unified into a standard format, thereby improving data consistency and accuracy.

[0089] 103. According to the target field alignment table, update the structured description information of the multiple historical data tables to generate the updated structured description information of the multiple historical data tables.

[0090] Among them, the update structured description information of the target field alignment table and the multiple historical data tables is used to query the target field name.

[0091] Exemplarily, according to the target field alignment table, the update structured description information of the multiple historical data tables is generated, that is, a unified dynamic Schema is generated. The dynamic Schema is a unified structure based on the integration of multi-data table fields, and contains fields related to the user query target (such as the unified fields and field values after alignment). The dynamic Schema aims to eliminate problems such as inconsistent field names and formats between multiple historical data tables, and provide consistent structured data support for subsequent queries and analyses.

[0092] Schema (schema / data model) refers to a dynamic and unified field specification framework, whose core function is to solve the standardization problem of multi-source heterogeneous data. It is not a fixed database table structure, but a set of mapping rules automatically generated according to the data mining results, which can unify the field formats of different data sources.

[0093] For example: automatically identify user identities (Identity, ID), customer numbers, and user_id in different tables as equivalent fields, and normalize them into a unified field name (such as standard_user_id). Standardize the field value format (such as unifying the date into the YYYY-MM-DD format).

[0094] Technical implementation:

[0095] Data mining algorithm: Discover field equivalence through field name similarity, value distribution pattern (such as regular matching), primary and foreign key relationships, etc.

[0096] Large language model: Use Natural Language Processing (NLP) to understand field semantics (such as identifying the synonymy of "mobile phone number" and "contact phone number").

[0097] In this technical solution, a natural language query conversion system based on dynamic Schema is provided, including steps of generating a dynamic Schema related to the user query target, and subsequently using the NL2SQL model to convert natural language questions into SQL query statements; this system can automatically generate structured data support, improve the accuracy and efficiency of queries, and lower the technical threshold for users. Through at least one of the large language model and the data mining algorithm, analyze the equivalence of field names, and further, normalize the field values corresponding to the field names to ensure that the automatically generated dynamic Schema has semantic logical consistency. A unified data framework is also provided, and through the automatic generation of the dynamic Schema, problems such as inconsistent field names and field value formats between multiple tables are solved.

[0098] The embodiments of the present application can be applied to scenarios such as intelligent question asking and ChatBI, which can improve the overall accuracy of question asking.

[0099] Among them, ChatBI is a revolutionary way of business intelligence interaction. Through natural language processing (NLP) and large language model (LLM) technologies, users can directly query and analyze data in a conversational manner without having to master SQL or complex business intelligence (BI) tool operations.

[0100] In the embodiments of the present application, historical data is obtained. The historical data includes multiple historical data tables, historical natural language query statements, and historical SQL query statements corresponding to the historical natural language query statements. Each historical data table includes at least one field name; the historical data is input into at least one of a large language model and a data mining algorithm to generate a target field alignment table, which includes the mapping relationship of different field names with equivalent semantics in the multiple historical data tables; according to the target field alignment table, the structured description information of the multiple historical data tables is updated to generate the updated structured description information of the multiple historical data tables; wherein, the target field alignment table and the updated structured description information of the multiple historical data tables are used to query the target field name. By using at least one of a large language model and a data mining algorithm to analyze and align different field names in multiple historical data tables, the data analysis efficiency is improved.

[0101] As Figure 2 shown, it is a schematic diagram of another embodiment of the data analysis method in the embodiments of the present application, which may include:

[0102] 201. Obtain historical data, where the historical data includes multiple historical data tables, historical natural language query statements, and historical SQL query statements corresponding to the historical natural language query statements. Each historical data table includes at least one field name.

[0103] 202. Input the historical data into at least one of a large language model and a data mining algorithm to generate a target field alignment table, which includes the mapping relationship of different field names with equivalent semantics in the multiple historical data tables.

[0104] 203. Update the structured description information of the multiple historical data tables according to the target field alignment table to generate the updated structured description information of the multiple historical data tables.

[0105] Among them, the update structured description information of the target field alignment table and the multiple historical data tables is used to query the target field name.

[0106] It should be noted that in the embodiment of the present application, steps 201-203 are similar to Figure 1 steps 101-103 in the illustrated embodiment, and will not be elaborated here.

[0107] 204. Obtain the current natural language query statement.

[0108] In some possible implementation manners, for the electronic device to obtain the current natural language query statement, it may include: the electronic device responds to the user's input operation and obtains the current natural language query statement.

[0109] Exemplarily, when the user inputs the current natural language query statement on the electronic device, the electronic device responds to the user's input operation and obtains the current natural language query statement. Here, the input operation may be the user's text input operation, or the user's voice input operation, or a picture uploaded by the user for picture recognition input operation, etc. The specific embodiment of the present application does not make a limitation.

[0110] In the technical solution of the present application, several specific implementation manners are provided for the electronic device to obtain the current natural language query statement, which improves the feasibility of the solution.

[0111] 205. Input the current natural language query statement into the natural language structured query language NL2SQL model to be converted into the current SQL query statement.

[0112] Exemplarily, the electronic device converts the current natural language query statement into a query statement that conforms to the SQL syntax specification based on a dedicated NL2SQL model, that is, inputs the current natural language query statement into the NL2SQL model to be converted into the current SQL query statement. The NL2SQL (Natural Language to SQL) model is a technology that converts natural language queries into SQL queries and is widely used in database interaction and data analysis.

[0113] Common NL2SQL models include but are not limited to the following models:

[0114] (1) Seq2SQL model

[0115] Use a sequence-to-sequence (Seq2Seq) model to generate SQL, and divide the SQL generation into three subtasks: SELECT, WHERE, and AGGREGATION.

[0116] SELECT is used to retrieve data from a database table. It can specify the columns to be queried or use wildcards to query all columns.

[0117] WHERE is used to filter data and only return records that meet the specified conditions.

[0118] Aggregate functions are used to calculate a set of values and return a single value. Common aggregate functions include COUNT (counting), SUM (calculating the total), AVG (calculating the average), MIN (finding the minimum value), and MAX (finding the maximum value).

[0119] (2) SQLNet model

[0120] Improve Seq2SQL, use the slot filling method to generate SQL, and reduce the dependence on a large amount of labeled data.

[0121] (3) BERT-based NL2SQL model

[0122] Based on the BERT pre-trained model, combined with database schema information to generate SQL, and performs excellently on datasets such as WikiSQL.

[0123] (4) T5-based NL2SQL model

[0124] Use the T5 (Text-to-Text Transfer Transformer) model to regard the NL2SQL task as a text generation task. Support complex query generation.

[0125] In this technical solution, common NL2SQL models are illustrated for easy understanding of the NL2SQL model and improve the feasibility of the solution.

[0126] In some possible implementation manners, the electronic device inputs the current natural language query statement into the natural language structured query language NL2SQL model and converts it into the current SQL query statement, which may include: the electronic device generates a target natural language query statement through the large language model according to the current natural language query statement; inputs the target natural language query statement into the natural language structured query language NL2SQL model and converts it into the current SQL query statement.

[0127] Exemplarily, after detecting the current natural language query statement input by the user, the electronic device can use the large language model to understand the question intention of the current natural language query statement and automatically generate a target natural language query statement. Then, based on the dedicated NL2SQL model, the electronic device converts the target natural language query statement into a query statement that conforms to the SQL syntax specification, that is, the current SQL query statement.

[0128] In this technical solution, according to the current natural language query statement, a target natural language query statement is generated through the large language model; the target natural language query statement is input into the natural language structured query language NL2SQL model to be converted into the current SQL query statement. The semantic understanding of the user's query intention and the efficient output of subsequent query results are realized.

[0129] 206. According to the current SQL query statement, the target field alignment table, and the updated structured description information of the multiple historical data tables, a query result is obtained.

[0130] Exemplarily, an electronic device can obtain a query result according to the SQL query statement, the target field alignment table, and the updated structured description information of the multiple historical data tables, and the query result includes query field names. Optionally, it also includes query field names and corresponding field values.

[0131] In some possible implementation manners, the query result is a cleaned query result.

[0132] Exemplarily, when a user inputs a current natural language query statement in an electronic device, the electronic device responds to the user's input operation to generate the current natural language query statement; then based on a dedicated NL2SQL model, the current natural language query statement is converted into the current SQL query statement, that is, a query statement that conforms to the SQL syntax specification. Then, according to the target field alignment table and the updated structured description information of the multiple historical data tables, the current SQL query statement is executed to return a query result.

[0133] Optionally, a large model is used to understand the question intention of the current natural language query statement input by the user, and combined with a dynamic Schema, the field names related to the current natural language query statement and the information of the target historical data tables are automatically generated; then the current SQL query statement is executed to return a query result.

[0134] The updated structured description information of the multiple historical data tables is generated according to the target field alignment table. The updated structured description information of the multiple historical data tables can be understood as a dynamic Schema, and a dynamic Schema refers to the database Schema information that is dynamically adjusted or generated according to the user's query intention and context, rather than relying on a fixed and predefined table structure.

[0135] In a database, a schema refers to the logical structure of the database, which defines the way data is organized in the database. It includes the definition of tables (table names, field names, field types, field lengths, etc.), the relationships between tables (such as primary key - foreign key relationships), the definition of views, etc. For example, the database schema of a student management system in a school may include a student table (fields include student ID, name, gender, age, etc.), a course table (course number, course name, credits, etc.), and an enrollment table (student ID, course number, grades, etc.).

[0136] 207. Output the query result.

[0137] In some possible implementation manners, the electronic device outputting the query result may include: the electronic device outputs the query result in a table form, or a picture form, or a multimodal form.

[0138] Exemplarily, after obtaining the query result according to the target natural language query statement request and the target field alignment table, the query result may be output, and multiple output forms (such as table, picture, chart, or multimodal visual display, etc.) are provided. It supports the user to further analyze and adjust the query result.

[0139] In this technical solution, the electronic device outputting the query result provides a specific description, improves the feasibility of the solution, and also improves the interactivity with the user.

[0140] In some possible implementation manners, the electronic device outputting the query result may include: the electronic device responds to the user's selection operation, selects a target output manner, and outputs the query result according to the target output manner, where the target output manner is a table form, or a picture form, or a multimodal form.

[0141] Exemplarily, after obtaining the query result according to the target natural language query statement request and the target field alignment table, the query result may be output, and multiple output forms (such as table, picture, chart, or multimodal visual display, etc.) are provided for the user to select. The user can select the target output manner according to their actual needs. The electronic device responds to the user's selection operation, selects the target output manner, and outputs the query result according to the target output manner.

[0142] In this technical solution, the target output manner of the electronic device outputting the query result can be controlled on the user side. The user can select the target output manner according to their actual needs, and then output the query result according to the target output manner, which improves the user experience of the user.

[0143] Embodiments of the present application can be applied to scenarios such as intelligent question asking and ChatBI, which can improve the overall accuracy of question asking.

[0144] Among them, ChatBI is a revolutionary business intelligence interaction method. Through natural language processing (NLP) and large language model (LLM) technologies, users can directly query and analyze data in a conversational manner without having to master SQL or the operation of complex business intelligence (BI) tools.

[0145] In an embodiment of the present application, historical data is obtained. The historical data includes multiple historical data tables, historical natural language query statements, and historical SQL query statements corresponding to the historical natural language query statements. Each historical data table includes at least one field name; the historical data is input into at least one of a large language model and a data mining algorithm to generate a target field alignment table, where the target field alignment table includes mapping relationships of different field names with equivalent semantics in the multiple historical data tables; according to the target field alignment table, the structured description information of the multiple historical data tables is updated to generate updated structured description information of the multiple historical data tables; among them, the target field alignment table and the updated structured description information of the multiple historical data tables are used to query the target field name. The current natural language query statement is obtained; the current natural language query statement is input into a natural language structured query language NL2SQL model to be converted into a current SQL query statement; according to the current SQL query statement, the target field alignment table, and the updated structured description information of the multiple historical data tables, a query result is obtained; the query result is output. The present application aims to solve the problem of inconsistent field names and field value formats in different data tables in the prior art, and provides an automated data analysis method based on at least one of data mining technology and a large language model. This method can automatically discover the equivalent relationship of field names, perform field name alignment and field value normalization, generate a consistent dynamic Schema, and further generate an SQL query statement in combination with the NL2SQL (natural language to SQL) model, and finally realize the semantic understanding of the user's query intention and the efficient output of the query result.

[0146] As Figure 3 shown, it is a schematic diagram of an embodiment of a data analysis device in an embodiment of the present application, which may include:

[0147] Acquisition module 301, configured to acquire historical data, where the historical data includes multiple historical data tables, historical natural language query statements, and historical SQL query statements corresponding to the historical natural language query statements, and each historical data table includes at least one field name;

[0148] Processing module 302, configured to input the historical data into at least one of a large language model and a data mining algorithm to generate a target field alignment table, where the target field alignment table includes mapping relationships of different field names with equivalent semantics in the multiple historical data tables; update the structured description information of the multiple historical data tables according to the target field alignment table to generate updated structured description information of the multiple historical data tables; wherein, the target field alignment table and the updated structured description information of the multiple historical data tables are used to query a target field name.

[0149] In some possible implementation manners, each historical data table further includes field values corresponding to the at least one field name;

[0150] The target field alignment table further includes field values corresponding to the different field names with equivalent semantics;

[0151] The target field alignment table and the updated structured description information of the multiple historical data tables are used to query a target field name and corresponding target field values.

[0152] In some possible implementation manners, the target field alignment table further includes distribution probabilities of field values corresponding to the different field names with equivalent semantics.

[0153] In some possible implementation manners, the different field names with equivalent semantics include a first field name and a second field name, and the format of a first field value corresponding to the first field name is the same as the format of a second field value corresponding to the second field name.

[0154] In some possible implementation manners, the different field names with equivalent semantics include a third field name, and a third field value corresponding to the third field name is a third field value obtained through correction processing.

[0155] In some possible implementation manners, the data analysis device further includes: an output module 303;

[0156] The acquisition module 301 is further configured to acquire a current natural language query statement;

[0157] The processing module 302 is further configured to input the current natural language query statement into a natural language structured query language NL2SQL model to convert it into a current SQL query statement; and obtain a query result according to the current SQL query statement, the target field alignment table, and the updated structured description information of the multiple historical data tables, where the query result includes query field names.

[0158] The output module 303 is configured to output the query result.

[0159] In some possible implementation manners, the output module 303 is specifically configured to output the query result in a table form, or a picture form, or a multimodal form.

[0160] As Figure 4 shown, it is a schematic diagram of an embodiment of an electronic device in an embodiment of the present application, which may include a data analysis device as Figure 3 shown.

[0161] As Figure 5 shown, it is a schematic diagram of another embodiment of a terminal device in an embodiment of the present application. The following specifically introduces each component of the mobile phone in the terminal device in combination with Figure 5 :

[0162] The RF circuit 510 can be used for receiving and sending signals during information reception or call processes. Specifically, after receiving the downlink information of the base station, it is given to the processor 580 for processing; in addition, the designed uplink data is sent to the base station. Usually, the RF circuit 510 includes, but is not limited to, antennas, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 510 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0163] The memory 520 can be used to store software programs and modules. The processor 580 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 520. The memory 520 may mainly include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory 520 may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0164] The input unit 530 can be used to receive input digital or character information and generate key signal inputs related to the user settings and function control of the mobile phone. Specifically, the input unit 530 may include a touch panel 531 and other input devices 532. The touch panel 531, also known as a touch screen, can collect touch operations of the user on or near it (such as operations of the user using a finger, a stylus, or any suitable object or accessory on or near the touch panel 531), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 531 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the touch position of the user and detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 580, and can receive and execute commands sent by the processor 580. In addition, various types such as resistive, capacitive, infrared, and surface acoustic wave can be used to implement the touch panel 531. In addition to the touch panel 531, the input unit 530 may also include other input devices 532. Specifically, the other input devices 532 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a trackball, a mouse, a joystick, etc.

[0165] The display unit 540 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 540 may include a display panel 541. Optionally, the display panel 541 can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 531 can cover the display panel 541. When the touch panel 531 detects a touch operation on or near it, it is transmitted to the processor 580 to determine the type of touch event. Subsequently, the processor 580 provides a corresponding visual output on the display panel 541 according to the type of touch event. Although in Figure 5 the touch panel 531 and the display panel 541 are implemented as two independent components to realize the input and input functions of the mobile phone, in some embodiments, the touch panel 531 and the display panel 541 can be integrated to realize the input and output functions of the mobile phone.

[0166] The mobile phone may further include at least one sensor 550, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. Among them, the ambient light sensor can adjust the brightness of the display panel 541 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 541 and / or the backlight when the mobile phone is moved to the ear. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity, and can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that the mobile phone can also be configured with, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be elaborated here.

[0167] The audio circuit 560, the speaker 561, and the microphone 562 can provide an audio interface between the user and the mobile phone. The audio circuit 560 can transmit the electrical signal converted from the received audio data to the speaker 561, and the speaker 561 converts it into a sound signal for output; on the other hand, the microphone 562 converts the collected sound signal into an electrical signal, which is received by the audio circuit 560 and then converted into audio data. After the audio data is output and processed by the processor 580, it is sent through the RF circuit 510 to, for example, another mobile phone, or the audio data is output to the memory 520 for further processing.

[0168] Wi-Fi belongs to short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse the web, and access streaming media through the Wi-Fi module 570. It provides users with wireless broadband Internet access. Although Figure 5The Wi-Fi module 570 is shown, but it can be understood that it does not belong to the essential components of the mobile phone and can be completely omitted within the scope of not changing the essence of the invention as needed.

[0169] The processor 580 is the control center of the mobile phone, connecting various parts of the entire mobile phone through various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 520, and by calling data stored in the memory 520, it executes various functions of the mobile phone and processes data, thereby monitoring the mobile phone as a whole. Optionally, the processor 580 may include one or more processing units; preferably, the processor 580 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into the processor 580 either.

[0170] The mobile phone also includes a power supply 590 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 580 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system.

[0171] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be elaborated here.

[0172] In the embodiment of the present application, the processor 580 is used to obtain historical data. The historical data includes multiple historical data tables and historical natural language query statements, as well as historical SQL query statements corresponding to the historical natural language query statements. Each historical data table includes at least one field name; input the historical data into at least one of a large language model and a data mining algorithm to generate a target field alignment table, where the target field alignment table includes mapping relationships of different field names with equivalent semantics in the multiple historical data tables; update the structured description information of the multiple historical data tables according to the target field alignment table to generate updated structured description information of the multiple historical data tables; wherein, the target field alignment table and the updated structured description information of the multiple historical data tables are used to query the target field name.

[0173] In some possible implementation manners, each historical data table further includes field values corresponding to the at least one field name;

[0174] The target field alignment table further includes field values corresponding to the different field names with equivalent semantics;

[0175] The target field alignment table and the updated structured description information of the multiple historical data tables are used to query the target field name and the corresponding target field value.

[0176] In some possible implementations, the target field alignment table further includes the distribution probabilities of the field values corresponding to the different field names with semantic equivalence.

[0177] In some possible implementations, the different field names with semantic equivalence include a first field name and a second field name, and the format of the first field value corresponding to the first field name is the same as the format of the second field value corresponding to the second field name.

[0178] In some possible implementations, the different field names with semantic equivalence include a third field name, and the third field value corresponding to the third field name is a third field value after being corrected.

[0179] In some possible implementations, the processor 580 is further configured to obtain the current natural language query statement; input the current natural language query statement into a natural language structured query language NL2SQL model to be converted into a current SQL query statement; and obtain a query result according to the current SQL query statement, the target field alignment table, and the updated structured description information of the multiple historical data tables, where the query result includes query field names.

[0180] The display unit 540 is configured to output the query result.

[0181] In some possible implementations of the present application, the display unit 540 is specifically configured to output the query result in a table form, or a picture form, or a multimodal form.

[0182] The embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, and when the computer instructions run on an electronic device, the electronic device is enabled to execute the method embodiment described above.

[0183] The above computer-readable storage medium may adopt any combination of one or more computer-readable media. The computer-readable media may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM), or a flash memory, an optical fiber, a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0184] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal may take various forms, including - but not limited to - an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0185] The program code contained on the computer-readable medium may be transmitted by any appropriate medium, including - but not limited to - wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the above.

[0186] Computer program code for performing the operations of this specification may be written in one or more programming languages or combinations thereof. The programming languages include resource-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0187] An embodiment of this application also provides a computer program product. When the computer program product runs on a computer, it causes the computer to execute some or all of the steps in the foregoing method embodiments.

[0188] An embodiment of this application provides a chip system. The chip system includes a processor and may also include a memory for implementing the functions of the electronic device in the foregoing method. The chip system may be composed of chips or may include chips and other discrete devices.

[0189] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0190] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other may be through some interfaces, and the indirect couplings or communication connections of the devices or units may be in electrical, mechanical, or other forms.

[0191] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed over multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0192] In addition, each functional unit in various embodiments of the present application may be integrated into one processing unit, may exist separately as individual physical units, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0193] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0194] As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of various embodiments of the present application.

Claims

1. A data analysis method, characterized in that, Including: Obtain historical data, where the historical data includes multiple historical data tables and historical natural language query statements, as well as historical structured query language (SQL) query statements corresponding to the historical natural language query statements. Each historical data table includes at least one field name; Input the historical data into at least one of a large language model and a data mining algorithm to generate a target field alignment table, where the target field alignment table includes mapping relationships of different field names with equivalent semantics in the multiple historical data tables; Update the structured description information of the multiple historical data tables according to the target field alignment table to generate updated structured description information of the multiple historical data tables; Among them, the target field alignment table and the updated structured description information of the multiple historical data tables are used to query the target field name.

2. The method according to claim 1, characterized in that Each historical data table further includes field values corresponding to the at least one field name; The target field alignment table further includes field values corresponding to the different field names with equivalent semantics; The target field alignment table and the updated structured description information of the multiple historical data tables are used to query the target field name and the corresponding target field value.

3. The method according to claim 2, characterized in that, The target field alignment table further includes the distribution probability of the field values corresponding to the different field names with equivalent semantics.

4. The method according to claim 2 or 3, characterized in that, The different field names with equivalent semantics include a first field name and a second field name, and the format of the first field value corresponding to the first field name is the same as the format of the second field value corresponding to the second field name.

5. The method according to claim 2 or 3, characterized in that, The different field names with equivalent semantics include a third field name, and the third field value corresponding to the third field name is a third field value after being corrected.

6. The method according to any one of claims 1-3, characterized in that, The method further includes: Obtain a current natural language query statement; Input the current natural language query statement into a natural language to structured query language (NL2SQL) model to convert it into a current SQL query statement; Obtain a query result according to the current SQL query statement, the target field alignment table, and the updated structured description information of the multiple historical data tables; Output the query result, where the query result includes the query field name.

7. The method according to claim 6, characterized in that, The outputting the query result includes: Output the query result in a table form, or a picture form, or a multi-modal form.

8. A data analysis device, characterized in that, Including: An obtaining module, configured to obtain historical data, where the historical data includes multiple historical data tables and historical natural language query statements, as well as historical structured query language (SQL) query statements corresponding to the historical natural language query statements. Each historical data table includes at least one field name; A processing module, configured to input the historical data into at least one of a large language model and a data mining algorithm to generate a target field alignment table, where the target field alignment table includes mapping relationships of different field names with equivalent semantics in the multiple historical data tables; Update the structured description information of the multiple historical data tables according to the target field alignment table to generate updated structured description information of the multiple historical data tables; among them, the target field alignment table and the updated structured description information of the multiple historical data tables are used to query the target field name.

9. An electronic device, characterized in that, Including: A memory, a processor, and a display, wherein the memory stores a computer program that runs on the processor, and the electronic device implements the method according to any one of claims 1-7 when executing the program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the method according to any one of claims 1-7 is implemented.

Citation Information

Patent Citations

  • SQL (Structured Query Language) statement processing method and device, equipment and storage medium

    CN116628011A

  • Data analysis method, device and equipment and computer readable medium

    CN117609233A

  • Automatic corresponding matching method and system for homologous heterogeneous data

    CN118820797A

  • Multi-format document table data extraction and alignment method based on large language model

    CN119203958A

Cited By

  • Table data comparison method and device, computer equipment and readable storage medium

    CN120654004A

  • Table data comparison method and device, computer device and readable storage medium

    CN120654004B