Construction method of table knowledge base, query method of data table and related devices

Abstract

The invention discloses a table knowledge base construction method, a data table query method and related devices which are applied to the technical field of data processing. In the construction method of the table knowledge base, table knowledge of a data table is extracted from historical query data. The table knowledge of the data table can reflect the content of the data table from the perspective of query requirements. And constructing a table knowledge base by using the extracted table knowledge. In the data table query method, a query problem is firstly obtained, and a target data table is determined according to table knowledge of the data table and semantic information of the query problem. And finally, generating a query result comprising the target data table. By means of the table knowledge, the data table can be matched with the query problem from the perspective of the query requirement, the target data table better meeting the query requirement of the query problem is obtained, and the accuracy degree of the target data table obtained through query is improved.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and specifically relates to a method for constructing a table knowledge base, a method for querying a data table, and related devices. Background Art

[0002] Data asset management refers to comprehensively managing various data assets owned by an enterprise or organization through a series of strategies, processes, technologies, and tools, covering the entire life cycle of data from creation, storage, classification, use, access to destruction. In the stage of identifying and classifying data assets, querying data tables from the already stored data tables is an important step. For the queried data tables, subsequent operations such as key data identification, classification management of table dimensions, quality inspection, and adjustment of storage methods can be performed to achieve data management.

[0003] Currently, with the growth of data scale and the increase of business scenarios, the number of data tables and the complexity of the information included in the data tables are constantly increasing. When users query data tables, it is difficult to query data tables that meet the query requirements from a large number of data tables with complex information. How to improve the accuracy of the queried data tables has become a technical problem to be solved. Summary of the Invention

[0004] In view of this, this application provides a method for constructing a table knowledge base, a method for querying a data table, a device, and related equipment, which can extract the information of the data table from the perspective of the query requirements, obtain table knowledge and construct a table knowledge base, and the data table obtained by querying based on the table knowledge included in the table knowledge base is more in line with the user's query requirements and has a higher accuracy.

[0005] The technical solutions provided by this application are as follows:

[0006] In a first aspect, this application provides a method for constructing a table knowledge base, and the method includes:

[0007] Obtain historical query data, where the historical query data is used to record the query process of historical query data tables;

[0008] Based on the historical query data, extract the table knowledge of the data table, where the table knowledge includes one or more of the business scenario information, query problem information, and field characteristic information of the data table, the business scenario information includes information related to the business scenario to which the data table belongs, the query problem information includes information related to the historical query problems of the data table, and the field characteristic information includes information related to the fields included in the data table;

[0009] Write the table knowledge into a table knowledge base, where the table knowledge base is used to assist in querying data tables that meet the query requirements.

[0010] In a second aspect, the present application provides a method for querying a data table, the method comprising:

[0011] Obtaining a query question for describing a query requirement;

[0012] Determining a target data table that matches the query question according to the table knowledge of the data tables included in the table knowledge base and the semantic information of the query question, where the table knowledge base is generated according to the table knowledge base construction method described in the first aspect, and the table knowledge includes one or more of the business scenario information, query question information, and field characteristic information of the data table, the business scenario information includes information related to the business scenario to which the data table belongs, the query question information includes information related to the historical query questions of the data table, and the field characteristic information includes information related to the fields included in the data table;

[0013] Generating a query result including the target data table.

[0014] In a third aspect, the present application provides a table knowledge base construction device, the device comprising:

[0015] An obtaining module, configured to obtain historical query data, where the historical query data is used to record the query process of historical query data tables;

[0016] An extracting module, configured to extract the table knowledge of the data table based on the historical query data, where the table knowledge includes one or more of the business scenario information, query question information, and field characteristic information of the data table, the business scenario information includes information related to the business scenario to which the data table belongs, the query question information includes information related to the historical query questions of the data table, and the field characteristic information includes information related to the fields included in the data table;

[0017] A constructing module, configured to write the table knowledge into a table knowledge base, where the table knowledge base is used to assist in querying data tables that meet query requirements.

[0018] In a fourth aspect, the present application provides a data table query device, the device comprising:

[0019] An obtaining module, configured to obtain a query question for describing a query requirement;

[0020] A query module, configured to determine a target data table that matches the query problem according to the table knowledge of the data tables included in the table knowledge base and the semantic information of the query problem. The table knowledge base is generated according to the table knowledge base construction method described in the first aspect. The table knowledge includes one or more of the business scenario information, query problem information, and field characteristic information of the data table. The business scenario information includes information related to the business scenario to which the data table belongs. The query problem information includes information related to the historical query problems of the data table. The field characteristic information includes information related to the fields included in the data table;

[0021] A generation module, configured to generate a query result including the target data table.

[0022] In a fifth aspect, the present application provides an electronic device, including:

[0023] One or more processors;

[0024] A storage device, on which one or more programs are stored,

[0025] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in the first aspect, or implement the method described in the second aspect.

[0026] In a sixth aspect, the present application provides a computer-readable medium, on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect is implemented, or the method described in the second aspect is implemented.

[0027] In a seventh aspect, the present application provides a computer program product. When the computer program product runs on a device, the device executes the method described in the first aspect, or implements the method described in the second aspect.

[0028] Thus, the present application has the following beneficial effects:

[0029] This application provides a method for constructing a table knowledge base. First, table knowledge of a data table is extracted from historical query data that records the historical query process of the data table. The table knowledge of the data table can reflect the content of the data table from the perspective of query requirements. The table knowledge of the data table can include one or more of business scenario information, query problem information, and field characteristic information. Among them, business scenario information and field characteristic information are information of the data table that users pay more attention to during queries. Query problem information can reflect the connection between the data table and historical query problems. By using the table knowledge extracted based on query requirements, when querying the data table, the matching degree between the data table and the query problem can be effectively analyzed, and the accuracy of the obtained data table can be improved. Finally, the extracted table knowledge is used to construct a table knowledge base. The table knowledge base is used to assist in querying the data table.

[0030] In addition, an embodiment of this application also provides a method for querying a data table. First, a query problem for describing a query requirement input by a user is obtained. The query problem is semantically parsed to obtain semantic information that effectively represents the query requirement. Then, the table knowledge of the data table included in the table knowledge base is obtained. The table knowledge of the data table includes pre-extracted information that can reflect the content of the data table from the perspective of query requirements. According to the table knowledge of the data table and the semantic information of the query problem, the data table and the query problem can be matched from the perspective of query requirements to obtain a target data table that more conforms to the query requirements of the query problem. Finally, a query result including the target data table is generated for the user to process the target data table subsequently. In this way, based on the table knowledge extracted from the perspective of query requirements, the target data table that meets the user's query requirements can be effectively screened, and the accuracy of the obtained target data table can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a schematic diagram of an application scenario provided by an embodiment of this application;

[0032] Figure 2 is a schematic flowchart of constructing a table knowledge base and querying a data table provided by an embodiment of this application;

[0033] Figure 3 is a flowchart of a method for constructing a table knowledge base provided by an embodiment of this application;

[0034] Figure 4 is a flowchart of a method for querying a data table provided by an embodiment of this application;

[0035] Figure 5 is a schematic structural diagram of a device for constructing a table knowledge base provided by an embodiment of this application;

[0036] Figure 6 is a schematic structural diagram of a device for querying a data table provided by an embodiment of this application;

[0037] Figure 7 This is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0038] To facilitate the understanding and explanation of the technical solution provided by the embodiment of the present application, the background technology of the present application will be described first below.

[0039] In the process of data asset management, users need to query data tables that meet the requirements from the storage system storing data tables according to the needs of processing data, and perform subsequent management operations on the data tables. Querying data tables can also be referred to as selecting data tables. Currently, there are five common methods for querying data tables. The following introduces these five query methods and the existing defects.

[0040] First, query data tables based on a metadata management system.

[0041] In the metadata management system, the basic metadata of data tables is recorded in the form of a data catalog (Data Catalog). The basic metadata includes, for example, basic contents such as table structure, lineage, and business scenario tags. The business scenario tags are generated by manual annotation.

[0042] When querying data tables, using the basic metadata and the user's query problem, the data tables that match the basic metadata and the query problem are used as the query results.

[0043] The metadata management system only records the basic metadata at the data level, lacking semantic information, and it is difficult to query appropriate data tables based on the semantics of the query problem, which easily leads to inaccurate query results and cannot meet the user's query needs. In addition, the business scenario tags need to be manually maintained, which has problems of high maintenance cost and time lag, further exacerbating the problem of inaccurate query results.

[0044] Second, query data tables by means of keyword search and recommendation.

[0045] In the query problem input by the user, it may include the table name or field name of the data table. Using the table name or field name of the data table as a keyword, a search engine is used to query the data tables that match the keyword.

[0046] The method of querying data tables using keywords has problems of semantic understanding deviation and weak generalization ability. For example, it cannot effectively distinguish synonyms, cannot clarify the semantics of keywords in combination with the context, and has poor recognition ability for synonyms with different expression methods. The method of querying data tables using keywords also has the problem of inaccurate query results.

[0047] Third, query data tables by means of a rule engine.

[0048] Query rules are pre-generated in a coded manner based on expert experience. When querying a data table, a rule engine is called to determine the data table that meets the query problem based on the query rules.

[0049] Query rules cannot fully cover all query scenarios. Especially for relatively complex query scenarios, it is difficult to summarize them with brief query rules. Manually written query rules also have difficulty adapting to the changing speed of data tables and query requirements, resulting in slow query rule maintenance efficiency and high costs. Querying data tables through query rules also makes it difficult to obtain accurate data tables that meet the query requirements.

[0050] Fourth, assist in querying data tables based on the query log statistics results.

[0051] Based on the historical query situations recorded in the query log, query log statistics results can be generated. The query log statistics results include information such as the query frequency of the data table and the portrait of the associated query scenario. When querying a data table, it is possible to recommend and sort the data tables that match the query problem based on the query log statistics results.

[0052] The query log statistics results overly focus on the query frequency of the data table, resulting in a significant Matthew effect. For example, data tables with high query frequencies will continue to receive recommendations, while newly created data tables or data tables with low query frequencies have a small probability of being recommended. The query results obtained in this way are usually data tables with high query frequencies, not necessarily the data tables that actually best match the query problem, and the obtained query results also have the problem of insufficient accuracy.

[0053] Fifth, use a machine learning model to query the data table.

[0054] By performing supervised training on the machine learning model, a machine learning model for determining the queried data table based on the query problem can be trained. The machine learning model is, for example, a collaborative filtering model or a classification model. The classification model is, for example, a support vector machine (SVM) model.

[0055] Supervised training requires a large amount of labeled data of <query problem, data table>, and the labeling cost is relatively high. In the case where the quantity of labeled data is small or the query scenarios covered by the labeled data are insufficient, the model accuracy of the trained machine learning model may not be high. The query results obtained using the machine learning model are not accurate enough. Moreover, it is difficult to provide a basis for selecting the data table determined by the machine learning model, which is not conducive to users performing subsequent management operations on the data table.

[0056] Based on the above introduction of the five existing methods for querying data tables, it can be seen that the currently commonly used methods for querying data tables all have certain defects, resulting in the inability to obtain a data table that meets the user's query requirements and has a high degree of accuracy.

[0057] Based on this, the embodiments of the present application provide a method for constructing a table knowledge base. Extract the table knowledge of the data table from the historical query data that records the historical query process of the data table. The table knowledge of the data table can include one or more of business scenario information, query problem information, and field characteristic information. The table knowledge of the data table can reflect the content of the data table from the perspective of the query requirements. Among them, the business scenario information and the field characteristic information are the information of the data table that users are more concerned about during the query. The query problem information can reflect the connection between the data table and the historical query problem. By using the table knowledge extracted based on the query requirements, when querying the data table, the matching degree between the data table and the query problem can be effectively analyzed, and the accuracy of the obtained data table can be improved. Finally, use the extracted table knowledge to construct a table knowledge base. The table knowledge base is used to assist in the query of the data table.

[0058] In addition, the embodiments of the present application also provide a method for querying a data table. First, obtain the query problem entered by the user to describe the query requirements. Perform semantic parsing on the query problem to obtain semantic information representing the query requirements. Then obtain the table knowledge of the data table included in the table knowledge base. The table knowledge of the data table includes the information that can reflect the content of the data table from the perspective of the query requirements extracted in advance. According to the table knowledge of the data table and the semantic information of the query problem, the target data table that meets the query requirements of the query problem can be more accurately determined. In this way, the table knowledge of the data table can be used to match the semantic information of the data table and the query problem from the perspective of the query requirements, and the target data table that more meets the query requirements of the query problem can be obtained. Finally, generate a query result including the target data table for the user to process the target data table subsequently.

[0059] The method for querying a data table based on a table knowledge base provided by the embodiments of the present application can, on the one hand, perform semantic analysis on the query problem entered by the user to effectively understand the user's query requirements; on the other hand, it can effectively screen and obtain the target data table that meets the user's query requirements based on the table knowledge extracted from the perspective of the query requirements, improving the accuracy of the target data table. In addition, the method for querying a data table provided by the embodiments of the present application does not require the user to manually screen a large number of data tables, can quickly locate the target data table that meets the query requirements, improve the efficiency of querying the data table, and greatly shorten the time for querying the data table.

[0060] To facilitate the understanding of the technical solutions provided by the embodiments of the present application, the application scenarios of the method for constructing a table knowledge base and the method for querying a data table provided by the embodiments of the present application are described below with reference to the accompanying drawings.

[0061] The method for constructing a table knowledge base and the method for querying a data table provided by the embodiments of the present application can be applied to a table query system. Refer to Figure 1 As shown, the table query system 100 is connected to the storage system. The table query system 100 is used to select a data table that meets the user's query requirements from the connected storage system. The table query system 100 can also be referred to as a table selection system. The storage system is used to store data tables. As an example, the storage system is a data warehouse, or a database system. The specific type of the storage system can be selected based on the requirements of the stored data, such as the size of the data volume, and the present application does not limit this here.

[0062] During the process of the user's historical query of the storage system, historical query data is generated. The historical query data is used to record the historical query process of querying the data table. The historical query data can be stored in the table query system 100, or in other storage systems connected to the table query system 100.

[0063] The table query system 100 obtains the historical query data. The table query system 100 analyzes the historical query data to obtain the table knowledge of the data table. The table knowledge includes one or more of the business scenario information, query problem information, and field characteristic information of the data table. The table knowledge of the data table can reflect the content of the data table from the perspective of the query requirements, which is convenient for accurately analyzing whether the data table meets the query requirements based on the table knowledge when querying the data table.

[0064] As an example, the table query system 100 can call the large language model (LLM) 101 to process the historical query data to obtain the table knowledge of the data table. The LLM 101 can be deployed in the table query system 100. Or, the LLM 101 is deployed in a computing system independent of the table query system 100, and provides an external call interface to provide external interactive services.

[0065] The table query system 100 constructs a table knowledge base based on the extracted table knowledge.

[0066] In the query stage, the table query system 100 obtains the query problem input by the user. The query problem can reflect the user's query requirements. The table query system 100 determines the target data table that meets the query requirements based on the table knowledge included in the table knowledge base and the semantic information of the query problem. The table query system 100 generates a query result including the target data table and feeds it back to the user, so that the user can perform subsequent data processing based on the query result.

[0067] As an example, the table lookup system 100 can call the large language model 102, process query questions based on the table knowledge base, and obtain the target data table. The large language model 101 used to construct the table knowledge base and the large language model 102 used to query the data table can be the same large language model or different large language models, which can be flexibly selected according to processing needs, and the present application does not make any limitations in this regard.

[0068] It should be noted that the above application scenarios are only used as a possible implementation method and do not limit the application scenarios of the method provided by the embodiments of the present application.

[0069] The construction method of the table knowledge base and the query method of the data table provided by the embodiments of the present application will be introduced below.

[0070] It should be noted that referring to Figure 2 As shown, the table knowledge base can be pre-constructed when the table lookup system is in an offline state, so as to provide a service for querying the data table when the table lookup system is in an online state. Figure 2 The specific content included will be described in detail below.

[0071] For the convenience of understanding, the construction method of the table knowledge base will be introduced first.

[0072] Referring to Figure 3 As shown, this figure is a flowchart of a construction method of a table knowledge base provided by an embodiment of the present application. The method includes:

[0073] S301: Obtain historical query data for recording the historical query process of the data table.

[0074] During the historical query process for the data table, historical query data is generated. The historical query data is used to record the historical query process for the data table. The historical query data at least includes the historical query questions input by the user and the corresponding historical query results. The historical query results include the data tables that match the historical query questions.

[0075] It should be noted that the historical query data is divided by data table dimension. That is to say, the historical query data for one data table is regarded as one piece of historical query data.

[0076] The embodiments of the present application do not limit the source of the historical query data. As an example, the historical query data is obtained from structured query record data.

[0077] The structured query record data adopts a structured data form, which is convenient for computer processing. For example, the structured query record can adopt SQL or a domain-specific language (DSL).

[0078] As an example, the structured query record data includes one or more of business dashboard SQL query data, data visualization platform query data, existing query links, and other query record data. Among them, the business dashboard SQL query data is used to record the process of querying relevant data tables through SQL queries using a visual dashboard. The data visualization platform query data is used to record the process of a user querying data tables through the data visualization platform. The existing query link is a link created by the user that displays the query results corresponding to the determined query problem after being triggered.

[0079] The historical query data obtained from the structured query record data may be described in a non-standardized language. Among them, the standardized language is a pre-set language that conforms to specifications in terms of statement format, symbols, grammar, and vocabulary. The standardized language can be determined based on the language type of the query problem input by the user in the actual scenario of querying data tables. For example, if the user inputs a query problem described in natural language, the standardized language is standardized natural language. Another example is that if the user inputs a DSL query problem, the standardized language is standardized DSL.

[0080] Convert the historical query data described in non-standardized language into historical query data described in standardized language, realizing the normalization processing of the language of historical query data, facilitating subsequent unified semantic understanding and analysis of historical query data, and being conducive to effectively extracting table knowledge.

[0081] The embodiments of this application do not limit the method of obtaining the historical query data described in standardized language. As an example, the conversion rules between non-standardized language and standardized language can be pre-set, and the non-standardized language is processed according to the conversion rules. As another example, a pre-trained language model can be called to convert the historical query data in non-standardized language into historical query data in standardized language. Among them, the language model is used to implement the conversion of language types.

[0082] S302: Extract the table knowledge of the data table based on the historical query data.

[0083] When a user queries a data table, the user will be more concerned about some information in the data table. For example, from a macroscopic dimension during the query, the user is more concerned about the business scenario to which the data table belongs. For example, the user usually needs to query the data table under a certain business scenario. From a microscopic dimension, the user is more concerned about some fields in the data table. For example, when the user queries the data table related to "order amount", the user is more concerned about the fields that have the same or similar semantics as "order amount".

[0084] For the query requirements of users, that is, the content that users are more concerned about when querying, by analyzing historical query data, it is possible to determine the information related to the query requirements included in the data table and generate table knowledge. Table knowledge can be represented by structured data, which is convenient for computer processing.

[0085] Table knowledge includes one or more of the business scenario information, query problem information, and field characteristic information of the data table. Among them, the business scenario information includes information related to the business scenario to which the data table belongs. The business scenario information describes the content of the data table from the aspect of the business scenario that users are more concerned about. The query problem information includes information related to historical query problems for the data table. The query problem information can reflect the query method. The field characteristic information includes information related to the fields included in the data table. The field characteristic information describes the content of the data table from the aspect of the fields that users are more concerned about.

[0086] For the business scenario information, query problem information, and field characteristic information, the extraction methods for different types of information are different. As some examples, the embodiments of the present application provide possible extraction methods for the above three types of information. For specific details, please refer to the following description.

[0087] In addition, in some possible implementation manners, a large language model (LLM) is called to extract table knowledge. The LLM processes historical query data to obtain the table knowledge of the data table fed back by the LLM. For the specific process of the LLM extracting table knowledge, please refer to the following introduction.

[0088] In this way, it is possible to analyze historical query data from the perspective of query requirements, extract the information of the data table that users are more concerned about during the query, and generate table knowledge. Furthermore, when querying the data table using the table knowledge, it is possible to better match the table knowledge of the database with the user's query problem, obtain the target data table that meets the user's query requirements, and thus improve the accuracy of the query result.

[0089] S303: Write the table knowledge into the table knowledge base.

[0090] The table knowledge base can include the table knowledge of each data table within the query scope. The query scope refers to the scope of data tables that can be queried by users. The query scope can be determined based on the data tables included in the storage system and the query permissions of the data tables.

[0091] For each data table included in the query scope, according to the above S301 and S302, obtain historical query data, extract the table knowledge from the historical query data, and write the obtained table knowledge into the table knowledge base to obtain a table knowledge base including the table knowledge of each data table within the query scope.

[0092] In some possible implementations, table knowledge can be generated in batches. As an example, based on the importance of the data tables, the historical query data of the more important data tables is processed first, and then the historical query data of other data tables is processed. This can relieve the pressure of processing historical query data and improve the stability of the table lookup system.

[0093] The extraction methods of business scenario information, query problem information, and field characteristic information will be introduced separately below.

[0094] First, business scenario information.

[0095] Business scenario information includes information related to the business scenario to which the data table belongs.

[0096] The embodiments of the present application do not limit the way of dividing business scenarios. As an example, business scenarios can be divided according to the dimension of the data's own attributes. As another example, business scenarios can be divided according to the dimension of the actual usage scenario.

[0097] For example, the business scenario to which the data table belongs can include one or more of the first business scenario indicated by the business attributes of the data table and the second business scenario in which the data table is frequently used. Among them, the first business scenario is the business scenario divided according to the dimension of the data's own attributes. The first business scenario is usually the business scenario determined when creating the data table. The second business scenario is the business scenario divided according to the dimension of the actual usage scenario. The first business scenario and the second business scenario may have the same content or different content.

[0098] The business scenario information of the data table can be extracted from business problems. A business problem is a historical query problem for querying the business in the historical query data. That is, the historical query problem including business information is used as a business problem. A business problem can only include the business information of the data table to be queried, or can also include other information, such as including the field information of the data table to be queried.

[0099] In a possible implementation, the historical query data is filtered to determine a set of business problems with the data table as the dimension. A set of business problems corresponds to a data table. The set of business problems includes business problems for the corresponding data table.

[0100] The business problems included in the set of business problems are analyzed to obtain the business scenario information of the data table.

[0101] As an example, calling the LLM to process the business problems included in the set of business problems to obtain the business scenario information of the data table fed back by the LLM. For example, based on the business problems and the instructions for extracting business scenario information, a first prompt is generated. The instructions for extracting business scenario information can be preset instructions, which are used to indicate the way for the LLM to extract business scenario information and the data format of the business scenario information (for example, structured data format). Call the LLM to process the first prompt to obtain the business scenario information extracted by the LLM from the set of business problems.

[0102] Furthermore, it can be divided into the extraction of business scenario information for a single data table and for multiple data tables according to the number of data tables for extracting business scenario information.

[0103] In the extraction method of business scenario information for a single data table, it is possible to analyze the business problems of a single data table to obtain the first business information of the data table. The first business information is used to describe the business scenario of the data table.

[0104] Specifically, call the LLM to process the business problems of a single data table, and use the LLM to analyze the business problems of a single data table to obtain the first business information.

[0105] In this way, it is possible to comprehensively analyze the business scenario of a single data table and the coverage of the business scenario. The obtained first business information can describe a relatively complete business portrait of the data table.

[0106] In addition to the extraction method of business scenario information for a single data table, it is also possible to extract the business scenario information for multiple data tables. Among them, multiple data tables include at least two data tables. In addition, multiple data tables can be data tables with relatively similar business scenarios.

[0107] In the extraction method of business scenario information for multiple data tables, it is possible to analyze the business problems of multiple data tables to obtain the second business information of each data table included in the multiple data tables. The second business information is used to describe the difference in the business scenario between the data table and other data tables in the multiple data tables.

[0108] Specifically, call the LLM to process the business problems of multiple data tables, and use the LLM to analyze the business problems of multiple data tables to obtain the second business information of each data table included in the multiple data tables.

[0109] Analyzing the business problems of multiple data tables can analyze the characteristics of the business problems corresponding to different data tables, clarify the business boundaries and specific business scenarios of different data tables, and effectively distinguish the business scenarios of different data tables.

[0110] It should be noted that the business problems of a single data table can be analyzed alone, the business problems of multiple data tables can be analyzed alone, or the business problems of a single data table can be analyzed first and then the business problems of multiple data tables can be analyzed. The method of analyzing the business problems of a single data table first and then the business problems of multiple data tables can be called the progressive business scenario information extraction method.

[0111] In the progressive business scenario information extraction method, when extracting the second business information of multiple data tables, the first business information of a single data table can also be combined. That is to say, analyze the business problems of multiple data tables and the first business information of each data table included in the multiple data tables to obtain the second business information of each data table. In this way, the business scenario information of a single data table can be determined first, and then the business scenarios between data tables can be compared, so that the differences in business scenarios between data tables can be determined more clearly, which helps to effectively distinguish the business scenarios of different data tables when querying data tables and reduce the possibility of data table confusion. The progressive business scenario information extraction method helps to sort out and distinguish the business scenarios of large-scale data tables.

[0112] In addition to analyzing business problems, it is also possible to analyze the fields included in the data table to determine the business scenario information of the data table through the fields.

[0113] As an example, first obtain the field basic information of the fields included in the data table. The field basic information is used to describe the fields. The sources of the field basic information are, for example, data dictionary documents, business knowledge bases, field naming specifications, and field annotation descriptions. Among them, the data dictionary document is used to record the fields corresponding to the data. The business knowledge base includes knowledge of fields related to the business. The field naming specification is used to define the field naming method. The field annotation description is used to describe the fields. Based on the field basic information, information such as field semantics and field lineage relationships can be analyzed.

[0114] In addition, obtain field problems from historical query data. The field problems are historical query problems for querying fields in the historical query data. That is to say, the historical query problems including field basic information are used as field problems. The field problems can only include the information of the data table to be queried in terms of fields, or can also include other information, such as including the business information of the data table to be queried. That is, the query problems including information in terms of fields and business information are both field problems and business problems.

[0115] Analyze the basic information and problems of fields to obtain the field knowledge of the data table. The field knowledge includes the relevant information of the fields in the data table. The field knowledge includes one or more of the field combination type, the relationship between fields, the field usage frequency, and the field type distribution. Among them, the field combination type refers to the combination type of two or more fields. The field combination type can be determined based on multiple fields included in the field problem. For example, if a field problem includes field A and field B, then field A and field B can be combined to obtain the field combination <field A, field B>. The field combination is related to the data table corresponding to the field problem. The relationship between fields refers to the relationship between the fields included in the data table. The field usage frequency is the frequency of a field appearing in the data table. The field type distribution refers to the distribution of different field types in the data table. The field type can be divided based on the data type of the field value.

[0116] The embodiments of the present application do not limit the manner of analyzing the basic information and problems of fields to obtain the field knowledge of the data table. As an example, call the LLM to process the basic information and problems of fields to obtain the field knowledge. Specifically, based on the basic information of fields, the problems of fields, and the instruction indicating the manner of generating field knowledge by the LLM, generate a second prompt. Call the LLM to process the second prompt to obtain the field knowledge of the data table feedback by the LLM.

[0117] Finally, the business scenario information can be generated according to the field knowledge of the data table.

[0118] Specifically, generating business scenario information according to the field knowledge of the data table includes one or more of the following:

[0119] Analyze the field combination type, and the business entity corresponding to the data table can be determined;

[0120] Analyze the relationship between fields to determine the business processing logic of the data table;

[0121] Analyze the field usage frequency to determine the business dimension to which the data table belongs.

[0122] Among them, the business entity, the business processing logic, and the business dimension all belong to the business scenario information of the table.

[0123] As an example, the analysis of field knowledge can be implemented through the LLM. For example, call the LLM to analyze the field combination type to determine the business entity corresponding to the data table.

[0124] Second, field characteristic information.

[0125] The field characteristic information can be obtained by analyzing the fields included in the data table.

[0126] As an example, by obtaining the basic field information and field problems, the field knowledge of the data table is obtained. The method of obtaining the basic field information and field problems to obtain the field knowledge of the data table is the same as the method introduced in the above business scenario information and will not be elaborated here.

[0127] Specifically, field characteristic information is generated based on the field knowledge of the data table, including one or more of the following:

[0128] Determine the data information of the data table according to the field type distribution;

[0129] Aggregate the field knowledge according to the field semantics to obtain the aggregated field knowledge;

[0130] Analyze the field knowledge of the same fields included in different data tables, which can realize the comparison of cross-table fields and obtain the table comparison result.

[0131] Among them, the data information, the aggregated field knowledge, and the table comparison result belong to the field characteristic information.

[0132] The data information is used to describe the characteristics of the data included in the data table. The characteristics of the data include, for example, the characteristics in terms of data type. As an example, the analysis of the field knowledge can be realized through the LLM. For example, call the LLM to analyze the field type distribution to determine the data information of the data table.

[0133] The table comparison result can describe the field-level differences and association relationships between different data tables. The table comparison result can be realized through the LLM. For example, call the LLM to analyze the field knowledge of the same fields included in different data tables to obtain the table comparison result.

[0134] The field characteristic information can include the results of multi-table comparison and the aggregated field knowledge, support the differentiation of data tables at the field level, meet the query requirements in more complex query scenarios, and realize a more comprehensive data table recommendation.

[0135] In addition to being able to be used to analyze and obtain table knowledge, the field knowledge can also be stored in the field knowledge base. The field knowledge base is independent of the table knowledge base. The field knowledge base can also be used to assist in querying the data table. The field knowledge base can assist in querying the data table at the field level.

[0136] Third, query problem information.

[0137] The query problem information includes information related to the historical query problems for the data table.

[0138] As an example, the query problem information includes the key feature words of the data table. The key feature words are the feature words with a relatively high degree of association with the data table included in the query problem.

[0139] In the process of determining the key feature words, first obtain the association relationship between the data table and the feature words.

[0140] Among them, the feature words are the word segments extracted from the historical query questions in the data table. In a possible implementation, first obtain the historical query question - data table association data set. The historical query question - data table association data set includes: <historical query question, associated table identifier list>. The data tables corresponding to the identifiers included in the associated table identifier list are the data tables included in the historical query results corresponding to the historical query questions.

[0141] Perform word segmentation on the historical query question to obtain the feature words. As an example, first perform preprocessing of word segmentation and stop word removal on the historical query question, and then extract the nouns included in the historical query question as the feature words.

[0142] According to the historical query results corresponding to the historical query questions, determine the co-occurrence frequency of the feature words and the data tables. Among them, the historical query results include the information of the data tables. Specifically, based on the historical query question - data table association data set, the co-occurrence frequency of the feature words and the data tables can be determined. The co-occurrence frequency refers to the frequency that the historical query question includes the feature words and the historical query results include the data tables. Based on the co-occurrence frequency of the feature words and the data tables, a bipartite graph of "feature word - table identifier" can be constructed. Among them, the table identifier is used to identify the data table. The table identifier is, for example, the name of the data table.

[0143] Construct an association relationship between the feature words and the data tables whose co-occurrence frequency meets the association conditions. The association conditions that the co-occurrence frequency meets are, for example, that the co-occurrence frequency is greater than or equal to the co-occurrence frequency threshold. As an example, an algorithm can be used to mine frequent item sets, that is, the feature words and the data tables whose co-occurrence frequency meets the conditions. It should be noted that the association relationship can include the association relationship between a feature word and a data table, or can also include the association relationship between multiple feature words and a data table. As an example, the constructed association relationship is expressed as: "{feature word 1, feature word 2} → table identifier". The association relationship constructed between the feature words and the data tables can be written into the association relationship library. The association relationship library can be a database independent of the table knowledge base.

[0144] After obtaining the association relationship between the data table and the feature words, analyze the association relationship between the data table and the feature words to determine the key feature words of the data table. The key feature words are the feature words whose association degree with the data table is greater than or equal to the association degree threshold.

[0145] In a possible implementation, the association degree value between the feature word and the data table can be determined. The feature words whose association degree value is greater than or equal to the association degree threshold are used as the key feature words.

[0146] The degree of association value is determined according to one or more of support, confidence, and coverage. As an example, the degree of association value is a weighted value of support, confidence, and coverage.

[0147] Among them, coverage is used to measure the generalization ability of the feature word to the semantics of the historical query question, that is, the coverage range of the feature word to the historical query question. Support is used to measure the probability of the co-occurrence of the feature word and the data table, that is, the co-occurrence universality of the feature word and the data table. Confidence is used to measure the probability that the historical query result corresponding to the historical query question includes the data table when the historical query question includes the feature word, that is, the prediction ability of the feature word to the data table. In the process of determining the key feature words using the degree of association value, feature values with a confidence lower than the confidence threshold can be filtered first, and then feature words with a key degree value greater than or equal to the degree of association threshold are used as key feature words. Among them, the degree of association threshold can be a preset value. Or, the degree of association threshold can be determined based on the sorting result of the key degree values of each feature word. For example, if N key feature words need to be selected, the key degree values of each feature word are sorted from large to small, and the degree of association value of the Nth feature word is used as the key degree threshold.

[0148] The obtained key feature words are used as query question information.

[0149] Extracting key feature words can be performed by the LLM. Specifically, the historical query question-data table association dataset and the instruction indicating the extraction of key feature words are combined to generate a third prompt word. The LLM is called to process the third prompt word to obtain the key feature words extracted by the LLM.

[0150] It should be noted that the LLM called in the above different steps of extracting table knowledge can be the same model or different models, which can be flexibly selected based on the needs of processing data.

[0151] In addition, manual review can also be performed on the key feature words. Based on the review results of the manual review, the key feature words are adjusted.

[0152] In this way, the association relationship between the query question and the data table can be mined, and then the key feature words of the data table can be efficiently and comprehensively extracted from the query question, further enriching the information types included in the table knowledge, thereby improving the accuracy of querying the data table using the table knowledge. In addition, based on the association relationship, an explanation basis can also be provided for the queried data table.

[0153] The above is an introduction to the generation process of table knowledge. In addition, after generating the table knowledge and writing it into the table knowledge base, the table knowledge base can also be maintained. The following is an introduction to the maintenance process of the table knowledge base.

[0154] In some possible implementations, the update conditions of the table knowledge base are preset. When the update conditions are met, the table knowledge included in the table knowledge base is updated.

[0155] As an example, the update conditions include that the content of the data table corresponding to the table knowledge changes. For example, the data table is detected. If it is determined that there is a schema change event for the data table, it is determined that the content of the data table has changed. The table knowledge of the changed data table is re-extracted. For example, if the changed content of the data table is a newly added field, the field knowledge of the newly added field is analyzed to obtain the updated table knowledge. Another example is that if the changed content of the data table is related to the business scenario, the business scenario information included in the table knowledge is updated.

[0156] The above example updates some of the table knowledge included in the table knowledge base. In another example, all the table knowledge included in the table knowledge base can be updated.

[0157] As another example, the update conditions include reaching a preset moment for full-scale updating of the table knowledge base. The preset moment can be the moment set for periodically updating the table knowledge base. In the case of full-scale updating of the table knowledge base, the historical query data can be re-analyzed to refresh the table knowledge included in the table knowledge base. In this way, the table knowledge of the data table can be updated, business scenarios can be added or deleted, and based on the development of the business and the changes of data assets, it is ensured that the table knowledge base adapts to business changes and continuously evolves, maintaining the effectiveness of the table knowledge base. In addition, automatically refining and updating the table knowledge reduces the workload of manually maintaining the table knowledge, can reduce the cost of data asset management, and improve the efficiency of data asset management.

[0158] As yet another example, during the query process, the user can evaluate the query results to generate post-evaluation content. The table knowledge base can be updated using the evaluation content. The update conditions include obtaining the evaluation content for the query results. For the method of updating the table knowledge base using the evaluation content, please refer to the introduction below.

[0159] After updating the table knowledge base, the quality of the table knowledge base can also be evaluated. As an example, the updated table knowledge included in the table knowledge base is selected. The query results determined based on the table knowledge before the update and the query results determined based on the updated table knowledge are obtained. The two query results are compared to obtain the update comparison result of the table knowledge. According to the update comparison result, the quality of the updated table knowledge can be evaluated, and the anomalies and errors introduced by updating the table knowledge can be identified.

[0160] In addition, it is also possible to record the update process of the table knowledge base, which facilitates the management of the update process of the table knowledge base. During the update of the table knowledge base, it is possible to coordinate partial updates and full updates to complete the update of table knowledge while ensuring that the table knowledge base stably provides services for assisting in querying data tables.

[0161] The above is an introduction to the construction process of the table knowledge base. Based on the constructed table knowledge base, data tables can be queried. Next, the process of querying data tables based on the table knowledge base will be described.

[0162] See Figure 4 As shown in the figure, this figure is a flowchart of a method for querying a data table provided by an embodiment of the present application. The method includes:

[0163] S401: Obtain a query problem for describing the query requirement.

[0164] The query problem is a problem provided by the user for describing the query requirement. The query problem is used to match the data table that the user wants to query. The query problem may include the business scenario of the data table that the user wants to query, the fields included in the data table, or other content of the data table. The present application does not limit the language used for the query problem. As an example, the query problem input by the user can be described in natural language or be an SQL query problem.

[0165] After obtaining the query problem, if the language used for the query problem is a non-standardized language, the query problem can be subjected to language conversion. Convert the non-standardized language used for the query problem into a standardized language. The standardized language is the standardized language used for processing historical query data when constructing the table knowledge base. As an example, the standardized language is natural language. Analyzing the query problem described in natural language facilitates user querying, reduces the threshold for users to understand data tables and field names, and enables non-technical users to efficiently use the table query system.

[0166] Language conversion can make the language of the query problem consistent with the language for processing historical query data, facilitating subsequent processing of the query problem and matching with table knowledge.

[0167] The embodiments of the present application do not limit the manner of obtaining a query problem described in a standardized language. As an example, conversion rules between non-standardized languages and standardized languages can be preset in advance, and the non-standardized language can be processed according to the conversion rules. As another example, a pre-trained language model can be called to convert the query problem in non-standardized language into a query problem in standardized language. The language model is used to implement the conversion of language types.

[0168] S402: Determine a target data table that matches the query problem based on the table knowledge of the data tables included in the table knowledge base and the semantic information of the query problem.

[0169] For the obtained query problem, perform semantic parsing on the query problem to obtain the semantic information of the query problem. The semantic information is used to describe the query requirements of the query problem. Based on the semantic information of the query problem, the query intention of the user can be better understood, the query requirements can be fully understood, and thus it is helpful to query the data table that meets the user's query requirements.

[0170] As an example, the LLM can be used to analyze the query problem to obtain the semantic information of the query problem. As another example, an embedding model is used to process the query problem, extract the features of the query problem, and obtain vectorized semantic information.

[0171] Based on the table knowledge included in the table knowledge base and the semantic information of the query problem, the data table and the query problem can be matched from a semantic perspective to obtain a target data table that matches the query problem. Among them, the table knowledge base is constructed by using the above-mentioned construction method of the table knowledge base.

[0172] It should be noted that step S402: Determine a target data table that matches the query problem based on the table knowledge of the data tables included in the table knowledge base and the semantic information of the query problem can be executed by the LLM. The large language model used to query the target data table can be the same model as the large language model for constructing the above-mentioned table knowledge base or a different model.

[0173] On the one hand, based on the semantic perspective for matching, the obtained target data table is more in line with the query requirements described by the query problem, that is, the semantics represented by the query problem. On the other hand, using the table knowledge extracted from the query requirement perspective can more fittingly match the table knowledge and the semantic information of the query problem from the perspective of the query requirements, so as to obtain a more effective and accurate matching result.

[0174] The embodiments of the present application provide possible implementation manners for determining a target data table that matches the query problem based on the table knowledge of the data tables included in the table knowledge base and the semantic information of the query problem. For details, please refer to the following introduction.

[0175] S403: Generate a query result including the target data table.

[0176] Based on the target data tables obtained by matching, query results can be generated for feedback to users. The query results at least include the target data tables. As an example, the query results can include the specific content of the target data tables, or information such as the identifiers or access links of the target data tables. Additionally, the query results can also include the reasons for selecting the target data tables. Based on the target data tables determined according to table knowledge, the matching degree between the target data tables and the query questions in terms of query requirements can be determined, and then the reasons for selecting the target data tables can be generated, enhancing users' trust in the query results.

[0177] When there are multiple target data tables, the query results can also include the matching values of each target data table, that is, the scores of the target data tables.

[0178] The generated query results are sent to the user. Specifically, the query results are displayed so that the user can obtain the target data tables obtained by the query.

[0179] The following introduces possible implementation methods for determining the target data tables that match the query questions.

[0180] In one possible implementation method, according to the table knowledge of the data tables included in the table knowledge base and the semantic information of the query questions, determining the target data tables that match the query questions includes the following two steps:

[0181] A1: According to the table knowledge of the data tables included in the table knowledge base and the semantic information of the query questions, determine the first matching value between the data tables and the query questions.

[0182] The first matching value is used to measure the matching degree between the candidate data tables and the query questions.

[0183] As an example, it is possible to directly calculate the similarity between the table knowledge of the data tables included in the table knowledge base and the semantic information of the query questions to obtain the first matching value between the data tables and the query questions.

[0184] For example, the semantic information of the query questions is the vectorized information extracted by an embedding model. Similarly, the embedding model is used to process the table knowledge of the data tables to extract the vectorized information of the table knowledge. Calculate the similarity between the semantic information of the query questions and the information of the table knowledge to obtain the first matching value between the query questions and the data tables.

[0185] As another example, when the table knowledge at least includes the business scenario information of the data tables, a hierarchical matching strategy can be adopted. First, candidate data tables are determined for the business scenario, and then the target data tables are determined from the candidate data tables.

[0186] The following specifically explains the hierarchical matching strategy. The process of executing the hierarchical matching strategy mainly includes the following steps A11 - A14.

[0187] A11: Determine the business similarity between the data table and the query problem based on the business scenario information of the data table included in the table knowledge and the semantic information of the query problem.

[0188] The business similarity between the data table and the query problem can be reflected in the matching degree between the business scenario aspects of the data table and the query problem. As an example, the semantic information of the query problem is the vectorized information extracted by the embedding model. The embedding model is used to process the business scenario information to extract the vectorized business scenario information. Calculate the similarity between the semantic information and the vectorized business scenario information to obtain the business similarity between the query problem and the data table.

[0189] A12: Use the data tables whose business similarity meets the business similarity condition as candidate data tables.

[0190] The business similarity condition is, for example, that the business similarity is greater than or equal to the business similarity threshold. When the value range of the business similarity is [0, 1], the business similarity threshold is, for example, 0.6.

[0191] By calculating the business similarity and selecting candidate data tables according to the business similarity, a coarse-grained screening of the data tables within the query range is achieved, excluding the data tables with low business similarity, that is, the data tables with low relevance to the business scenario. The range of the determined candidate data tables is smaller than the initial query range, which is convenient for further querying the target data table in the candidate data tables. On the one hand, it can save the resources for query processing, and on the other hand, it can effectively screen the data tables and improve the accuracy of the finally determined target data table.

[0192] In a possible implementation manner, after determining the candidate data tables, the candidate data tables can be further screened according to the fields included in the data tables.

[0193] Specifically, determine the query fields included in the query problem. The query fields included in the query problem are the fields that are important for the data table to be queried and are the core fields related to the query requirements. The query fields can be the fields directly included in the text of the query problem. As an example, entity recognition of the fields of the query problem is performed to determine the query fields. In addition, the query fields can also be the fields with the same semantics extracted from the semantic information of the query problem. As an example, according to the semantic information of the query problem and the field knowledge, the query fields are matched.

[0194] If the data table does not include the query fields, obviously it cannot be used as the final target data table and is not within the scope of further screening of the candidate data tables.

[0195] Check whether the candidate data table includes the query field. Exclude the candidate data tables that do not include the query field. That is, delete the candidate data tables that do not include the query field from the scope of the candidate data tables and no longer participate in the selection process of the target data table for this time.

[0196] A13: For various types of information included in the table knowledge of the candidate data table, determine the sub - matching values of the candidate data table for each type of information with respect to the query problem.

[0197] Based on the above introduction, it can be known that the table knowledge can include three types of information, namely one or more of business scenario information, query problem information, and field characteristic information. When the table knowledge includes at least business scenario information, the table knowledge can also include one or more of query problem information and field characteristic information.

[0198] Based on each type of information included in the table knowledge, the sub - matching values of the candidate data table and the query problem can be determined respectively. As an example, when the table knowledge includes business scenario information, query problem information, and field characteristic information, determine the sub - matching value for business scenario information, the sub - matching value for query problem information, and the sub - matching value for field characteristic information respectively. Another example, when the table knowledge includes business scenario information and query problem information, determine the sub - matching value for business scenario information and the sub - matching value for query problem information respectively.

[0199] The sub - matching value is used to measure the matching degree between the candidate data table and the query problem in the dimension of the corresponding information type.

[0200] The embodiments of this application do not limit the calculation method of the sub - matching value. As an example, the calculation method of the sub - matching value is similar to the calculation method of the above - mentioned business similarity and will not be elaborated here. Among them, the business similarity can be used as the sub - matching value of the business scenario information.

[0201] A14: According to the sub - matching value, determine the second matching value between the candidate data table and the query problem.

[0202] As an example, the weighted value of the sub - matching value can be calculated as the second matching value between the candidate data table and the query problem. The weight of the sub - matching value can be set based on the query requirements.

[0203] In this way, first conduct a preliminary screening based on the business scenario, and then determine the second matching value between the candidate data table and the query problem with fine - grainedness in the dimension of multiple information types, realizing hierarchical matching with different granularities. Hierarchical matching can continuously narrow the scope of the data tables to be queried and improve the efficiency of querying data tables.

[0204] The obtained second matching value between the candidate data table and the query problem can be used as the first matching value to determine the target data table.

[0205] In addition, after obtaining the second matching value between the candidate data table and the query question, the matching value can be optimized from one or more aspects of semantic dynamic expansion and field comparison. The optimization methods of semantic dynamic expansion and field comparison are introduced below.

[0206] First, dynamic expansion of semantics.

[0207] Taking into account that the query question may have implicit query requirements, the query question can also be semantically expanded.

[0208] Specifically, a target field related to the query's semantic information is first determined. The semantics of the target field are somewhat correlated with the query's semantic information. The target field is a field expanded based on the query's semantic information. The degree of match between the target field's semantics and the query's semantic information is lower than the degree of match between the query field's semantics and the query's semantic information.

[0209] As an example, the LLM can be called to perform semantic analysis on the query question and mine target fields related to the semantic information of the query question. The number of target fields can be one or more.

[0210] Query the target field's field knowledge. This field knowledge can be retrieved from the aforementioned field knowledge base or table knowledge base. This field knowledge describes the target field. If no field knowledge for the target field is found, the LLM can be invoked to dynamically expand the semantics of the target field, obtaining the target field's semantic content as the target field's field knowledge.

[0211] Generate an expanded question based on the target field's field knowledge. Based on the semantic information of the query question, the expanded question is used to query the data table. The expanded question is used to query the data table related to the target field.

[0212] Then, based on the table knowledge, the third matching value between the data table and the expansion question is determined. The third matching value is used to measure the matching degree between the expansion question and the data table.

[0213] It should be noted that the data table for which the expanded similarity value calculation is performed with the expanded question is the data table included in the largest query scope. In other words, the third matching value with the expanded question can be calculated for both candidate data tables and non-candidate data tables.

[0214] The embodiment of the present application does not limit the calculation method of the third matching value between the data table and the expansion question.

[0215] As an example, the semantic information of the augmented question is extracted through the embedding model. The embedding model is used to process the table knowledge of the data table, and the vectorized table knowledge information is extracted. The similarity between the semantic information of the augmented question and the information of the table knowledge is calculated to obtain the third matching value between the augmented question and the data table.

[0216] As another example, according to the table knowledge and target fields of the data table, the field relevance and field coverage are calculated. The weighted value of the field relevance and field coverage is used as the third matching value.

[0217] Among them, the field relevance is the degree of relevance between the target field and the data table. Specifically, the field relevance is the degree of relevance between the fields included in the data table and the target field. The field relevance can be determined based on the field characteristic information in the table knowledge of the data table and the target field. The field coverage is used to measure the generalization degree of the target field to the data table. The field coverage is, for example, the ratio of the number of target fields included in the data table to the number of fields in the data table.

[0218] The third matching value between the augmented question and the data table is used to optimize the first matching value between the query question and the data table. For example, the weighted value of the third matching value and the already determined first matching value can be calculated to obtain the optimized first matching value. The weights of the third matching value and the already determined first matching value can be flexibly set according to needs.

[0219] For the scenario of determining the candidate data table mentioned above. If the third matching value of the candidate data table is determined through semantic dynamic augmentation, the second matching value between the candidate data table and the query question can be used as the first matching value, and the third matching value is used to optimize the first matching value, such as calculating the weighted value, to obtain the optimized first matching value.

[0220] As another example, if the third matching value of the non-candidate data table is determined through semantic dynamic augmentation, the third matching value can be directly used as the first matching value, enabling the non-candidate data table to participate in the selection of the target data table, realizing the supplementation of other data tables other than the candidate data tables.

[0221] Finally, based on the optimized first matching value, the data table whose first matching value meets the matching condition is used as the target data table. The method for determining the target data table can refer to the following step A2: the specific description of using the data table whose first matching value meets the matching condition as the target data table.

[0222] Through semantic dynamic augmentation, the deviation between the expression of the user's query question and the existing fields is compensated, the hidden semantics included in the query question are effectively extracted, the range of query data tables is expanded, which is conducive to selecting a suitable target data table.

[0223] In addition, after finally determining the target data table, it is also possible to obtain the user's feedback on the expanded query results. Based on the user feedback, the strategy for extracting target fields and generating expanded questions can be optimized. For the new fields with high quality confirmed by the user, the field knowledge of the new fields can be extracted and written into the table knowledge base and the field knowledge base to achieve automatic expansion of the content of the knowledge base. In this way, continuous learning and evolution of the query strategy can be realized, enabling the table lookup system and the table knowledge base to adapt to business changes and the continuous growth of data assets, and maintaining the timeliness and accuracy of the recommended results.

[0224] Second, field comparison.

[0225] Some data tables are relatively similar in terms of fields. When determining the target data table, precise comparison and differentiation need to be carried out at the field level to avoid problems such as incorrect selection of data tables or confusion.

[0226] As an example, after determining the first matching values of multiple data tables, first determine at least one similar data table combination from the multiple data tables. The similar data table combination includes at least two similar data tables. The at least two similar data tables are data tables whose difference from the first matching value of the query problem is less than or equal to the difference threshold. For example, the difference between the first matching value of data table A and the first matching value of candidate data table B is less than or equal to the difference threshold. Data table A and data table B are two similar data tables belonging to the same similar data table combination.

[0227] For each similar data table combination, it is necessary to perform field-level comparison on the similar data tables included in the similar data table combination.

[0228] As an example, the field-level comparison includes one or more of field semantic comparison, field statistical feature comparison, field function comparison, and field distribution comparison.

[0229] Among them, the field semantic comparison is to compare the semantics of the fields included in the similar data tables. Specifically, semantic analysis can be performed based on the field basic information included in the table knowledge of the similar data tables. In addition, calculate the semantic similarity between fields and identify the difference points in semantics between fields. In addition, the context of the fields can be combined to clarify the boundary of the business scenario of the similar data tables. The context of the fields is the name of the similar data table to which the fields belong and the field combinations including the fields, etc. In a possible implementation, a large language model is called to perform semantic analysis and comparison on the fields to obtain the field semantic comparison result.

[0230] Field statistical feature comparison refers to comparing the statistical features of fields. As an example, the statistical features of fields include statistical features such as value distribution, type, missing rate, and uniqueness. Among them, the value distribution refers to the distribution of field values in similar data tables. The type refers to the field type. The missing rate refers to the ratio of the number of missing field values to the total number of fields in the similar data table. Uniqueness means that the values of the field must be unique and no duplicate values can appear.

[0231] Based on the above statistical features, it is possible to analyze the fields of similar data tables and determine the differences between different similar data tables at the field level. As an example, call a large language model to analyze and compare the statistical features of fields to obtain the field statistical feature comparison result.

[0232] Field role comparison can analyze the usage methods of fields and identify the roles played by fields in different similar data tables. As an example, call a large language model to analyze the usage frequency, combination type, etc. of fields to identify the roles of fields in similar data tables. Among them, information such as field usage frequency and field combination type can be extracted from the table knowledge of similar data tables.

[0233] Field distribution comparison is to compare the fields included in similar data tables, determine the same fields and different fields included in different similar data tables, and obtain the field distribution comparison result. The field distribution comparison result can be represented in the form of a matrix. Representing it in the form of a matrix can facilitate display as a basis for table selection, help users understand the differences between similar data tables through visualization, enhance users' trust in query results, and also facilitate users to understand the reasons for selecting the target data table.

[0234] Perform field-level comparison on similar data tables to obtain the field comparison result. The field comparison result includes one or more of the field semantic comparison result, field statistical feature comparison result, field role comparison result, and field distribution comparison result. In this way, through the mining of information such as field semantic comparison, statistical feature analysis, and field usage methods, similar data tables can be accurately distinguished, reducing the possibility of recommendation confusion.

[0235] Using the field comparison result, it is possible to further optimize the first matching value of the data table and the query problem.

[0236] As an example, according to the field comparison result, it is possible to determine the data table in the similar data table combination that is more suitable for the query problem. For example, based on the field semantic comparison result included in the field comparison result, it is possible to determine the similar data table that is more semantically matched with the semantic information of the query problem in terms of semantics. Thus, it is possible to increase the first matching value of the similar data table that is more suitable for the query problem and achieve the optimization of the first matching value.

[0237] Finally, based on the optimized first matching value, the data table whose first matching value meets the matching condition can be used as the target data table. The method for determining the target data table can be referred to in step A2 below: the specific description of using the data table whose first matching value meets the matching condition as the target data table.

[0238] Based on the difference analysis of similar data tables at the table knowledge base and field levels, the target data table that precisely matches the user's query requirements can be determined, avoiding the error of selecting a table caused by the confusion of similar data tables or the misunderstanding of fields.

[0239] A2: Use the data table whose first matching value meets the matching condition as the target data table.

[0240] The matching condition is the condition for selecting the target data table. As an example, the matching condition is that the first matching value is greater than or equal to the matching value threshold. As another example, the matching condition is that the first matching values are sorted from largest to smallest, and the matching values with the first M sorting serial numbers are selected. M is a positive integer. Both the matching value threshold and M can be flexibly set based on the requirements of querying the data table.

[0241] After generating the query result, the user can also evaluate the query result to obtain the evaluation content. For example, the evaluation content includes the data table in the target data table selected by the user that most conforms to the query requirements, and the supplementary description content provided by the user for the business scenario, fields, etc. As an example, the interface for displaying the query result also includes a feedback control. The user can trigger the feedback control to input the evaluation content for the query result.

[0242] Obtain the evaluation content of the user for the query result. The table knowledge base can be optimized using the evaluation content.

[0243] As an example, perform semantic analysis on the evaluation content to extract knowledge related to the business scenario, fields, etc. Convert the extracted knowledge into structured data.

[0244] In addition, the knowledge extracted from the evaluation content can also be verified. As an example, use the method of cross-validation to test whether the knowledge is valid and has universality.

[0245] Write the knowledge with relatively high quality determined through verification into the table knowledge base to achieve the optimization of the table knowledge base. The table knowledge base can be continuously optimized during use to adapt to changes in the business scenario and data assets, and improve the long-term use effect.

[0246] Based on the method for constructing a table knowledge base provided in the above method embodiments, the embodiments of the present application also provide an apparatus for constructing a table knowledge base. The apparatus for constructing a table knowledge base will be described below with reference to the accompanying drawings.

[0247] SeeFigure 5 As shown in the figure, this is a schematic structural diagram of a table knowledge base construction device provided by an embodiment of the present application. As Figure 5 shown, the table knowledge base construction device includes:

[0248] An acquisition module 501, configured to acquire historical query data, where the historical query data is used to record the query process of a historical query data table;

[0249] An extraction module 502, configured to extract table knowledge of the data table based on the historical query data, where the table knowledge includes one or more of business scenario information, query problem information, and field characteristic information of the data table, the business scenario information includes information related to the business scenario to which the data table belongs, the query problem information includes information related to the historical query problems of the data table, and the field characteristic information includes information related to the fields included in the data table;

[0250] A construction module 503, configured to write the table knowledge into a table knowledge base, where the table knowledge base is used to assist in querying data tables that meet query requirements.

[0251] In a possible implementation manner, the table knowledge includes the business scenario information of the data table, and the extraction module 502 is specifically configured to:

[0252] Obtain a set of business problems of the data table from the historical query data, where the set of business problems includes business problems for the data table, and the business problems are historical query problems for querying the business;

[0253] Analyze the business problems included in the set of business problems to obtain the business scenario information of the data table.

[0254] In a possible implementation manner, the extraction module 502 is configured to analyze the business problems included in the set of business problems to obtain the business scenario information of the data table, including:

[0255] The extraction module 502 is configured to analyze the business problems of a single data table to obtain first business information of the data table, where the first business information is used to describe the business scenario of the data table;

[0256] And / or,

[0257] For multiple data tables, analyze the business problems of the multiple data tables to obtain second business information of each data table included in the multiple data tables, where the second business information is used to describe the difference in the business scenario between the data table and other data tables in the multiple data tables.

[0258] In a possible implementation, the table knowledge includes one or more of field characteristic information and the business scenario information. The extraction module 502 is specifically configured to:

[0259] Obtain the field basic information of the fields included in the data table, and obtain field problems from the historical query data. The field basic information is used to describe the fields, and the field problems are historical query problems for querying the fields.

[0260] Analyze the field basic information and the field problems to obtain the field knowledge of the data table. The field knowledge includes one or more of field combination types, relationships between fields, field usage frequencies, and field type distributions.

[0261] Generate the table knowledge according to the field knowledge of the data table.

[0262] In a possible implementation, the table knowledge includes the field characteristic information, and the field knowledge includes the field type distribution. The extraction module 502 is used to generate the table knowledge according to the field knowledge of the data table, including:

[0263] The extraction module 502 is used to determine the data information of the data table according to the field type distribution. The data information is used to describe the data included in the data table.

[0264] And / or

[0265] Aggregate the field knowledge according to the semantics of the fields to obtain the aggregated field knowledge.

[0266] And / or

[0267] Analyze the field knowledge of the same fields included in different data tables to obtain a table comparison result. The table comparison result is used to describe the differences and association relationships at the field level between different data tables.

[0268] The data information, the aggregated field knowledge, and the table comparison result belong to the field characteristic information.

[0269] In a possible implementation, the table knowledge includes the business scenario information. The extraction module 502 is used to generate the table knowledge according to the field knowledge of the data table, including:

[0270] The extraction module 502 is used to analyze the field combination type to determine the business entity corresponding to the data table.

[0271] And / or

[0272] Analyze the relationships between the fields to determine the business processing logic of the data table;

[0273] and / or,

[0274] Analyze the usage frequency of the fields to determine the business dimension to which the data table belongs;

[0275] The business entity corresponding to the data table, the business processing logic of the data table, and the business dimension to which the data table belongs belong to the business scenario information of the table.

[0276] In a possible implementation, the table knowledge includes the query problem information, and the extraction module 502 is used to extract the table knowledge of the data table based on the historical query data, including:

[0277] The extraction module 502 is used to obtain the association relationship between the data table and the feature words, and the feature words are extracted from the historical query problems of the data table;

[0278] Analyze the association relationship between the data table and the feature words to determine the key feature words of the data table. The key feature words of the data table belong to the query problem information, and the key feature words are feature words whose association degree with the table is greater than or equal to the association degree threshold.

[0279] In a possible implementation, the extraction module 502 is used to obtain the association relationship between the data table and the feature words, including:

[0280] The extraction module 502 is used to perform word segmentation on the historical query problems to obtain feature words; determine the co-occurrence frequency of the feature words and the data table based on the historical query results corresponding to the historical query problems; construct the association relationship between the feature words and the data table that satisfies the association condition.

[0281] In a possible implementation, the extraction module 502 is used to analyze the association relationship between the data table and the feature words to determine the key feature words of the data table, including:

[0282] The extraction module 502 is used to determine the association degree value between the feature word and the data table. The association degree value is determined according to one or more of support, confidence, and coverage. The support is used to measure the probability that the feature word and the data table co-occur. The confidence is used to measure the probability that the historical query results corresponding to the historical query problems include the data table when the historical query problems include the feature word. The coverage is used to measure the semantic generalization ability of the feature word for the historical query problems; the feature words with an association degree value greater than or equal to the association degree threshold are used as key feature words.

[0283] In a possible implementation, the extraction of the table knowledge of the data table based on the historical query data is performed by a large language model.

[0284] In a possible implementation, the apparatus further includes:

[0285] An update module, configured to update the table knowledge included in the table knowledge base if an update condition is satisfied, where the update condition includes one or more of the following:

[0286] Determine that the content of the data table has changed, reach a preset moment for full - volume updating of the table knowledge base, and obtain evaluation content for the query result.

[0287] Based on the query method for a data table provided in the above - mentioned method embodiment, an embodiment of the present application further provides a query apparatus for a data table. The query apparatus for the data table will be described below with reference to the accompanying drawings.

[0288] See Figure 6 As shown, this figure is a schematic structural diagram of a query apparatus for a data table provided in an embodiment of the present application. As Figure 6 shown, the query apparatus for the data table includes:

[0289] An acquisition module 601, configured to acquire a query question for describing a query requirement;

[0290] A query module 602, configured to determine a target data table that matches the query question according to the table knowledge of the data table included in the table knowledge base and the semantic information of the query question. The table knowledge base is generated according to the above - mentioned construction method of the table knowledge base. The table knowledge includes one or more of the business scenario information, query question information, and field characteristic information of the data table. The business scenario information includes information related to the business scenario to which the data table belongs. The query question information includes information related to the historical query questions of the data table. The field characteristic information includes information related to the fields included in the data table;

[0291] A generation module 603, configured to generate a query result including the target data table.

[0292] In a possible implementation, the query module 602 is specifically configured to:

[0293] Determine a first matching value between the data table and the query question according to the table knowledge of the data table included in the table knowledge base and the semantic information of the query question. The first matching value is used to measure the matching degree between the data table and the query question;

[0294] Use the data table whose matching value meets the matching condition as the target data table.

[0295] In a possible implementation, the table knowledge at least includes the business scenario information of the data table, and the query module 602 is specifically configured to:

[0296] Determine the business similarity between the data table and the query problem according to the business scenario information of the data table included in the table knowledge and the semantic information of the query problem;

[0297] Use the data tables whose business similarity meets the business similarity condition as candidate data tables;

[0298] Determine the sub-matching values of the candidate data tables and the query problem for various types of information included in the table knowledge of the candidate data tables;

[0299] Determine the second matching value between the candidate data table and the query problem according to the sub-matching value, and the second matching value is used to measure the matching degree between the candidate data table and the query problem.

[0300] In a possible implementation, the query module 602 is further configured to:

[0301] Determine the query fields included in the query problem, and exclude the candidate data tables that do not include the query fields.

[0302] [[ID=D23]]In a possible implementation, the device further includes:

[0303] A first optimization module, configured to determine target fields related to the semantic information of the query problem; generate an extended problem according to the field knowledge of the target fields, where the extended problem is used to query data tables related to the target fields, and the field knowledge of the target fields is used to describe the target fields; determine the third matching value between the data table and the extended problem, where the third matching value is used to measure the matching degree between the extended problem and the data table, and the third matching value is used to optimize the first matching value between the query problem and the data table.

[0304] In a possible implementation, the device further includes:

[0305] A second optimization module, configured to determine a combination of similar data tables, where the combination of similar data tables includes at least two similar data tables, and the at least two similar data tables are data tables with a difference in a first matching value less than or equal to a difference threshold; perform a field-level comparison on the at least two similar data tables according to the field characteristic information of the similar data tables to obtain a field comparison result, where the field comparison result includes one or more of a field semantic comparison result, a field statistical feature comparison result, a field function comparison result, and a field distribution comparison result, and the field comparison result is used to optimize the first matching value.

[0306] In a possible implementation manner, the device further includes a calling module, configured to call a large language model so that the large language model executes the steps performed by each module included in the query device for the data table.

[0307] Reference is made below Figure 7 , which shows a schematic structural diagram of an electronic device 700 suitable for implementing embodiments of the present application. The terminal device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (portable android devices, tablet computers), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs (televisions), desktop computers, etc. Figure 7 The electronic device shown is only an example and should not impose any limitations on the functions and usage scopes of the embodiments of the present application.

[0308] As Figure 7 shown, the electronic device 700 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 701, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0309] Generally, the following devices can be connected to the I / O interface 705: input devices including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and a communication device 709. The communication device 709 can allow the electronic device 700 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 7 the electronic device 700 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices can be implemented or had.

[0310] In particular, according to an embodiment of the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above functions defined in the method for constructing a table knowledge base or the method for querying a data table of the embodiments of the present application are executed.

[0311] The electronic device provided by the embodiment of the present application and the method for constructing a table knowledge base and the method for querying a data table provided by the above embodiment belong to the same inventive concept. Technical details not described in detail in this embodiment can be seen in the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0312] Based on the method for constructing a table knowledge base and the method for querying a data table provided by the above method embodiment, an embodiment of the present application provides a computer storage medium, on which a computer program is stored, wherein when the program is executed by a processor, the method for constructing a table knowledge base or the method for querying a data table as described in any of the above embodiments is implemented.

[0313] It should be noted that the above-mentioned computer-readable medium in the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, and this computer-readable signal medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0314] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (Hyper Text Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed network.

[0315] The above-mentioned computer-readable medium can be included in the above-mentioned electronic device; or it can exist separately without being assembled into the electronic device.

[0316] The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device is caused to execute the above-mentioned method for constructing a table knowledge base or the method for querying a data table.

[0317] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0318] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks can occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0319] The units involved in the embodiments described in this application can be implemented in software or in hardware. Among them, the name of the unit / module does not constitute a limitation to the unit itself in some cases. For example, the voice data acquisition module can also be described as the "data acquisition module".

[0320] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, the exemplary types of hardware logic components that can be used include: field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), system on a chip (SOC), complex programmable logic devices (CPLD), and so on.

[0321] In the context of the present application, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0322] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. For the same or similar parts among the various embodiments, reference may be made to each other. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and reference may be made to the description in the method part for the relevant parts.

[0323] It should be understood that in the present application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" may mean: only A exists, only B exists, and both A and B exist simultaneously. Here, A and B may be singular or plural. The character " / " generally indicates an "or" relationship between the associated objects before and after. "At least one (one) of the following" or its similar expression refers to any combination of these items, including any combination of single items (ones) or plural items (ones). For example, at least one (one) of a, b, or c may mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c may be single or multiple.

[0324] It should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0325] The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for constructing a knowledge base of a watch, characterized in that, The method includes: Obtaining historical query data, which is used to record the query process of a historical query data table; Based on the historical query data, extracting table knowledge of the data table, where the table knowledge includes one or more of the business scenario information, query problem information, and field characteristic information of the data table. The business scenario information includes information related to the business scenario to which the data table belongs, the query problem information includes information related to the historical query problems of the data table, and the field characteristic information includes information related to the fields included in the data table; Writing the table knowledge into a table knowledge base, which is used to assist in querying data tables that meet the query requirements.

2. The method according to claim 1, characterized in that, The table knowledge includes the business scenario information of the data table. The extracting the table knowledge of the data table based on the historical query data includes: Obtaining, from the historical query data, a set of business problems of the data table, where the set of business problems includes business problems for the data table, and the business problems are historical query problems for querying the business; Analyzing the business problems included in the set of business problems to obtain the business scenario information of the data table.

3. The method according to claim 2, wherein The analyzing the business problems included in the set of business problems to obtain the business scenario information of the data table includes: For a single data table, analyzing the business problems of the data table to obtain first business information of the data table, where the first business information is used to describe the business scenario of the data table; And / or For multiple data tables, analyzing the business problems of the multiple data tables to obtain second business information of each data table included in the multiple data tables, where the second business information is used to describe the difference in the business scenario between the data table and other data tables in the multiple data tables.

4. The method according to claim 1, wherein The table knowledge includes one or more of the field characteristic information and the business scenario information. The extracting the table knowledge of the data table based on the historical query data includes: Obtaining the basic field information of the fields included in the data table, and obtaining field problems from the historical query data, where the basic field information is used to describe the fields, and the field problems are historical query problems for querying the fields; Analyzing the basic field information and the field problems to obtain the field knowledge of the data table, where the field knowledge includes one or more of the field combination type, the relationship between fields, the field usage frequency, and the field type distribution; Generating the table knowledge according to the field knowledge of the data table.

5. The method according to claim 4, wherein The table knowledge includes the field characteristic information, and the field knowledge includes the field type distribution. The generating the table knowledge according to the field knowledge of the data table includes: Determining the data information of the data table according to the field type distribution, where the data information is used to describe the data included in the data table; And / or Aggregating the field knowledge according to the semantics of the fields to obtain the aggregated field knowledge; And / or Analyze the field knowledge of the same fields included in different data tables to obtain a table comparison result, where the table comparison result is used to describe the differences and association relationships at the field level between different data tables; The data information, the aggregated field knowledge, and the table comparison result belong to the field characteristic information.

6. The method according to claim 4, characterized in that, The table knowledge includes the business scenario information. Generating the table knowledge based on the field knowledge of the data table includes: Analyze the field combination type to determine the business entity corresponding to the data table; and / or Analyze the relationship between fields to determine the business processing logic of the data table; and / or Analyze the field usage frequency to determine the business dimension to which the data table belongs; The business entity corresponding to the data table, the business processing logic of the data table, and the business dimension to which the data table belongs belong to the business scenario information of the table.

7. The method according to claim 1, wherein The table knowledge includes the query problem information. Extracting the table knowledge of the data table based on the historical query data includes: Obtain the association relationship between the data table and the feature words, where the feature words are extracted from the historical query problems of the data table; Analyze the association relationship between the data table and the feature words to determine the key feature words of the data table. The key feature words of the data table belong to the query problem information, and the key feature words are feature words whose association degree with the table is greater than or equal to the association degree threshold.

8. The method according to claim 7, wherein The analyzing the association relationship between the data table and the feature words to determine the key feature words of the data table includes: Determine the association degree value between the feature word and the data table. The association degree value is determined according to one or more of support, confidence, and coverage. The support is used to measure the probability of co-occurrence of the feature word and the data table. The confidence is used to measure the probability that the historical query result corresponding to the historical query problem includes the data table when the historical query problem includes the feature word. The coverage is used to measure the semantic generalization ability of the feature word for the historical query problem; Use the feature words with an association degree value greater than or equal to the association degree threshold as key feature words.

9. The method according to any one of claims 1 to 8, characterized in that, The extracting the table knowledge of the data table based on the historical query data is executed by calling a large language model.

10. The method according to any one of claims 1-8, characterized in that, The method further includes: If an update condition is met, update the table knowledge included in the table knowledge base. The update condition includes one or more of the following: Determine that the content of the data table has changed, reach a preset moment for full-update of the table knowledge base, and obtain the evaluation content for the query result.

11. A query method for a data table, characterized in that, The method includes: Obtain a query problem for describing the query requirement; Determine a target data table that matches the query problem based on the table knowledge of the data tables included in the table knowledge base and the semantic information of the query problem. The table knowledge base is generated according to the construction method of the table knowledge base described in any one of claims 1-10. The table knowledge includes one or more of the business scenario information, query problem information, and field characteristic information of the data table. The business scenario information includes information related to the business scenario to which the data table belongs. The query problem information includes information related to the historical query problems of the data table. The field characteristic information includes information related to the fields included in the data table; Generate a query result including the target data table.

12. The method according to claim 11, wherein The determining of a target data table that matches the query problem according to the table knowledge of the data tables included in the table knowledge base and the semantic information of the query problem includes: Determine a first matching value between the data table and the query problem according to the table knowledge of the data tables included in the table knowledge base and the semantic information of the query problem. The first matching value is used to measure the matching degree between the data table and the query problem; Use the data tables whose matching values meet the matching conditions as the target data tables.

13. The method according to claim 12, wherein The table knowledge at least includes the business scenario information of the data table. The determining of a target data table that matches the query problem according to the table knowledge of the data tables included in the table knowledge base and the semantic information of the query problem includes: Determine the business similarity between the data table and the query problem according to the business scenario information of the data tables included in the table knowledge and the semantic information of the query problem; Use the data tables whose business similarities meet the business similarity conditions as candidate data tables; For each type of information included in the table knowledge of the candidate data tables, determine the sub-matching values between the candidate data tables and the query problem for each type of information; According to the sub-matching values, determine a second matching value between the candidate data tables and the query problem. The second matching value is used to measure the matching degree between the candidate data tables and the query problem.

14. The method according to any one of claims 11-13, characterized in that, The method further includes: Determine target fields related to the semantic information of the query problem; Generate an extended problem according to the field knowledge of the target fields. The extended problem is used to query data tables related to the target fields. The field knowledge of the target fields is used to describe the target fields; Determine a third matching value between the data table and the extended problem. The third matching value is used to measure the matching degree between the extended problem and the data table. The third matching value is used to optimize the first matching value between the query problem and the data table.

15. The method according to any one of claims 11-13, characterized in that The method further includes: Determine a combination of similar data tables. The combination of similar data tables includes at least two similar data tables. The at least two similar data tables are data tables whose difference in the first matching value is less than or equal to the difference threshold; Perform a field-level comparison of the at least two similar data tables according to the field characteristic information of the similar data table to obtain a field comparison result, where the field comparison result includes one or more of a field semantic comparison result, a field statistical feature comparison result, a field function comparison result, and a field distribution comparison result, and the field comparison result is used to optimize the first matching value.

16. The method according to any one of claims 11 - 13, characterized in that, The query method of the data table is executed by invoking a large language model.

17. An apparatus for constructing a table knowledge base, characterized in that The device includes: An acquisition module, configured to acquire historical query data, where the historical query data is used to record the query process of the historical query data table; An extraction module, configured to extract the table knowledge of the data table based on the historical query data, where the table knowledge includes one or more of the business scenario information, query problem information, and field characteristic information of the data table, the business scenario information includes information related to the business scenario to which the data table belongs, the query problem information includes information related to the historical query problem of the data table, and the field characteristic information includes information related to the fields included in the data table; A construction module, configured to write the table knowledge into a table knowledge base, where the table knowledge base is used to assist in querying a data table that meets the query requirements.

18. A query device for a data table, characterized in that, The device includes: An acquisition module, configured to acquire a query problem for describing the query requirements; A query module, configured to determine a target data table that matches the query problem according to the table knowledge of the data tables included in the table knowledge base and the semantic information of the query problem, where the table knowledge base is generated according to the table knowledge base construction method described in any one of claims 1-10, and the table knowledge includes one or more of the business scenario information, query problem information, and field characteristic information of the data table, the business scenario information includes information related to the business scenario to which the data table belongs, the query problem information includes information related to the historical query problem of the data table, and the field characteristic information includes information related to the fields included in the data table; A generation module, configured to generate a query result including the target data table.

19. An electronic device, characterized in that, Includes: One or more processors; A storage device, on which one or more programs are stored. When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any one of claims 1-10, or implement the method described in any one of claims 11-16.

20. A computer-readable medium, characterized in that, A computer program is stored thereon, where the program, when executed by a processor, implements the method described in any one of claims 1-10, or implements the method described in any one of claims 11-16.

Images