Natural language data query method and apparatus, electronic device, and storage medium

Abstract

The application provides a natural language data query method and device, electronic equipment and storage medium, which are applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation and vehicle-mounted devices. The method comprises the following steps: performing entity extraction on a natural language query text according to preset entity resources in a preset target field to obtain a target entity set; obtaining a mapping column value corresponding to each target entity in the target entity set by constructing a mapping relationship between each target entity in the target entity set and a column value in a preset data table; performing a structured query statement prediction on the natural language query text based on the target entity set, the mapping column value corresponding to each target entity and a column name in the preset data table to obtain a to-be-queried statement; and performing a query on the preset data table according to the to-be-queried statement to obtain a query result corresponding to the natural language text. Through the application, the accuracy of data query using natural language can be improved.

Description

Technical Field

[0001] This application relates to artificial intelligence technology, and more particularly to a natural language data query method, apparatus, electronic device, and storage medium. Background Technology

[0002] Natural Language to SQL (NL2SQL) is a type of semantic parsing task that aims to convert user-input natural language questions into structured query language (SQL) statements that can be used to manipulate databases. This enables interaction with the database and retrieval of information from the database using natural language. NL2SQL methods typically rely on semantic recognition of natural language, extracting column values ​​from the natural language based on table column names to generate SQL statements. However, NL2SQL is often used in specialized fields such as finance and education. When migrating NL2SQL to general domains like TableQA and DuSQL, the semantic recognition performance in the new domain may be poor, leading to lower accuracy in column value extraction based on semantic recognition. This reduces the accuracy of generating SQL statements from natural language and consequently, the accuracy of data retrieval using SQL statements. Summary of the Invention

[0003] This application provides a natural language data query method, apparatus, electronic device, and storage medium, which can improve the accuracy of natural language data query.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a natural language data query method, including:

[0006] Based on the preset entity resources of the preset target domain, entity extraction is performed on the natural language query text to obtain the target entity set;

[0007] By constructing a mapping relationship between each target entity in the target entity set and column values ​​in a preset data table, the mapping column value corresponding to each target entity is obtained;

[0008] Based on the target entity set, the mapping column value corresponding to each target entity, and the column name in the preset data table, the natural language query text is used to predict the structured query statement to obtain the query statement;

[0009] The query is performed in the preset data table according to the query statement to obtain the query result corresponding to the natural language text.

[0010] This application provides a natural language data query device, including:

[0011] The extraction module is used to extract entities from natural language query text based on preset entity resources in a preset target domain, and obtain a target entity set.

[0012] The mapping module is used to obtain the mapping column value corresponding to each target entity by constructing a mapping relationship between each target entity in the target entity set and the column value in a preset data table;

[0013] The prediction module is used to predict the structured query statement of the natural language query text based on the target entity set, the mapping column value corresponding to each target entity and the column name in the preset data table, so as to obtain the query statement;

[0014] The query module is used to perform a query on the preset data table based on the query statement to obtain the query results corresponding to the natural language text.

[0015] In the above apparatus, the preset entity resources include at least one of a preset entity library and a preset entity recognition model; the preset entity recognition model is a network model trained using entity data in the preset target domain; the extraction module is further configured to match entities in the preset entity library in the natural language query text, and if the entity exists in the natural language query text, use the entity as the target entity to obtain a target entity set; and / or, through the preset entity recognition model, perform entity recognition and extraction on the natural language query text to obtain the target entity set.

[0016] In the above-described device, the mapping module is further configured to, for target entities obtained from the preset entity library, obtain the mapping column value corresponding to each target entity according to a preset correspondence between entities and column values; the preset correspondence between entities and column values ​​is a pre-constructed correspondence between each entity in the preset entity library and the column value in the preset data table;

[0017] For each target entity obtained according to the preset entity recognition model, a first similarity is calculated between each target entity and each column value in the preset data table, and the mapping column value corresponding to each target entity is obtained based on the first similarity.

[0018] In the aforementioned apparatus, the prediction module is further configured to: perform semantic encoding and decoding processing on the natural language query text and each target entity to obtain a statement vector representation corresponding to the natural language query text and an entity vector representation of each target entity; select target column values ​​from each column of data in the preset data table, and perform semantic encoding and decoding processing on the column name and the target column value of each column of data to obtain a column name vector representation and a target column value vector representation; combine the column name vector representation and the target column value vector representation to obtain a column vector representation of each column of data; and perform structured query statement prediction on the natural language query text based on the statement vector representation, the entity vector representation of each target entity, and the column vector representation of each column of data, combined with the mapping column value corresponding to each target entity, to obtain the query statement.

[0019] In the above apparatus, the prediction module is further configured to predict the query field in the structured query statement prediction based on the statement vector representation and the column vector representation of each column of data, to obtain the query field prediction result; predict the condition field in the structured query statement prediction based on the statement vector representation, the entity vector representation of each target entity, and the column vector representation of each column of data, combined with the mapping column value corresponding to each target entity, to obtain the condition field prediction result; and combine the query field prediction result and the condition field prediction result to obtain the query statement.

[0020] In the above apparatus, the prediction module is further configured to perform classification prediction on the statement vector representation for at least one preset query quantity, to obtain a first probability that the statement vector representation corresponds to each preset query quantity; take the preset query quantity corresponding to the highest first probability as the number of query fields corresponding to the structured query statement; perform query target prediction on each column of data in the preset data table according to the column vector representation, to obtain a second probability that each column of data is a query target; select the top number of query field columns from each column of data as target column data according to the second probability from high to low, and take the column name of the target column data as the target query field, to obtain a target query field set; and obtain the query field prediction result based on the target query field set.

[0021] In the above-described apparatus, the prediction module is further configured to perform classification prediction on the column vector representation corresponding to each target query field in the target query field set using at least one preset aggregation function to obtain a third probability for each target query field corresponding to each preset aggregation function; take the preset aggregation function corresponding to the highest third probability as the target aggregation function corresponding to each target query field; and combine each target query field according to the target aggregation function corresponding to each target query field to obtain the prediction result of the query field.

[0022] In the above apparatus, the prediction module is further configured to perform classification prediction on the statement vector representation using at least one preset number of conditions to obtain a fourth probability corresponding to each preset number of conditions; take the preset number of conditions corresponding to the highest fourth probability as the number of condition fields corresponding to the query statement; based on the number of condition fields, obtain the matching relationship between the target entity and the column data from the correspondence obtained by matching the entity vector representation of each target entity with the column vector representation; update the target entity in the matching relationship based on the mapping column value corresponding to each target entity to obtain the condition matching relationship between the column data and the mapping column value; perform at least one preset condition operator prediction on the condition matching relationship to obtain the target condition operator corresponding to the condition matching relationship; and obtain the condition field prediction result based on the target condition operator and the condition matching relationship.

[0023] In the above-described apparatus, the prediction module is further configured to perform similarity calculation between the column vector representation of each column of data and the entity vector representation of each target entity to obtain a second similarity between each column of data and each target entity; for each target entity, the column data corresponding to the maximum second similarity is used as the column data corresponding to each target entity to obtain the correspondence between each target entity and the column data; from the correspondence between each target entity and the column data, the number of correspondences of the precondition fields with high second similarity are selected as the matching relationship between the target entity and the column data.

[0024] In the above-described apparatus, the prediction module is further configured to predict the connection relationship of the condition matching relationship based on the statement vector representation, thereby obtaining the connection relationship between the condition matching relationships; and to obtain the condition field prediction result by combining the connection relationship, the target condition operator, and the condition matching relationship.

[0025] This application provides an electronic device, including:

[0026] Memory, used to store executable instructions;

[0027] The processor, when executing executable instructions stored in the memory, implements the natural language data query method provided in the embodiments of this application.

[0028] This application provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implement the natural language data query method provided in this application.

[0029] This application provides a computer program product, including a computer program or instructions, characterized in that, when the computer program or instructions are executed by a processor, they implement the natural language data query method provided in this application.

[0030] The embodiments of this application have the following beneficial effects:

[0031] By extracting target entities from predefined entity resources within a predefined target domain, the accuracy of target entities extracted from natural language can be greatly improved by utilizing predefined entity resources that are strongly related to the domain. Furthermore, by establishing a mapping relationship between target entities and column values ​​in a predefined data table to be queried, a set of mapped column values ​​is obtained. By combining the set of mapped column values ​​to generate the query statement, the column value information of the predefined data table can be used to effectively correct the column value information in the query statement, thereby improving the accuracy of the predicted query statement and ultimately improving the accuracy of natural language data query based on the query statement. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the current natural language data query methods using related technologies;

[0033] Figure 2 This is an optional structural diagram of the natural language data query system architecture provided in the embodiments of this application;

[0034] Figure 3 This is an optional structural diagram of the natural language data query device provided in the embodiments of this application;

[0035] Figure 4 This is an optional flowchart illustrating the natural language data query method provided in the embodiments of this application;

[0036] Figure 5 This is an optional flowchart illustrating the natural language data query method provided in the embodiments of this application;

[0037] Figure 6 This is an optional flowchart illustrating the natural language data query method provided in the embodiments of this application;

[0038] Figure 7This is an optional flowchart illustrating the natural language data query method provided in the embodiments of this application;

[0039] Figure 8 This is an optional flowchart illustrating the natural language data query method provided in the embodiments of this application;

[0040] Figure 9 This is a schematic diagram of an optional module in the structured query language prediction process of the natural language data query method provided in this application embodiment;

[0041] Figure 10 This is a schematic diagram illustrating an application process of the natural language data query method provided in this application embodiment as an intelligent analysis assistant in a real-world scenario;

[0042] Figure 11 This is a modular flowchart illustrating the natural language processing and intelligent analysis process of the intelligent analysis assistant provided in this application embodiment;

[0043] Figure 12 This is a schematic diagram illustrating the effect of a query result display format provided in an embodiment of this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0046] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0047] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0049] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0050] 1) Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology within computer science used to grasp the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence. Furthermore, AI is used to study the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities. Moreover, AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision technology, speech processing technology, natural language data query technology, and machine learning (ML) / deep learning.

[0051] 2) Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It refers to the study of theories and methods that enable effective communication between humans and computers using natural language. Therefore, natural language data querying is a science integrating linguistics, computer science, and mathematics; consequently, research in this field involves natural language, i.e., the language people use in daily life, thus it is closely related to linguistic research. Natural language data querying technologies typically include Machine Reading Comprehension (MRC), text processing, semantic understanding, machine translation, robot question answering, and knowledge graphs.

[0052] 3) Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory, among others. It studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence. Its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0053] 4) Named Entity Recognition (NER), also known as entity segmentation and entity extraction, is used to locate and classify named entities in text into predefined categories, such as people, organizations, locations, time expressions, quantities, currency values, percentages, etc. Typically, the task of named entity recognition is to identify three main categories (entity, time, and number) and seven subcategories (person names, organization names, place names, time, date, currency, and percentage) of named entities in the text to be processed. In this embodiment, named entity recognition is used to obtain entities of preset entity types, such as person names and place names.

[0054] 5) Bidirectional Encoder Representations from Transformers (BERT): A pre-training technique for NLP. The goal of the BERT model is to train on a large-scale unlabeled corpus to obtain a semantic representation of the text containing rich semantic information, then fine-tune the semantic representation of the text for a specific NLP task, and finally apply it to that NLP task.

[0055] 6) Domain adaptability: refers to the ability of neural network models to adapt quickly to different application domains, such as finance and education.

[0056] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0057] The solutions provided in this application involve technologies such as natural language processing in artificial intelligence, and are specifically illustrated through the following embodiments:

[0058] Currently, such as Figure 1 As shown, related technologies typically transform the SQL statement prediction task into two parts: Select and Where prediction tasks. The Select prediction task may include selecting columns and their corresponding aggregate functions; the Where prediction task may include selecting columns from the Select list, extracting column values ​​from natural language, and predicting the relationships between column values ​​(OP operator relationships). When extracting column values ​​from natural language, related technologies usually use column names as indexes to extract entities from the original natural language. This method often fails to obtain appropriate entity boundaries, especially in domain migration scenarios. Furthermore, completing the above tasks based solely on column names relies entirely on semantic results, making it difficult to guarantee the model's correctness. Therefore, current NL2SQL technologies, when applied to domain migration scenarios, struggle to improve the model's domain adaptability, thus reducing the accuracy of the SQL statements obtained from NL2SQL transformations in the new domain, and consequently reducing the accuracy of data queries based on these SQL statements.

[0059] This application provides a natural language data query method, apparatus, electronic device, and storage medium, which can improve the efficiency and accuracy of natural language data query. The following describes exemplary applications of the electronic device provided in this application. The electronic device provided in this application can be implemented as various types of user terminals such as smartphones, smartwatches, laptops, tablets, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), intelligent voice interaction devices, smart home appliances, and in-vehicle terminals, or it can be implemented as a server. The following describes exemplary applications when the electronic device is implemented as a server.

[0060] See Figure 2 , Figure 2This is an optional architecture diagram of the natural language data query system 100 provided in the embodiments of this application. The terminal 400 is connected to the server 200 through the network 300, which can be a wide area network or a local area network, or a combination of the two.

[0061] The terminal 400 has a web client or application 410 running on it, which is used to receive natural language input by the user through voice or text, obtain the natural language query text corresponding to the natural language, and send the natural language query text to the server 200.

[0062] Server 200 is used to extract entities from natural language query text based on preset entity resources in a preset target domain, obtaining a target entity set; by constructing a mapping relationship between each target entity in the target entity set and column values ​​in a preset data table, the corresponding mapping column value for each target entity is obtained; the preset data table can be stored in database 500; based on the target entity set, the mapping column value corresponding to each target entity, and the column names in the preset data table, structured query statement prediction is performed on the natural language query text to obtain the query statement; the query statement is then used to query the preset data table to obtain the query results corresponding to the natural language text. Furthermore, server 200 can push the data query results to terminal 400 for display in the web client or application 410 of terminal 400.

[0063] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Terminal 400 may be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, smart voice interaction device, smart home appliance, or in-vehicle terminal, but is not limited to these. Terminals and servers can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0064] See Figure 3 , Figure 3 This is a schematic diagram of the structure of the server 200 provided in the embodiments of this application. Figure 3The server 200 shown includes at least one processor 210, memory 250, at least one network interface 220, and a user interface 230. The various components in server 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to implement communication between these components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 3 The general labeled all buses as Bus System 240.

[0065] Processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0066] User interface 230 includes one or more output devices 231 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 230 also includes one or more input devices 232, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0067] The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 250 may optionally include one or more storage devices physically located away from the processor 210.

[0068] The memory 250 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 250 described in this application embodiment is intended to include any suitable type of memory.

[0069] In some embodiments, memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0070] Operating system 251 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0071] The network communication module 252 is used to reach other computing devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0072] Presentation module 253 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 231 associated with user interface 230 (e.g., a display screen, a speaker, etc.).

[0073] The input processing module 254 is used to detect and translate one or more user inputs or interactions from one or more input devices 232.

[0074] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 3 A natural language data query device 255 stored in memory 250 is shown. It can be software in the form of programs and plug-ins, including the following software modules: extraction module 2551, mapping module 2552, prediction module 2553 and query module 2554. These modules are logical and can therefore be arbitrarily combined or further split according to the functions implemented.

[0075] The functions of each module will be explained below.

[0076] In other embodiments, the apparatus provided in this application can be implemented in hardware. For example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the natural language data query method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0077] In some embodiments, the terminal or server can implement the natural language data query method provided in this application by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), that is, a program that needs to be installed in the operating system to run, such as a social application APP or a messaging APP; it can also be a mini-program, that is, a program that only needs to be downloaded to a browser environment to run; or it can be a mini-program or web client program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plugin.

[0078] The natural language data query method provided in this application will be described in conjunction with exemplary applications and implementations of the server provided in the embodiments of this application. Furthermore, the natural language data query method provided in this application can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and in-vehicle systems.

[0079] See Figure 4 , Figure 4 This is an optional flowchart illustrating the natural language data query method provided in this application embodiment, which will be combined with... Figure 4 The steps shown are explained.

[0080] S101. Based on the preset entity resources of the preset target domain, extract entities from the natural language query text to obtain the target entity set.

[0081] The natural language data query method provided in this application can be applied to scenarios where natural language is used to query data in a database in a preset target domain. For example, users can have human-computer dialogue with AI-based electronic customer service, or use a table-based intelligent question-and-answer or dialogue system, etc. The specific method can be selected according to the actual situation, and this application does not limit it.

[0082] In this embodiment of the application, the preset target domain is the application domain corresponding to the data query, and the preset entity resources of the preset target domain can be entity data resources constructed based on the relevant application data in the preset target domain.

[0083] In some embodiments, the preset entity resources may include a preset entity library. For example, if the preset target domain is a multimedia domain, the corresponding preset entity library may be a multimedia type entity library, which contains multiple multimedia type entities, or it may also contain alias information corresponding to the entities.

[0084] In this embodiment of the application, when the preset entity resource is a preset entity library, the server can match each entity in the preset entity library with the natural language query text. If the entity exists in the natural language query text, the match is determined to be successful. If the match is successful, the entity is taken as the target entity, thereby obtaining the target entity set.

[0085] In some embodiments, the server can pre-build and train an entity matching extraction model based on an entity library, and then use the entity matching extraction model to perform entity matching on the natural language query text according to the entities in the preset entity library, and extract the successfully matched entities to obtain the target entity set.

[0086] In some embodiments, the preset entity resource may include a preset entity recognition model pre-trained in a preset target domain. Here, the preset entity recognition model is trained using entity data from the preset target domain and is capable of recognizing the corresponding target entities from natural language query text using the learned feature representations or semantic representations of entities in the preset target domain.

[0087] In some embodiments, both the entity matching extraction model and the preset entity recognition model described above can be implemented using a sequence-to-sequence (Seq2Seq) neural network architecture. For example, the preset entity recognition model can be implemented by constructing a character-word union convolutional neural network (CWCNN); or by constructing a long short-term memory (LSTM) network combined with a conditional random field (CRF) model, or other Seq2Seq type neural network models. The specific choice depends on the actual situation, and this application embodiment does not limit the choice.

[0088] In this embodiment, the server can use a preset entity library to extract entities from the natural language query text to obtain a target entity set; it can also use a preset entity recognition model to extract entities from the natural language query text to obtain a target entity set; or it can combine the extraction results of the preset entity library and the preset entity recognition model. For example, the extraction results of the preset entity library and the preset entity recognition model can be merged to obtain a target entity set. The specific choice is made according to the actual situation, and this embodiment does not limit it.

[0089] It is understood that, since the embodiments of this application use preset entity resources in a preset target domain to extract natural language query text, compared with the related technologies that use column names of data tables to extract column values, the target entity set obtained by the method of the embodiments of this application has stronger domain relevance, thereby helping to improve domain adaptability and product compatibility.

[0090] S102. By constructing a mapping relationship between each target entity in the target entity set and the column values ​​in the preset data table, the mapping column value corresponding to each target entity is obtained.

[0091] In this embodiment of the application, when the server obtains the target entity set, it can map each target entity contained in the natural language to the column value contained in the preset data table to obtain the column value corresponding to each target entity in the preset data table, and use it as the mapping column value corresponding to each target entity, thereby obtaining the set of mapping column values ​​corresponding to the target entity set, and constructing the mapping relationship between the target entities in the natural language and the column values ​​in the preset data table.

[0092] In some embodiments, the server can pre-construct a correspondence between each entity in the entity library and column values ​​in a preset data table, thereby obtaining the corresponding column value for each entity in the entity library in the preset data table. Exemplarily, this can be achieved using the BM25 algorithm, or other text similarity algorithms, depending on the specific circumstances; this application does not limit the choice. Thus, for target entities extracted from the preset entity library within the target entity set, the server can obtain the mapped column value for each target entity based on the preset correspondence between entities and column values.

[0093] In some embodiments, the server may pre-build and train a preset similarity model. For example, the preset similarity model may be a BERT model or a classification model based on text features, which can be used to infer the similarity between the target entity and each column value in a preset data table, as the first similarity. Thus, the server can obtain the mapping column value corresponding to each target entity based on the first similarity. For example, the server may use the column value with the highest first similarity to the target entity in the preset data table as the target column value; or, the server may use column values ​​greater than or equal to a preset similarity threshold as the mapping column value corresponding to the target entity; or, the server may combine the highest similarity with the preset similarity threshold to obtain the mapping column value. The specific choice depends on the actual situation, and this application embodiment does not limit the specific choice.

[0094] S103. Based on the target entity set, the mapping column value corresponding to each target entity and the column name in the preset data table, perform structured query statement prediction on the natural language query text to obtain the query statement.

[0095] In this embodiment, the target entity set represents the query information extracted from the user's natural language query text that is strongly related to a preset data table in a preset target domain. The server can match each target entity in the target entity set with the corresponding mapping column value and the column name in the preset data table to determine the column name and column value corresponding to the query information. Furthermore, through multiple classification prediction tasks, the query relationship between column names and the conditional relationship between column names and column values ​​are predicted. The output results of multiple tasks are combined to form an SQL statement, thereby realizing the prediction of structured query statements from natural language query text and obtaining the query statement.

[0096] In some embodiments, the server can extract semantic features from the target entity set and column names in a preset data table, such as performing semantic encoding and decoding to obtain a semantic representation of each target entity and each column name. For example, the semantic representation can be embedding information in vector form. Thus, the server can predict which columns in the preset data table need to be queried for the natural language query text based on the semantic representation of each target entity and each column name. For example, predicting the Select clause portion of the SQL statement, including predicting the Select column selection and the aggregate function corresponding to the Select column, etc. The server can also predict the query conditions contained in the natural language query text, i.e., the query conditions between column names and column values, and the relationships between various query conditions, based on the semantic representation of each target entity and each column name, combined with the mapping column value corresponding to each target entity. For example, predicting the WHERE clause portion of the SQL statement can include predicting WHERE column selection, the relationship between columns and column values ​​in the WHERE condition (op relationship), and the connection relationship between multiple WHERE conditions. The server can combine the prediction results of the Select clause portion and the WHERE clause portion to obtain a complete structured query statement as the query statement to be queried.

[0097] For example, the natural language query text entered by the user could be "the number of chemistry teachers at the First Affiliated High School of Peking University". The preset data table could be a teacher data table, as shown in Table 1 below, with column names including "School Name", "Year", "School District", "Gender", "Teaching Subject", and "Teacher ID", as follows:

[0098]

[0099] Table 1

[0100] When the server receives the natural language query text, it can extract target entities such as "First University Affiliated High School," "Chemistry Teacher," and "Quantity" from the natural language query text through the process S101 described above. Then, through the process S102 described above, it obtains the mapping column value "First University Affiliated High School" corresponding to "First University Affiliated High School," thereby mapping the entities in the natural language to column values ​​in a pre-set data table and resolving the aliasing problem. Using "First University Affiliated High School," "Chemistry Teacher," and "Quantity" as the extracted column values, the SQL statement prediction is decomposed into multiple tasks, and the structured query statement is predicted using at least one pre-trained classification prediction network. For example, the server can use a quantity classification prediction network to predict the number of Select and WHERE fields in the SQL statement, resulting in a prediction that the number of Select fields is 1 and the number of WHERE fields is 2. Furthermore, using a binary classification prediction network, the server can predict that the probability of Select selection for each column in Table 1 is [1, 0, 0, 0] and the probability of WHERE selection is [0, 0.5, 0.5, 0], with the WHERE operator (op) for each column being [None, =, =, None]. Using a multi-class prediction network, the server can predict that the Select aggregate function for each column in Table 1 is [Count, None, None, None], and the relationship between the WHERE condition fields is "AND". Finally, by concatenating the prediction results from multiple tasks, the server obtains the final SQL statement "Select Count(Teacher id)where School Name = First University Affiliated Middle School and Teaching Subject = Chemistry", which serves as the query statement.

[0101] S104. Perform a query in the preset data table based on the query statement to obtain the query results corresponding to the natural language text.

[0102] In this embodiment of the application, the server can use the query statement to query in a preset data table to obtain the query result of the SQL statement, which is used as the query result corresponding to the natural language text.

[0103] It is understandable that extracting target entities by using preset entity resources in a predefined target domain can greatly improve the accuracy of target entities extracted from natural language by utilizing these domain-relevant resources. Furthermore, by establishing a mapping relationship between target entities and column values ​​in a predefined data table to be queried, a set of mapped column values ​​is obtained. By combining this set of mapped column values ​​to generate the query statement, the column value information in the predefined data table can be used to effectively correct the column value information in the query statement, thereby improving the accuracy of the predicted query statement and ultimately enhancing the accuracy of natural language data queries based on the query statement.

[0104] In some embodiments, based on Figure 4 See Figure 5 , Figure 4 S103 can be implemented through S201-S204, which will be explained in conjunction with each step.

[0105] S201. By performing semantic encoding and decoding on the natural language query text and each target entity, the statement vector representation corresponding to the natural language query text and the entity vector representation of each target entity are obtained.

[0106] In this embodiment, the server can perform semantic encoding and decoding on the natural language query text to obtain a semantic representation in vector form corresponding to the entire natural language query text, which serves as the statement vector identifier. The server can also perform semantic encoding and decoding on each target entity in the target entity set to obtain a semantic vector representation corresponding to each target entity, which serves as the entity vector representation corresponding to each target entity.

[0107] In some embodiments, the server can use a neural network model containing encoder and decoder structures, such as the Transformer model or the BERT model, to perform semantic encoding and decoding processing on natural language query text and target entities. Alternatively, it can use related semantic encoding and decoding algorithms. The specific choice depends on the actual situation and is not limited in the embodiments of this application.

[0108] S202. Select the target column value from each column of data in the preset data table, and perform semantic encoding and decoding processing on the column name and target column value of each column to obtain the column name vector representation and the target column value vector representation.

[0109] In this embodiment of the application, the server may select the target column value from each column of data in the preset data table in a random manner, or select the column value with the high frequency of historical queries in each column of data as the target column value based on historical queries, or select the target column value for each column of data based on other selection strategies. The specific selection is made according to the actual situation, and this embodiment of the application does not limit it.

[0110] In this embodiment of the application, the number of target column values ​​can be one or more, and can be set according to factors such as the processing capacity of the device, resource usage, and latency limitations. This embodiment of the application does not impose any restrictions on this.

[0111] In this embodiment of the application, the server can perform semantic encoding and decoding processing on each column name and the corresponding selected target column value in each column data to obtain the column name vector representation and the target column value vector representation corresponding to each column name.

[0112] S203. Combining the column name vector representation and the target column value vector representation, we obtain the column vector representation of each column of data in the preset data table.

[0113] In this embodiment of the application, the server can combine the column name vector representation with the target column value vector representation, thereby using the column values ​​in each column of data to enhance the semantic representation of the column data, and obtain the column vector representation of each column of data in the preset data table.

[0114] In some embodiments, the server may use the average of the column name vector representation and the target column value vector representation as the column vector representation of each column of data. Alternatively, the server may also perform weighted processing on the column name vector representation and the target column value vector representation according to preset weights to obtain the column vector representation of each column of data. The specific choice depends on the actual situation, and this application embodiment does not limit it.

[0115] S204. Based on the statement vector representation, the entity vector representation of each target entity, and the column vector representation of each column of data, and combined with the mapping column value corresponding to each target entity, perform structured query statement prediction on the natural language query text to obtain the query statement.

[0116] In this embodiment, the server can perform various query task predictions based on the statement vector representation, the entity vector representation of each target entity, and the column vector representation of each column of data obtained in the above process, combined with the mapping column value corresponding to each target entity, thereby realizing structured query statement prediction of natural language query text and obtaining the query statement.

[0117] In some embodiments, S204 can be implemented by S2041-S2043, which will be described in conjunction with each step.

[0118] S2041. Based on the statement vector representation and the column vector representation of each column of data, perform query field prediction in structured query statement prediction to obtain the query field prediction result.

[0119] In this embodiment of the application, the server can predict which columns of data in a preset data table the natural language query text will query based on the statement vector representation and the column vector representation of each column of data, that is, perform query field prediction in structured query statement prediction, and obtain query field prediction results.

[0120] In some embodiments, the predicted result of a query field can be a SELECT statement portion of an SQL statement.

[0121] S2042. Based on the statement vector representation, the entity vector representation of each target entity, and the column vector representation of each column of data, combined with the mapping column value corresponding to each target entity, the condition field prediction in the structured query statement prediction is performed to obtain the condition field prediction result.

[0122] In this embodiment of the application, the server can predict the query conditions contained in the natural language query text and the relationship between the various query conditions based on the statement vector representation, the entity vector representation of each target entity, and the column vector representation of each column of data, combined with the mapping column value corresponding to each target entity. That is, it can perform condition field prediction in structured query statement prediction and obtain the condition field prediction result.

[0123] In some embodiments, the prediction result of the condition field can be the WHERE clause portion of the SQL statement.

[0124] S2043. Combine the prediction results of the query field with the prediction results of the condition field to obtain the query statement.

[0125] In this embodiment of the application, the server can combine the prediction results of the query field with the prediction results of the condition field by splicing or connecting them to obtain the query statement.

[0126] It is understandable that by selecting target column values ​​from each column of data and combining the semantic representations of the target column values ​​and column names to form the semantic representation of the column data, the accuracy of column vector representation can be greatly improved, thereby improving the accuracy of query statement prediction based on column vector representation. Furthermore, compared to representing columns by column names in existing technologies, the method in this embodiment can significantly improve the generalization ability of tables. When applied to new domains, especially highly specialized ones, it can ensure the accuracy of extracting query fields and condition fields from natural language based on column vector representation, improving the model's domain adaptability. Moreover, by incorporating the entity information of the preset target domain added during the semantic encoding stage in this embodiment, the model's domain transfer capability can be further improved.

[0127] In some embodiments, such as Figure 6 As shown, S2041 can be implemented through S301-S305, which will be explained in conjunction with each step.

[0128] S301. Perform at least one preset query quantity classification prediction on the statement vector representation to obtain the first probability of the statement vector representation corresponding to each preset query quantity.

[0129] S302. The preset number of queries corresponding to the highest first probability is used as the number of query fields corresponding to the structured query statement.

[0130] In this embodiment, the server performs at least one preset query quantity classification prediction on the statement vector representation that represents the overall semantic representation of the natural speech query text, so as to predict the number of query fields contained in the natural speech query text and obtain the probability of each preset query quantity in the statement vector representation corresponding to at least one preset query quantity, as the first probability; then, the server takes the preset query quantity corresponding to the highest first probability as the number of query fields corresponding to the structured query statement.

[0131] In some embodiments, the server can utilize a multi-class classification prediction network to predict the number of queries. Here, the at least one preset query quantity corresponding to the multi-class classification prediction network can represent the preset number of categories that the multi-class network can predict and the maximum number of categories. For example, the at least one preset query quantity can include values ​​such as 2, 3, 5, etc., that can represent the number of Select fields in the SQL statement. The statement vector representation can be the embedding at the [CLS] position of the natural language query text output by the BERT model. The server uses the multi-class classification prediction network to predict the embedding at the [CLS] position, and uses the preset query quantity with the highest probability in the output prediction results as the predicted value of the Select num part in the SQL statement, i.e., the number of query fields.

[0132] S303. Based on the column vector representation, predict the query target for each column of data in the preset data table to obtain the second probability that each column of data is the query target.

[0133] In this embodiment of the application, the server can predict whether each column of data in the preset data table is the query target corresponding to the natural language query text based on the column vector representation that combines the column name and the target column value, and obtain the probability that each column of data is the query target, as a second probability.

[0134] In some embodiments, the server can utilize a pre-trained binary classification network to predict the probability of each column of data being selected as the query target based on the column vector representation, i.e., perform binary classification prediction. During the training phase of the binary classification network, the server can obtain a training data table, in which each column of the training data table corresponds to the labeled probability of that column being selected as the query target. For example, if the Select section contains two columns, the labeled probabilities of the selected column data in the training data table are 0.5 and 0.5 respectively, and the labeled probabilities of other columns are 0. The server can use the training data table to perform model fitting and model training on the initial binary classification network using loss functions such as KL (Kullback-Leibler divergence) or cross-entropy loss until the preset training conditions are met, thus obtaining the trained binary classification network.

[0135] In this way, the server can use the trained binary classification network to predict whether each column of data in the preset data table is the query target corresponding to the natural language query text, and obtain the second probability that each column of data is the query target.

[0136] S304. Based on the second probability from high to low, select the number of columns of data from each column of data that are the first query fields as the target column data, and use the column name of the target column data as the target query field to obtain the target query field set.

[0137] In some embodiments, the server can select the K columns with the highest second probability as the columns to be selected based on the predicted value K of Select num. For example, when K=2, the Select part contains two columns, and the server can select the two columns with the highest probability from the second probability corresponding to each column of data, and use the column names corresponding to these two columns of data as the target query fields to obtain two target query fields.

[0138] Here, the target query field set contains the number of target query fields.

[0139] S305. Based on the target query field set, obtain the query field prediction results.

[0140] In this embodiment of the application, the server can predict the query field operation part of the query statement based on the obtained target query field set, and obtain the query field prediction result, such as the prediction result of the Select part in the SQL statement.

[0141] In some embodiments, when there is an aggregation relationship between the number of query fields in the target query field set and the target query fields, based on Figure 6 ,like Figure 7 As shown, S305 can be implemented through S3051-S3053, and will be explained in conjunction with each step.

[0142] S3051. Perform classification prediction using at least one preset aggregation function on the column vector representation corresponding to each target query field in the target query field set to obtain the third probability of each target query field corresponding to each preset aggregation function.

[0143] S3052. Use the preset aggregation function corresponding to the highest third probability as the target aggregation function for each target query field.

[0144] In this embodiment, the server can utilize a multi-classification network for classifying and predicting at least one preset aggregation function. Based on the column vector representation corresponding to the target query field, it predicts the probability that each target query field belongs to each preset aggregation function, which is then used as the third probability for each target query field corresponding to each preset aggregation function. Furthermore, the preset aggregation function corresponding to the highest third probability is taken as the target aggregation function for each target query field.

[0145] S3053. Based on the target aggregation function corresponding to each target query field, combine each target query field to obtain the query field prediction result.

[0146] In this embodiment, the server can combine or concatenate a number of target query fields according to the target aggregation function corresponding to each target query field to obtain the query field prediction result. For example, the query field prediction result can be a SELECT statement portion included in an SQL statement.

[0147] In some embodiments, at least one preset aggregation function may include six types: "", "AND", "MAX", "MIN", "COUNT", and "SUM". The server can construct a 6-classification task for the column vector representation corresponding to each target query field, select the largest of the six categories as the aggregation function for the current target query field, and combine the prediction results of the Select column to complete the full prediction of the Select part.

[0148] In some embodiments, such as Figure 8 As shown, S2042 can be implemented through S401-S406, which will be explained in conjunction with each step.

[0149] S401. Perform classification prediction on the statement vector representation using at least one preset number of conditions to obtain the fourth probability of the statement vector representation corresponding to each preset number of conditions.

[0150] S402. Take the number of preset conditions corresponding to the highest fourth probability as the number of condition fields corresponding to the query statement.

[0151] In this embodiment of the application, the server can classify and predict the fields of the conditional relationship type contained in the statement vector representation to obtain the probability of the statement vector representation corresponding to each of the at least one preset condition quantity, which is used as the fourth probability of the statement vector representation corresponding to each preset condition quantity.

[0152] In some embodiments, the classification prediction of the number of conditions represents the prediction of the Where num part in the SQL statement. The server can also use a multi-classification network and predefine the number of condition categories, i.e., there is an upper limit to the maximum number of conditions. The multi-classification network is used to predict the embedding of the statement vector representation, such as the [CLS] position in the BERT output, and the preset number of conditions with the highest probability is used as the predicted value of Where num.

[0153] S403. Based on the number of condition fields, obtain the matching relationship between the target entity and the column data from the correspondence obtained by matching the entity vector representation and the column vector representation of each target entity.

[0154] In this embodiment of the application, the server can use a similarity calculation method to obtain the matching degree between each column of data and each target entity based on the entity vector representation of each column and the entity vector representation of each target entity, thereby determining the column data with the highest matching degree with each target entity. Each target entity corresponds to one column of data, thus obtaining the correspondence between the target entity and the column data.

[0155] In some embodiments, the server can calculate the similarity between the column vector representation of each data column and the entity vector representation of each target entity to obtain a second similarity between each data column and each target entity. For each target entity, the column data corresponding to the maximum second similarity is used as the column data corresponding to each target entity, thus obtaining a correspondence between each target entity and the column data. From the correspondence between each target entity and the column data, the number of correspondences with high second similarity among the precondition fields is selected as the matching relationship between the target entity and the column data. That is, the number of matching relationships can be the number of condition fields.

[0156] For example, the server can calculate the difference between the embedding represented by the column vector and the embedding represented by each entity vector using vector distance calculation methods, such as cosine distance calculation, to obtain the second similarity between each column of data and each target entity. Then, for each target entity, the column data corresponding to the highest second similarity among the second similarities between the target entity and each column of data is taken as the column data corresponding to that target entity, thus obtaining the correspondence between each target entity and column data. The server then selects the correspondence with the highest second similarity among the corresponding correspondences of each target entity and column data, based on the second similarity from high to low, as the matching relationship between the target entity and the column data.

[0157] S404. Update the target entities in the matching relationship based on the mapping column values ​​corresponding to each target entity to obtain the conditional matching relationship between the column data and the mapping column values.

[0158] In this embodiment of the application, the server can update the target entity in the matching relationship by using the mapping column value corresponding to each target entity according to the mapping relationship between each target entity and the column value in the preset data table constructed in S102. For example, the target entity "Yida Fuzhong" is updated to "Yida Fuzhong" according to its corresponding mapping value, thereby realizing the correction of the column value extraction result of natural language, i.e. the target entity, according to the real column value in the data table, and obtaining the conditional matching relationship between column data and mapping column value.

[0159] Here, since the mapped column values ​​are the actual column values ​​in the preset data table, updating the matching relationship between the target entity and the column data based on the mapped column values ​​can greatly improve the accuracy of predicting query statements using the updated matching relationship, i.e., the conditional matching relationship.

[0160] S405. For the conditional matching relationship, predict at least one preset conditional operator to obtain the target conditional operator corresponding to the conditional matching relationship.

[0161] In this embodiment of the application, the server can predict at least one preset condition operator for the obtained condition matching relationship, thereby predicting the condition connection relationship between each column data and the matched mapping column value, and obtaining the target condition operator corresponding to each condition matching relationship.

[0162] In some embodiments, at least one preset conditional operator may include five types: ">", "<", "=", "!=", and "". The server may also use a multi-classification network to predict the conditional operator and select the preset conditional operator with the highest probability as the target conditional operator corresponding to each conditional matching relationship.

[0163] S406. Based on the target conditional operator and the conditional matching relationship, the prediction result of the conditional field is obtained.

[0164] In this embodiment, the server can combine the column name and mapped column value of each conditional matching relationship according to the target conditional operator to obtain the conditional field prediction result. For example, the conditional field prediction result can be the WHERE clause portion of an SQL statement.

[0165] In some embodiments, the server can also use a multi-classification network to predict the connection relationships between conditional matching relationships based on the statement vector representation, such as the embedding at the [CLS] position output by BERT, to obtain the connection relationships between conditional matching relationships. For example, the preset connection relationships may include two categories: AND and OR, and the server can use the category with the highest probability as the predicted value of the Where conditional relationship. Furthermore, the server can obtain the prediction result of the conditional field by combining the connection relationships, the target conditional operator, and the conditional matching relationships.

[0166] In some embodiments, based on the above Figures 5-8 The method described in this application, the structured query prediction process, can be as follows: Figure 9 As shown below:

[0167] The server can utilize the BERT model to perform semantic encoding and decoding on the natural language query text, candidate values ​​1, 2, and 3 in the candidate value set, and the column name (e.g., column name 1) and the target column value selected from that column (e.g., column value 1). Here, the candidate value set is the aforementioned target entity set, and candidate values ​​1, 2, and 3 are each target entity in the target entity set. The server uses the embedding output by the BERT model at the CLS position of the natural language query text as the statement vector representation. Furthermore, based on the positions of candidate values ​​1, 2, and 3 in the natural language query text, the average value of the token embeddings at corresponding positions after BERT processing is calculated to obtain the corresponding candidate value vector representations for candidate values ​​1, 2, and 3, as well as the column name vector representation and the corresponding target column value vector representation for each column (e.g., column name 1 and column value 1, column name 2 and column value 2, not shown in the diagram). The server uses the average of the column name vector representation and the column value vector representation as the column vector representation of the column data. Then, the server can predict the query fields in structured query statement prediction based on the statement vector representation and the column vector representation of each column of data, and obtain the query field prediction results.

[0168] like Figure 9As shown, the server can perform column value matching between each column of data and candidate value vectors corresponding to each candidate value, respectively, based on the column vector representation of each column of data and the candidate value vector representation of each candidate value. The server selects the candidate value with the highest similarity to the column of data as the corresponding candidate value, thus obtaining the matching relationship between the column data and the candidate values. Furthermore, the server uses a pre-constructed correspondence between candidate values ​​and actual column values ​​in a preset data table to refine the matching relationship between the column data and the candidate values, obtaining the conditional matching relationship between the column data and the actual column values. Based on the conditional matching relationship, the server performs prediction of at least one preset conditional operator to obtain the target conditional operator corresponding to each conditional matching relationship. Based on the target conditional operator and the conditional matching relationship, the server obtains the conditional field prediction result. The server can then combine the obtained query field prediction result and conditional field prediction result to obtain the query statement.

[0169] It is understood that, in the embodiments of this application, by correcting the column value extraction results of the target entities extracted from natural language through real mapping column values, the accuracy of column value extraction results can be greatly improved, thereby improving the accuracy of predicting the query statement based on the column value extraction results, and further improving the accuracy of natural language data query based on the query statement.

[0170] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0171] Figure 10 This application provides an exemplary application of the method described in this embodiment to an intelligent analysis assistant in an intelligent human-computer question-answering scenario, such as... Figure 10 As shown, the intelligent analysis assistant can be an application that implements intelligent human-computer question-and-answer based on data tables. It can obtain data conclusions, i.e., query results, through processes such as speech recognition, natural language processing, and intelligent analysis. The natural language data query method provided in this application embodiment can be applied to the natural language processing and intelligent analysis process of the intelligent analysis assistant. Users can launch the intelligent analysis assistant on their terminal and enter... Figure 10 In the question-and-answer dialogue interface shown, the intelligent analysis assistant can first send a welcome message to the user's terminal, such as "Hello, the intelligent analysis assistant is at your service." The user can then input their desired query via voice, such as "How many teachers were there at the First Affiliated High School in 2020?" The intelligent analysis assistant converts the voice query into natural language query text through voice recognition. Then, using the method described in this embodiment, it performs natural language processing and intelligent analysis on the natural language query text to obtain the query result, i.e., the data conclusion, which is then displayed on the user's terminal in a dialogue format. For example, the dialog box displays the user's question and its corresponding answer, such as... Figure 10 As shown in region 600. In some embodiments, a user satisfaction rating control 601 may also be displayed to collect user satisfaction with the search results, thereby further improving model performance.

[0172] Here, the above-mentioned natural language processing and intelligent analysis process can be described as follows: Figure 11 The modular process shown is used to achieve this. The intelligent analysis assistant inputs the natural language query text and table information from the data table into the natural language understanding module. Here, the natural language understanding module can be a functional module used to implement the S101-S102 processes described above. The natural language understanding module obtains a candidate value set, where each candidate value in the set is the target entity, and corresponds to a pre-established mapping column value in the data table. Furthermore, based on the candidate value set and combined with the natural language query text and table information, the NL2SQL module is used. For example, the NL2SQL module can be implemented as described above. Figure 9 As shown, structured query statement prediction is performed to obtain the SQL statement, i.e., the query statement to be executed. For example, for the natural language query text "How many teachers were there at No. 1 High School in 2020?", an SQL statement such as "select count(teacher ID) from teacher_data_table where year=2020 and school_name=No.1 High School" can be obtained. Furthermore, the intelligent analysis assistant can use the SQL statement to perform data query operations on the data table and obtain data conclusions.

[0173] In some embodiments, the intelligent analysis assistant can also display the query results in the form of charts or graphs. For example, the query results obtained from the natural language query text "Query the distribution of full-time teachers in each district of XX city" can be displayed as follows: Figure 12 The diagram 120 shown is designed to enhance the richness of the query results display methods.

[0174] It is understood that in this embodiment of the application, by constructing a natural language understanding module, combining natural language and table information to obtain a set of candidate values, and then using the set of candidate values ​​to perform NL2SQL prediction tasks to obtain SQL statements, when adding a table-based question-answering module to a dialogue system built for a specific domain, it is only necessary to construct a corresponding natural language understanding module for the target domain, and other modules can reuse the results obtained from training in the general domain. This allows for the rapid construction of a table-based question-answering module for the target domain, improves the domain adaptability of the model, enhances the ability of NL2SQL technology to be implemented in different scenarios, and improves the adaptability of the product.

[0175] The following description continues to illustrate the exemplary structure of the natural language data query device 255 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 3 As shown, the software modules stored in the natural language data query device 255 in the memory 250 may include:

[0176] Extraction module 2551 is used to extract entities from natural language query text based on preset entity resources in a preset target domain, and obtain a target entity set;

[0177] The mapping module 2552 is used to obtain the mapping column value corresponding to each target entity by constructing a mapping relationship between each target entity in the target entity set and the column value in the preset data table;

[0178] Prediction module 2553 is used to predict the structured query statement of the natural language query text based on the target entity set, the mapping column value corresponding to each target entity and the column name in the preset data table, so as to obtain the query statement to be queried;

[0179] The query module 2554 is used to perform a query on the preset data table based on the query statement to obtain the query result corresponding to the natural language text.

[0180] In some embodiments, the preset entity resources include at least one of a preset entity library and a preset entity recognition model; the preset entity recognition model is a network model trained using entity data in the preset target domain; the extraction module 2551 is used to match entities in the preset entity library in the natural language query text, and if the entity exists in the natural language query text, to use the entity as the target entity, thereby obtaining a target entity set; and / or, to perform entity recognition and extraction on the natural language query text through the preset entity recognition model to obtain the target entity set.

[0181] In some embodiments, the mapping module 2552 is configured to, for target entities obtained from the preset entity library, obtain a mapping column value corresponding to each target entity according to a preset correspondence between entities and column values; the preset correspondence between entities and column values ​​is a pre-constructed correspondence between each entity in the preset entity library and the column values ​​in the preset data table; for target entities obtained according to the preset entity recognition model, calculate a first similarity between each target entity and each column value in the preset data table, and obtain a mapping column value corresponding to each target entity based on the first similarity.

[0182] In some embodiments, the prediction module 2553 is further configured to: perform semantic encoding and decoding processing on the natural language query text and each target entity to obtain a statement vector representation corresponding to the natural language query text and an entity vector representation of each target entity; select target column values ​​from each column of data in the preset data table, and perform semantic encoding and decoding processing on the column name and the target column value of each column of data to obtain a column name vector representation and a target column value vector representation; combine the column name vector representation and the target column value vector representation to obtain a column vector representation of each column of data; and perform structured query statement prediction on the natural language query text based on the statement vector representation, the entity vector representation of each target entity, and the column vector representation of each column of data, combined with the mapping column value corresponding to each target entity, to obtain the query statement.

[0183] In some embodiments, the prediction module 2553 is further configured to predict query fields in the structured query statement prediction based on the statement vector representation and the column vector representation of each column of data, to obtain query field prediction results; predict condition fields in the structured query statement prediction based on the statement vector representation, the entity vector representation of each target entity, and the column vector representation of each column of data, combined with the mapping column value corresponding to each target entity, to obtain condition field prediction results; and combine the query field prediction results and the condition field prediction results to obtain the query statement.

[0184] In some embodiments, the prediction module 2553 is further configured to perform classification prediction on the statement vector representation for at least one preset query quantity to obtain a first probability that the statement vector representation corresponds to each preset query quantity; take the preset query quantity corresponding to the highest first probability as the number of query fields corresponding to the structured query statement; perform query target prediction on each column of data in the preset data table according to the column vector representation to obtain a second probability that each column of data is a query target; select the top number of query field columns from each column of data as target column data according to the second probability from high to low, and take the column name of the target column data as the target query field to obtain a target query field set; and obtain the query field prediction result based on the target query field set.

[0185] In some embodiments, the prediction module 2553 is further configured to perform classification prediction on the column vector representation corresponding to each target query field in the target query field set using at least one preset aggregation function to obtain a third probability for each target query field corresponding to each preset aggregation function; take the preset aggregation function corresponding to the highest third probability as the target aggregation function corresponding to each target query field; and combine each target query field according to the target aggregation function corresponding to each target query field to obtain the prediction result of the query field.

[0186] In some embodiments, the prediction module 2553 is further configured to perform classification prediction on the statement vector representation for at least one preset number of conditions to obtain a fourth probability corresponding to each preset number of conditions; take the preset number of conditions corresponding to the highest fourth probability as the number of condition fields corresponding to the query statement; obtain the matching relationship between the target entity and the column data from the correspondence obtained by matching the entity vector representation of each target entity with the column vector representation based on the number of condition fields; update the target entity in the matching relationship based on the mapping column value corresponding to each target entity to obtain the condition matching relationship between the column data and the mapping column value; perform at least one preset condition operator prediction on the condition matching relationship to obtain the target condition operator corresponding to the condition matching relationship; and obtain the condition field prediction result based on the target condition operator and the condition matching relationship.

[0187] In some embodiments, the prediction module 2553 is further configured to perform similarity calculation between the column vector representation of each column of data and the entity vector representation of each target entity to obtain a second similarity between each column of data and each target entity; for each target entity, the column data corresponding to the maximum second similarity is used as the column data corresponding to each target entity to obtain the correspondence between each target entity and the column data; from the correspondence between each target entity and the column data, the number of correspondences of the precondition fields with high second similarity are selected as the matching relationship between the target entity and the column data.

[0188] In some embodiments, the prediction module 2553 is further configured to predict the connection relationship of the condition matching relationship based on the statement vector representation, and obtain the connection relationship between the condition matching relationships; and combine the connection relationship, the target condition operator and the condition matching relationship to obtain the condition field prediction result.

[0189] It should be noted that the description of the above device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0190] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to perform the method provided in this application, for example... Figure 4-8 The method shown in the figure.

[0191] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0192] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0193] As an example, executable instructions may, but do not necessarily, correspond to files in the file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborative files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0194] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0195] In summary, the embodiments of this application extract target entities by utilizing preset entity resources within a preset target domain. This leverages domain-relevant preset entity resources, significantly improving the accuracy of target entities extracted from natural language. Furthermore, by establishing a mapping relationship between target entities and column values ​​in a preset data table to be queried, a set of mapped column values ​​is obtained. This set is then used to generate the query statement. The column value information from the preset data table can be used to effectively correct the column value information in the query statement, thereby improving the accuracy of the predicted query statement and ultimately enhancing the accuracy of natural language data queries based on the query statement. Moreover, by selecting target column values ​​from each column of data and combining the semantic representation of the target column values ​​with the semantic representation of the column names, the accuracy of column vector representation can be greatly improved, thereby enhancing the accuracy of query statement prediction based on column vector representation. Furthermore, compared to existing technologies that represent columns by column names, the method in this embodiment can greatly improve the generalization ability of the table. When applied to new domains, especially highly specialized ones, it can ensure the accuracy of extracting query fields and condition fields from natural language based on column vector representations, thereby improving the model's domain adaptability. Moreover, by incorporating the entity information of the preset target domain added during the semantic encoding stage in this embodiment, the model's domain transfer capability can be further improved.

[0196] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A natural language data query method, characterized in that, include: Based on the preset entity resources of the preset target domain, entity extraction is performed on the natural language query text to obtain the target entity set; By constructing a mapping relationship between each target entity in the target entity set and column values ​​in a preset data table, the mapping column value corresponding to each target entity is obtained; By performing semantic encoding and decoding on the natural language query text and each target entity, the statement vector representation corresponding to the natural language query text and the entity vector representation of each target entity are obtained. Select target column values ​​from each column of data in the preset data table, and perform semantic encoding and decoding processing on the column name and the target column value of each column to obtain column name vector representation and target column value vector representation; By combining the column name vector representation and the target column value vector representation, the semantic representation of each column of data is enhanced using the target column value, resulting in a column vector representation of each column of data. Based on the statement vector representation, the entity vector representation of each target entity, and the column vector representation of each column of data, the structured query statement is predicted for the natural language query text by combining the mapping column value corresponding to each target entity, and the query statement is obtained. The query is performed in the preset data table according to the query statement to obtain the query result corresponding to the natural language query text.

2. The method according to claim 1, characterized in that, The preset entity resources include at least one of a preset entity library and a preset entity recognition model; the preset entity recognition model is a network model trained using entity data in the preset target domain; the step of extracting entities from natural language query text based on the preset entity resources of the preset target domain to obtain a target entity set includes at least one of the following: The entities in the preset entity library are matched against the natural language query text. If the entity exists in the natural language query text, the entity is taken as the target entity, thereby obtaining a target entity set. The target entity set is obtained by performing entity recognition and extraction on the natural language query text using the preset entity recognition model.

3. The method according to claim 2, characterized in that, The step of constructing a mapping relationship between each target entity in the target entity set and column values ​​in a preset data table to obtain the mapping column value corresponding to each target entity includes at least one of the following: For a target entity obtained from the preset entity library, a mapping column value corresponding to each target entity is obtained according to the preset correspondence between entities and column values; the preset correspondence between entities and column values ​​is the correspondence between each entity in the preset entity library and the column value in the preset data table, which is pre-built. For each target entity obtained according to the preset entity recognition model, a first similarity is calculated between each target entity and each column value in the preset data table, and the mapping column value corresponding to each target entity is obtained based on the first similarity.

4. The method according to claim 1, characterized in that, The step of predicting the structured query statement of the natural language query text based on the statement vector representation, the entity vector representation of each target entity, and the column vector representation of each column of data, combined with the mapping column value corresponding to each target entity, to obtain the query statement includes: Based on the statement vector representation and the column vector representation of each column of data, the query field prediction in the structured query statement prediction is performed to obtain the query field prediction result; Based on the statement vector representation, the entity vector representation of each target entity, and the column vector representation of each column of data, combined with the mapping column value corresponding to each target entity, the condition field prediction in the structured query statement prediction is performed to obtain the condition field prediction result. The query statement is obtained by combining the prediction results of the query field and the prediction results of the condition field.

5. The method according to claim 4, characterized in that, The process of predicting query fields in the structured query statement prediction based on the statement vector representation and the column vector representation of each column of data, to obtain query field prediction results, includes: Perform at least one preset query count classification prediction on the statement vector representation to obtain a first probability of the statement vector representation corresponding to each preset query count; The number of queries corresponding to the highest first probability is taken as the number of query fields corresponding to the structured query statement; Based on the column vector representation, query target prediction is performed on each column of data in the preset data table to obtain a second probability that each column of data is the query target; Based on the second probability from high to low, select the number of columns of data that are the first number of query fields from each column of data as the target column data, and use the column name of the target column data as the target query field to obtain the target query field set; Based on the target query field set, the prediction result of the query field is obtained.

6. The method according to claim 5, characterized in that, The step of obtaining the prediction result of the query field based on the target query field set includes: For each column vector representation corresponding to the target query field in the target query field set, perform classification prediction using at least one preset aggregation function to obtain the third probability of each target query field corresponding to each preset aggregation function; The preset aggregation function corresponding to the highest third probability is used as the target aggregation function for each target query field; Based on the target aggregation function corresponding to each target query field, the target query fields are combined to obtain the prediction result of the query field.

7. The method according to claim 4, characterized in that, The condition field prediction in the structured query statement prediction is performed based on the statement vector representation, the entity vector representation of each target entity, and the column vector representation of each column of data, combined with the mapping column value corresponding to each target entity, to obtain the condition field prediction result, including: Perform at least one preset number of classification predictions on the statement vector representation to obtain a fourth probability of the statement vector representation corresponding to each preset number of conditions; The number of preset conditions corresponding to the highest fourth probability is taken as the number of condition fields corresponding to the query statement; Based on the number of condition fields, the matching relationship between the target entity and the column data is obtained from the correspondence obtained by matching the entity vector representation of each target entity with the column vector representation; Based on the mapping column value corresponding to each target entity, the target entities in the matching relationship are updated to obtain the conditional matching relationship between column data and mapping column values; For the conditional matching relationship, at least one preset conditional operator is predicted to obtain the target conditional operator corresponding to the conditional matching relationship; Based on the matching relationship between the target condition operator and the condition, the prediction result of the condition field is obtained.

8. The method according to claim 7, characterized in that, The step of obtaining the matching relationship between the target entity and the column data based on the number of condition fields, from the correspondence obtained by matching the entity vector representation of each target entity with the column vector representation, includes: The similarity between the column vector representation of each column of data and the entity vector representation of each target entity is calculated to obtain the second similarity between each column of data and each target entity; For each target entity, the column data corresponding to the maximum second similarity is used as the column data corresponding to each target entity, thus obtaining the correspondence between each target entity and the column data. From the correspondence between each target entity and column data, select the number of correspondences with the second highest similarity in the precondition fields, and use them as the matching relationships between the target entity and the column data.

9. The method according to claim 7 or 8, characterized in that, The step of obtaining the prediction result of the condition field based on the matching relationship between the target condition operator and the condition includes: Based on the statement vector representation, the connection relationship prediction is performed on the condition matching relationship to obtain the connection relationship between the condition matching relationships; By combining the connection relationship, the target condition operator, and the condition matching relationship, the prediction result of the condition field is obtained.

10. A natural language data query device, characterized in that, include: The extraction module is used to extract entities from natural language query text based on preset entity resources in a preset target domain, and obtain a target entity set. The mapping module is used to obtain the mapping column value corresponding to each target entity by constructing a mapping relationship between each target entity in the target entity set and the column value in a preset data table; The prediction module is used to obtain the statement vector representation corresponding to the natural language query text and the entity vector representation of each target entity by performing semantic encoding and decoding processing on the natural language query text and each target entity; Select target column values ​​from each column of data in the preset data table, and perform semantic encoding and decoding processing on the column name and the target column value of each column to obtain column name vector representation and target column value vector representation; By combining the column name vector representation and the target column value vector representation, the semantic representation of each column of data is enhanced using the target column value, resulting in a column vector representation of each column of data. The prediction module is used to perform structured query statement prediction on the natural language query text based on the statement vector representation, the entity vector representation of each target entity, and the column vector representation of each column of data, combined with the mapping column value corresponding to each target entity, to obtain the query statement; The query module is used to perform a query on the preset data table based on the query statement to obtain the query results corresponding to the natural language query text.

11. The apparatus according to claim 10, characterized in that, The preset entity resources include at least one of a preset entity library and a preset entity recognition model; the preset entity recognition model is a network model trained using entity data in the preset target domain. The extraction module is also used to match entities in the preset entity library with the natural language query text, and if the entity exists in the natural language query text, to use the entity as the target entity, thereby obtaining a target entity set; The target entity set is obtained by performing entity recognition and extraction on the natural language query text using the preset entity recognition model.

12. The apparatus according to claim 11, characterized in that, The mapping module is further configured to, for target entities obtained from the preset entity library, obtain the mapping column value corresponding to each target entity according to a preset correspondence between entities and column values; the preset correspondence between entities and column values ​​is a pre-constructed correspondence between each entity in the preset entity library and the column value in the preset data table. For each target entity obtained according to the preset entity recognition model, a first similarity is calculated between each target entity and each column value in the preset data table, and the mapping column value corresponding to each target entity is obtained based on the first similarity.

13. The apparatus according to claim 10, characterized in that, The prediction module is also used to predict the query fields in the structured query statement prediction based on the statement vector representation and the column vector representation of each column of data, and to obtain the query field prediction result. Based on the statement vector representation, the entity vector representation of each target entity, and the column vector representation of each column of data, combined with the mapping column value corresponding to each target entity, the condition field prediction in the structured query statement prediction is performed to obtain the condition field prediction result. The query statement is obtained by combining the prediction results of the query field and the prediction results of the condition field.

14. The apparatus according to claim 13, characterized in that, The prediction module is further configured to perform classification prediction on the statement vector representation for at least one preset number of queries, and obtain a first probability that the statement vector representation corresponds to each preset number of queries; The number of queries corresponding to the highest first probability is taken as the number of query fields corresponding to the structured query statement; Based on the column vector representation, query target prediction is performed on each column of data in the preset data table to obtain a second probability that each column of data is the query target; Based on the second probability from high to low, select the number of columns of data that are the first number of query fields from each column of data as the target column data, and use the column name of the target column data as the target query field to obtain the target query field set; Based on the target query field set, the prediction result of the query field is obtained.

15. The apparatus according to claim 14, characterized in that, The prediction module is further configured to perform classification prediction on the column vector representation corresponding to each target query field in the target query field set using at least one preset aggregation function to obtain the third probability of each target query field corresponding to each preset aggregation function; The preset aggregation function corresponding to the highest third probability is used as the target aggregation function for each target query field; Based on the target aggregation function corresponding to each target query field, the target query fields are combined to obtain the prediction result of the query field.

16. The apparatus according to claim 13, characterized in that, The prediction module is further configured to perform classification prediction on the statement vector representation using at least one preset number of conditions, to obtain a fourth probability of the statement vector representation corresponding to each preset number of conditions; The number of preset conditions corresponding to the highest fourth probability is taken as the number of condition fields corresponding to the query statement; Based on the number of condition fields, the matching relationship between the target entity and the column data is obtained from the correspondence obtained by matching the entity vector representation of each target entity with the column vector representation; Based on the mapping column value corresponding to each target entity, the target entities in the matching relationship are updated to obtain the conditional matching relationship between column data and mapping column values; For the conditional matching relationship, at least one preset conditional operator is predicted to obtain the target conditional operator corresponding to the conditional matching relationship; Based on the matching relationship between the target condition operator and the condition, the prediction result of the condition field is obtained.

17. The apparatus according to claim 16, characterized in that, The prediction module is further configured to perform similarity calculation between the column vector representation of each column of data and the entity vector representation of each target entity to obtain a second similarity between each column of data and each target entity; For each target entity, the column data corresponding to the maximum second similarity is used as the column data corresponding to each target entity, thus obtaining the correspondence between each target entity and the column data. From the correspondence between each target entity and column data, select the number of correspondences with the second highest similarity in the precondition fields, and use them as the matching relationships between the target entity and the column data.

18. The apparatus according to claim 16 or 17, characterized in that, The prediction module is further configured to predict the connection relationship of the condition matching relationship based on the statement vector representation, so as to obtain the connection relationship between the condition matching relationships; By combining the connection relationship, the target condition operator, and the condition matching relationship, the prediction result of the condition field is obtained.

19. An electronic device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the method according to any one of claims 1 to 9.

20. A computer-readable storage medium, characterized in that, It stores executable instructions for implementing the method according to any one of claims 1 to 9 when executed by a processor.

21. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the method described in any one of claims 1 to 9.

Images