Voice-based performance queries using non-semantic databases

Abstract

Query of a database using natural language (e.g., storing performance data of an industrial system) may be hindered by database elements in the database using non-semantic names. Although these non-semantic names can be modified, this entails modifying the database and modifying applications that utilize the database. Accordingly, the embodiments disclosed herein train and apply a text-to-SQL model that translates natural language queries into structured query language (SQL) queries that utilize semantic names for database elements. These SQL queries utilize a database view that maps native names for database elements to their semantic names. In addition, domain-specific pre-processing may be performed on text-to-SQL model inputs. In a word, the accuracy of conversion from the text to the SQL is improved, a database or an application program does not need to be modified, and meanwhile natural language query based on voice can be achieved.

Description

Background Art Technical Field

[0001] Embodiments described herein relate generally to database querying using natural language, and more particularly to natural language querying of non-semantic databases (eg, regarding performance of a system). Background Art

[0003] Industrial systems such as network management systems (NMS) can use relational database management systems (RDBMS) to store operational data and measurement data in a database. Such data can be retrieved from the database by executing queries on the database.

[0004] Typically, queries are written by users or applications in Structured Query Language (SQL). Especially for users who do not have expertise in the software, it would be beneficial to be able to ask queries in natural language, either spoken or written text. This would eliminate the need for users to have to acquire expertise in a query language such as SQL.

[0005] Recent advances in artificial intelligence (AI) can convert natural language text questions into SQL queries that can be executed on a database. However, this text-to-SQL conversion assumes that the database utilizes semantic names for its database elements (e.g., tables and columns). This assumption is not always correct, especially for legacy databases that may have evolved over decades. While it is possible to rename database elements from their original native names to semantic names, this may break links with existing applications that utilize the native names. In other words, not only does the database need to be modified, but every application that queries the database may also need to be modified.

[0006] Additionally, there are many scenarios where voice-based queries are required. For example, when a field engineer mounts a wireless router on top of a pole and wants to query the network status, they may not be able to feasibly or safely type text into their device. In this case, it would be beneficial if the field engineer could simply speak a natural language query. However, when natural language queries are derived from spoken text, the intermediate speech-to-text conversion process may incorrectly transcribe one or more domain-specific values ​​or add other noise.

[0007] Therefore, there is a need to be able to achieve text-to-SQL conversion for non-semantic databases while denoising speech-to-text conversion to enable real-time speech-based queries (e.g., queries on the performance of industrial systems). Summary of the invention

[0008] Thus, systems, methods, and non-transitory computer-readable media for natural language querying of non-semantic databases are disclosed.

[0009] In an embodiment, a method includes using at least one hardware processor to perform the following operations: adding a database view to a database, wherein the database view maps native names of database elements in the database to semantic names of the database elements; obtaining a natural language text input representing a natural language query; applying a machine learning text-to-SQL model to the natural language text input to generate a structured query language (SQL) query, the SQL query including a semantic name for each database element referenced in the SQL query; and executing the SQL query on the database using the database view to map the semantic name for each database element referenced in the SQL query to the native name of the database element. The database elements may include tables and columns within tables. The natural language query and the SQL query may include a request for a value of at least one performance parameter of a network. The at least one performance parameter of the network may be a utility network. The utility network may be an industrial wireless mesh network or a substation network, wherein performance measurements of the network are stored in the database for querying using natural language.

[0010] The method may also include using the at least one hardware processor to perform the following operations: receiving a natural language text string; and normalizing the natural language text string to obtain the natural language text input. Normalizing the natural language text string may include: identifying a text transcription of a network address in the natural language text string; and replacing the text transcription of the network address with a representation of the network address in a standard format. Normalizing the natural language text string may include: identifying a text transcription of one or both of a date and a time in the natural language text string; and replacing the text transcription of one or both of the date and time with a timestamp value representing one or both of the date and time. Normalizing the natural language text string may include: identifying a text transcription of a term in the natural language text string for an entity represented in the database; and replacing the text transcription of the term with a standard term for the entity.

[0011] The natural language text string may be received via an input of a graphical user interface, and the method may further include using the at least one hardware processor to: receive a result of executing the SQL query on the database; and displaying a representation of the result in the graphical user interface. The natural language text string may be received from an external system, and the method may further include using the at least one hardware processor to: receive a result of executing the SQL query on the database; and returning a representation of the result to the external system.

[0012] In an embodiment, a method includes using at least one hardware processor to perform the following operations: generating a representation of a database view of a database, wherein the database view maps native names for database elements in the database to semantic names for the database elements; and training a machine learning text-to-SQL model using a training data set to generate a structured query language (SQL) query from a natural language text input representing a natural language query, the SQL query including the semantic name of each database element referenced in the SQL query, the training data set including the natural language text input labeled with the SQL query. The database elements may include tables and columns within tables. One or more of the natural language queries in the training data set may include a request for a value of at least one performance parameter of a network. The at least one performance parameter of the network may be a utility network. The utility network may be an industrial wireless mesh network or a substation network, wherein performance measurements of the network are stored in the database for querying using natural language.

[0013] The method may further include using the at least one hardware processor to generate the training data set by: obtaining an existing data set including natural language text input tagged with an existing SQL query, the existing SQL query including a native name for each database element referenced in the existing SQL query; identifying semantic names for the native names in the existing SQL query; and replacing the native names in the existing SQL query with the identified semantic names to generate a modified SQL query, wherein the training data set includes natural language text input tagged with the modified SQL query from the existing data set. The method may further include using the at least one hardware processor to perform the following operations: using another training data set including natural language phrases tagged with semantic names to train an entity extraction model to extract the semantic names of database elements from natural language questions. Generating the representation of the database view may include: applying the trained entity extraction model to the natural language text input in the existing data set to extract the semantic names of the database elements; and associating the extracted semantic names with corresponding native names in the native names in the existing SQL query to generate the database view.

[0014] The method may further include using the at least one hardware processor to perform the following operations: applying the representation of the database view to the database to add the database view to the database; and for each of one or more natural language text inputs specified by the user, applying the trained machine learning text-to-SQL model to the user-specified natural language text input to generate a SQL query, and executing the generated SQL query on the database using the database view. The method may further include using the at least one hardware processor to perform the following operations for each of the one or more natural language text inputs specified by the user: receiving a result of executing the generated SQL query on the database; and returning the result to the user system. The method may further include using the at least one hardware processor to perform the following operations for each of the one or more natural language text inputs specified by the user: receiving a natural language text string specified by the user; and normalizing the natural language text string to obtain the natural language text input specified by the user. Normalizing the natural language text string may include: identifying a text transcription of a specific item in the natural language text string; and replacing the text transcription of the specific item with a standardized representation of the specific item.

[0015] It should be understood that any feature of the above method can be implemented alone or in any combination with any subset of other features. Therefore, even if the attached claims indicate that there are specific dependencies between features, the disclosed embodiments are not limited to these specific dependencies. On the contrary, any feature described herein can be combined with any other feature described herein, or implemented in any combination of any features without any one or more other features described herein. In addition, any method described above and elsewhere herein can be embodied in an executable software module of a processor-based system (such as a server) and / or in executable instructions stored in a non-transitory computer-readable medium, either alone or in any combination. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The details of the invention, both as to structure and operation, may be gleaned in part by examination of the accompanying drawings, in which like reference numerals refer to like parts, and in which:

[0017] Figure 1 illustrates an example infrastructure in which one or more processes described herein may be implemented according to an embodiment;

[0018] Figure 2 illustrates an example processing system that may be used to perform one or more of the processes described herein according to an embodiment;

[0019] Figure 3 illustrates a process for converting text into SQL according to an embodiment;

[0020] Figure 4 illustrates a process for training a text-to-SQL model and generating database view(s) according to an embodiment;

[0021] Figure 5 illustrates an example implementation of a sub-process for mapping native names to semantic names according to an embodiment;

[0022] Figure 6 illustrates the architecture of a text-to-SQL model according to an embodiment; and

[0023] Figure 7 Illustrated are example screens of a graphical user interface according to an embodiment. DETAILED DESCRIPTION

[0024] In an embodiment, a system, method and non-transient computer-readable medium for natural language query of a non-semantic database are disclosed. After reading this specification, it will be clear to those skilled in the art how to implement the present invention in various alternative embodiments and alternative applications. However, although various embodiments of the present invention will be described herein, it should be understood that these embodiments are presented only by way of example and illustration and not limitation. Therefore, this detailed description of various embodiments should not be interpreted as limiting the scope or breadth of the present invention set forth in the appended claims.

[0025] 1. System Overview

[0026] 1.1. Infrastructure

[0027] Figure 1An example infrastructure in which one or more disclosed processes may be implemented according to an embodiment is illustrated. The infrastructure may include a platform 110 (e.g., one or more servers) that hosts and / or performs one or more of the various functions, processes, methods, and / or software modules described herein. The platform 110 may include a dedicated server, or alternatively may be implemented in a computing cloud, where the resources of one or more servers may be dynamically and elastically allocated to multiple tenants based on demand. In either case, the servers may be co-located and / or distributed in different geographic locations. The platform 110 may also include or be communicatively connected to a server application 112 and / or one or more databases 114. In addition, the platform 110 may be communicatively connected to one or more user systems 130 via one or more networks 120. The platform 110 may also be communicatively connected to one or more external systems 140 (e.g., other platforms, websites, etc.) via one or more networks 120.

[0028] The network(s) 120 may include the Internet, and the platform 110 may communicate with the user system(s) 130 over the Internet using standard transfer protocols, such as Hypertext Transfer Protocol (HTTP), HTTP Secure (HTTPS), File Transfer Protocol (FTP), FTP Secure (FTPS), Secure Shell FTP (SFTP), etc., as well as proprietary protocols. Although the platform 110 is illustrated as being connected to various systems via a single set of network(s) 120, it should be understood that the platform 110 may be connected to various systems via a different set of one or more networks. For example, the platform 110 may be connected to a subset of the user systems 130 and / or external systems 140 via the Internet, but may also be connected to one or more other user systems 130 and / or external systems 140 via an intranet. Furthermore, although only a few user systems 130 and external systems 140, one server application 112, and a set of database(s) 114 are illustrated, it should be understood that the infrastructure may include any number of user systems, external systems, server applications, and databases.

[0029] The user systems 130 may include any one or more types of computing devices capable of wired and / or wireless communication, including but not limited to desktop computers, laptop computers, tablet computers, smart phones or other mobile phones, servers, game consoles, televisions, set-top boxes, electronic self-service terminals, and / or point-of-sale terminals, etc. However, it is contemplated that the user systems 130 will typically be personal computers, workstations, or mobile devices (e.g., smart phones, field tablet computers, etc.) of the users. Each user system 130 may include or be communicatively connected to a client application 132 and / or one or more local databases 134. The client application 132 may communicate with the server application 112 via the network(s) 120.

[0030] The platform 110 may include a web server that hosts one or more websites and / or web services. In an embodiment where a website is provided, the website may include a graphical user interface, including, for example, one or more screens (e.g., web pages) generated in hypertext markup language (HTML) or other languages. The platform 110 transmits or provides one or more screens of a graphical user interface in response to a request from (multiple) user systems 130. In some embodiments, these screens may be provided in the form of a wizard, in which case two or more screens may be provided in a sequential manner, and one or more sequential screens may depend on the interaction of the user or user system 130 with one or more previous screens. Requests to the platform 110 and responses from the platform 110 (including screens of a graphical user interface) may be transmitted through (multiple) networks 120 using standard communication protocols (e.g., HTTP, HTTPS, etc.), which may include the Internet. These screens (e.g., web pages) may include a combination of content and elements such as text, images, videos, animations, references (e.g., hyperlinks), frames, inputs (e.g., text boxes, text areas, check boxes, radio buttons, drop-down menus, buttons, tables, etc.), scripts (e.g., JavaScript), etc., including elements composed of or derived from data stored in one or more databases (e.g., database(s) 114) accessible locally and / or remotely to the platform 110. The platform 110 may also respond to other requests from the user system(s) 130.

[0031] The platform 110 may include or be communicatively coupled to or otherwise access one or more databases 114. For example, the platform 110 may include one or more database servers (e.g., RDBMS) that manage one or more databases 114. Server applications 112 executing on the platform 110, client applications 132 executing on a user system 130, and / or applications executing on an external system 140 may submit data (e.g., user data, table data, etc.) to be stored in (multiple) databases 114, and / or request access to data stored in (multiple) databases 114. Of particular relevance to the present application, data may be requested from (multiple) databases 114 using queries in an appropriate query language (e.g., SQL). Any suitable database may be utilized, including but not limited to MySQL. TM 、Oracle TM 、IBM TM , Microsoft SQL TM 、Access TM , PostgreSQL TM etc., including cloud-based databases and proprietary databases. For example, data can be sent to the platform 110 using the well-known POST request supported by HTTP, and / or via FTP, etc. The data and other requests can be processed, for example, by server-side network technologies executed by the platform 110, such as server applet or other software module (e.g., included in the server application 112).

[0032] In an embodiment providing a network service, the platform 110 may receive a request from (multiple) external systems 140 and provide a response in an extensible markup language (XML), JavaScript object notation (JSON), and / or any other suitable or desired format. In such an embodiment, the platform 110 may provide an application programming interface (API) that defines the manner in which (multiple) user systems 130 and / or (multiple) external systems 140 may interact with the network service. Thus, (multiple) user systems 130 and / or (multiple) external systems 140 (which may themselves be servers) may define their own user interfaces and rely on the network service to implement or otherwise provide the backend processes, methods, functions, and / or storage described herein, etc. For example, in such an embodiment, the external system 140 may execute SQL queries on (multiple) databases 114 via the API of the platform 110. As another example, a client application 132 executing on one or more user systems 130 may interact with a server application 112 executing on the platform 110 to perform one or more or a portion of one or more of the various functions, processes, methods, and / or software modules described herein.

[0033] The client applications 132 may be "thin", in which case the processing is primarily performed on the server side by the server applications 112 on the platform 110. A basic example of a thin client application 132 is a browser application that simply requests, receives, and renders web pages at the user system(s) 130, while the server applications 112 on the platform 110 are responsible for generating the web pages and managing database functions. Alternatively, the client application may be "thick", in which case the processing is primarily performed on the client side by the user system(s) 130. It should be understood that the client application 132 may perform the amount of processing relative to the server applications 112 on the platform 110 at any point along this spectrum between "thin" and "thick", depending on the design goals of a particular implementation. In any case, the software described herein, which may reside entirely on the platform 110 (e.g., in which case the server application 112 performs all processing) or on (multiple) user systems 130 (e.g., in which case the client application 132 performs all processing) or distributed between the platform 110 and (multiple) user systems 130 (e.g., in which case both the server application 112 and the client application 132 perform processing), may include one or more executable software modules that include instructions for implementing one or more processes, methods, or functions described herein.

[0034] 1.2. Example Processing Equipment

[0035] Figure 2 is a block diagram illustrating an example wired or wireless system 200 that may be used in conjunction with the various embodiments described herein. For example, system 200 may be used as or in conjunction with one or more of the functions, processes, or methods described herein (e.g., for storing and / or executing software), and may represent components of platform 110, user system(s) 130, external system(s) 140, and / or other processing devices described herein. System 200 may be a server or any conventional personal computer, or any other processor-enabled device capable of wired or wireless data communications. It will be apparent to those skilled in the art that other computer systems and / or architectures may also be used.

[0036] The system 200 preferably includes one or more processors 210. The processor(s) 210 may include a central processing unit (CPU). Additional processors may be provided, such as a graphics processing unit (GPU) (e.g., for training any disclosed model, operations or inferences performed by any disclosed model, etc.), auxiliary processors for managing input / output, auxiliary processors for performing floating point math operations, dedicated microprocessors (e.g., digital signal processors) with an architecture suitable for fast execution of signal processing algorithms, slave processors (e.g., back-end processors) subordinate to a main processing system, additional microprocessors or controllers for dual-processor or multi-processor systems, and / or coprocessors. Such auxiliary processors may be discrete processors or may be integrated with the processor 210. Examples of processors that may be used with the system 200 include, but are not limited to, any processor provided by Intel Corporation of Santa Clara, California (e.g., Pentium TM , Core i7 TM , Xeon TM any processor provided by Advanced Micro Devices, Inc. (AMD) of Santa Clara, California, any processor provided by Apple Inc. of Cupertino (e.g., A-series, M-series, etc.), any processor provided by Samsung Electronics Co., Ltd. of Seoul, South Korea (e.g., Exynos TM ), any processor provided by NXP Semiconductors of Eindhoven, the Netherlands, etc.

[0037] The processor 210 is preferably connected to a communication bus 205. The communication bus 205 may include a data channel for facilitating the transfer of information between the storage devices and other peripheral components of the system 200. In addition, the communication bus 205 may provide a set of signals for communicating with the processor 210, including a data bus, an address bus, and / or a control bus (not shown). The communication bus 205 may include any standard or non-standard bus architecture, such as a bus architecture that conforms to the Industry Standard Architecture (ISA), the Extended Industry Standard Architecture (EISA), the Micro Channel Architecture (MCA), the Peripheral Component Interconnect (PCI) local bus, a bus architecture that conforms to standards promulgated by the Institute of Electrical and Electronics Engineers (IEEE) including the IEEE 488 General Purpose Interface Bus (GPIB) and / or IEEE 696 / S-100, etc.

[0038] The system 200 preferably includes a main memory 215, and may also include a secondary memory 220. The main memory 215 provides instructions and data storage for programs executed on the processor 210 (such as any software discussed herein). It should be understood that the programs stored in the memory and executed by the processor 210 can be written and / or compiled according to any suitable language, including but not limited to C / C++, Java, JavaScript, Perl, Visual Basic, NET, etc. The main memory 215 is typically a semiconductor-based memory, such as a dynamic random access memory (DRAM) and / or a static random access memory (SRAM). Other semiconductor-based memory types include, for example, synchronous dynamic random access memory (SDRAM), Rambus dynamic random access memory (RDRAM), ferroelectric random access memory (FRAM), etc., including read-only memory (ROM).

[0039] The secondary memory 220 is a non-transitory computer-readable medium having computer executable code (e.g., any software disclosed herein) and / or other data stored thereon. The computer software or data stored on the secondary memory 220 is read into the main memory 215 for execution by the processor 210. The secondary memory 220 may include, for example, semiconductor-based memory such as a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), and flash memory (a block-oriented memory similar to EEPROM).

[0040] The secondary storage 220 may optionally include internal media 225 and / or removable media 230. The removable media 230 is read and / or written in any well-known manner. The removable storage medium 230 may be, for example, a tape drive, a compact disk (CD) drive, a digital versatile disk (DVD) drive, other optical drives, and / or a flash memory drive, etc.

[0041] In alternative embodiments, secondary memory 220 may include other similar means for allowing computer programs or other data or instructions to be loaded into system 200. Such means may include, for example, a communication interface 240 that allows software and data to be transferred to system 200 from an external storage medium 245. Examples of external storage medium 245 include an external hard drive, an external optical drive, and / or an external magneto-optical drive, etc.

[0042] As mentioned above, the system 200 may include a communication interface 240. The communication interface 240 allows software and data to be transferred between the system 200 and an external device (e.g., a printer), a network, or other information source. For example, computer software or executable code may be transferred from a network server (e.g., platform 110) to the system 200 via the communication interface 240. Examples of the communication interface 240 include a built-in network adapter, a network interface card (NIC), a Personal Computer Memory Card International Association (PCMCIA) network card, a cardbus network adapter, a wireless network adapter, a universal serial bus (USB) network adapter, a modem, a wireless data card, a communication port, an infrared interface, an IEEE 1394 FireWire, and any other device capable of interfacing the system 200 with a network (e.g., network(s) 120) or another computing device. The communication interface 240 preferably implements industry-published protocol standards, such as Ethernet IEEE 802 standards, Fibre Channel, Digital Subscriber Line (DSL), Asynchronous Digital Subscriber Line (ADSL), Frame Relay, Asynchronous Transfer Mode (ATM), Integrated Services Digital Network (ISDN), Personal Communications Service (PCS), Transmission Control Protocol / Internet Protocol (TCP / IP), Serial Line Internet Protocol / Point-to-Point Protocol (SLIP / PPP), etc., but customized or non-standard interface protocols may also be implemented.

[0043] Software and data transmitted via communication interface 240 are typically in the form of electrical communication signals 255. These signals 255 may be provided to communication interface 240 via communication channel 250. In embodiments, communication channel 250 may be a wired or wireless network (e.g., network(s) 120), or any other communication link of any kind. Communication channel 250 carries signals 255 and may be implemented using a variety of wired or wireless communication means, including wire or cable, optical fiber, conventional telephone line, cellular telephone link, wireless data communication link, radio frequency ("RF") link, or infrared link, to name a few.

[0044] Computer executable code (e.g., a computer program, such as the disclosed software) is stored in the main memory 215 and / or the secondary memory 220. The computer executable code may also be received via the communication interface 240 and stored in the main memory 215 and / or the secondary memory 220. Such a computer program, when executed, enables the system 200 to perform various functions of the disclosed embodiments as described elsewhere herein.

[0045] In this specification, the term "computer-readable medium" is used to refer to any non-transitory computer-readable storage medium used to provide computer executable code and / or other data to or within the system 200. Examples of such media include main memory 215, auxiliary memory 220 (including internal memory 225 and / or removable media 230), external storage media 245, and any peripheral devices (including network information servers or other network devices) communicatively coupled to communication interface 240. These non-transitory computer-readable media are means for providing software and / or other data to system 200.

[0046] In embodiments implemented using software, the software may be stored on a computer-readable medium and loaded into the system 200 via the removable media 230, the I / O interface 235, or the communication interface 240. In such embodiments, the software is loaded into the system 200 in the form of an electrical communication signal 255. The software, when executed by the processor 210, preferably causes the processor 210 to perform one or more of the processes and functions described elsewhere herein.

[0047] In an embodiment, the I / O interface 235 provides an interface between one or more components of the system 200 and one or more input and / or output devices. Example input devices include, but are not limited to, sensors, keyboards, touch screens or other touch-sensitive devices, cameras, biometric sensing devices, computer mice, trackballs, and / or pen-based pointing devices, etc. Examples of output devices include, but are not limited to, other processing devices, cathode ray tubes (CRTs), plasma displays, light emitting diode (LED) displays, liquid crystal displays (LCDs), printers, vacuum fluorescent displays (VFDs), surface conduction electron emitter displays (SEDs), and / or field emission displays (FEDs), etc. In some cases, input and output devices may be combined, such as in the case of a touch panel display (e.g., in a smartphone, tablet computer, or other mobile device).

[0048] System 200 may also include an optional wireless communication component that facilitates wireless communication over a voice network and / or a data network (e.g., in the case of user system 130). The wireless communication component includes antenna system 270, radio system 265, and baseband system 260. In system 200, radio frequency (RF) signals are transmitted and received over the air by antenna system 270 under the management of radio system 265.

[0049] In an embodiment, the antenna system 270 may include one or more antennas and one or more multiplexers (not shown) that perform switching functions to provide transmit and receive signal paths for the antenna system 270. In the receive path, the received RF signal may be coupled from the multiplexer to a low noise amplifier (not shown) that amplifies the received RF signal and sends the amplified signal to the radio system 265.

[0050] In an alternative embodiment, the radio system 265 may include one or more radios configured to communicate on various frequencies. In an embodiment, the radio system 265 may combine a demodulator (not shown) and a modulator (not shown) in one integrated circuit (IC). The demodulator and modulator may also be separate components. In the incoming path, the demodulator removes the RF carrier signal, leaving a baseband receive audio signal that is sent from the radio system 265 to the baseband system 260.

[0051] If the received signal contains audio information, the baseband system 260 decodes the signal and converts it into an analog signal. The signal is then amplified and sent to a speaker. The baseband system 260 also receives analog audio signals from a microphone. These analog audio signals are converted into digital signals and encoded by the baseband system 260. The baseband system 260 also encodes the digital signals for transmission and generates a baseband transmit audio signal, which is routed to the modulator portion of the radio system 265. The modulator mixes the baseband transmit audio signal with an RF carrier signal to generate an RF transmit signal, which is routed to the antenna system 270 and can pass through a power amplifier (not shown). The power amplifier amplifies the RF transmit signal and routes it to the antenna system 270, where the signal is switched to the antenna port for transmission.

[0052] The baseband system 260 is also communicatively coupled to the processor(s) 210. The processor(s) 210 may access the data storage areas 215 and 220. The processor(s) 210 are preferably configured to execute instructions (i.e., computer programs, such as the disclosed software) which may be stored in the main memory 215 or the secondary memory 220. The computer programs may also be received from the baseband processor 260 and stored in the main memory 210 or the secondary memory 220, or executed upon receipt. Such computer programs, when executed, may enable the system 200 to perform the various functions of the disclosed embodiments.

[0053] 2. Process Overview

[0054] An embodiment of a process for natural language query of a non-semantic database will now be described in detail. It should be understood that the described process may be embodied as one or more software modules executed by one or more hardware processors (e.g., processor 210), for example, as a software application (e.g., server application 112, client application 132, and / or a distributed application including both server application 112 and client application 132), which may be executed entirely by (multiple) processors of platform 110, entirely by (multiple) processors of user system 130, or distributed across platform 110 and user system 130, so that some parts or modules of the software application are executed by platform 110, while other parts or modules of the software application are executed by user system 130. The described process may be implemented as instructions represented by source code, object code, and / or machine code. These instructions may be executed directly by (multiple) hardware processor 210, or alternatively, may be executed by a virtual machine running between the object code and (multiple) hardware processor 210. Additionally, the disclosed software may be built upon or interfaced with one or more existing systems.

[0055] Alternatively, the described process can be implemented as a hardware component (e.g., a general purpose processor, an integrated circuit (IC), an application specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic, etc.), a combination of hardware components, or a combination of hardware components and software components. In order to clearly illustrate the interchangeability of hardware and software, various illustrative components, blocks, modules, circuits and steps are generally described herein according to their functions. Whether such functions are implemented as hardware or software depends on the specific application and the design constraints imposed on the entire system. The technician can use different ways to implement the described functions for each specific application, but such implementation decisions should not be interpreted as causing departure from the scope of the present invention. In addition, the functional grouping within the components, blocks, modules, circuits or steps is for ease of description. Without departing from the present invention, a specific function or step can be moved from one component, block, module, circuit or step to another.

[0056] In addition, although the processes described herein are illustrated by certain arrangements and sequencing of subprocesses, each process can be implemented by fewer, more, or different subprocesses and different arrangements and / or sequencing of subprocesses. In addition, it should be understood that any subprocess that is not dependent on the completion of another subprocess can be performed before, after, or in parallel with the other independent subprocess, even if the subprocesses are described or illustrated in a particular order.

[0057] 2.1. operate

[0058] Figure 3 A process 300 for converting text to SQL according to an embodiment is illustrated. The process 300 assumes that the conversion starts with spoken text 305, which is converted from speech to text in a subprocess 310 to produce a natural language text string 315. However, in an alternative example in which the natural language question is typed rather than spoken, the subprocess 310 may be omitted. In other words, in this case, the process 300 may start with the natural language text string 315.

[0059] In subprocess 310, spoken text 305 is received and converted into a natural language text string 315. Spoken text 305 may be received in real time (i.e., as the text is spoken by the user) via an audio interface (e.g., one or more microphones) of user system 130. For example, a user (e.g., a field engineer) may initiate a recording function of client application 132 on user system 130. The recording function may be initiated via input in a graphical user interface of client application 132, via a hardware button on user system 130, in response to a spoken command recognized by a background function executing on user system 130, automatically whenever speech is detected by a background function executing on user system 130, and so forth. Subprocess 310 may utilize any suitable technology for speech-to-text conversion and may be performed as a module of client application 132 on user system 130, as a module of server application 112 on platform 110 (e.g., as a cloud-based service), or as a module of an edge device between user system 130 and platform 110. Natural language text string 315 may consist of a text transcription of spoken text 305 .

[0060] In subprocess 320, natural language text string 315 can be preprocessed to produce natural language text input 325. In particular, natural language text string 315 can be normalized according to one or more normalization functions to obtain natural language text input 325. The specific normalization function utilized can depend on the specific domain of the utilization process 300. For example, the text representation of entities in certain domains is significantly different from non-technical text. Such domain-specific entities include network addresses, date-time values, certain terms, etc. The general speech-to-text conversion function in subprocess 310 may not perform well when transcribing these domain-specific entities, which may ultimately cause SQL queries to be incorrect. On the other hand, it will take a lot of effort to establish a domain-specific speech-to-text conversion function from scratch. Therefore, one or more normalization functions can be used to correct the text transcription of these domain-specific entities or any other specific items. It should be understood that subprocess 320 may use one of the disclosed normalization functions, use all of the disclosed normalization functions, use no disclosed normalization functions, or use any subset of the disclosed normalization functions in any combination, as well as use other normalization functions not specifically disclosed herein.

[0061] In the field of utilizing network addresses, subprocess 320 may include a normalization function that identifies the text transcription of the network address in the natural language text string 315 and replaces the text transcription of the network address with a representation of the network address in a standard format. The network address may include an Internet Protocol (IP) address, a Media Access Control (MAC) address, and / or other identifiers. For example, the spoken text 305 may include a question such as "what is the current latency of gateway 10.0.0.1?" . The corresponding natural language text string 315 transcribed by subprocess 310 may be "what is the current latency ofgateway 10.0dot 0.1". Since IP addresses are not typically represented in this non-standard format in the database 114, the resulting SQL query will not return correct results. Therefore, the normalization function of subprocess 320 may detect this incorrect format and convert it to a standard format. Incorrect formats can be detected using regular expression pattern matching, such as: r'\b(\d+\.\d+)[\s]*(\d+\.\d+)\b'. In this case, the regular expression groups can be concatenated to obtain the correctly formatted IP address. It should be understood that the normalization function can attempt to match multiple different regular expressions to identify and correct multiple different potentially incorrectly formatted network addresses.

[0062] In the field of utilizing date and time, subprocess 320 may include a normalization function that recognizes a text transcription of one or both of a date and a time in a natural language text string 315 and replaces the text transcription of one or both of the date and the time with a timestamp value representing one or both of the date and the time. The database 114 typically stores date-time values ​​as timestamps representing an epoch. For example, the Unix epoch represents a date-time value as the number of seconds since 00:00:00 Coordinated Universal Time (UTC) on January 1, 1970. While this is convenient for computers, humans cannot naturally express in terms of epochs. Therefore, the normalization function of subprocess 320 may use regular expressions for pattern matching to detect date-time values ​​of different formats and convert any such date-time values ​​into a timestamp value representing a standard epoch (e.g., the Unix epoch).

[0063] Some fields may utilize terms to represent entities, which are typically represented in the database using classification terms or standard terms. Therefore, subprocess 320 may include a normalization function that identifies the text transcription of the term in the natural language text string 315 for the entity represented in the database 114 and replaces the text transcription of the term with the standard term for the entity. These terms may be identified using a keyword search (e.g., from a list of terms) or regular expressions (e.g., for detecting multiple possible grammatical variants of each term) for pattern matching. As an example, in the field of wireless mesh networks, database elements may utilize terms such as "Node" and "Gateway" to indicate device types. However, the natural language text string 315 may include a request to "show me the IP addresses of allgateways". In this case, the normalization function may identify the term "gateways" and replace it with the term "Gateway" to obtain the natural language text input 325 of "show me the IPaddresses of all Gateways". In an embodiment, one or more (possibly including all) plural nouns in the natural language text string 315 may be replaced with the corresponding singular noun.

[0064] The machine learning text-to-SQL model 330 can be applied to the natural language text input 325 output by the text preprocessing in the subprocess 320 to generate an SQL query 335. The SQL query 335 includes a semantic name for each database element (e.g., a table, a column within a table, etc.) referenced in the SQL query 335, rather than a native name. As used herein, the term "semantic" refers to a name that describes an entity according to its meaning, while the term "native" refers to the name of the actual database element used to represent the entity in the database 114. For example, a native name may be "num_neighbor", and the corresponding semantic name may be "neighbor count" or "neighbor_count". In some cases, the semantic name and the native name may be the same. However, in many cases, the native name is selected for convenience, and it does not convey the correct meaning of the entity represented by the database element, or the way it is expressed is difficult to understand.

[0065] In subprocess 340, SQL query 335 may be adjusted to suit the relevant database, if necessary. In particular, different database types (e.g., SQLite TM ,MySQL TM 、Oracle TM etc.) may have different query syntax. For example, if the results of the query should be limited to a single row, SQL query 335 should be used in querying MySQL TM Specify "LIMIT 1" when querying Oracle TM To perform this adjustment, a list of known patterns may be associated with each database type or database 114 and used to pattern match SQL query 335 to replace portions of SQL query 335, add portions to SQL query 335, delete portions from SQL query 335, and / or otherwise modify SQL query 335 to conform to the query syntax of the associated database 114. In embodiments that support only a single database type or where the supported database types all use the same query syntax, subprocess 340 may be omitted.

[0066] In subprocess 350, SQL query 335 can be executed on database 114 (e.g., as adjusted by subprocess 340 in an embodiment including subprocess 340, or as output by subprocess 330 in an embodiment not including subprocess 340). For example, SQL query 335 can be submitted to database 114 via RDBMS for execution. Likewise, SQL query 335 can utilize semantic names of database elements in database 114. Database 114 can include one or more database views 115. Database view 115 is a logical table, virtual table, or proxy table that does not actually exist in database 114. For each actual table in database 114, or at least for each table in database 114 that will be the subject of a natural language query, a database view 115 can exist. Each database view 115 maps native names of database elements (e.g., columns) in a table of database 114 to corresponding semantic names for these database elements. In addition, database view 115 can be named using semantic names that correspond to native names of actual tables in database 114 corresponding to database view 115. Thus, during execution of the SQL query 335, which may act on the database view 115 of the relevant tables, each semantic name in the SQL query 335 is mapped to a corresponding native name in the database 114, so that the query can be executed against the native names in the database 114. For example, the semantic name of each referenced table may be mapped to a particular database view 115, and the semantic name of each referenced column in each referenced table may be mapped to the native name of the column in the database 114. After executing the SQL query 335, the RDBMS of the database 114 will return the result of executing the SQL query 335 on the database 114.

[0067] In subprocess 360, the results of executing SQL query 335 may be returned in response to spoken text 305 or natural language text string 315. For example, subprocess 360 may display a representation of the results in a graphical user interface on user system 130 (e.g., generated by client application 132, or generated by server application 112 and rendered by client application 132). Thus, in an embodiment, a user may speak a natural language query to his or her user system 130 (e.g., as a question, request, or statement), and user system 130 may display the results to the user. Alternatively, the user may type a natural language query into an input of a graphical user interface displayed on user system 130, and user system 130 may display the results to the user in the graphical user interface.

[0068] It should be understood that process 300 may be performed entirely by server application 112, in which case spoken text 305 may be recorded and transmitted to server application 112, which may implement subprocesses 320, 330, 340, and 350 and transmit the results in subprocess 360. Alternatively, process 300 may be performed entirely by client application 132 on user system 130. As yet another alternative, process 300 may be performed in part by client application 132 and in part by server application 112. For example, client application 132 may implement subprocess 310 and transmit natural language text string 315 to server application 112 to implement the remaining subprocesses, or the client application may implement subprocesses 310 and 320 and transmit natural language text input 325 to server application 112 to implement the remaining subprocesses.

[0069] 2.2. Training of text-to-SQL models and generation of database views

[0070] Figure 4 A process 400 for training a text-to-SQL model 330 and generating (multiple) database views 115 according to an embodiment is illustrated. The process 400 may begin by obtaining an existing data set 410, which includes a natural language text input 412 labeled with existing SQL queries 414, which include native names for each referenced database element. It should be understood that each existing SQL query 414 represents a real SQL query corresponding to the natural language text input 412 associated therewith. The existing data set 410 may be a public data set or may be obtained by any other means.

[0071] An entity extraction model 420 is applied to a natural language text input 412 in an existing dataset 410 to extract potential semantic names 425 of data elements represented in the natural language text input 412. The entity extraction model 420 may also output a confidence value for each potential semantic name 425. The entity extraction model 420 may have been previously trained to extract semantic names of database elements from natural language questions using a training dataset that includes natural language phrases labeled with the actual semantic names represented by the natural language phrases.

[0072] The entity extraction model 420 may include a discriminative undirected probabilistic graphical model that takes into account context, such as a conditional random field (CRF). TM Used to train entity extraction model 420. Rasa TMis an open source natural language understanding (NLU) software for training AI models. In this embodiment, the entity extraction model 420 includes a dual intent and entity transformer (DIET), as described in Bunk et al., "DIET: Lightweight Language Understanding for Dialogue Systems", Computational Research Paper Library (CoRR), abs / 2004.09936, doi:10.48550 / ARXIV.2004.09936 (2020), the entire contents of which are hereby incorporated by reference herein as fully set forth. DIET implements a natural language processing (NLP) architecture that uses an encoder-decoder model with a self-attention layer. The encoder includes a stack of encoder layers, each of which includes a self-attention layer and a fully connected two-layer feed-forward network. In particular, DIET is a multi-task transformer that classifies intents and recognizes entities. In an embodiment, the output of DIET is input into a CRF layer to produce a latent semantic name 425.

[0073] Subprocess 430 may map native names used in existing SQL query 414 to semantic names from latent semantic names 425 to produce a mapping 435 between native names and semantic names. Each latent semantic name 425 may be mapped to a native name that corresponds to the same entity (i.e., represents the same database element) in existing SQL query 414. In some cases, a single native name may be mapped to multiple different latent semantic names.

[0074] Figure 5 An example implementation of a subprocess 430 according to an embodiment is illustrated. Initially, in a subprocess 432, the latent semantic names 425 are grouped by their corresponding native names in a SQL query 414. The following table illustrates some examples of mappings extracted from a natural language text input 412 and their corresponding SQL queries 414, where the extracted latent semantic column names are indicated with brackets and their mapped native column names (if any) are indicated with the same number of brackets:

[0075]

[0076] This results in the following potential mapping table output by the entity extraction model 420, grouped by native name and depicted with their respective confidence values:

[0077] # Native column names Semantic column names Confidence value 1 t_cell.id ID 0.97 2 t_cell.id deviceid 0.68 2 t_cell.numneighbor neighbors count 0.48 1 t_cell.num_neighbor neighbors count 0.35 1 t_cell.start_time timestamp 0.99 2 t_cell.start_time timestamp 0.99 3 t_cell.tx_rate1 traffictransmitted (transmitted traffic) 0.67

[0078] In subprocess 434, the first K potential semantic names are identified for each native name. K can be set to any suitable number, such as ten, five, three, etc. The potential semantic names can be sorted according to parameters, such as the confidence value associated with each potential semantic name and / or the number of words in each potential semantic name. For example, the ranking of semantic names with higher confidence values ​​and / or fewer words can be higher than the semantic names with lower confidence values ​​and / or more words. It should be understood that these are just examples, and other parameters or parameter combinations can be used to sort the potential semantic names. Alternatively or in addition, subprocess 434 can exclude all potential semantic names with confidence values ​​lower than a predefined threshold, and / or can exclude any potential semantic names with a word count greater than the maximum word count (or only when there is a potential semantic name with a word count less than the maximum word count for the same native name). The maximum word count should be a small positive integer, such as one, two or three. In general, using too many words (e.g., more than three) may lead to ambiguity when comparing the names of database elements, and therefore, semantic names with fewer words are preferred over semantic names with more words. It should be understood that for some native names, there may be fewer than K potential semantic names. It should also be understood that for a given native name, the same potential semantic name can be collapsed into a single potential semantic name. For example, the native column name "num_neighbor" is associated with two instances of the same potential semantic column name "neighbors count", and the native column name "start_time" is associated with two instances of the same potential semantic column name "timestamp". However, the native column name "id" is associated with different potential semantic column names "IDs" and "device id".

[0079] In subprocess 436, a single best semantic name is selected for each native name from the top K potential semantic names to produce mapping 435. In the event that only a single potential semantic name exists for a given native name, the single potential semantic name can be automatically (e.g., without user intervention), semi-automatically (e.g., after user confirmation), or manually (e.g., after user selection) selected and mapped to the native name in mapping 435. For example, the semantic column name "neighbors count" can be selected for the native column name "num_neighbor", and the semantic column name "traffic transmitted" can be selected for the native column name "tx_rate1". In a semi-automatic or manual embodiment, the administrator may choose to type in a different semantic name for the native name instead of the semantic name provided in potential semantic name 425.

[0080] In the event that there are multiple potential semantic names for a given native name, a single potential semantic name may be selected automatically, semi-automatically, or manually using any technique. In an automatic embodiment, the highest ranked semantic name may be selected. Using the above example, subprocess 436 may select "IDs" as the semantic column name for the native column name "id" because its confidence value is higher than "device id" and / or its number of words is less than "device id". In a semi-automatic embodiment, the highest ranked semantic name may be recommended in a graphical user interface, and the administrator may select the recommended semantic name, select any other semantic name from the top K potential semantic names, or enter a new semantic name via the graphical user interface. For example, the graphical user interface may recommend "IDs" as the semantic column name for the native column name "id", but the administrator may select "device id" instead. In a manual embodiment, the user may view the top K potential semantic names sorted by rank in the graphical user interface and select a semantic name or enter a new semantic name via the graphical user interface.

[0081] In an alternative embodiment, subprocess 434 may be omitted, and in subprocess 436, a single semantic name may be automatically, semi-automatically, or manually selected from all available potential semantic names 425 for a given native name, rather than selecting from only the top K potential semantic names. In the automatic or semi-automatic case, the semantic name with the highest ranking (e.g., in terms of confidence value and / or number of words) may be selected or recommended. In the manual case, the potential semantic names may be listed in order of ranking for selection, with the highest ranked potential semantic name listed first and the lowest ranked potential semantic name listed last.

[0082] In an embodiment, an administrator can manually add additional mappings to mapping 435 if necessary. For example, entity extraction model 420 may not be able to extract semantic names for all database elements that may be queried. In this case, the administrator can use a graphical user interface to enter semantic names into mapping 435 to obtain the native names of any database elements that have not been paired with semantic names. Generally, for a given database table, it is preferred that each database view 115 has the semantic name of each column in the database table.

[0083] In an alternative embodiment, mapping 435 can be generated completely manually. For example, an administrator can view the description of the relevant database elements and manually derive the semantic names of these database elements to generate mapping 435 between the semantic names and native names of the database elements.

[0084] In subprocess 440, one or more database views 115 can be generated from the mapping 435. One database view 115 can be generated for each database table involved in the existing data set 410. Using the above example, a database view 115 can be created for the "t_cell" table. Each database view 115 represents the relevant database elements from the mapping 435 in a format that can be added to the database 114. For example, each database view 115 for a given table maps the native names of the columns in the table to the semantic names of these columns. In addition, each database view 115 for a given table can be named using the semantic names mapped to the native names of the table. Optionally, additional logical columns can be added to one or more database views 115 to handle domain specific queries. The generated (multiple) database views 115 can be added to the database 114 so that the mapped database elements can be queried using the semantic names.

[0085] An example of a representation of a database view 115 that may be added to a database 114 is provided below:

[0086]

[0087] Among them, "t_Gateway_Specific_Data" is the native name of the actual database table, and "gateways" is the semantic name of the database table, and therefore also the name of the database view 115. In addition, "timestamp", "id", "ip_address", "downstream_throughput", "upstream_throughput", and "latency" are semantic names used for the native names "START_TIME", "ID", "IP", "DOWN_T_FROM_B", "UP_T_TO_B", and "AVG_LAT_FROM_G", respectively. It is worth noting that semantic names with two or more words can use underscores instead of spaces between words in order to comply with the naming requirements of SQL. However, compared with native names, semantic names are more descriptive of tables and columns, easier for ordinary people to understand, and more likely to be used in natural language.

[0088] In subprocess 450, mapping 435 may also be used to generate SQL queries 464 having semantic names from existing SQL queries 414 having native names. In particular, subprocess 450 may process each existing SQL query 414 to replace each instance of the native name in mapping 435 with the semantic name mapped in mapping 435 to produce a corresponding modified SQL query 464 having the semantic name. It should be understood that in all other respects, the modified SQL query 464 may be identical to the existing SQL query 414. The resulting modified SQL query 464 having the semantic name instead of the native name may be used as a label for the natural language text input 412 in the new training data set 460. In other words, the existing data set 410 is modified so that each natural language text input 412 is labeled with a SQL query 464 having the semantic name of the database element instead of the corresponding SQL query 414 having the native name of the database element.

[0089] In subprocess 470, text-to-SQL model 330 is trained to generate SQL queries 335 with semantic names from natural language text input 325 using training data set 460. As discussed above, training data set 460 includes natural language text input 412 labeled with SQL queries 464 with semantic names. Training data set 460 should contain the schema of database 114. Once trained, text-to-SQL model 330 can be deployed for use in process 300.

[0090] 2.3. Text to SQL Model

[0091] The text-to-SQL model 330 may be a machine learning model trained via supervised learning. In an embodiment, the text-to-SQL model 330 includes a deep learning neural network, such as a recurrent neural network (RNN). In particular, the RNN may be a sequence-to-sequence (seq2seq) model, such as a semi-autoregressive bottom-up semantic parsing (SmBoP) model, as described in Rubin et al., "SmBoP: Semi-autoregressive Bottom-up Semantic Parsing," in Proceedings of the 5th NLP Workshop on Structured Prediction, Association for Computational Linguistics, pp. 12-21 (August 2021), the entire contents of which are hereby incorporated by reference herein as if fully set forth.

[0092] Figure 6The architecture of the text-to-SQL model 330 according to an embodiment utilizing a seq2seq model is illustrated. The purpose of the text-to-SQL conversion task is to map natural language text into SQL queries. To this end, the seq2seq model includes an encoder 610 and a decoder 620. The encoder 610 encodes the natural language text input 325 into an internal representation 615. The decoder 620 decodes the internal representation 615 into a SQL query 335 with a semantic name. It should be understood that the internal representation 615 essentially encodes the meaning of the natural language text input 325, such as in an internal state vector.

[0093] Encoder 610 may include a Relation-Aware Transformer (RAT-SQL), as described in Wang et al., “RAT-SQL: Relation-aware Schema Encoding and Linking for Text-to-SQL Parsers,” in Proceedings of the 58th Annual Conference of the Association for Computational Linguistics, pp. 7567-7578 (2020), the entire contents of which are hereby incorporated by reference herein as if fully set forth, and GraPPa, as described in Yu et al., “GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing,” in International Conference on Learning Representations (2021), the entire contents of which are hereby incorporated by reference herein as if fully set forth. RAT-SQL uses a transformer as described in Vaswani et al., “Attention Is All You Need,” Advances in Neural Information Processing Systems, pp. 5998-6008 (2017), the entire contents of which are hereby incorporated by reference herein as if fully set forth, to jointly encode natural language queries, relevant database columns, and the schema of the underlying database 114. GraPPa provides a grammar-based semantic parsing mechanism for pre-training.

[0094] 3. Example GUI

[0095] In an embodiment, a graphical user interface may be provided by the server application 112 or the client application 132 to access the functionality of the process 300 . Figure 7An example screen 700 of a graphical user interface according to an embodiment is illustrated. The screen 700 may include an input 710 (e.g., a text box) for entering a natural language text string 315 and an input 712 for submitting the entered natural language text string 315. Additionally or alternatively, the screen 700 may include an input 720 for entering spoken text 305 via an audio interface (e.g., a microphone) of the user system 130. In a preferred embodiment, the screen 700 includes all of the inputs 710, 712, and 720. The screen 700 may also include a text bubble 730 that includes instructions for entering a natural language text string 315 and / or spoken text 305.

[0096] The user may type the natural language text string 315 into input 710 and then select input 712 to submit the natural language text string 315 to a function implementing process 300 (e.g., starting from subprocess 320). Alternatively, the user may select input 720 and then speak a natural language query into an audio interface of user system 130. The natural language query may be recorded as spoken text 305 and submitted to a function implementing process 300 (e.g., starting from subprocess 310). It should be understood that typing the natural language text string 315 into input 710 and the selection of inputs 712 and / or 720 may be performed via an input of user system 130, such as a touch panel display (e.g., of a smart phone or tablet computer).

[0097] In additional or alternative embodiments, the wake-up word detection model can operate continuously in the background to listen to ambient sounds captured by the audio interface of the user system 130 to detect a "wake-up" phrase consisting of one or more words, similar to "Ok Google" or "Hey Alexa". If the wake-up word detection model detects a wake-up phrase, the functionality of process 300 can be activated responsively. The graphical user interface can provide a settings screen that enables the user to train the wake-up word detection model (e.g., using machine learning) to detect a custom user-specified wake-up phrase and / or improve the detection of wake-up phrases in user speech. The wake-up word detection model can be based on a convolutional neural network (CNN) transformer, such as those described in Wang et al., "Wake Word Detection with Streaming Transformers," 2021 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 5864-5868, the entire contents of which are hereby incorporated by reference herein as fully set forth, or any other suitable architecture. This user interface provides completely hands-free voice-enabled operation, which is convenient for field engineers working in austere environments. It should be understood that in this embodiment, screen 700 can omit inputs (e.g., no inputs 710, 712, or 720). Alternatively, in addition to having inputs 710, 712, and / or 720, wake-up word detection can also be used.

[0098] Regardless of how the natural language query is entered, the corresponding natural language text string 315 (whether entered into input 710 or converted by subprocess 310) can be displayed in a text bubble 740 on screen 700 along with the date and time the natural language query was submitted. This provides the user with a time-stamped record of the natural language query. In addition, the functionality of implementing process 300 will pre-process the natural language text string 315 into natural language text input 325 in subprocess 320, convert the natural language text input 325 into a SQL query 335 with a semantic name using a text-to-SQL model 330, make adjustments to the SQL query 335 if necessary in subprocess 340, execute the SQL query 335 on the database 114 using the database view(s) 115 in subprocess 350, and return the results of the SQL query 335 in subprocess 360.

[0099] The results of the SQL query 335 can be displayed in a text bubble 750 on the screen 700, which is located below the text bubble 740 in which the corresponding natural language text string 315 is displayed. The text bubble 750 can include the date and time when the results of the SQL query 335 are returned, so that the relevant time of the results is easy to determine. In the illustrated example, the natural language text string 315 includes "What is the latency of gateway id 216548?" In the illustrated example, the SQL query 335 can include:

[0100]

[0101] Where "gateways" is the name of the database view 115 assigned to the relevant table (e.g., "gateways" is a semantic name that maps to the native name of the relevant table), and "id" and "timestamp" represent semantic column names that map to native column names in the database view 115. As shown in the text bubble 750, the result of the SQL query 335 is a "latency" of "1.6". In a particular embodiment, through the text-to-SQL model 330, the results are returned in about 2 to 3 seconds with an accuracy of up to 92%.

[0102] 4. Example Embodiments

[0103] In an embodiment, the text-to-SQL model 330 is trained to convert natural language text input 325 into SQL queries 335 that utilize semantic names of database elements in the database 114. The text-to-SQL model 330 may be trained using a training dataset 460 that includes natural language text input 412 that has been labeled with SQL queries 464 with semantic names. The SQL queries 464 may be generated from an existing dataset 410 by mapping semantic names (e.g., output by the entity extraction model 420) to native names and replacing native names in existing SQL queries 414 with their mapped semantic names. In an embodiment, prior to applying the text-to-SQL model 330, the natural language text strings 315 are preprocessed into the natural language text input 325 so as to normalize the input to the text-to-SQL model 330 in a manner specific to a particular domain and reduce or eliminate noise. Additionally, database views 115 are generated and added to the database 114 to map semantic names of database elements to native names during execution of the SQL queries 335.

[0104] The combination of text-to-SQL model 330 and database view(s) 115 enables semantic names to be used in SQL queries 335. Using semantic names instead of native names in SQL queries 335 improves the performance of the transformations output by text-to-SQL model 330 (e.g., up to 92% accuracy). Additionally, this allows database 114 to continue to use traditional names for database elements, which means that database 114 and legacy applications that utilize database 114 do not have to be modified to realize the improvements in transformation performance provided by text-to-SQL model 330.

[0105] In an embodiment, database 114 may store performance data of an industrial system, such as a network (e.g., a mesh network). Thus, a natural language query (e.g., whether via spoken text 305 or natural language text string 315) may represent a query about the performance of the industrial system. The ability to use natural language queries enables quick and natural acquisition of such performance data, thereby allowing engineers to easily identify problems in industrial systems and reduce troubleshooting time. Additionally, the use of speech-to-text conversion (e.g., subprocess 310) enables such performance queries to be performed safely, for example, in the field, e.g., without requiring a field engineer to use their hands.

[0106] Embodiment 1: A method, comprising using at least one hardware processor to perform the following operations: adding a database view to a database, wherein the database view maps native names of database elements in the database to semantic names of the database elements; obtaining a natural language text input representing a natural language query; applying a machine learning text-to-SQL model to the natural language text input to generate a structured query language (SQL) query, wherein the SQL query includes a semantic name for each database element referenced in the SQL query; and using the database view to map the semantic name for each database element referenced in the SQL query to the native name of the database element to execute the SQL query on the database.

[0107] Embodiment 2: The method as described in Embodiment 1, wherein the database elements include tables and columns within the tables.

[0108] Embodiment 3: The method as described in any of the preceding embodiments further includes using the at least one hardware processor to perform the following operations: receiving a natural language text string; and normalizing the natural language text string to obtain the natural language text input.

[0109] Embodiment 4: A method as described in Embodiment 3, wherein normalizing the natural language text string comprises: identifying a text transcription of a network address in the natural language text string; and replacing the text transcription of the network address with a representation of the network address in a standard format.

[0110] Example 5: A method as described in Example 3 or 4, wherein normalizing the natural language text string includes: identifying a text transcription of one or both of a date and a time in the natural language text string; and replacing the text transcription of one or both of the date and the time with a timestamp value representing one or both of the date and the time.

[0111] Example 6: A method as described in any one of Examples 3 to 5, wherein normalizing the natural language text string includes: identifying a text transcription of a term in the natural language text string for an entity represented in the database; and replacing the text transcription of the term with a standard term for the entity.

[0112] Embodiment 7: A method as described in any one of Embodiments 3 to 6, wherein the natural language text string is received via input of a graphical user interface, and wherein the method further comprises using the at least one hardware processor to: receive a result of executing the SQL query on the database; and display a representation of the result in the graphical user interface.

[0113] Embodiment 8: A method as described in any one of Embodiments 3 to 7, wherein the natural language text string is received from an external system, and wherein the method further includes using the at least one hardware processor to perform the following operations: receiving the result of executing the SQL query on the database; and returning a representation of the result to the external system.

[0114] Embodiment 9: A method as described in any of the preceding embodiments, wherein the natural language query and the SQL query include a request for a value of at least one performance parameter of the network.

[0115] Embodiment 10: The method of embodiment 9, wherein the at least one performance parameter of the network is a utility network. The utility network may be an industrial wireless mesh network or a substation network, wherein the performance measurement results of the network are stored in the database for query using natural language.

[0116] Embodiment 11: A system comprising: at least one hardware processor; and software, wherein the software is configured to perform the method as described in any one of Embodiments 1 to 10 when executed by the at least one hardware processor.

[0117] Embodiment 12: A non-transitory computer-readable medium having instructions stored thereon, wherein the instructions, when executed by a processor, cause the processor to perform the method as described in any one of Embodiments 1 to 10.

[0118] Embodiment 13: A method comprising using at least one hardware processor to perform the following operations: generating a representation of a database view of a database, wherein the database view maps native names for database elements in the database to semantic names for the database elements; and using a training data set to train a machine learning text-to-SQL model to generate a structured query language (SQL) query from a natural language text input representing a natural language query, wherein the SQL query includes the semantic name of each database element referenced in the SQL query, wherein the training data set includes the natural language text input labeled with the SQL query.

[0119] Embodiment 14: The method as described in Embodiment 13, wherein the database elements include tables and columns within the tables.

[0120] Embodiment 15: The method as described in Embodiment 13 or 14 further includes using the at least one hardware processor to generate the training data set by the following operations: obtaining an existing data set including natural language text input labeled with an existing SQL query, the existing SQL query including a native name for each database element referenced in the existing SQL query; identifying semantic names used for the native names in the existing SQL query; and replacing the native names in the existing SQL query with the identified semantic names to produce a modified SQL query, wherein the training data set includes natural language text input labeled with the modified SQL query from the existing data set.

[0121] Embodiment 16: The method as described in Embodiment 15 also includes using the at least one hardware processor to perform the following operations: using another training data set including natural language phrases labeled with semantic names to train an entity extraction model to extract semantic names of database elements from natural language questions.

[0122] Embodiment 17: A method as described in Embodiment 16, wherein generating the representation of the database view includes: applying the trained entity extraction model to the natural language text input in the existing data set to extract the semantic names of the database elements; and associating the extracted semantic names with corresponding native names in the native names in the existing SQL query to produce the database view.

[0123] Embodiment 18: The method as described in any one of Embodiments 13 to 17, wherein one or more of the natural language queries in the training data set includes a request for a value of at least one performance parameter of the network.

[0124] Embodiment 19: The method of embodiment 18, wherein the at least one performance parameter of the network is a utility network. The utility network may be an industrial wireless mesh network or a substation network, wherein the performance measurement results of the network are stored in the database for query using natural language.

[0125] Embodiment 20: The method as described in any one of Embodiments 13 to 19 further includes using the at least one hardware processor to perform the following operations: applying the representation of the database view to the database to add the database view to the database; and for each of one or more natural language text inputs specified by the user, applying the trained machine learning text-to-SQL model to the user-specified natural language text input to generate a SQL query, and executing the generated SQL query on the database using the database view.

[0126] Embodiment 21: The method as described in Embodiment 20 further includes using the at least one hardware processor to perform the following operations for each of the one or more natural language text inputs specified by the user: receiving the result of executing the generated SQL query on the database; and returning the result to the user system.

[0127] Embodiment 22: The method as described in Embodiment 20 or 21 further includes using the at least one hardware processor to perform the following operations for each of the one or more natural language text inputs specified by the user: receiving a natural language text string specified by the user; and normalizing the natural language text string to obtain the natural language text input specified by the user.

[0128] Embodiment 23: A method as described in Embodiment 22, wherein normalizing the natural language text string includes: identifying a text transcription of a specific item in the natural language text string; and replacing the text transcription of the specific item with a standardized representation of the specific item.

[0129] Embodiment 24: A system comprising: at least one hardware processor; and software, wherein the software is configured to perform the method as described in any one of Embodiments 13 to 23 when executed by the at least one hardware processor.

[0130] Embodiment 25: A non-transitory computer-readable medium having instructions stored thereon, wherein the instructions, when executed by a processor, cause the processor to perform the method as described in any one of Embodiments 13 to 23.

[0131] The above description of the disclosed embodiments is provided to enable any person skilled in the art to implement or use the present invention. It will become apparent to those skilled in the art that various modifications to these embodiments will become apparent, and the general principles described herein may be applied to other embodiments without departing from the spirit or scope of the present invention. Therefore, it should be understood that the description and accompanying drawings presented herein represent the present invention's current preferred embodiments, and therefore represent the subject matter of the present invention's broad considerations. It should be further understood that the scope of the present invention fully encompasses other embodiments that are apparent to those skilled in the art, and therefore the scope of the present invention is not limited.

[0132] Combinations such as "at least one of A, B, or C", "one or more of A, B, or C", "at least one of A, B and C", "one or more of A, B, and C", and "A, B, C, or any combination thereof" described herein include any combination of A, B, and / or C, and may include multiple A, multiple B, or multiple C. Specifically, combinations such as "at least one of A, B, or C", "one or more of A, B, or C", "at least one of A, B, and C", "one or more of A, B, and C", and "A, B, C, or any combination thereof" may be only A, only B, only C, A and B, A and C, B and C, or A, B, and C, and any such combination may contain one or more members of its constituent parts A, B, and / or C. For example, the combination of A and B may include one A and multiple Bs, multiple A and one B, or multiple A and multiple Bs.

Claims

1. A method comprising using at least one hardware processor to perform the following operations: Add a database view to the database where, The database view maps native names of database elements in the database to semantic names of the database elements; obtaining a natural language text input representing a natural language query; applying a machine learning text-to-SQL model to the natural language text input to generate a structured query language (SQL) query, the SQL query including a semantic name for each database element referenced in the SQL query; and The SQL query is executed on the database using the database view to map the semantic name for each database element referenced in the SQL query to the native name of the database element.

2. The method of claim 1, wherein: The database elements include tables and columns within the tables.

3. The method of claim 1 , further comprising using the at least one hardware processor to: receiving a natural language text string; and The natural language text string is normalized to obtain the natural language text input.

4. The method of claim 3, wherein: Normalizing the natural language text string includes: identifying a textual transcription of a network address in the natural language text string; and The textual transcription of the network address is replaced with a representation of the network address in a standard format.

5. The method of claim 3, wherein: Normalizing the natural language text string includes: identifying a textual transcription of one or both of a date and a time in the natural language text string; and The textual transcription of one or both of the date and time is replaced with a timestamp value representing one or both of the date and time.

6. The method of claim 3, wherein: Normalizing the natural language text string includes: identifying a textual transcription of a term in the natural language text string for an entity represented in the database; and Replace the text transcription of the term with the standard term for the entity.

7. The method of claim 3, wherein: The natural language text string is received via an input of a graphical user interface, and wherein the method further comprises using the at least one hardware processor to: receiving a result of executing the SQL query on the database; and A representation of the results is displayed in the graphical user interface.

8. The method of claim 3, wherein: The natural language text string is received from an external system, and wherein the method further comprises using the at least one hardware processor to: receiving a result of executing the SQL query on the database; and A representation of the results is returned to the external system.

9. The method of claim 1, wherein: The natural language query and the SQL query include a request for a value of at least one performance parameter of a network.

10. The method of claim 9, wherein: The at least one performance parameter of the network is a utility network, wherein the utility network is an industrial wireless mesh network or a substation network, wherein performance measurements of the network are stored in the database for querying using natural language.

11. A non-transitory computer readable medium having instructions stored thereon, wherein: When executed by a processor, the instructions cause the processor to perform the following operations: adding a database view to a database, wherein the database view maps native names of database elements in the database to semantic names of the database elements; obtaining a natural language text input representing a natural language query; applying a machine learning text-to-SQL model to the natural language text input to generate a structured query language (SQL) query, the SQL query including a semantic name for each database element referenced in the SQL query; and The SQL query is executed on the database using the database view to map the semantic name for each database element referenced in the SQL query to the native name of the database element.

12. A method comprising using at least one hardware processor to: Generates a representation of a database view of a database where The database view maps native names for database elements in the database to semantic names for the database elements; and A machine learning text-to-SQL model is trained using a training dataset to generate a structured query language (SQL) query from a natural language text input representing a natural language query, the SQL query including a semantic name for each database element referenced in the SQL query, the training dataset including the natural language text input labeled with the SQL query.

13. The method of claim 12, wherein: The database elements include tables and columns within the tables.

14. The method of claim 12, further comprising using the at least one hardware processor to generate the training data set by: obtaining an existing data set comprising a natural language text input tagged with an existing SQL query, the existing SQL query comprising a native name for each database element referenced in the existing SQL query; identifying a semantic name for a native name used in the existing SQL query; as well as replacing native names in the existing SQL query with the identified semantic names to generate a modified SQL query, The training data set includes natural language text input from the existing data set marked with the modified SQL query.

15. The method of claim 14, further comprising using the at least one hardware processor to perform the following operations: using another training data set including natural language phrases labeled with semantic names to train an entity extraction model to extract semantic names of database elements from natural language questions.

16. The method of claim 15, wherein: Generating the representation of the database view comprises: applying the trained entity extraction model to natural language text input in the existing dataset to extract semantic names of the database elements; and The extracted semantic names are associated with corresponding ones of the native names in the existing SQL query to generate the database view.

17. The method of claim 12, wherein: One or more of the natural language queries in the training data set include a request for a value of at least one performance parameter of the network.

18. The method of claim 17, wherein: The at least one performance parameter of the network is a utility network, wherein the utility network is an industrial wireless mesh network or a substation network, wherein performance measurements of the network are stored in the database for querying using natural language.

19. The method of claim 12, further comprising using the at least one hardware processor to: applying a representation of the database view to the database to add the database view to the database; and For each of the one or more natural language text inputs specified by the user, applying the trained machine learning text-to-SQL model to the user-specified natural language text input to generate a SQL query, and The generated SQL query is executed on the database using the database view.

20. The method of claim 19, further comprising, for each of the one or more natural language text inputs specified by the user, using the at least one hardware processor to: receiving a result of executing the generated SQL query on the database; and The results are returned to the user system.

21. The method of claim 19, further comprising, for each of the one or more natural language text inputs specified by the user, using the at least one hardware processor to perform the following operations: receiving a natural language text string specified by a user; and The natural language text string is normalized to obtain the user-specified natural language text input.

22. The method of claim 21, wherein: Normalizing the natural language text string includes: identifying a text transcript of a particular term in the natural language text string; and The text transcription of the particular term is replaced with a standardized representation of the particular term.

Images