Method for performing data analysis according to natural language query by using generative ai, and electronic device for performing same

Abstract

This method for performing data analysis, according to a natural language query of a user, comprises the steps of: receiving, from a user terminal, a user input including a natural language query in which data analysis is requested; determining, as a target database, at least one database from among a plurality of databases on the basis of the user input; generating a prompt on the basis of the user input and the target database; acquiring a structured query language (SQL) syntax by inputting the prompt into a code generation model; outputting a data analysis result for the target database by executing the SQL syntax; and transmitting the data analysis result to the user terminal, wherein the data analysis result can be displayed on a screen of the user terminal.

Description

A method for performing data analysis based on natural language queries using generative AI and an electronic device for performing the same

[0001] The present disclosure relates to a method for performing data analysis based on a user's natural language query, and more particularly, to a method for performing analysis of data stored in a database by generating an SQL statement corresponding to the request using a generative AI model when the user requests data analysis in natural language, and executing the SQL statement, and to an electronic device and system for performing the same.

[0002] Recently, vast amounts of data are being produced across various fields, and the amount of data produced is steadily increasing. To utilize this data, it must be analyzed according to specific purposes or criteria. Analyzing data stored in databases requires SQL (Structured Query Language) syntax. However, for general users without specialized knowledge, writing SQL syntax is difficult without time-consuming and costly professional training.

[0003] If a system or service can be implemented to perform data analysis based on a user's request for data analysis through a natural language query, user convenience will be improved and more users will be able to utilize the data.

[0004] According to one aspect of the present disclosure, a method for performing data analysis based on a user's natural language query includes the steps of: receiving a user input including a natural language query requesting data analysis from a user terminal; determining at least one database among a plurality of databases as a target database based on the user input; generating a prompt based on the user input and the target database; obtaining a Structured Query Language syntax by inputting the prompt into a code generation model; outputting a data analysis result for the target database by executing the SQL syntax; and transmitting the data analysis result to the user terminal, wherein the data analysis result can be displayed on a screen of the user terminal.

[0005] According to one aspect of the present disclosure, an electronic device for performing data analysis based on a user's natural language query includes a memory storing one or more instructions and at least one processor functionally coupled to the memory, wherein the at least one processor executes the at least one instruction, whereby the electronic device receives a user input including a natural language query requesting data analysis from a user terminal, determines at least one database among a plurality of databases as a target database based on the user input, generates a prompt based on the user input and the target database, obtains a Structured Query Language syntax by inputting the prompt into a code generation model, and outputs a data analysis result for the target database by executing the SQL syntax, and then transmits the data analysis result to the user terminal, and the data analysis result can be displayed on a screen of the user terminal.

[0006] A computer-readable recording medium disclosed as a technical means for achieving a technical task may have stored thereon a program for executing at least one of the embodiments of the disclosed method on a computer.

[0007] A computer program disclosed as a technical means for achieving a technical task may be stored on a medium for performing at least one of the embodiments of the disclosed method on a computer.

[0008] The above and other aspects, features and advantages according to specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0009] FIG. 1 is a diagram illustrating a system for performing data analysis based on a user's natural language query according to one or more embodiments of the present disclosure.

[0010] FIG. 2 is a drawing for explaining the hardware configuration included in the electronic device of FIG. 1.

[0011] FIG. 3 is a diagram illustrating a method for training a code generation model according to one or more embodiments of the present disclosure.

[0012] FIG. 4 is a diagram illustrating a method for training a similarity judgment model according to one or more embodiments of the present disclosure.

[0013] FIG. 5 is a diagram illustrating low-code data for training a code generation model according to one or more embodiments of the present disclosure.

[0014] FIG. 6 is a diagram illustrating the results of performing preprocessing for learning on the low code data of FIG. 5 according to one or more embodiments.

[0015] FIG. 7 is a diagram illustrating instruction tuning data for training a code generation model according to one or more embodiments of the present disclosure.

[0016] FIG. 8 is a diagram illustrating a UI screen for a user to select an analysis history, displayed during a process of performing a data analysis method according to one or more embodiments of the present disclosure.

[0017] FIG. 9 is a diagram illustrating analysis history data stored in a history repository of a system according to one or more embodiments of the present disclosure.

[0018] FIG. 10 is a diagram illustrating a situation in which details of an analysis history are displayed on a UI screen for selecting an analysis history according to one or more embodiments of the present disclosure.

[0019] FIG. 11 is a diagram illustrating a UI screen for a user to input a natural language query requesting data analysis, displayed during a process of performing a data analysis method according to one or more embodiments of the present disclosure.

[0020] FIG. 12 is a diagram illustrating metadata of databases according to one or more embodiments of the present disclosure.

[0021] FIGS. 13A to 13C are diagrams illustrating UI screens displaying metadata and tables of databases according to one or more embodiments of the present disclosure.

[0022] FIG. 14 is a diagram for explaining a method for determining a target database based on similarity judgment in the process of performing a data analysis method according to one or more embodiments of the present disclosure.

[0023] FIG. 15 is a diagram illustrating the structure of a prompt generated in the process of performing a data analysis method according to one or more embodiments of the present disclosure.

[0024] FIG. 16 is a diagram illustrating details of a portion related to analysis history 1 included in the prompt of FIG. 15 according to one or more embodiments.

[0025] FIG. 17 is a diagram illustrating details of a portion related to analysis history 2 included in the prompt of FIG. 15 according to one or more embodiments.

[0026] FIG. 18 is a diagram illustrating details of a current natural language query related portion included in the prompt of FIG. 15 according to one or more embodiments.

[0027] FIG. 19 is a diagram illustrating an example in which a code generation model generates SQL statements when a prompt is input to the code generation model during a process of performing a data analysis method according to one or more embodiments of the present disclosure.

[0028] FIG. 20 is a diagram illustrating a UI screen that outputs the results of data analysis by executing a generated SQL statement during a process of performing a data analysis method according to one or more embodiments of the present disclosure.

[0029] FIG. 21 is a drawing for explaining a method of visualizing data analysis results in the form of a graph or diagram, etc., during the process of performing a data analysis method according to one or more embodiments of the present disclosure.

[0030] FIG. 22 is a flowchart illustrating a method of analyzing data according to a user's natural language query according to one or more embodiments of the present disclosure.

[0031] FIG. 23 is a flowchart illustrating detailed steps included in step 2201 of FIG. 22 according to one or more embodiments.

[0032] FIG. 24 is a flowchart illustrating detailed steps included in step 2202 of FIG. 22 according to one or more embodiments.

[0033] FIG. 25 is a flowchart illustrating detailed steps included in step 2203 of FIG. 22 according to one or more embodiments.

[0034] FIG. 26 is a flowchart illustrating detailed steps included in step 2205 of FIG. 22 according to one or more embodiments.

[0035] In this disclosure, the expression “at least one of a, b or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “all of a, b and c”, or variations thereof.

[0036] In describing this disclosure, descriptions of technical details that are well-known in the technical field to which this disclosure pertains and are not directly related to this disclosure will be omitted. This is to avoid obscuring the gist of this disclosure by omitting unnecessary explanations and to convey it more clearly. Furthermore, the terms described below are defined based on their functions in this disclosure and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the contents of this specification as a whole.

[0037] For the same reason, some components in the attached drawings are exaggerated, omitted, or schematically depicted. Furthermore, the dimensions of each component do not entirely reflect its actual size. Identical or corresponding components in each drawing are assigned the same reference numbers.

[0038] The advantages and features of the present disclosure, and methods for achieving them, will become clearer with reference to the embodiments described below in detail with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. The disclosed embodiments are provided to ensure that the disclosure of the present disclosure is complete and to fully inform those skilled in the art of the present disclosure of the scope of the disclosure. One or more embodiments of the present disclosure may be defined according to the claims. Like reference numerals denote like elements throughout the specification. In addition, when describing one or more embodiments of the present disclosure, if a detailed description of a related function or configuration is determined to unnecessarily obscure the gist of the present disclosure, the detailed description thereof will be omitted. In addition, the terms described below are terms defined in consideration of the functions of the present disclosure and may vary depending on the intentions or customs of the user or operator. Therefore, the definitions should be made based on the contents throughout the specification.

[0039] In one or more embodiments, each block of the flowchart diagrams and combinations of the flowchart diagrams may be implemented by computer program instructions. The computer program instructions may be installed on a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, and the instructions, when executed by the processor of the computer or other programmable data processing apparatus, may create means for performing the functions described in the flowchart block(s). The computer program instructions may also be stored in a computer-available or computer-readable memory that can direct a computer or other programmable data processing apparatus to implement the functions in a particular manner, and the instructions stored in the computer-available or computer-readable memory may also produce an article of manufacture that includes instruction means for performing the functions described in the flowchart block(s). The computer program instructions may also be installed on a computer or other programmable data processing apparatus.

[0040] Additionally, each block in the flowchart diagram may represent a module, segment, or portion of code that includes one or more executable instructions for performing a specified logical function(s). In one or more embodiments, the functions described in the blocks may occur out of order. For example, two blocks depicted in succession may be executed substantially simultaneously or, depending on the function, may be executed in reverse order.

[0041] The term '~ unit' used in one or more embodiments of the present disclosure may represent software or a hardware component such as a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC), and the '~ unit' may perform a specific role. Meanwhile, the '~ unit' is not limited to software or hardware. The '~ unit' may be configured to be on an addressable storage medium and may be configured to play one or more processors. In one or more embodiments, the '~ unit' may include components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided by a specific component or a specific '~ unit' may be combined to reduce the number of components or separated into additional components. Furthermore, in one or more embodiments, the '~ unit' may include one or more processors.

[0042] Below, exemplary meanings of terms used in this disclosure are explained.

[0043] In one or more examples, "generative AI" may refer to artificial intelligence technology capable of generating new text, images, etc. in response to input data (e.g., text, images, etc.). Representative examples of generative AI are described in the "Generative Model" section below.

[0044] In one or more examples, a 'generative model' may refer to a neural network model that implements generative AI technology. The generative model can generate new data having similar characteristics to the input data or new data corresponding to the input data by learning the patterns and structures of training data. For example, if the input data is text containing a question, the generative model can generate and output an answer to the question. In one or more examples, if the input data is text containing a request, the generative model can output text or an image generated according to the request. A 'code generation model' according to one or more embodiments of the present disclosure is a generative model that generates and outputs code (SQL statements) when a generated prompt is input based on a user's input (e.g., analysis history selection, natural language query, etc.). Instead of 'generative model' or 'code generation model', terms such as 'generative artificial intelligence model', 'language model', or 'neural network model' may be used.

[0045] In one or more examples, a "natural language query" may mean a text composed in natural language that requests the performance of a specific action or requests specific information. According to one or more embodiments of the present disclosure, a user may input a natural language query requesting data analysis to a system or electronic device for performing data analysis. Instead of a "natural language query," terms such as "natural language instruction," "natural language input," "natural language request," "analysis query," "analysis request," or "instruction" may also be used.

[0046] In one or more examples, 'SQL syntax (Structured Query Language syntax)' may refer to code written according to SQL syntax for performing data analysis. For reference, 'SQL (Structured Query Language)' may refer to a standard search language that connects a user and a database. For example, an SQL syntax may include content that instructs to aggregate, extract, classify, or sort information (e.g., data values) contained in a database according to specific criteria. According to one or more embodiments of the present disclosure, an electronic device may request information from a database through an SQL syntax. Furthermore, according to one or more embodiments of the present disclosure, an electronic device may perform analysis on data stored in a database by executing an SQL syntax in the database. According to one or more embodiments of the present disclosure, a code generation model, which is a generative AI model, may generate an SQL syntax. Instead of 'SQL syntax', terms such as 'SQL query', 'SQL-based request', 'query', or 'code' may also be used. Embodiments of the present disclosure are not limited to SQL as understood by those of ordinary skill in the art, and may include any suitable database language known to those of ordinary skill in the art.

[0047] In one or more examples, a 'database' may refer to a space where data is stored. According to one or more embodiments of the present disclosure, a 'table' created using data may be stored in the 'database'. Furthermore, according to one or more embodiments of the present disclosure, the 'database' may also refer to a 'table' created using data stored therein. A database according to one or more embodiments of the present disclosure may be a relational database (RDB), and a relational database may refer to a collection of tables composed of rows and columns that are gathered while forming relationships with other tables. Data stored in a relational database may be referred to as a relational table.

[0048] In one or more examples, the term "target database" may refer to a database in which data to be analyzed is stored. According to one or more embodiments of the present disclosure, an electronic device may select a target database from among a plurality of databases based on user input (e.g., analysis history selection and natural language query), and generate SQL statements and perform data analysis based on the selected target database. Instead of the term "target database," terms such as "analysis database," "associated database," or "related database" may also be used.

[0049] In one or more examples, a "database management system (DBMS)" can refer to a configuration that manages a database, providing an environment in which applications can share and use the database. Typically, applications do not directly manipulate the database; instead, separate software is used to manipulate the database, which can be referred to as a database management system.

[0050] In one or more examples, 'analysis history' may mean analysis work on previously performed data. The analysis history may be structured in a hierarchical structure, and the analysis history of a lower layer may include analysis work corresponding to the analysis history of a higher layer. This will be described in detail below with reference to the drawings. A system or electronic device according to one or more embodiments of the present disclosure may store 'data corresponding to the analysis history', and the data corresponding to the analysis history may include the name of the analysis work, a natural language query input when performing the analysis work, an SQL statement generated when performing the analysis work, an execution result of the SQL statement, and the like. Instead of 'analysis history', terms such as 'analysis work', 'analysis task', or 'history' may be used.

[0051] In one or more examples, "metadata" of a database may refer to data describing the database. Metadata may include a table catalog and a table schema.

[0052] In one or more examples, a 'table catalog' may include descriptions of the tables contained in the database (e.g., what information is stored in the tables) and descriptions of each column in the tables. Examples of specific table catalogs are described in detail below with reference to the drawings.

[0053] In one or more examples, a 'table schema' may be information defining the structure and rules of tables included in a database. For example, a table schema may refer to a logical structure indicating how information is stored within a database. For example, a table schema may include information regarding which tables to create, which columns to include in each table, what constraints each column has, how relationships between tables are to be structured, etc. According to one or more embodiments of the present disclosure, a table schema may include commands created to create tables. Specific examples of table schemas are described in detail below with reference to the drawings.

[0054] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0055] Embodiments of the present disclosure relate to a method for performing data analysis by generating SQL statements corresponding to a user's request using a generative AI model when a user requests data analysis in natural language, and executing the generated SQL statements in a database, and an electronic device or system for performing the same.

[0056] FIG. 1 is a diagram illustrating a system for performing data analysis based on a user's natural language query, according to one or more embodiments of the present disclosure.

[0057] Referring to FIG. 1, a system according to one or more embodiments may include a user terminal (20), an electronic device (200), and a plurality of databases (31, 32, 33), and the electronic device (200) may include a frontend server (110), a backend server (120), a database management system (DBMS) (130), a search module (140), a history storage (150), a code generation model (11), and a similarity judgment model (12).

[0058] The user terminal (20) may be a device that provides an interface for data analysis to the user (1). The user (1) may request data analysis and check the data analysis results through the input / output interface (e.g., display panel, keyboard, mouse, etc.) of the user terminal (20). In the present disclosure, UI screens (UI screens illustrated in the drawings) output during the process of performing data analysis may be displayed on the user terminal (20) at the request of the electronic device (200). According to one or more embodiments of the present disclosure, an application or program for data analysis may be installed on the user terminal (20), and when the user (1) executes the application or program, the user terminal (20) may access the electronic device (200) and request execution of processes for data analysis. The user terminal (20) may be implemented as various types of electronic devices, and for example, a laptop, a desktop PC, a tablet, a mobile phone, etc. may be used as the user terminal (20).

[0059] A plurality of databases (31, 32, 33) may store various data that are subject to analysis in embodiments of the present disclosure. According to one or more embodiments, each database may store at least one table created using the data. The plurality of databases (31, 32, 33) may be controlled by a database management system (130).

[0060] The electronic device (200) is a device for analyzing data stored in databases (31, 32, 33) according to a request from a user (1) received through a user terminal (20).

[0061] The detailed components (11, 12, 110, 120, 130, 140, 150) included in the electronic device (200) of FIG. 1 are components classified based on their functions or roles. In FIG. 1, one electronic device (200) is illustrated as including all of the detailed components (11, 12, 110, 120, 130, 140, 150), and in reality, the electronic device (200) may be implemented in this manner. However, the present invention is not limited thereto, and the detailed components (11, 12, 110, 120, 130, 140, 150) may each be implemented as an independent hardware device, or two or more detailed components may be integrated, or one detailed component may be implemented to be included in another detailed component. As will be understood by those skilled in the art, each server illustrated in FIG. 1 may be implemented as one or more servers. Additionally, the databases (31, 32, 33) may be located remotely from the electronic device (200).

[0062] For example, the front-end server (110) and the back-end server (120) may each be independent hardware devices, or may be hardware / software configurations included in a single hardware device. If the front-end server (110) and the back-end server (120) are each implemented as independent hardware devices, the electronic device (200) may be an electronic system. The back-end server (120) may be configured to include a code generation model (11), a similarity determination model (12), and a search module (140), as illustrated in FIG. 1.

[0063] For example, the history storage (150) may be a separate memory device for storing data, or may be implemented to be included in the backend server (120).

[0064] For example, the code generation model (11) may refer to an independent hardware device that executes a generative AI model that generates code, or may refer to a generative AI model executed by a backend server (120). Also, for example, the similarity judgment model (12) may refer to an independent hardware device that executes an AI model that determines similarity between text data, or may refer to an AI model executed by a backend server (120).

[0065] For example, the database management system (130) may be implemented as a single independent server and may include multiple databases (31, 32, 33).

[0066] According to one or more embodiments of the present disclosure, some of the detailed configurations (11, 12, 110, 120, 130, 140, 150) may be implemented to be included in the user terminal (20).

[0067] In this way, the detailed configurations (11, 12, 110, 120, 130, 140, 150) included in the system according to one or more embodiments of the present disclosure may be hardware configurations or software configurations, and may be implemented in the form of various electronic devices (e.g., one electronic device or a combination of two or more electronic devices).

[0068] In the present disclosure, embodiments are described assuming that one electronic device (200) includes all detailed components (11, 12, 110, 120, 130, 140, 150). Therefore, in the embodiments below, the operations described as being performed by the detailed components (11, 12, 110, 120, 130, 140, 150) of FIG. 1 can be viewed as being performed by the processor (220) of the electronic device (200) illustrated in FIG. 2 executing a program or at least one instruction stored in the memory (230).

[0069] FIG. 2 is a diagram illustrating a hardware configuration included in the electronic device of FIG. 1. Referring to FIG. 2, an electronic device (200) according to one or more embodiments may include a communication interface (210), a processor (220), and a memory (230). However, the components of the electronic device (200) are not limited to the above-described examples, and the electronic device (200) may include more or fewer components than the above-described components. Some or all of the communication interface (210), the processor (220), and the memory (230) may be implemented in the form of a single chip. According to one or more embodiments of the present disclosure, the electronic device (200) may be a server operated by a business operator providing a data analysis service. In one or more examples, the electronic device (200) may correspond to a user terminal (20). In one or more examples, the electronic device (200) may be functionally similar to the user terminal (20), wherein the processor (220) may perform the functions of a front-end server (110), a back-end server (120), and a database management system (130), and the memory (230) may store the same information as the history storage (150).

[0070] In one or more examples, the communication interface (210) may be implemented to include a communication chipset that supports various communication protocols as a configuration for transmitting and receiving signals (such as control commands and data) with an external device via wire or wirelessly. The communication interface (210) may receive a signal from the outside and output it to the processor (220), or transmit a signal output from the processor (220) to the outside. The electronic device (200) may communicate with a user terminal (20) or a plurality of databases (31, 32, 33) via the communication interface (210).

[0071] In one or more examples, the electronic device (200) may transmit information for displaying a UI screen to a user terminal (20) via a communication interface (210) and receive an input requesting data analysis from the user terminal (20). In addition, the electronic device (200) may access at least one of a plurality of databases (31, 32, 33) via the communication interface (210) to perform data analysis and obtain analysis results.

[0072] In one or more examples, the processor (220) may be configured with one or more processors as a component that controls a series of processes so that the electronic device (200) operates according to the embodiments described below. The one or more processors included in the processor (220) may be circuitry such as a System on Chip (SoC), an Integrated Circuit (IC), etc. The one or more processors included in the processor (220) may be a general-purpose processor such as a Central Processing Unit (CPU), a Micro Processor Unit (MPU), an Application Processor (AP), a Digital Signal Processor (DSP), a graphics-only processor such as a Graphics Processing Unit (GPU), a Vision Processing Unit (VPU), an artificial intelligence-only processor such as a Neural Processing Unit (NPU), or a communication-only processor such as a Communication Processor (CP). When the one or more processors included in the processor (220) are artificial intelligence-only processors, the artificial intelligence-only processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0073] The processor (220) can write data to the memory (230) or read data stored in the memory (230), and in particular, process data according to predefined operation rules or artificial intelligence models by executing a program or at least one instruction stored in the memory (230). Accordingly, the processor (220) can perform operations described in the following embodiments, and operations described as being performed by the electronic device (200) or detailed components (11, 12, 110, 120, 130, 140, 150) included in the electronic device (200) in the following embodiments can be regarded as being performed by the processor (220) unless otherwise specifically described.

[0074] In one or more examples, the memory (230) may be configured as a storage medium or a combination of storage media such as a ROM, a RAM, a hard disk, a CD-ROM, and a DVD, as a configuration for storing various programs or data. The memory (230) may not exist separately and may be configured to be included in the processor (220). The memory (230) may be configured as a volatile memory, a non-volatile memory, or a combination of volatile memory and non-volatile memory. A program or at least one instruction for performing operations according to embodiments described below may be stored in the memory (230). The memory (230) may also provide stored data to the processor (220) upon request of the processor (220).

[0075] According to embodiments of the present disclosure, when a user (1) inputs a natural language query requesting data analysis through a user terminal (20), the electronic device (200) generates an SQL statement corresponding to the natural language query using a code generation model (11) and executes the generated SQL statement in databases (31, 32, 33) to perform data analysis. In one or more examples, the natural language query may be a query that allows the user to use general human language, such as "Search for all data generated last month." Hereinafter, embodiments in which the electronic device (200) performs data analysis according to a user's natural language query will be described in detail.

[0076] First, the roles of the two neural network models (11, 12) included in the electronic device (200) and the method for training them are described, and then the operations of generating SQL statements corresponding to natural language queries and performing data analysis using the trained neural network models (11, 12) are described.

[0077] 1. Role and learning of the code generation model (11)

[0078] According to one or more embodiments, the code generation model (11) may be a generative AI model for generating SQL statements (code) corresponding to a natural language query of a user (1). The prompt input to the code generation model (11) includes the natural language query input by the user (1), and may additionally include data corresponding to an analysis history (previous analysis work) selected by the user (1) and metadata (table catalog and table schema) of the target database.

[0079] According to one or more embodiments of the present disclosure, a target database may be determined based on a natural language query entered by a user (1). Furthermore, according to one or more embodiments of the present disclosure, the target database may also be determined based on a natural language query entered by the user (1) and an analysis history selected by the user (1). A specific method for generating a prompt to be input into a code generation model (11) and examples of the prompts will be described in detail below with reference to the corresponding drawings.

[0080] FIG. 3 is a diagram illustrating a method for training a code generation model according to one or more embodiments of the present disclosure. Referring to FIG. 3, raw code data (310), instruction tuning data (320), and natural data (330) may be used to train a code generation model (11), and preprocessing may be performed on at least some of the training data (310, 320, 330) to increase learning efficiency.

[0081] (1) Collection of training data

[0082] In one or more examples, the raw code data (310) may be data used to learn what format code is generally in. Since the code generation model (11) must generate code, various types of codes can be used to teach the code generation model (11) the general format of the code.

[0083] A specific example of low-code data (310) is illustrated in FIG. 5. Referring to FIG. 5, low-code data (500) may include various codes (SQL statements) collected arbitrarily. According to one or more embodiments of the present disclosure, low-code data (500) may be collected from a repository where various codes are stored, such as GitHub. As will be understood by those skilled in the art, the codes are not limited to SQL and may include any suitable code known to those skilled in the art for querying a database.

[0084] In one or more examples, the instruction tuning data (320) is data used to learn which code (SQL statement) corresponds to which instruction in natural language. Learning of the code generation model (11) using the instruction tuning data (320) can be said to be a concept of semantically aligning codes corresponding to instructions in natural language. Since the instruction tuning data (320) is training data for fine tuning, it can be configured to include a much smaller amount of data compared to the low-code data (310).

[0085] A specific example of instruction tuning data (320) is illustrated in FIG. 7. Referring to FIG. 7, the instruction tuning data (700) may include natural language instructions (710), comments (720), SQL statements (730), and comments (740). The instruction tuning data (700) may be collected to include pairs of natural language instructions (710) and corresponding SQL statements (730). The comments (720, 740) may be text describing a task or situation and may be generated according to a preset format and criteria. In one or more examples, the instruction tuning data (320) may be supervised learning data in which comments are predetermined and added to the instruction tuning data (320). In one or more examples, the instruction tuning data (320) may be unsupervised learning data derived using an AI model (e.g., a large-scale language model) in which the annotations are not predetermined, but instead trained to take natural language queries as input and output annotations.

[0086] The code generation model (11) can operate as a chat model that allows a user (1) to issue commands in natural language by being trained using instruction tuning data (320).

[0087] Natural language data (330) may be a collection of texts in natural language that can be used in general language model training. Since the code generation model (11) basically corresponds to a language model that processes natural language, training using natural language data (330) may also be performed. According to one or more embodiments of the present disclosure, natural language data (330) may be collected from various sources on the Internet.

[0088] (2) Preprocessing of training data

[0089] According to one or more embodiments, at least some of the training data (310, 320, 330) described above may be subjected to preprocessing to increase learning efficiency.

[0090] According to one or more embodiments of the present disclosure, preprocessing may be performed on low code data (310), as described with reference to FIG. 6.

[0091] Fig. 6 is a diagram illustrating the results of performing preprocessing for learning on the low-code data (500) of Fig. 5. Referring to the preprocessed low-code data (600) illustrated in Fig. 6, it can be seen that the following preprocessing steps have been performed.

[0092] 1) Remove codes that are too short or too long.

[0093] Data shorter than a certain standard (621) and data longer than another certain standard (622) were removed as they may have a negative effect on learning.

[0094] 2) Attach a title including the repository name and storage path to the front.

[0095] In order to transform it into a form suitable for learning, a title (610) configured to include a repository name, a storage path, and a file name is attached to the front of the low-code data (600). At this time, the file name can be determined so that the function of the file (low-code data) can be inferred, and the file extension can be set to a predetermined specific value (e.g., ".sql") to clearly indicate that it is a file regarding SQL statements.

[0096] According to one or more examples, duplicate data may be removed from the data included in the low code data (600).

[0097] According to one or more embodiments of the present disclosure, preprocessing may also be performed on natural language data (330). For example, parts of the natural language data (330), such as special characters or abbreviations, that are not helpful for natural language learning, may be removed through preprocessing.

[0098] (3) Learning using training data

[0099] The code generation model (11) can be trained using training data (310, 320, 330) collected and preprocessed as described above.

[0100] The method for training a code generation model (11) using low-code data (310) is as follows. Referring to the preprocessed low-code data (600) of Fig. 6, the code generation model (11) can be trained to infer tokens that constitute the remaining code when a portion of the code in the front part of the low-code data (600) is given.

[0101] The method for training a code generation model (11) using instruction tuning data (320) is as follows. Referring to the instruction tuning data (700) of Fig. 7, the code generation model (11) can be trained to infer the following comment (720), SQL syntax (730), and comment (740) when the preceding natural language instruction (710) is given.

[0102] A method for training a code generation model (11) using natural language data (330) is as follows. When a portion of the front part of the natural language data (330) of the code generation model (11) is given, the model can be trained to infer the remaining portion.

[0103] The code generation model (11) can be trained according to the method described above, and the training of the code generation model (11) can be performed by the electronic device (200) or by another external device.

[0104] 2. Role and learning of the similarity judgment model (12)

[0105] The similarity judgment model (12) may be a neural network model for judging the similarity between data. The similarity judgment model (12) may be used when determining a target database to be analyzed among a plurality of databases (31, 32, 33). According to one or more embodiments of the present disclosure, the search module (140) may use the similarity judgment model (12) to measure the similarity between the input of the user (1) (e.g., natural language query and analysis history selection) and the metadata of the databases (31, 32, 33), and select the target database based on the measurement result.

[0106] To perform this role, the similarity judgment model (12) can be trained to measure the similarity between text data (e.g., calculate a score). Accordingly, the similarity judgment model (12) can be trained using natural language data (410), as illustrated in FIG. 4.

[0107] The collection and preprocessing of natural language data (410) used for learning the similarity judgment model (12), and the learning of the similarity judgment model (12) using the natural language data (410) can be equally applied to the learning of the code generation model (11) using the natural language data (330).

[0108] According to one or more embodiments of the present disclosure, the similarity judgment model (12) may be implemented as an encoder.

[0109] 3. A brief overview of the overall process of performing data analysis in response to natural language queries.

[0110] According to one or more embodiments of the present disclosure, when a user (1) inputs a natural language query through a user terminal (20), the electronic device (200) generates a prompt based on the natural language query, inputs the generated prompt into a code generation model (11), thereby obtaining an SQL statement generated by the code generation model (11), and then executing the SQL statement to perform data analysis. In one or more examples, the prompt may be a natural language query (e.g., a text string) that is converted into a format suitable for the code generation model. For example, the prompt may include additional data that the code generation model (11) can use to generate the SQL statement together with the text string.

[0111] That is, the electronic device (200) according to one or more embodiments of the present disclosure can perform data analysis by automatically generating an SQL statement corresponding to the query when the user (1) inputs a query requesting data analysis in natural language.

[0112] Below, the operations performed step by step by the detailed components (11, 12, 110, 120, 130, 140, 150) included in the electronic device (200) are described in detail.

[0113] 4. Receiving user input (selection of analysis history and natural language queries)

[0114] The front-end server (110) can provide a UI screen for receiving user input requesting data analysis to the user (1) through the user terminal (20), and can receive user input from the user terminal (20) and transmit it to the back-end server (120).

[0115] According to one or more embodiments of the present disclosure, the user input received by the electronic device (200) from the user (1) may only include a natural language query requesting data analysis. In this case, the electronic device (200) may perform processes such as selecting a target database and generating SQL statements based on the natural language query received from the user (1).

[0116] Alternatively, according to one or more embodiments of the present disclosure, the user input received by the electronic device (200) from the user (1) may additionally include an input for selecting an analysis history in addition to a natural language query requesting data analysis. In this case, the electronic device (200) may perform processes such as selecting a target database and generating an SQL statement based on the analysis history selected by the user (1) and the natural language query entered by the user (1). By utilizing the analysis history selected by the user (1), the electronic device (200) can more faithfully reflect the intention of the user (1) and improve work efficiency. In the present disclosure, embodiments are described assuming that the user input received by the electronic device (100) from the user (1) includes both an input for selecting an analysis history and a natural language query requesting data analysis. However, as will be understood by those skilled in the art, the embodiments are not limited to these configurations.

[0117] The front-end server (110) and the back-end server (120) can provide the user (1) with a history of previously performed analysis tasks through a UI screen. According to one or more embodiments of the present disclosure, the front-end server (110) and the back-end server (120) first output a UI screen for login through the user terminal (20), and when the user (1) successfully logs in, the front-end server (110) and the back-end server (120) can control the UI screen for selecting analysis history to be displayed on the user terminal (20) by retrieving data for analysis history stored in the history storage (150). Data corresponding to the analysis history stored in the history storage (150) (hereinafter, “analysis history data”) will be described in detail below with reference to FIG. 9.

[0118] The reason why the electronic device (200) allows the user (1) to select an analysis history is as follows. The natural language query entered by the user (1) alone may not sufficiently reflect the user's intention, but the electronic device (200) can more accurately understand the user's intention through the analysis history selected by the user (1). In addition, the electronic device (200) can increase work efficiency by utilizing the results of previously performed analysis work (analysis work corresponding to the analysis history selected by the user (1)) when analyzing data according to the natural language query entered by the user (1).

[0119] An example of a UI screen for a user (1) to select an analysis history is illustrated in FIG. 8. Referring to FIG. 8, a plurality of analysis histories (810, 820, 830, 840, 850, 860) are displayed on the first UI screen (800). The analysis histories (810, 820, 830, 840, 850, 860) displayed on the first UI screen (800) may be information stored to correspond to the account of the logged-in user (1), or may be information shared by all users using the system. For example, when a user (1) logs in, only analysis works previously executed by the user (1) may be displayed as analysis histories, or all analysis works previously executed by other users may be displayed as analysis histories.

[0120] For convenience of explanation, below, characteristic 1 (810) of a purchasing customer is referred to as 'analysis history 1', and characteristic 2 (820) of a purchasing customer is referred to as 'analysis history 2'.

[0121] FIG. 9 illustrates a specific example of analysis history data stored in the history storage (150). The analysis history data (900) illustrated in FIG. 9 may include a first portion (910) corresponding to the analysis task of analysis history 1 (810) of FIG. 8 and a second portion (920) corresponding to the analysis task of analysis history 2 (820). Analysis history 1 (810) corresponds to the analysis history of the upper layer of analysis history 2 (820). As described above, the analysis history of the lower layer may include the analysis task corresponding to the analysis history of the upper layer. The analysis history data (900) illustrated in FIG. 9 is data corresponding to analysis history 2 (820), and therefore may include information on the analysis task corresponding to analysis history 1 (810), which is the upper layer.

[0122] Referring to FIG. 9, a first part (910) of analysis history data (900) may include an ID (911) for identifying a history, an ID (912) for identifying a history of a higher layer (previous history), a title (913) of the analysis history, a natural language query (914) entered when performing an analysis task included in the analysis history, an SQL statement (915) generated by a code generation model (11) when performing an analysis task included in the analysis history, an execution result (916) of the SQL statement, and an ID (917) for identifying a graph visualizing the data analysis result. A second part (920) of the analysis history data (900) may also have the same structure as the first part (910).

[0123] A user (1) can select at least one of a plurality of analysis histories (810, 820, 830, 840, 850, 860) through a first UI screen (800). For example, the user (1) can select one of the analysis histories (810, 820, 830, 840, 850, 860) by looking at the names of the analysis histories displayed on the first UI screen (800). Or, for example, the user (1) can select an analysis history after checking the details of the analysis history. According to one or more embodiments of the present disclosure, when the user (1) enlarges the first area (80) of the first UI screen (800), the front-end server (110) and the back-end server (120) can control the details of the analysis histories (810, 820) included in the first area (80) to be displayed on the user terminal (20). The UI screen displayed when the user (1) enlarges the first area (80) of the first UI screen (800) is illustrated in FIG. 10.

[0124] FIG. 10 is a diagram illustrating a situation in which details of an analysis history are displayed on a UI screen for selecting an analysis history according to one or more embodiments of the present disclosure. Referring to FIG. 10 , a second UI screen (1000) may display details of analysis history 1 (1010) and details of analysis history 2 (1020).

[0125] The details of the analysis history 1 (1010) may include a natural language query (1011) input when performing an analysis task corresponding to the analysis history 1 (1010), an SQL statement (1012) generated based on the natural language query (1011), and an execution result (1013) of the SQL statement. The execution result (1013) of the SQL statement may include a table representing the data analysis result and a graph visualizing the same.

[0126] Similarly, the details of the analysis history 2 (1020) may include a natural language query (1021) input when performing an analysis task corresponding to the analysis history 2 (1020), an SQL statement (1022) generated based on the natural language query (1021), and an execution result (1023) of the SQL statement.

[0127] A user (1) can understand the details of analysis histories (1010, 1020) through the second UI screen (1000) and select an analysis history to be reflected in data analysis. In one or more embodiments of the present disclosure, the user (1) is assumed to have selected analysis history 2 (1020), and the subsequent processes are described.

[0128] When a user (1) selects analysis history 2 (1020) on the second UI screen (1000), the front-end server (110) and the back-end server (120) can control the user terminal (20) to display an input window for entering a natural language query. A UI screen displaying an input window for entering a natural language query is illustrated in Fig. 11.

[0129] Fig. 11 is a diagram illustrating a UI screen for a user to input a natural language query requesting data analysis. Referring to Fig. 11, a third UI screen (1100) may display details of analysis history 2 (1120) selected by a user (1), and an input window (1130) for inputting a natural language query may be displayed. The user (1) may input a natural language query (1131) requesting data analysis through the input window (1130).

[0130] According to the process described above, the user (1) can select the analysis history that he or she wants to reflect in the data analysis and then input a natural language query requesting data analysis.

[0131] 5. Selecting the target database

[0132] According to one or more embodiments, when the front-end server (110) receives user input (selection of analysis history and natural language query) through the user terminal (20), it transmits the same to the back-end server (120), and the back-end server (120) transmits data related to the user input to the search module (140) while requesting the search module (140) to select a target database. At this time, the back-end server (120) may obtain metadata of the databases (31, 32, 33) from the database management system (130) and transmit the metadata to the search module (140).

[0133] The search module (140) can select at least one of the databases (31, 32, 33) as a target database by using data related to user input and metadata of the databases (31, 32, 33). For example, the search module (140) can determine a target database to be used for analysis based on an analysis history selected by the user (1) and a natural language query entered by the user (1).

[0134] In detail, the search module (140) can measure the similarity between natural language data related to user input (e.g., natural language queries included in the selected analysis history and natural language queries entered by the user (1)) and metadata of multiple databases (31, 32, 33) using a similarity judgment model (12), and select a target database based on the measured similarity.

[0135] According to one or more embodiments of the present disclosure, 'natural language data related to user input' may include at least one natural language query related to an analysis history selected by the user (1) (a natural language query input when performing at least one analysis task included in the selected analysis history) and a natural language query input by the user (1). Alternatively, according to one or more embodiments of the present disclosure, 'natural language data related to user input' may include only the natural language query input by the user (1). In the present disclosure, it is assumed that 'natural language data related to user input' includes at least one natural language query related to an analysis history selected by the user (1) and a natural language query input by the user (1).

[0136] The process of selecting a target database using the similarity judgment model (12) by the search module (140) is described in detail with reference to FIGS. 12 to 14.

[0137] Figure 12 illustrates metadata of multiple databases (31, 32, 33). As previously described, the metadata of the databases (31, 32, 33) may include a table catalog and a table schema. The table catalog and table schema will be described in detail with reference to Figure 12.

[0138] Referring to FIG. 12, metadata (1210) of DB1 (31) may include a table catalog (1211) and a table schema (1212).

[0139] The table catalog (1211) may include a description of a table included in DB1 (31) (e.g., what information is stored in the table). In addition, the table catalog (1211) may include a description of each column of the table included in DB1 (31) (e.g., what each column of the table means). Referring to the table catalog (1211) of FIG. 12, the table called "sales" included in DB1 (31) stores information on which smartphones customers purchased, and the names of each column of the table are "product name", "guid (globally unique identifier)", and "color", respectively, and the information recorded in each column is "name of product", "customer ID", and "color of product", respectively.

[0140] The table schema (1212) can define the structure and rules of the tables included in DB1 (31). For example, referring to FIG. 12, the table schema (1212) can include a command for creating a table, and thus can include information on which table to create, which columns are included in each table, and what the constraints are for each column. Looking at the table schema (1212) of FIG. 12, it can be seen that a table called "sales" is created in DB1 (31), the names of each column of the table are "product name", "guid", and "color", and the format of the information recorded in each column is text, integer, and text, respectively.

[0141] Fig. 12 also illustrates metadata (1220) of DB2 (32) and metadata (1230) of DB3 (33). The metadata (1220) of DB2 (32) may also include a table catalog (1221) and a table schema (1222), and the metadata (1230) of DB3 (33) may also include a table catalog (1231) and a table schema (1232). Since the metadata (1220) of DB2 (32) and the metadata (1230) of DB3 (33) have similar forms and characteristics to the metadata (1210) of DB1 (31), a detailed description thereof will be omitted.

[0142] According to one or more embodiments of the present disclosure, the backend server (120) may obtain metadata of the databases (31, 32, 33) from the database management system (130) whenever necessary. Alternatively, according to one or more embodiments of the present disclosure, the backend server (120) may obtain metadata by updating the metadata of the databases (31, 32, 33) locally (e.g., cache memory within the backend server (120)) at regular intervals and caching the metadata whenever necessary, thereby improving efficiency and speed.

[0143] FIGS. 13A to 13C are diagrams illustrating UI screens on which metadata and tables of databases according to one or more embodiments of the present disclosure are displayed. The front-end server (110) and the back-end server (120) may control the UI screens (UI screens corresponding to databases) of FIGS. 13A to 13C to be displayed on the user terminal (20), if necessary. For example, when a target database is selected by the search module (140), the front-end server (110) and the back-end server (120) may control the UI screen corresponding to the selected target database to be displayed on the user terminal (20). In addition, for example, the front-end server (110) and the back-end server (120) may control the UI screen corresponding to a database that the search module (140) did not select as a target database to be also displayed on the user terminal (20), thereby providing the user (1) with an opportunity to directly select the target database.

[0144] According to one or more embodiments of the present disclosure, the backend server (120) may display a UI screen corresponding to a database selected by the search module (140) and a UI screen corresponding to other databases in a manner that is distinct (e.g., by displaying them differently in size or contrast, etc.), thereby allowing the user (1) to check the selection result of the search module (140). In this case, the user (1) may exclude some of the databases selected by the search module (140) from the target database, or conversely, may include some of the databases not selected by the search module (140) in the target database.

[0145] The UI screen (1310) corresponding to DB1 (31) illustrated in FIG. 13a may include a table catalog (1311) of DB1 (31) and a table (1312) corresponding to DB1 (31). As described above, the table catalog (1311) may include a description of the table (1312) and a description of each column of the table (1312). The table (1312) includes table values ​​corresponding to each column for each of a plurality of entities (configurations corresponding to rows), and since the table (1312) is created according to the structure and rules determined by the table schema, the UI screen (1310) may be viewed as including a table schema and table values.

[0146] Similarly, the UI screen (1320) corresponding to DB2 (32) illustrated in FIG. 13b may include a table catalog (1321) of DB2 (32) and a table (1322) corresponding to DB2 (32). In addition, the UI screen (1330) corresponding to DB3 (33) illustrated in FIG. 13c may include a table catalog (1331) of DB3 (33) and a table (1332) corresponding to DB3 (33). The UI screen (1320) corresponding to DB2 (32) and the UI screen (1330) corresponding to DB3 (33) have similar forms and characteristics to the UI screen (1310) corresponding to DB1 (31), and therefore, a detailed description thereof will be omitted.

[0147] FIG. 14 is a diagram for explaining a method for determining a target database based on similarity judgment in the process of performing a data analysis method according to one or more embodiments of the present disclosure.

[0148] Referring to FIG. 14, the search module (140) can measure the similarity between natural language data (1410) related to user input and table catalogs (1311, 1321, 1331) of a plurality of databases (31, 32, 33) using a similarity judgment model (12), and select at least one of the databases (31, 32, 33) as a target database based on the measured similarity.

[0149] At this time, natural language data (1410) related to user input may include a natural language query related to the analysis history selected by the user (1) and a natural language query input by the user (1). This will be described in detail with reference to FIGS. 10 and 11. In FIG. 10, when the user (1) selects analysis history 2 (1020), the backend server (120) may extract a natural language query (1021) included in analysis history 2 (1020) and a natural language query (1011) included in analysis history 1 (1010), which is an upper layer, for use in selecting a target database. When a user (1) selects analysis history 2 (1020) and then inputs a natural language query (1131) into the input window (1130) of the UI screen (1100) of FIG. 11, the backend server (120) may request selection of a target database while transmitting the natural language queries (1011, 1021) extracted from the analysis histories (1010, 1020) and the natural language query (1131) input by the user (1) to the search module (140).

[0150] At the request of the search module (140), the similarity judgment model (12) can measure the similarity score between the natural language data (1410) related to the user input received from the backend server (120) and the table catalogs (1311, 1321, 1331) of the plurality of databases (31, 32, 33). Referring to FIG. 14, the similarity score measured by the similarity judgment model (12) between the natural language data (1410) related to the user input and the table catalog (1311) of DB1 (31) is 0.9586. The similarity score between the natural language data (1410) related to the user input and the table catalog (1321) of DB2 (32) is 0.9784, and the similarity score between the natural language data (1410) related to the user input and the table catalog (1331) of DB3 (33) is 0.4324.

[0151] The search module (140) can select at least one of the plurality of databases (31, 32, 33) as a target database based on the calculated similarity score. The search module (140) can select a target database according to preset rules or criteria. For example, the search module (140) can select a database with a similarity score greater than a preset threshold as a target database. Alternatively, for example, the search module (140) can select a preset number of databases in descending order of similarity score as target databases. Alternatively, for example, the search module (140) can perform a normalization operation on the calculated similarity score and then select a database with a similarity score greater than a preset threshold as a target database.

[0152] In the embodiment illustrated in FIG. 14, it is assumed that the search module (140) selects DB1 (31) and DB2 (32) as target databases, which have a similarity score higher than a threshold value (e.g., 0.8). Once the search module (140) completes the selection of the target database, the front-end server (110) and the back-end server (120) can cause the selection results to be displayed on the screen of the user terminal (20).

[0153] According to one or more embodiments of the present disclosure, the front-end server (110) and the back-end server (120) can control the display of a UI screen corresponding to a database selected as a target database by the search module (140) on the user terminal (20). The UI screens corresponding to each database according to one or more embodiments of the present disclosure are illustrated in FIGS. 13A to 13C . Of course, the information included in the UI screen corresponding to the database or the layout of the UI screen may be configured differently.

[0154] As shown in Fig. 14, when the search module (140) selects DB1 (31) and DB2 (32) as target devices, the front-end server (110) and the back-end server (120) can control the UI screen (1310) corresponding to DB1 (31) shown in Fig. 13a and the UI screen (1320) corresponding to DB2 (32) shown in Fig. 13b to be displayed on the user terminal (20). The user (1) can check information on the databases selected as target databases through the user terminal (20).

[0155] In addition, as described above, according to one or more embodiments of the present disclosure, the front-end server (110) and the back-end server (120) may also display a UI screen corresponding to a database that the search module (140) did not select on the user terminal (20), thereby providing the user (1) with an opportunity to edit the target database. At this time, the back-end server (120) may display the selection result so that the databases selected by the search module (140) and the databases that the search module (140) did not select are distinguished. For example, the back-end server (120) may display the UI screen corresponding to the database selected by the search module (140) more clearly than the UI screen corresponding to the database that the search module (140) did not select.

[0156] The user (1) can exclude some of the databases selected by the search module (140) from the target database, or conversely, can include some of the databases not selected by the search module (140) in the target database.

[0157] 6. Creating a prompt (for generating SQL statements)

[0158] Once the selection of the target database is completed, the backend server (120) of the electronic device (200) can generate a prompt based on the user input and the target database. At this time, the generated prompt is a prompt for generating an SQL statement. That is, when the backend server (120) inputs the prompt generated based on the user input and the target database into the code generation model (11), the code generation model (11) can generate and output an SQL statement for performing data analysis.

[0159] The backend server (120) can generate prompts based on user input and a target database, which, as described above, may include only natural language queries or may further include input for selecting analysis histories.

[0160] If the user input includes an input for selecting an analysis history and a natural language query, the backend server (120) can generate a prompt based on the metadata of the target database, the natural language query included in the user input, and the selected analysis history.

[0161] The structure of a prompt generated by the backend server (120) based on user input (selection of analysis history and natural language query) and a target database is illustrated in FIG. 15. Referring to FIG. 15, the prompt (1500) may include an analysis history 1 related portion (1510), an analysis history 2 related portion (1520), and a current natural language query related portion (1530).

[0162] As explained above, if a user (1) selects analysis history 2, data corresponding to analysis history 1, which is a higher layer of the selected analysis history 2, can also be used when generating a prompt (1500).

[0163] The current natural language query related part (1530) may mean a part generated using data related to a natural language query (current natural language query) entered through the input window (1130) of FIG. 11.

[0164] If the user input received through the user terminal (20) does not include an input for selecting an analysis history and only includes a natural language query, the prompt (1500) may be configured to include only the relevant portion (1530) of the current natural language query.

[0165] According to one or more embodiments of the present disclosure, the backend server (120) can generate an analysis history related portion (1510, 1520) using metadata of a database related to the selected analysis history, a natural language query related to the selected analysis history, and an SQL statement related to the selected analysis history.

[0166] At this time, the database related to the selected analysis history may mean a database used when performing at least one analysis task included in the selected analysis history.

[0167] Additionally, at this time, the natural language query related to the selected analysis history may mean a natural language query entered when performing at least one analysis task included in the selected analysis history.

[0168] Additionally, at this time, the SQL statement related to the selected analysis history may mean a SQL statement generated by the code generation model when performing at least one analysis task included in the selected analysis history.

[0169] According to one or more embodiments of the present disclosure, the backend server (120) can generate a current natural language query relevant portion (1530) using metadata (table catalog and table schema) of the target database and a natural language query entered by a user (1).

[0170] Below, the detailed configuration of the prompt (1500) is described in detail with reference to FIGS. 16 to 18.

[0171] FIG. 16 is a diagram for explaining details of a portion related to analysis history 1 included in the prompt (1500) of FIG. 15. Referring to FIG. 16, it can be seen that the portion related to analysis history 1 (1510) includes data related to an analysis task of 'aggregating the number of purchasers by product' (hereinafter, 'first analysis task').

[0172] Since DB1 (31) was used when performing the first analysis task (analysis task corresponding to analysis history 1), the analysis history 1 related part (1510) may include the table catalog (1511) of DB1 and the table schema (1512) of DB1.

[0173] Additionally, the analysis history 1 related portion (1510) may include a preset instruction (1513). The preset instruction (1513) may be text that instructs, 'Refer to the metadata of the database, and generate an SQL statement corresponding to a natural language query.'

[0174] In addition, the analysis history 1 related part (1510) may include a natural language query (1514) of the analysis history 1 and an SQL statement (1515) of the analysis history 1. The natural language query (1514) of the analysis history 1 may be a natural language query input when performing the first analysis task. In addition, the SQL statement (1515) of the analysis history 1 may be an SQL statement generated by the code generation model (11) when performing the first analysis task.

[0175] FIG. 17 is a diagram illustrating details of a portion related to analysis history 2 included in the prompt (1500) of FIG. 15. Referring to FIG. 17, it can be seen that the portion related to analysis history 2 (1520) includes data related to an analysis task that "aggregates by additionally considering gender" (hereinafter, "second analysis task"). The second analysis task corresponds to a task additionally performed using the results of the first analysis task.

[0176] Since DB1 (31) and DB2 (32) were used when performing the second analysis task (analysis task corresponding to analysis history 2), the analysis history 2 related part (1520) may include the table catalog (1521) of DB1, the table catalog (1522) of DB2, the table schema (1523) of DB1, and the table schema (1524) of DB2.

[0177] Additionally, the analysis history 2 related part (1520) may include a preset instruction (1525). The preset instruction (1513) may be a text that instructs, 'Refer to the metadata of the database, and generate an SQL statement corresponding to a natural language query.'

[0178] In addition, the part (1520) related to analysis history 2 may include a natural language query (1526) of analysis history 2 and an SQL statement (1527) of analysis history 2. The natural language query (1526) of analysis history 2 may be a natural language query input when performing the second analysis task. In addition, the SQL statement (1527) of analysis history 2 may be an SQL statement generated by a code generation model (11) when performing the second analysis task.

[0179] FIG. 18 is a diagram illustrating details of a current natural language query-related portion included in a prompt (1500) of FIG. 15. Referring to FIG. 18, it can be seen that the current natural language query-related portion (1530) includes metadata of a target database and a natural language query (current natural language query) entered by a user (1). In one or more examples, a prompt may be formed by retrieving metadata of a target database and adding it to a natural language query.

[0180] Since DB1 (31) and DB2 (32) are selected as target databases, the current natural language query related part (1530) may include the table catalog (1531) of DB1, the table catalog (1532) of DB2, the table schema (1533) of DB1, and the table schema (1534) of DB2.

[0181] Additionally, the current natural language query related part (1530) may include a preset instruction (1535). The preset instruction (1535) may be text that instructs, 'Refer to the metadata of the database, and generate an SQL statement corresponding to the natural language query.'

[0182] Additionally, the current natural language query related part (1530) may include a current natural language query (1536). The current natural language query (1536) may be a natural language query entered by a user (1) through the input window (1130) of FIG. 11.

[0183] 7. Enter the generated prompt into the code generation model to obtain the SQL statement.

[0184] FIG. 19 is a diagram illustrating an example in which a code generation model generates SQL statements when a prompt is input to the code generation model during a process of performing a data analysis method according to one or more embodiments of the present disclosure.

[0185] As illustrated in FIG. 19, when the backend server (120) inputs a prompt (1500) into the code generation model (11), the code generation model (11) can generate and output an SQL statement corresponding to the current natural language query (1536).

[0186] For example, the SQL statement output by the code generation model (11) in Fig. 19 is a code that uses the results of the first analysis task (aggregation of the number of purchasers by product) and the second analysis task (aggregation by additionally considering gender) to additionally perform a third analysis task (aggregation by additionally considering age group in units of 10 years).

[0187] 8. Execute the acquired SQL statement to perform data analysis.

[0188] The backend server (120) can perform data analysis by executing SQL statements obtained from the code generation model (11). Specifically, the backend server (120) can perform analysis on data stored in the target database (31, 32) by executing SQL statements in the database management system (130), and receive data analysis results from the database management system (130).

[0189] The front-end server (110) and back-end server (120) can display data analysis results on the user terminal (20). A UI screen that outputs the results of data analysis by executing the generated SQL statement is illustrated in FIG. 20.

[0190] Referring to FIG. 20, the fourth UI screen (2000) may display the execution result of the current natural language query in the natural language query input window (2030) along with the details of the analysis history 2 (2020). The input window (2030) may display the natural language query (current natural language query) (2031) previously entered by the user (1), and below that, an SQL statement (2032) corresponding to the current natural language query (2031) may be displayed, and below that, a table (2033) showing the result of data analysis performed according to the SQL statement (2032) may be displayed.

[0191] The fourth UI screen (2000) of FIG. 20 is only an example, and the UI screen showing the data analysis results may be configured in various layouts.

[0192] 9. Visualize and output data analysis results

[0193] According to one or more embodiments of the present disclosure, the front-end server (110) and the back-end server (120) may visualize the data analysis results in the form of a graph or diagram and display the visualized results on the user terminal (20).

[0194] FIG. 21 is a diagram for explaining a method of visualizing data analysis results as a graph or diagram, etc., in the process of performing a data analysis method according to one or more embodiments of the present disclosure. The backend server (120) can generate a prompt (2110) for generating visualization code using a natural language query (current natural language query) input by a user (1) and a generated SQL syntax. Referring to FIG. 21, the prompt (2110) for generating visualization code can include the SQL syntax (1900) generated in FIG. 19, a preset instruction (2111), and the current natural language query (1536) included in the prompt (1500). The preset instruction (2111) can be text that instructs, 'Write visualization code for a natural language query by referring to the SQL syntax.'

[0195] When the backend server (120) inputs a prompt (2110) for generating visualization code into the code generation model (11), the code generation model (11) can generate and output visualization code (2120).

[0196] When the backend server (120) executes the visualization code (2120) obtained from the code generation model (11), graphs and diagrams (2131, 2132) representing data analysis results according to the SQL statement (1900) can be generated. When the backend server (120) transmits the generated graphs and diagrams (2131, 2132) to the frontend server (110), the frontend server (110) can display them on the screen of the user terminal (20).

[0197] Hereinafter, a method for performing data analysis based on natural language queries according to embodiments of the present disclosure will be described with reference to the flowcharts of FIGS. 22 to 26. The steps included in the flowcharts of FIGS. 22 to 26 are performed by the electronic device (200) of FIG. 1, and therefore, the contents previously described with reference to FIGS. 1 to 21 may be equally applied to FIGS. 22 to 26, even if omitted below.

[0198] Referring to FIG. 22, in step 2201, the electronic device (200) may receive user input including a natural language query requesting data analysis. The user input may include only the natural language query, or may additionally include an input for selecting an analysis history. Detailed steps included in step 2201 are illustrated in FIG. 23.

[0199] Referring to FIG. 23, in step 2301, the electronic device (200) can receive an input from the user to select an analysis history. Specifically, when the electronic device (200) outputs a UI screen for selecting an analysis history through the user terminal (20), the user (1) can select an analysis history to be reflected in data analysis through the UI screen.

[0200] At step 2302, the electronic device (200) can receive a natural language query from a user.

[0201] Returning to FIG. 22, in step 2202, the electronic device (200) may determine at least one database among a plurality of databases as a target database based on user input. Specifically, the electronic device (200) may determine the target database using data related to the user input and metadata of the databases. In other words, the electronic device (200) may determine the target database to be used for analysis based on the analysis history selected by the user (1) and the natural language query entered by the user (1). Detailed steps included in step 2202 are illustrated in FIG. 24.

[0202] Referring to FIG. 24, in step 2401, the electronic device (200) may use a similarity judgment model to determine the similarity between at least one natural language query related to the selected analysis history, the received natural language query, and metadata of multiple databases. The similarity judgment model may measure the similarity as a score and output it.

[0203] At step 2402, the electronic device (200) may determine at least one of a plurality of databases as a target database based on the similarity determination result. As described above, the electronic device (200) may select the target database by comparing the similarity score with a preset threshold value, or by selecting a certain number of databases in order of increasing similarity score.

[0204] Returning to FIG. 22, at step 2203, the electronic device (200) can generate a prompt based on user input and the target database. Detailed steps included in step 2203 are illustrated in FIG. 25.

[0205] The prompt may include a portion related to the analysis history and a portion related to the current natural language query. In step 2501, the electronic device (200) may generate the portion related to the analysis history using metadata of the database related to the selected analysis history, a natural language query related to the selected analysis history, and an SQL statement related to the selected analysis history. In step 2502, the electronic device (200) may generate the portion related to the current natural language query using metadata of the target database and the received natural language query.

[0206] Returning to FIG. 22 again, at step 2204, the electronic device (200) can obtain SQL statements by inputting the generated prompt into the code generation model.

[0207] In step 2205, the electronic device (200) can output data analysis results for the target database by executing SQL statements. According to one or more embodiments of the present disclosure, the electronic device (200) can output the data analysis results by visualizing them in a graph or diagram, and detailed steps of step 2205 for this purpose are illustrated in FIG. 26.

[0208] Referring to FIG. 26, in step 2601, the electronic device (200) can generate a prompt for visualizing the data analysis results using the received natural language query and generated SQL syntax.

[0209] At step 2602, the electronic device (200) can obtain visualization code by inputting the generated prompt into the code generation model.

[0210] At step 2603, the electronic device (200) can visualize and output the data analysis results as at least one of a graph or diagram by executing a visualization code.

[0211] According to the embodiments described above, data analysis is performed based on the user's natural language query, so that even users who do not know SQL grammar can easily analyze data, thereby improving user convenience and enabling more users to utilize the data.

[0212] A method for performing data analysis based on a user's natural language query according to one or more embodiments of the present disclosure includes the steps of: receiving a user input including a natural language query requesting data analysis from a user terminal; determining at least one database from among a plurality of databases as a target database based on the user input; generating a prompt based on the user input and the target database; obtaining a Structured Query Language syntax by inputting the prompt into a code generation model; outputting a data analysis result for the target database by executing the SQL syntax; and transmitting the data analysis result to the user terminal, wherein the data analysis result can be displayed on a screen of the user terminal.

[0213] According to one or more embodiments, the step of receiving user input may include the step of receiving an input from the user for selecting an analysis history and the step of receiving the natural language query from the user.

[0214] According to one or more embodiments, the step of determining the at least one database as a target database may include the step of determining, using a similarity determination model, a similarity between at least one natural language query related to the selected analysis history and the received natural language query and metadata of the plurality of databases, and the step of determining at least one of the plurality of databases as the target database based on a result of the similarity determination.

[0215] According to one or more embodiments, the metadata includes a table catalog and a table schema, the table catalog including a description of a table included in a database corresponding to the metadata and a description of each column of the table, and the table schema may define a structure and rules of a table included in a database corresponding to the metadata.

[0216] According to one or more embodiments, the prompt includes an analysis history related portion and a current natural language query related portion, and the step of generating the prompt may include a step of generating the analysis history related portion using metadata of a database related to the selected analysis history, a natural language query related to the selected analysis history, and an SQL statement related to the selected analysis history, and a step of generating the current natural language query related portion using metadata of the target database and the received natural language query.

[0217] According to one or more embodiments, the database related to the selected analysis history may be a database used when performing at least one analysis task included in the selected analysis history, the natural language query related to the selected analysis history may be a natural language query input when performing at least one analysis task included in the selected analysis history, and the SQL statement related to the selected analysis history may be a SQL statement generated by the code generation model when performing at least one analysis task included in the selected analysis history.

[0218] According to one or more embodiments, the analysis history related portion and the current natural language query related portion may each include instructions to generate an SQL statement corresponding to a natural language query by referencing metadata of at least one database among a plurality of databases.

[0219] According to one or more embodiments, the analysis history may include at least one analysis work already performed on at least one database among the plurality of databases.

[0220] According to one or more embodiments, the step of outputting the data analysis result may include the step of generating a prompt for visualizing the data analysis result using the received natural language query and the generated SQL statement, the step of obtaining visualization code by inputting the prompt for visualizing the data analysis result into the code generation model, and the step of visualizing and outputting the data analysis result as at least one of a graph or a diagram by executing the visualization code.

[0221] According to one or more embodiments, the prompt for visualizing the data analysis result may include an instruction to generate visualization code corresponding to the received natural language query by referencing the generated SQL statement.

[0222] An electronic device for performing data analysis according to a user's natural language query according to one or more embodiments of the present disclosure includes a memory storing one or more instructions and at least one processor functionally coupled to the memory, wherein the at least one processor executes the at least one instruction, whereby the electronic device receives a user input including a natural language query requesting data analysis from a user terminal, determines at least one database among a plurality of databases as a target database based on the user input, generates a prompt based on the user input and the target database, inputs the prompt into a code generation model to obtain a Structured Query Language (SQL) syntax, executes the SQL syntax to output a data analysis result for the target database, and then transmits the data analysis result to the user terminal, whereby the data analysis result can be displayed on a screen of the user terminal.

[0223] According to one or more embodiments, the electronic device may receive the natural language query from the user after receiving an input for selecting an analysis history from the user by executing the at least one instruction by the at least one processor when receiving the user input.

[0224] According to one or more embodiments, when the at least one instruction is executed by the at least one processor, the electronic device may determine the at least one database as a target database by using a similarity determination model to determine a similarity between at least one natural language query related to the selected analysis history and the received natural language query and metadata of the plurality of databases, and then determine at least one of the plurality of databases as a target database based on a result of the similarity determination.

[0225] According to one or more embodiments, the metadata includes a table catalog and a table schema, the table catalog including a description of a table included in a database corresponding to the metadata and a description of each column of the table, and the table schema can define a structure and rules of a table included in a database corresponding to the metadata.

[0226] According to one or more embodiments, the prompt includes an analysis history related portion and a current natural language query related portion, and when the at least one instruction is executed by the at least one processor, the electronic device can generate the analysis history related portion by using metadata of a database related to the selected analysis history, a natural language query related to the selected analysis history, and an SQL statement related to the selected analysis history, and then generate the current natural language query related portion by using metadata of the target database and the received natural language query.

[0227] According to one or more embodiments, the database related to the selected analysis history may be a database used when performing at least one analysis task included in the selected analysis history, the natural language query related to the selected analysis history may be a natural language query input when performing at least one analysis task included in the selected analysis history, and the SQL statement related to the selected analysis history may be a SQL statement generated by the code generation model when performing at least one analysis task included in the selected analysis history.

[0228] According to one or more embodiments, the analysis history related portion and the current natural language query related portion may each include instructions to generate an SQL statement corresponding to a natural language query by referencing metadata of at least one database among a plurality of databases.

[0229] According to one or more embodiments, the analysis history may include at least one analysis work already performed on at least one database among the plurality of databases.

[0230] According to one or more embodiments, when the at least one instruction is executed by the at least one processor, the electronic device may generate a prompt for visualizing the data analysis result using the received natural language query and the generated SQL statement, input the prompt for visualizing the data analysis result into the code generation model to obtain visualization code, and then execute the visualization code to visualize the data analysis result and output it as at least one of a graph or a diagram.

[0231] Various embodiments of the present disclosure may be implemented or supported by one or more computer programs, and the computer programs may be formed from computer-readable program code and embodied in a computer-readable medium. In the present disclosure, "application" and "program" may refer to one or more computer programs, software components, instruction sets, procedures, functions, objects, classes, instances, associated data, or portions thereof suitable for implementation in computer-readable program code. "Computer-readable program code" may include various types of computer code, including source code, object code, and executable code. "Computer-readable medium" may include various types of media that can be accessed by a computer, such as read-only memory (ROM), random access memory (RAM), a hard disk drive (HDD), a compact disc (CD), a digital video disc (DVD), or various types of memory.

[0232] Additionally, a device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, a 'non-transitory storage medium' is a tangible device and may exclude wired, wireless, optical, or other communication links that transmit temporary electrical or other signals. Meanwhile, this 'non-transitory storage medium' does not distinguish between cases where data is permanently stored in the storage medium and cases where it is temporarily stored. For example, a 'non-transitory storage medium' may include a buffer where data is temporarily stored. A computer-readable medium may be any available medium that can be accessed by a computer, and may include both volatile and non-volatile media, and removable and non-removable media. A computer-readable medium includes a medium on which data can be permanently stored and a medium on which data can be stored and later overwritten, such as a rewritable optical disk or an erasable memory device.

[0233] According to one or more embodiments, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0234] The above description of the present disclosure is for illustrative purposes only, and those skilled in the art will appreciate that the present disclosure can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present disclosure. For example, suitable results can be achieved even if the described techniques are performed in a different order than the described method, and / or components of the systems, structures, devices, circuits, etc. described are combined or combined in a different form than the described method, or are replaced or substituted by other components or equivalents. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined form.

[0235] The scope of the present disclosure is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present disclosure.

Claims

1. A method for performing data analysis based on a user's natural language query, A step of receiving a user input including a natural language query requesting data analysis from a user terminal; A step of determining at least one database among a plurality of databases as a target database based on the user input; A step of generating a prompt based on the user input and the target database; A step of obtaining a Structured Query Language syntax by inputting the above prompt into a code generation model; A step of outputting data analysis results for the target database by executing the above SQL statement; and A step of transmitting the above data analysis results to the user terminal is included. A method in which the results of the above data analysis are displayed on the screen of the user terminal.

2. In paragraph 1, The step of receiving the above user input is: A step of receiving an input from a user for selecting an analysis history; and A method characterized by comprising the step of receiving the natural language query from the user.

3. In either of paragraphs 1 and 2, The step of determining at least one of the above databases as a target database is: A step of using a similarity judgment model to determine the similarity between at least one natural language query related to the selected analysis history and the received natural language query and metadata of the plurality of databases; and A method characterized by comprising a step of determining at least one of the plurality of databases as a target database based on the result of the similarity determination.

4. In any one of paragraphs 1 to 3, The above metadata includes a table catalog and a table schema. The above table catalog includes a description of a table included in a database corresponding to the above metadata and a description of each column of the table, A method characterized in that the above table schema defines the structure and rules of a table included in a database corresponding to the above metadata.

5. In any one of paragraphs 1 to 4, The above prompt includes parts related to the analysis history and parts related to the current natural language query. The steps to generate the above prompt are: A step of generating a part related to the analysis history by using metadata of a database related to the selected analysis history, a natural language query related to the selected analysis history, and an SQL statement related to the selected analysis history; and A method characterized by comprising a step of generating a relevant portion of the current natural language query using metadata of the target database and the received natural language query.

6. In any one of paragraphs 1 to 5, The database related to the above-mentioned selected analysis history is a database used when performing at least one analysis task included in the above-mentioned selected analysis history, The natural language query related to the above-mentioned selected analysis history is a natural language query entered when performing at least one analysis task included in the above-mentioned selected analysis history, A method characterized in that the SQL statement related to the above-mentioned selected analysis history is a SQL statement generated by the code generation model when performing at least one analysis task included in the above-mentioned selected analysis history.

7. In any one of paragraphs 1 to 6, The above analysis history is, A method characterized by including at least one analysis work already performed on at least one database among the plurality of databases.

8. In any one of paragraphs 1 to 7, The step of outputting the above data analysis results is: A step of generating a prompt for visualizing the data analysis results using the received natural language query and the generated SQL statement; A step of obtaining a visualization code by inputting a prompt for visualizing the data analysis results into the code generation model; and A method characterized by including a step of visualizing and outputting the data analysis results as at least one of a graph or diagram by executing the visualization code.

9. A computer-readable recording medium having recorded thereon a program for performing the method of any one of clauses 1 to 8 on a computer.

10. An electronic device for performing data analysis based on a user's natural language query, A memory (230) in which at least one instruction is stored; and comprising at least one processor (220) functionally coupled to the memory; The electronic device (200) executes the at least one instruction by the at least one processor (220), Receiving user input from a user terminal that includes a natural language query requesting data analysis, Based on the above user input, at least one database among multiple databases (31, 32, 33) is determined as a target database, Generate a prompt based on the user input and the target database; By inputting the above prompt into the code generation model (11), a SQL syntax (Structured Query Language syntax) is obtained. After executing the above SQL statement, the data analysis results for the target database are output. The above data analysis results are transmitted to the user terminal, An electronic device in which the results of the above data analysis are displayed on the screen of the user terminal.

11. In paragraph 10, The electronic device (200) receives the user input by executing the at least one instruction by the at least one processor (220). After receiving input from the user to select analysis history, An electronic device characterized by receiving a natural language query from said user.

12. In either of paragraphs 10 and 11, By executing the at least one instruction by the at least one processor (220), the electronic device (200) determines the at least one database as a target database. Using the similarity judgment model (12), the similarity between at least one natural language query related to the selected analysis history and the received natural language query and the metadata of the plurality of databases (31, 32, 33) is judged. An electronic device characterized in that, based on the results of the similarity determination, at least one of the plurality of databases (31, 32, 33) is determined as a target database.

13. In any one of paragraphs 10 to 12, The above metadata includes a table catalog and a table schema. The above table catalog includes a description of a table included in a database corresponding to the above metadata and a description of each column of the table, An electronic device, characterized in that the above table schema defines the structure and rules of a table included in a database corresponding to the above metadata.

14. In any one of paragraphs 10 to 13, The above prompt includes parts related to the analysis history and parts related to the current natural language query. The electronic device (200) generates the prompt by executing the at least one instruction by the at least one processor (220). After generating the analysis history-related part using the metadata of the database related to the selected analysis history, the natural language query related to the selected analysis history, and the SQL statement related to the selected analysis history, An electronic device characterized by generating a relevant portion of the current natural language query using metadata of the target database and the received natural language query.

15. In any one of paragraphs 10 to 14, By executing the at least one instruction by the at least one processor (220), the electronic device (200) outputs the data analysis result. Using the received natural language query and the generated SQL statement, a prompt is generated for visualizing the data analysis results, After obtaining the visualization code by inputting the prompt for visualizing the above data analysis results into the above code generation model, An electronic device characterized in that it outputs the data analysis results by visualizing them in at least one of a graph or diagram by executing the visualization code.

Images