Data processing method, electronic equipment, storage medium and computer program product

Abstract

The invention discloses a data processing method, electronic equipment, a storage medium and a computer program product, and the method comprises the steps: determining first metadata corresponding to each piece of first training data in one or more pieces of first training data, the first training data being used for training a first model, the first model is used for assisting a second model in generating a query statement, the query statement is used for querying in a first database, and the first metadata is used for describing attributes of data in the first database; and under the condition that one or more pieces of first metadata in the first database are changed, adjusting the first training data corresponding to the one or more pieces of first metadata. Through the method, the accuracy of generating the query statement by the model can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, electronic device, storage medium, and computer program product. Background Technology

[0002] In related technologies, large language models (LLMs) are used to generate query statements. During the query generation process of the LLM model, relevant models can be used to generate prompts to assist the large language model in generating more accurate query statements. For example, this relevant model could be a retrieval-augmented generation (RAG) model. However, the accuracy of the prompt generation model is relatively low, which leads to a lower accuracy rate in the generated query statements. Summary of the Invention

[0003] This application provides a data processing method, electronic device, storage medium, and computer program product that can improve the accuracy of model-generated query statements.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a data processing method, the method comprising: Determine first metadata corresponding to each of one or more first training data, the first training data being used to train a first model, the first model being used to assist a second model in generating a query statement, the query statement being used to perform a query in a first database, and the first metadata being used to describe the attributes of the data in the first database; If one or more of the first metadata in the first database change, the first training data corresponding to one or more of the first metadata shall be adjusted.

[0005] In the above scheme, the first training data includes one or more of the following: A data definition language used to define objects in the first database; The first example includes one or more questions and a corresponding Structured Query Language (SQL) statement for each question; The first statement is used to define the synonym.

[0006] In the above scheme, when the first training data is the first example, determining the first metadata corresponding to each of the one or more first training data includes: Analyze each of the first examples in one or more of the first examples to obtain a first abstract syntax tree corresponding to each first example, and each first abstract syntax tree includes multiple nodes; The first metadata corresponding to each first instance is obtained by parsing from each of the multiple nodes of the first abstract syntax tree.

[0007] In the above scheme, when the first training data is defined by a data definition language, the first metadata includes one or more of the following: The database name in the data definition language; The table field names in the data definition language; This refers to the field types in the data definition language.

[0008] In the above scheme, when the first training data is the first statement, the first metadata includes table field names or table field comments.

[0009] The above scheme further includes the following method: Based on whether each of the one or more listeners detects the first event, it is determined whether each of the first metadata in the first database has changed. Each listener is used to listen for a first event of the first metadata, and the first event is used to indicate that the corresponding first metadata has changed. When the listener detects the first event, the first metadata corresponding to the listener has changed. When the listener does not detect the first event, the first metadata corresponding to the listener has not changed. The above scheme further includes the following method: Run a first service, which is used to determine whether each of the first metadata in the first database has changed, and if one or more of the first metadata have changed, the server of the first service sends the first event; The first event sent by the server of the first service is listened for by one or more listeners.

[0010] This application provides a data processing apparatus, including: A determining unit is configured to determine first metadata corresponding to each of one or more first training data sets, the first training data sets being used to train a first model, the first model being used to assist a second model in generating a query statement, the query statement being used to perform a query in a first database, and the first metadata being used to describe the attributes of the data in the first database. The adjustment unit is used to adjust the first training data corresponding to one or more of the first metadata in the first database when one or more of the first metadata changes.

[0011] This application provides an electronic device, the electronic device comprising: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the method provided in the embodiments of this application.

[0012] This application provides a storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the data processing method provided in this application.

[0013] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the data processing method provided in this application.

[0014] The embodiments of this application have the following beneficial effects: In this embodiment, the second model is a model for generating query statements. The query statements generated by the second model are used for querying in the first database. For example, the second model can be a large language model. The first model is a model that assists the second model in generating query statements. For example, the first model assists the second model in generating query statements by generating prompt words. In this embodiment, it is necessary to determine the first metadata corresponding to the training data (i.e., the first training data) of the first model. This first metadata is used to describe the attributes of the data in the first database. When one or more first metadata in the first database changes, that is, when the attributes of the data in the first database change, the first training data corresponding to one or more of the first metadata is adjusted. It can be seen that in this application, when the attributes of the data in the database that needs to be queried by the query statement change, that is, when the attributes of the data in the first database change, this embodiment automatically updates the training data of the first model, so that the training data can be adapted to the attributes of the data in the first database, avoiding the introduction of too much dirty data into the first model, thereby enabling better training of the first model through the training data, thereby improving the accuracy of the first model, and thus improving the accuracy of the query statements generated by the second model. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the structure of the relevant LLM model provided in the embodiments of this application; Figure 2 A first flowchart illustrating a data processing method provided for an application embodiment of this application. Figure 3 A schematic diagram of the first abstract syntax tree provided for application embodiments of this application. Figure 4 A schematic diagram illustrating the RAG model training data processing provided in an application embodiment of this application; Figure 5 A schematic diagram of a display page provided for an application embodiment of this application; Figure 6 A schematic diagram illustrating LLM model data preprocessing and input provided for an application embodiment of this application; Figure 7 A second flowchart illustrating a data processing method provided for an application embodiment of this application. Figure 8 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application; Figure 9 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.

[0016] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0018] It should be understood that the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "one or more" in this document refers to any combination of at least two of any one or more elements from a set of A, B, and C. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set of A, B, and C.

[0019] It should also be understood that the term "instruction" mentioned in the embodiments of this application can be a direct instruction, an indirect instruction, or an indication of a relationship. For example, A instructing B can mean that A directly instructs B, such as B being able to obtain information through A; it can also mean that A indirectly instructs B, such as A instructing C, so B can obtain information through C; or it can mean that there is a relationship between A and B.

[0020] It should also be understood that the term "correspondence" mentioned in the embodiments of this application may indicate a direct or indirect correspondence between the two, or an association between the two, or a relationship of instruction and being instructed, configuration and being configured, etc.

[0021] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0022] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0023] The Text-to-SQL technology used in this application is described below: Text-to-SQL technology is used to convert text into SQL language. More specifically, it can transform the natural language (NL) problem in the database domain into a structured query language (SQL) that can be executed in relational databases.

[0024] In related technologies, Text-To-SQL technology can generate SQL statements using large LLM-like models. To reduce the number of tokens and improve accuracy, the user question can generally be processed through prompt engineering before it is input into the LLM model, forming a context prompt for the user question. This context prompt can be generated using RAG models or other retrieval enhancement models.

[0025] Please see Figure 1 The RAG model is used to generate context prompts. During the training of the RAG model, the training set includes: data definition language (DDL), field synonyms, questions, and examples composed of SQL. For example, the field synonyms can be Chinese synonyms of table fields, and the examples can be examples composed of question text and correct SQL text. The RAG model is trained using this training set, and the training data is vectorized and inserted into the vector library in RAG.

[0026] During the user's question-asking process, the question is segmented into words, then vectorized. A similarity search is performed on the vector database within the RAG model, and a prompt is generated as the context of the question. This prompt is then input into an LLM model, such as ChatGpT or Board, to ultimately generate the corresponding SQL statement for the question. Figure 1As shown, the black arrow represents the RAG model training process, and the gray arrow represents the process of converting user queries into SQL.

[0027] The accuracy of the RAG model directly determines the precision of the context prompt, which in turn affects the output of the LLM model. Currently, the input of the training set for the RAG model is incremental, relying on manual adjustments by the model maintainers. For example, if the data table initially has two fields, A and B, and the data table is updated to A and C, the model maintainers need to manually modify the rules. The issues involved in field B and the examples composed of SQL also need to be manually retrieved and modified.

[0028] When database tables undergo numerous or frequent updates, model maintainers may miss some modifications to the training data, resulting in some dirty data in the vector database. This can lead to a decrease in the accuracy of the RAG model and affect the output of SQL statements after calling the LLM model.

[0029] For example, based on hints from the order table, instruct the LLM model to generate the SQL statement corresponding to the question "Query the province with the highest sales volume".

[0030] The order table has the following structure: province is defined as the province, and pay_count is defined as the sales volume of the province.

[0031] The following is the SQL statement generated by LLM to query the province with the highest sales volume: SELECT province, SUM(pay_count) AS total_pay_count FROM order GROUP BY province ORDER BY total_pay_count DESC LIMIT 1; When the sales volume field name in the province table changes, the training data cannot be automatically adjusted, which leads to inaccurate generated context prompts and consequently SQL errors generated by the LLM.

[0032] Based on this, this application provides a data processing method. In this method, the second model is a model for generating query statements. The query statements generated by the second model are used for querying in a first database. For example, the second model can be a large language model. The first model is a model that assists the second model in generating query statements. For example, the first model assists the second model in generating query statements by generating prompt words. In this application, it is necessary to determine the first metadata corresponding to the training data (i.e., the first training data) of the first model. The first metadata is used to describe the attributes of the data in the first database. When one or more first metadata in the first database changes, that is, when the attributes of the data in the first database change, the first training data corresponding to one or more of the first metadata is adjusted. It can be seen that in this application, when the attributes of the data in the database that needs to be queried by the query statement change, that is, when the attributes of the data in the first database change, this application automatically updates the training data of the first model, so that the training data can be adapted to the attributes of the data in the first database, avoiding the introduction of too much dirty data into the first model, thereby enabling better training of the first model through the training data, thereby improving the accuracy of the first model, and thus improving the accuracy of the second model in generating query statements.

[0033] First, we introduce the application scenarios of the embodiments of this application. This application can be applied to one or more of the following scenarios: intelligent data analysis, answer system, intelligent customer service, data mining, data quality inspection, intelligent data development, etc.

[0034] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0035] This application provides a data processing method, see [link to relevant documentation] Figure 2 The method includes: Step 201: Determine the first metadata corresponding to each of the one or more first training data.

[0036] The first training data is used to train the first model, the first model is used to assist the second model in generating query statements, the query statements are used to perform queries in the first database, and the first metadata is used to describe the attributes of the data in the first database.

[0037] In practical applications, the first training data is the set of data samples used to train the first model. The first model constructs its own parameters and structure by learning the data features and patterns in the first training data.

[0038] For example, when the first model is a RAG model, the first training data is used to input into the RAG model to train the RAG model, and the first training data is vectorized and inserted into the vector retrieval library of the RAG model.

[0039] It should be noted that the term "metadata" in this application is an exemplary term and can be replaced with any possible term, such as intermediary data, relay data, or data description.

[0040] In practical applications, the first metadata includes: database name, table name, table field name, column name, comment information, or field type.

[0041] For example, the first metadata may be the database name of the first database, the name of the data table in the first database, the name of the table field in the first database, the column name of the table in the first database, or the comment information in the first database or the field type of the field in the first database.

[0042] For example, if the first database is a database of sales information of a province, then the first metadata can be "sales information of province". For example, if a table in the first database is named "order", then the first metadata can be "sales information of province". For example, if a field in the "order" table in the first database is named "province", then the first metadata can be "province". For example, if a column in the "order" table in the first database is named "pay_count", then the first metadata can be "pay_count". For example, if the comment information for a field in the "order" table in the first database is "province name", then the first metadata can be "province name". For example, if the field type of a field in the "order" table in the first database is int(8), then the first metadata can be int(8).

[0043] In practical applications, the first model is the neural network model.

[0044] In practical applications, a question is input into the first model, which can generate relevant prompts, such as contextual prompts. The prompts generated by the first model can be input into the second model as context for the question, guiding the second model to generate the corresponding query.

[0045] For example, in a typical application of this application, the first model is the RAG model. This application does not limit the first model to be the RAG model. The first model can also be any other possible model, such as the Cache-Augmented Generation (CAG) model.

[0046] In practical applications, the second model is a neural network model.

[0047] In practical applications, the second model can be used to generate SQL query statements.

[0048] For example, when this application is used in scenarios such as answering systems or intelligent customer service, the second model can convert the natural language text input by the user into an SQL query statement. When the first model and the second model are used together, the second model can input the information input by the user (e.g., user questions) and the prompts generated by the first model into the second model, and the second model can generate the SQL query statement corresponding to the information input by the user.

[0049] For example, in a typical application of this application, the second model is an LLM model. This application does not limit the second model to an LLM model. The first model can also be any other possible model, such as a model based on the Seq2Seq architecture.

[0050] In practical applications, the first database is the business database, the original data, the upstream database, the data source, or the upstream data source.

[0051] In practical applications, the first training data can be obtained based on the first database. For example, the first training data can be a portion of historical data in the first database, or it can be obtained by processing the data in the first database. For example, a portion of the data in the first training data can be obtained by annotating the table fields in the first database with synonyms.

[0052] In practical applications, when the query statement is derived from the user's input question by the second model, the answer to the user's question can be retrieved from the first database using this query statement.

[0053] In practical applications, the first training data includes one or more of the following: A data definition language used to define objects in the first database; The first example includes one or more questions and the corresponding SQL statement for each question; The first statement is used to define the synonym.

[0054] It should be noted that the data definition language in this application is an exemplary term and can be replaced with any possible term, such as a table creation language.

[0055] In the above application, the first training data was divided into three types: data definition language, first example, and first statement.

[0056] In practical applications, the first statement can be used to define synonyms for table fields in the first dataset. These synonyms can be Chinese synonyms or synonyms in other languages. For example, if sales volume is a table field in the first dataset, the first statement can be: sales volume defined as sales volume, or the first statement can also be: number of orders defined as sales volume.

[0057] In practical applications, the first metadata corresponding to each first training data includes: the metadata included in each first training data, the metadata involved in each first training data, or the metadata that each first training data depends on.

[0058] To help understand the above implementation method, let's take an example: Table 1 Examples of the first training data

[0059] In the example above, we take a table from the first database, namely the province sales table (region_sales), as an example.

[0060] In the example above, DDL is used to create the region_sales table and define the table field names, field types, and comment information for the table fields.

[0061] In the example above, the first statement defines revenue as a synonym for sales. Here, sales is a table field in the province `region_sales`.

[0062] In the above examples, the first example asks for the province with the most orders yesterday. The corresponding SQL query for the first example is: Select province_name from region_sales where sale_day=CURDATE() -INTERVAL 1 DAY order by order_count desc limit1, where province_name and sale_day are fields of the region_sales table.

[0063] In practical applications, the first training data can be user input. In this case, the user-input first training data is obtained, and then the first metadata corresponding to the first training data is determined.

[0064] For the three cases where the first training data is DDL, the first statement, and the first example, this application provides corresponding examples for determining the first metadata corresponding to the first training data. These examples can be performed independently or in combination. Please refer to the following examples for details: Example 1 In practical applications, when the first training data is the first example, determining the first metadata corresponding to each of the one or more first training data includes: Analyze each of the first examples in one or more of the first examples to obtain a first abstract syntax tree corresponding to each first example, and each first abstract syntax tree includes multiple nodes; The first metadata corresponding to each first instance is obtained by parsing from each of the multiple nodes of the first abstract syntax tree.

[0065] Based on the above implementation method, the first training data can be transformed into a first abstract syntax tree. By building the first abstract syntax tree, the first training data can be transformed into an intermediate structure that is easier to analyze, optimize and process, and the first metadata corresponding to the first training data can be quickly and accurately determined.

[0066] In practical applications, the first example can be analyzed using tools such as ANTLR4, including lexical analysis, syntax analysis, and semantic analysis, to obtain the first abstract syntax tree corresponding to the first example.

[0067] In practical applications, lexical analysis can be used to transform the initial training data into meaningful lexical units, such as identifiers or keywords. For example, deterministic finite automata (DFA) can be used to identify the initial metadata corresponding to the initial training data.

[0068] In practical applications, during syntax analysis, the token sequence can be converted into a tree structure according to syntax rules.

[0069] In practical applications, the first metadata corresponding to each first example is parsed from each of the multiple nodes of the first abstract syntax tree, including: traversing multiple nodes in the first abstract syntax tree, parsing the first metadata on which the first training data depends, and forming a mapping relationship between the first training data and the first metadata.

[0070] In practical applications, the visitor model can be used to traverse multiple nodes of the abstract syntax tree.

[0071] In practical applications, multiple nodes include: SelectItem, FromItem, and WhereItem. SelectItem is the object associated with Select in the first abstract syntax tree, FromItem is the object associated with from in the first abstract syntax tree, and WhereItem is the object associated with where in the first abstract syntax tree.

[0072] like Figure 3 As shown, Figure 3 Taking the first example, which focuses on the city with the most orders yesterday, the corresponding metadata for the first example is province_name, region_sales, and sale_day. SelectItem includes the node containing province_name, WhereItem includes the node containing region_sales, and WhereItem includes the node containing sale_day.

[0073] In practical applications, the first metadata corresponding to the first example includes: metadata describing the database, metadata of the table, or metadata of the field.

[0074] Example 2 In practical applications, when the first training data defines a language for that data, the first metadata includes one or more of the following: The database name in the data definition language; The table field names in the data definition language; This refers to the field types in the data definition language.

[0075] Taking the data definition language as CREATE TABLE 'region_sales' ('sale_day' int(8) COMMENT='date', 'order_count' Int(8) COMMENT='order quantity', 'province_name' varchar(255) COMMENT='province name', 'pay_cnt' BigInt(16) COMMENT='sales', ) COMMENT='regional sales table' as an example, the first metadata includes: sale_day, Int(8), date, order_count, order quantity, province_name, varchar(255), province name, pay_cnt, BigInt(16), sales or regional sales table, etc.

[0076] Example 3 In practical applications, when the first training data is the first statement, the first metadata includes table field names or table field comments.

[0077] Taking the first statement as "daily order volume define order number" as an example, this first statement defines daily order volume as a synonym for order number. Here, order number is the comment of the table field named order_count. Then the metadata that this training set depends on includes order_count.

[0078] In practical applications, the method further includes: determining whether each of the first metadata in the first database has changed based on whether each of the one or more listeners has detected the first event, wherein each listener is used to listen for a first event of the first metadata, the first event being used to indicate that the corresponding first metadata has changed; when the listener detects the first event, the first metadata corresponding to the listener has changed; when the listener does not detect the first event, the first metadata corresponding to the listener has not changed. In practical applications, this listener is an event listener, such as an event listener in JavaScript used to monitor user interactions. When a user clicks, types on the keyboard, or scrolls the page, specific events are triggered. Through event listeners, developers can specify handler functions for these events, thereby enabling dynamic interaction.

[0079] In practical applications, the first event is a change event. For example, the first event includes: deleting the first metadata, updating the first metadata, or adding new first metadata, etc.

[0080] Table 2 Field Change Events

[0081] Taking Table 2 as an example, the first event can be a field change event. For example, the first metadata can be a comment on a table field, such as sales, and the first event can be updating the sales to the sales amount.

[0082] In practical applications, each listener is mapped to a primary metadata object.

[0083] For practical applications, please refer to Figure 4 When one set of first training data corresponds to one or more sets of first metadata, then one set of first training data corresponds to one or more listeners. For example, the listener is located at... Figure 4 Metadata change monitoring module in the system.

[0084] In practical applications, one listener corresponds to one first metadata, one first metadata can correspond to one or more first training data, and one listener can correspond to one or more first training data.

[0085] In practical applications, after determining the first metadata corresponding to each of the first training data in one or more first training data sets, a listener is registered for each first metadata, and each listener is used to listen for a first event of that first metadata.

[0086] In practical applications, this involves displaying the mapping relationship between the listener and the initial training data. For example, the mapping relationship between the listener and the initial training data can be shown to the user on a webpage.

[0087] In practical applications, a data table in the first database has a many-to-many relationship with the first metadata, and a data table in the first database has a many-to-many relationship with the first training data. Whenever a new dataset is entered into the first model, a new mapping relationship is added according to the method described above for determining the first metadata corresponding to each of the first training data in one or more first training data. This mapping relationship is the mapping relationship between the first training data and the listener, and registration is performed. Figure 5 This is a mapping page between listeners and the first training set. Each listener listens for the first event in the first database.

[0088] In practical applications, when changes in the primary metadata necessitate adjustments to the primary training data corresponding to one or more primary metadata sets, a first notification can be sent. This first notification informs the user that the primary training data requires adjustment. For example, the primary training data requiring adjustment can be highlighted in red by displaying the mapping relationship between the display listener and the primary training data.

[0089] In practical applications, after receiving the first event, the listener will send a first notification, which could, for example, notify downstream devices to change the training set. Simultaneously, the first training data that needs to be changed will be highlighted in red on the page. This first training data that needs to be changed is the first training data whose first metadata in the corresponding first database has been modified.

[0090] Understandably, since the listener detects the first event, it indicates that the first metadata has changed, so the first training data corresponding to one or more first metadata needs to be adjusted, and therefore the first notification needs to be sent.

[0091] Regarding how to determine whether each piece of first metadata in the first database has changed, this application provides several possible implementation methods, which can be found in one or more of the following implementation methods: Implementation method 1: Run the first service to determine whether each of the first metadata in the first database has changed.

[0092] In practical applications, a first service is run to determine whether each piece of first metadata in the first database has changed. If one or more pieces of first metadata have changed, the server of the first service sends a first event. The first event sent by the server of the first service is listened to by one or more listeners.

[0093] In practical applications, the primary server can send a dump request to the primary server based on the interaction protocol between the primary and secondary servers. Upon receiving the dump request, the primary server sends log information to the primary server. The primary server parses this log information to obtain the change information of the primary metadata; for example, this change information could be the relationship between fields before and after the change. Then, based on this relationship, a primary event is formed.

[0094] For example, based on the MySQL slave interaction protocol, the first service server impersonates a MySQL slave and sends a dump request to the MySQL master. Upon receiving the dump request, the MySQL master begins pushing binary logs to the data listening service. The first service server parses the byte stream formed by the binary log, establishes the relationship between the changed fields as the first event, and sends a change notification.

[0095] In practical applications, binarylog records changes to the table structure and modifications to the table data in the first database.

[0096] Implementation Method 2: The first module determines whether each piece of first metadata in the first database has changed.

[0097] In the above scheme, the method further includes: a first module determining whether each of the first metadata in the first database has changed; if one or more of the first metadata have changed, the first module sends the first event; and one or more listeners listen for the first event sent by the first module.

[0098] In practical applications, the first module can be a module on the device that executes the method of this application. For example, the device that executes the method of this application can be a server, and the first module can be a module on the server. The listener is also a module configured on the server.

[0099] In practical applications, the first module can send a dump request to the master server based on the interaction protocol between the master and slave servers. After receiving the dump request, the master server sends log information to the first module. The first server parses this log information to obtain the change information of the first metadata. For example, the change information of the first metadata can be the relationship between fields before and after the change, and then the first event is formed based on the relationship between the fields before and after the change.

[0100] For example, based on the MySQL slave interaction protocol, the first module impersonates a MySQL slave and sends a dump request to the MySQL master. Upon receiving the dump request, the MySQL master begins pushing binary logs to the data listening service. The first module parses the byte stream formed by the binary log, establishes the relationship between the changed fields as the first event, and sends a change notification.

[0101] Step 202: If one or more of the first metadata in the first database changes, adjust the first training data corresponding to one or more of the first metadata.

[0102] In practical applications, after each first metadata listener receives the first event, it sends an update event to the downstream training set update service. Figure 5 Using the data in Table 2 as an example, the listener will send at least one of three update events to the training set update service. These update events include: for the first example, the first example whose question is to query the province with the highest sales volume needs to be updated; for the first statement, the first statement "revenue defined as sales" needs to be updated; or, for the data definition language, the data definition language for creating the regional sales table needs to be updated.

[0103] In practical applications, when one or more primary metadata entries in the primary database change, and adjustments are made to the primary training data corresponding to those primary metadata entries, the fields in the primary training data associated with those primary metadata entries can be updated. These fields can be one or more primary metadata entries from the primary training data.

[0104] Still with Figure 5 Using the data in Table 2 as an example, when the training set update service receives at least one of the three update events, the training set update service will perform at least one of the following update steps: Update "Revenue defined as sales" to "Revenue defined as sales amount"; Update "select procince_name" to "selectprovince" in the SQL statement of the first example; Change “procince_name” to “province” in the data definition statement, and change “sales” to “sales amount”.

[0105] For example, in the SQL statement "Select province_name from region_saleswhere sale_day=CURDATE() - INTERVAL 1 DAY order byorder_count desc limit1" in the first example, "select procince_name" can be updated to "selectprovince".

[0106] For example, the data definition language statement "CREATE TABLE 'region_sales' ('sale_day' int(8) COMMENT='Date','order_count' Int(8) COMMENT='Order Quantity','province_name'varchar(255)COMMENT='Province Name','pay_cnt'BigInt(16) COMMENT='Sales',) COMMENT='Regional Sales Table'" can be updated to: "CREATE TABLE 'region_sales' ('sale_day' int(8) COMMENT='Date','order_count' Int(8) COMMENT='Order Quantity','province' varchar(255)COMMENT='Province Name','pay_cnt'BigInt(16) COMMENT='Sales Amount',) COMMENT='Regional Sales Table'".

[0107] In practical applications, after adjusting the first training data, a second training data is obtained. This second training data is then used as the training data for the first model, training it to generate the prompts for the second model. Taking the first model as a RAG model as an example, the second training data can be used as the training data for the RAG model, and the second training data vector is inserted into the vector library within the RAG model.

[0108] In practical applications, the second training data is used as the training data for the first model. After the first model is trained to generate the prompts for the second model, the user's question is input into the first model, and the prompts for the user's question are generated by the first model.

[0109] In practical applications, the prompt word and the user's question are input into the second model, which generates a query statement corresponding to the user's question. This query statement then retrieves the answer to the user's question from the first database.

[0110] Taking the application of this application's embodiments to Text-To-SQL technology as an example, where the first model in this application is the RAG model and the second model is the LLM model, Text-To-SQL technology mainly uses the RAG model for retrieval enhancement. Please refer to [link to relevant documentation]. Figure 6 The input to an LLM model, and the data processing before input, include the following three steps: Step 1 is used to vectorize the user's question text and perform similarity retrieval in the vector database. Step 1 includes: vectorizing the question text using the CBOW model under word2vec to generate a word vector embedding matrix, performing similarity retrieval in the vector database, calculating the similarity score between the query vector and all stored vectors using the cosine theorem, and returning the record with the highest score. Step 2 is used for generating the large model prompt context. Step 2 includes: using the top N most similar vectors as background knowledge, and generating a prompt with the user question and task description; Step 3 is used to input the prompt and user question into the LLM model: The prompt context generated in Step 2 is used as the question background and input together with the user question into the LLM model for final SQL generation.

[0111] In practical applications, embodiments of this application can be used to preprocess the first training data of a RAG model to obtain second training data. This second training data is used to train the RAG model to generate prompt words for an LLM. The context of these prompt words is used as the question background and, together with the user question, is input into the LLM model to finally generate the SQL statement for the user question.

[0112] As can be seen from the above, this application proposes a method for determining the metadata corresponding to the training data and automatically updating the training data of the first model when the metadata in the first database changes. This allows the training data to be adapted to the attributes of the data in the first database, avoiding the introduction of too much dirty data into the first model. This enables better training of the first model using the training data, thereby improving the accuracy of the first model and ultimately improving the accuracy of the query statements generated by the second model. Taking the first model as a RAG model as an example, this application can automatically update the training data of the RAG model when the metadata in the upstream database changes, thereby improving the quality of background knowledge posed to the LLM model and ultimately improving the accuracy of Text-to-SQL queries.

[0113] The present application will be further described in detail below with reference to application embodiments.

[0114] This application provides a data processing method through its application embodiments; see [link to relevant documentation]. Figure 7 , Figure 7 For this application to Text-To-Sql technology, the first model is a RAG model example. The data processing method can mainly include the following steps: Step 701: Obtain the first training data that needs to be entered into the RAG model.

[0115] The first training data is used to train the first model, the first model is used to assist the second model in generating query statements, and the query statements are used to perform queries in the first database.

[0116] In practical applications, the first training data is the set of data samples used to train the first model. The first model constructs its own parameters and structure by learning the data features and patterns in the first training data.

[0117] For example, when the first model is a RAG model, the first training data is used to input into the RAG model to train the RAG model, and the first training data is vectorized and inserted into the vector retrieval library of the RAG model.

[0118] In practical applications, the first model is the neural network model.

[0119] In practical applications, a question is input into the first model, which can generate relevant prompts, such as contextual prompts. The prompts generated by the first model can be input into the second model as context for the question, guiding the second model to generate the corresponding query.

[0120] For example, in a typical application of this application, the first model is the RAG model. This application does not limit the first model to be the RAG model. The first model can also be any other possible model, such as the Cache-Augmented Generation (CAG) model.

[0121] In practical applications, the second model is a neural network model.

[0122] In practical applications, the second model can be used to generate SQL query statements.

[0123] For example, when this application is used in scenarios such as answering systems or intelligent customer service, the second model can convert the natural language text input by the user into an SQL query statement. When the first model and the second model are used together, the second model can input the information input by the user (e.g., user questions) and the prompts generated by the first model into the second model, and the second model can generate the SQL query statement corresponding to the information input by the user.

[0124] For example, in a typical application of this application, the second model is an LLM model. This application does not limit the second model to an LLM model. The first model can also be any other possible model, such as a model based on the Seq2Seq architecture.

[0125] In practical applications, the first database is the business database, the original data, the upstream database, the data source, or the upstream data source.

[0126] In practical applications, the first training data can be obtained based on the first database. For example, the first training data can be a portion of historical data in the first database, or it can be obtained by processing the data in the first database. For example, a portion of the data in the first training data can be obtained by annotating the table fields in the first database with synonyms.

[0127] In practical applications, when the query statement is derived from the user's input question by the second model, the answer to the user's question can be retrieved from the first database using this query statement.

[0128] In practical applications, the first training data includes one or more of the following: A data definition language used to define objects in the first database; The first example includes one or more questions and the corresponding SQL statement for each question; The first statement is used to define the synonym.

[0129] In practical applications, the first statement can be used to define synonyms for table fields in the first dataset. These synonyms can be Chinese synonyms or synonyms in other languages. For example, if sales volume is a table field in the first dataset, the first statement can be: sales volume defined as sales volume, or the first statement can also be: number of orders defined as sales volume.

[0130] It should be noted that step 701 above is an optional step, and this application does not require that step 701 be performed.

[0131] Step 702: Determine the first metadata corresponding to each of the one or more first training data sets.

[0132] The first metadata is used to describe the attributes of the data in the first database.

[0133] In practical applications, the first metadata includes: database name, table name, table field name, column name, comment information, or field type.

[0134] For example, the first metadata may be the database name of the first database, the name of the data table in the first database, the name of the table field in the first database, the column name of the table in the first database, or the comment information in the first database or the field type of the field in the first database.

[0135] In practical applications, the first metadata corresponding to each first training data includes: the metadata included in each first training data, the metadata involved in each first training data, or the metadata that each first training data depends on.

[0136] In practical applications, when the first training data is the first example, determining the first metadata corresponding to each of the one or more first training data includes: Analyze each of the first examples in one or more of the first examples to obtain a first abstract syntax tree corresponding to each first example, and each first abstract syntax tree includes multiple nodes; The first metadata corresponding to each first instance is obtained by parsing from each of the multiple nodes of the first abstract syntax tree.

[0137] In practical applications, the first example can be analyzed using tools such as ANTLR4, including lexical analysis, syntax analysis, and semantic analysis, to obtain the first abstract syntax tree corresponding to the first example.

[0138] In practical applications, lexical analysis can be used to transform the initial training data into meaningful lexical units, such as identifiers or keywords. For example, deterministic finite automata (DFA) can be used to identify the initial metadata corresponding to the initial training data.

[0139] In practical applications, during syntax analysis, the token sequence can be converted into a tree structure according to syntax rules.

[0140] In practical applications, the first metadata corresponding to each first example is parsed from each of the multiple nodes of the first abstract syntax tree, including: traversing multiple nodes in the first abstract syntax tree, parsing the first metadata on which the first training data depends, and forming a mapping relationship between the first training data and the first metadata.

[0141] In practical applications, the visitor model can be used to traverse multiple nodes of the abstract syntax tree.

[0142] In practical applications, multiple nodes include: SelectItem, FromItem, and WhereItem. SelectItem is the object associated with Select in the first abstract syntax tree, FromItem is the object associated with from in the first abstract syntax tree, and WhereItem is the object associated with where in the first abstract syntax tree.

[0143] In practical applications, the first metadata corresponding to the first example includes: metadata describing the database, metadata of the table, or metadata of the field.

[0144] In practical applications, when the first training data defines a language for that data, the first metadata includes one or more of the following: The database name in the data definition language; The table field names in the data definition language; This refers to the field types in the data definition language.

[0145] In practical applications, when the first training data is the first statement, the first metadata includes table field names or table field comments.

[0146] Step 703: Register a listener for each piece of metadata.

[0147] In practical applications, the change of each first metadata in the first database is determined by whether each of the one or more listeners detects the first event. Each listener is used to listen for a first event of the first metadata, which indicates that the corresponding first metadata has changed. When the listener detects the first event, the first metadata corresponding to the listener has changed; when the listener does not detect the first event, the first metadata corresponding to the listener has not changed. In practical applications, this listener is an event listener, such as an event listener in JavaScript used to monitor user interactions. When a user clicks, types on the keyboard, or scrolls the page, specific events are triggered. Through event listeners, developers can specify handler functions for these events, thereby enabling dynamic interaction.

[0148] In practical applications, the first event is a change event. For example, the first event includes: deleting the first metadata, updating the first metadata, or adding new first metadata, etc.

[0149] In practical applications, a data table in the first database has a many-to-many relationship with the first metadata, and a data table in the first database has a many-to-many relationship with the first training data. Whenever a new dataset is entered into the first model, a new mapping relationship is added according to the method described above for determining the first metadata corresponding to each of the first training data in one or more first training data. This mapping relationship is the mapping relationship between the first training data and the listener, and registration is performed. Figure 5 This is a mapping page between listeners and the first training set. Each listener listens for the first event in the first database.

[0150] In practical applications, when changes in the primary metadata necessitate adjustments to the primary training data corresponding to one or more primary metadata sets, a first notification can be sent. This first notification informs the user that the primary training data requires adjustment. For example, the primary training data requiring adjustment can be highlighted in red by displaying the mapping relationship between the display listener and the primary training data.

[0151] It should be noted that step 703 above is an optional step, and this application does not require that step 703 must be performed.

[0152] Step 704: Listen for the first event using a listener.

[0153] In practical applications, a first service is run to determine whether each piece of first metadata in the first database has changed. If one or more pieces of first metadata have changed, the server of the first service sends a first event. The first event sent by the server of the first service is listened to by one or more listeners.

[0154] For example, based on the MySQL slave interaction protocol, the first service server impersonates a MySQL slave and sends a dump request to the MySQL master. Upon receiving the dump request, the MySQL master begins pushing binary logs to the data listening service. The first service server parses the byte stream formed by the binary log, establishes the relationship between the changed fields as the first event, and sends a change notification.

[0155] In practical applications, binarylog records changes to the table structure and modifications to the table data in the first database.

[0156] Step 705: When the first event is detected, adjust the corresponding first training data.

[0157] In practical applications, after each first metadata listener receives the first event, it sends an update event to the downstream training set update service. Figure 5 Using the data in Table 2 as an example, the listener will send at least one of three update events to the training set update service. These update events include: for the first example, the first example whose question is to query the province with the highest sales volume needs to be updated; for the first statement, the first statement "revenue defined as sales" needs to be updated; or, for the data definition language, the data definition language for creating the regional sales table needs to be updated.

[0158] In practical applications, when one or more primary metadata entries in the primary database change, and adjustments are made to the primary training data corresponding to those primary metadata entries, the fields in the primary training data associated with those primary metadata entries can be updated. These fields can be one or more primary metadata entries from the primary training data.

[0159] As described above, this application addresses the current situation where the data table structure changes and training data cannot be updated in a timely manner in the context of Text-To-Sql technology scenarios involving user responses. It proposes a method for automatically updating training data, automatically parsing the SQL statements in the training data to form a relationship between the training data (i.e., the first training data) and metadata. This method binds the training data to the metadata and registers listeners to monitor changes in metadata from upstream data sources, updating the training data promptly to improve the quality of the training set. This, in turn, improves the quality of background knowledge used to ask questions of the LLM model, ultimately enhancing the accuracy of Text-To-Sql. This application, targeting the Text-To-Sql technology scenario involving user responses, divides the training data into three types: DDL, synonym definitions (first statement, first example), and parses the relationship between each type of training data and its metadata. Based on the metadata upon which the training data depends, this application establishes a many-to-many relationship between the listener monitoring metadata changes and the training data, providing a tree-structured display page of the relationship mapping. Training data that has been automatically updated due to changes in the upstream data structure is highlighted in red, facilitating management and maintenance by model developers.

[0160] Based on the embodiments described above, this application also provides a data processing apparatus, see [link to previous document]. Figure 8 The data processing device includes: The first determining unit 801 is used to determine first metadata corresponding to each of the first training data in one or more first training data, the first training data being used to train a first model, the first model being used to assist a second model in generating a query statement, the query statement being used to perform a query in a first database, and the first metadata being used to describe the attributes of the data in the first database. The adjustment unit 802 is used to adjust the first training data corresponding to one or more of the first metadata in the first database when one or more of the first metadata in the first database changes.

[0161] In one embodiment, the first training data includes one or more of the following: A data definition language used to define objects in the first database; The first example includes one or more questions and a corresponding Structured Query Language (SQL) statement for each question; The first statement is used to define the synonym.

[0162] In one embodiment, when the first training data is a first example, the first determining unit 801 determines the first metadata corresponding to each of the one or more first training data, including: Analyze each of the first examples in one or more of the first examples to obtain a first abstract syntax tree corresponding to each first example, and each first abstract syntax tree includes multiple nodes; The first metadata corresponding to each first instance is obtained by parsing from each of the multiple nodes of the first abstract syntax tree.

[0163] In one embodiment, when the first training data is in a data definition language, the first metadata includes one or more of the following: The database name in the data definition language; The table field names in the data definition language; This refers to the field types in the data definition language.

[0164] In one embodiment, when the first training data is the first statement, the first metadata includes table field names or table field comments.

[0165] In one embodiment, the data processing apparatus further includes: a second determining unit, the second determining unit being configured to: Based on whether each of the one or more listeners detects the first event, it is determined whether each of the first metadata in the first database has changed. Each listener is used to listen for a first event of the first metadata, and the first event is used to indicate that the corresponding first metadata has changed. When the listener detects the first event, the first metadata corresponding to the listener has changed. When the listener does not detect the first event, the first metadata corresponding to the listener has not changed. In one embodiment, the data processing apparatus further includes a processing unit, the processing unit being configured to: Run a first service, which is used to determine whether each of the first metadata in the first database has changed, and if one or more of the first metadata have changed, the server of the first service sends the first event; The first event sent by the server of the first service is listened for by one or more listeners.

[0166] In practical applications, the first determining unit 801, the adjusting unit 802, the second determining unit, and the processing unit can be implemented by the processor in the data processing device.

[0167] It should be noted that the data processing apparatus provided in the above embodiments is only illustrated by the division of the above program modules. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the apparatus can be divided into different program modules to complete all or part of the processing described above. In addition, the data processing apparatus and data processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0168] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, this application also provides an electronic device, see [link to relevant documentation]. Figure 9 The electronic device includes: The first communication interface 1 is capable of exchanging information with other devices; The first processor 2 is configured to determine first metadata corresponding to each of one or more first training data sets, the first training data sets being used to train a first model, the first model being used to assist a second model in generating a query statement, the query statement being used to perform a query in a first database, and the first metadata being used to describe the attributes of the data in the first database; and to adjust the first training data corresponding to one or more of the first metadata in the first database when one or more of the first metadata in the first database changes. In one embodiment, the first training data includes one or more of the following: A data definition language used to define objects in the first database; The first example includes one or more questions and a corresponding Structured Query Language (SQL) statement for each question; The first statement is used to define the synonym.

[0169] In one embodiment, when the first training data is a first example, the first processor 2 determines first metadata corresponding to each of the one or more first training data sets, including: Analyze each of the first examples in one or more of the first examples to obtain a first abstract syntax tree corresponding to each first example, and each first abstract syntax tree includes multiple nodes; The first metadata corresponding to each first instance is obtained by parsing from each of the multiple nodes of the first abstract syntax tree.

[0170] In one embodiment, when the first training data is in a data definition language, the first metadata includes one or more of the following: The database name in the data definition language; The table field names in the data definition language; This refers to the field types in the data definition language.

[0171] In one embodiment, when the first training data is the first statement, the first metadata includes table field names or table field comments.

[0172] In one embodiment, the first processor 2 is further configured to: Based on whether each of the one or more listeners detects the first event, it is determined whether each of the first metadata in the first database has changed. Each listener is used to listen for a first event of the first metadata, and the first event is used to indicate that the corresponding first metadata has changed. When the listener detects the first event, the first metadata corresponding to the listener has changed. When the listener does not detect the first event, the first metadata corresponding to the listener has not changed. In one embodiment, the first processor 2 is further configured to: Run a first service, which is used to determine whether each of the first metadata in the first database has changed, and if one or more of the first metadata have changed, the server of the first service sends the first event; The first event sent by the server of the first service is listened for by one or more listeners.

[0173] It should be noted that the specific processing procedure of the first communication interface 1 can be understood by referring to the above method.

[0174] Of course, in practical applications, the various components in an electronic device are coupled together through bus system 4. It can be understood that bus system 4 is used to achieve communication and connection between these components. In addition to the data bus, bus system 4 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 9 The general will label all buses as Bus System 4.

[0175] The first memory 3 in this embodiment is used to store various types of data to support operation in the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0176] The methods disclosed in the embodiments of this application can be applied to the first processor 2, or implemented by the first processor 2. The first processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware or by instructions in the form of software in the first processor 2. The first processor 2 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The first processor 2 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the first memory 3. The first processor 2 reads the information in the first memory 3 and completes the steps of the aforementioned method in combination with its hardware.

[0177] In an exemplary embodiment, the electronic device may be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general-purpose processors, controllers, MCUs, microprocessors, or other electronic components to perform the aforementioned method.

[0178] It is understood that the first memory 3 in the embodiments of this application can be volatile memory or non-volatile memory, or both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); the magnetic surface memory can be disk storage or magnetic tape storage. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0179] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a first memory 3 storing a computer program, which can be executed by a first processor 2 to complete the steps described in the aforementioned method.

[0180] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a first processor 2 to perform the steps described in the foregoing method.

[0181] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A data processing method, characterized in that, The method includes: Determine first metadata corresponding to each of one or more first training data, the first training data being used to train a first model, the first model being used to assist a second model in generating a query statement, the query statement being used to perform a query in a first database, and the first metadata being used to describe the attributes of the data in the first database. If one or more of the first metadata in the first database changes, the first training data corresponding to one or more of the first metadata is adjusted.

2. The method according to claim 1, characterized in that, The first training data includes one or more of the following: A data definition language, which is used to define objects in the first database; A first example, comprising one or more questions and a corresponding Structured Query Language (SQL) statement for each question; The first statement is used to define synonyms.

3. The method according to claim 2, characterized in that, When the first training data is a first example, determining the first metadata corresponding to each of the one or more first training data includes: Analyze each of the first examples in one or more of the first examples to obtain a first abstract syntax tree corresponding to each first example, and each first abstract syntax tree includes multiple nodes; The first metadata corresponding to each first example is obtained by parsing from each of the plurality of nodes of the first abstract syntax tree.

4. The method according to claim 2 or 3, characterized in that, When the first training data is in the Data Definition Language, the first metadata includes one or more of the following: The database name in the data definition language; The table field names in the data definition language; Field types in the data definition language.

5. The method according to any one of claims 2 to 4, characterized in that, In the case where the first training data is the first statement, the first metadata includes table field names or table field comments.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Based on whether each of one or more listeners detects a first event, it is determined whether each of the first metadata in the first database has changed. Each listener is used to listen for a first event of the first metadata. The first event is used to indicate that the corresponding first metadata has changed. When the listener detects the first event, the first metadata corresponding to the listener has changed. When the listener does not detect the first event, the first metadata corresponding to the listener has not changed.

7. The method according to claim 6, characterized in that, The method further includes: Run a first service, which is used to determine whether each of the first metadata in the first database has changed, and if one or more of the first metadata have changed, the server of the first service sends the first event; The first event triggered by the server of the first service is monitored by one or more of the listeners.

8. An electronic device, characterized in that, include: The processor and the memory used to store computer programs that can run on the processor. When the processor is used to run the computer program, it performs the steps of the method according to any one of claims 1 to 7.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images