Data processing method, computing device, storage medium and program product

Abstract

The embodiments of the present application provide a data processing method, computing device, storage medium, and program product. The method comprises obtaining a target query text; recalling at least one sample pair from a sample collection, the sample pair comprising a sample query text and a sample query statement corresponding to the sample query text; determining database description information that matches the target query text; and generating a target query statement corresponding to the target query text using a large model based on the database description information, the at least one sample pair, and the target query text. The technical solution provided by the embodiments of the present application can improve the accuracy of text-converted query statements and reduce the computational cost of text-converted query statements.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and in particular to a data processing method, computing device, storage medium, and program product. Background Art

[0002] With the development of big data and cloud computing, converting text into query statements has become a key research topic in database management and data query. In the era of big data, enterprises have accumulated vast amounts of data, and effectively leveraging this data to drive decision-making has become crucial. Traditional manual query writing methods are difficult for non-technical personnel to master, limiting the efficiency of data utilization. Therefore, developing capabilities that automatically convert text into query statements is crucial for improving data access efficiency.

[0003] The emergence of big model technology has made it possible to use large models to convert text into query statements. However, the conventional implementation method usually uses training corpus to fine-tune the pre-trained large language model, which makes the current text-to-query statement conversion solution have the problem of high computational cost. Summary of the Invention

[0004] The embodiments of the present application provide a data processing method, a computing device, a storage medium, and a program product to solve the technical problem of high computational cost of text conversion query statements in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a data processing method, comprising:

[0006] Get the target query text;

[0007] Recalling at least one sample pair from the sample set, the sample pair comprising a sample query text and a sample query statement corresponding to the sample query text;

[0008] Determining database description information that matches the target query text;

[0009] A target query statement corresponding to the target query text is generated using a large model according to the database description information, the at least one sample pair, and the target query text.

[0010] Optionally, recalling at least one sample pair from the sample set includes:

[0011] At least one sample pair that meets the relevance requirement with the target query text is recalled from the sample collection.

[0012] Optionally, recalling at least one sample pair from the sample set includes:

[0013] From the sample set, unrecalled sample pairs that are relevant to the target query text and irrelevant to the recalled sample pairs are recalled to obtain a plurality of sample pairs.

[0014] Optionally, recalling unrecalled sample pairs that are relevant to the target query text and unrelated to the recalled sample pairs from the sample set to obtain a plurality of sample pairs includes:

[0015] From the sample set, unrecalled sample pairs whose similarity to the target query text satisfies a similarity condition and whose difference from the recalled sample pairs satisfies a difference condition are recalled to obtain a plurality of sample pairs.

[0016] Optionally, recalling unrecalled sample pairs that are relevant to the target query text and unrelated to the recalled sample pairs from the sample set to obtain a plurality of sample pairs includes:

[0017] For any unrecalled sample pair in the sample set, calculating a first similarity between the target query text and the sample query text in the unrecalled sample pair, and a maximum second similarity between the target query text and the sample query text in the recalled sample pair, and subtracting the maximum second similarity from the first similarity to obtain a similarity difference;

[0018] The unrecalled sample pairs with the largest similarity difference are recalled.

[0019] Optionally, recalling unrecalled sample pairs that are relevant to the target query text and unrelated to the recalled sample pairs from the sample set to obtain a plurality of sample pairs includes:

[0020] For any unrecalled sample pair in the sample set, calculate the first similarity between the target query text and the sample query text in the unrecalled sample pair, as well as the maximum second similarity with the sample query text in the recalled sample pair, and find the sample pair that maximizes the first similarity and minimizes the maximum second similarity for recall.

[0021] Optionally, determining the database description information that matches the target query text includes:

[0022] Identify entities and attributes involved in the target query text;

[0023] According to the entity and the attribute, the hit database description information is queried.

[0024] Optionally, generating a target query statement corresponding to the target query text using a large model according to the database description information, the at least one sample pair, and the target query text includes:

[0025] Generate prompt information based on the database description information, the target query text and the at least one sample pair according to a prompt template;

[0026] The prompt information is input into a large model, so as to utilize the large model to perform context learning from the at least one sample pair, and to generate a query statement corresponding to the target query text based on the database description information.

[0027] Optionally, it also includes:

[0028] The target query text and the target query sentence are taken as a sample pair and added to the sample set.

[0029] Optionally, recalling at least one sample pair from the sample set includes:

[0030] Performing clustering processing on the sample set to obtain multiple sample combinations;

[0031] At least one sample pair whose similarity to the target query text meets a similarity condition is recalled from the multiple sample combinations.

[0032] Optionally, obtaining the target query text includes:

[0033] Obtain the target query text submitted by the user;

[0034] After generating the target query statement, the method further includes:

[0035] Execute the target query statement to obtain a query result, and notify the user of the query result; or notify the user of the target query statement.

[0036] Optionally, obtaining the target query text includes:

[0037] Obtain the target query voice submitted by the user;

[0038] The target query speech is recognized to obtain a corresponding target query text.

[0039] In a second aspect, an embodiment of the present application provides a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement the data processing method described in the first aspect above.

[0040] In a third aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processing component, the data processing method described in the first aspect above is implemented.

[0041] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processing component, implements the data processing method described in the first aspect above.

[0042] In an embodiment of the present application, a target query text is obtained; at least one sample pair is retrieved from a sample collection, the sample pair including the sample query text and a sample query statement corresponding to the sample query text; database description information matching the target query text is determined; and based on the database description information, the at least one sample pair, and the target query text, a target query statement corresponding to the target query text is directly generated using an existing large model. In this process, by obtaining the target query text, the query requirements can be accurately captured, providing a foundation for subsequent query statement construction. By recalling at least one sample pair from the sample set, the existing sample pairs can be used as context information to guide the generation of target query statements, thereby increasing diversity and coverage while ensuring relevance, making the recall results more comprehensive, ensuring the quality of the recalled sample pairs while taking into account the recall efficiency; by determining the database description information that matches the target query text, it is ensured that the large model can understand the database structure to improve the accuracy of the generated target query statement; based on the database description information, at least one sample pair, and the target query text, the existing large model is directly used to generate the target query statement corresponding to the target query text. By combining the sample pairs, the target query text and the database description information, the large model is used to generate accurate and effective query statements. This not only fully utilizes the powerful generation capabilities of the large model and ensures that the generated query statements are both accurate and efficient, but more importantly, this method avoids the expensive fine-tuning process of the large model and greatly reduces the computational cost and technical threshold.

[0043] These and other aspects of the present application will become more readily apparent from the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0045] Figure 1 A flow chart showing an embodiment of a data processing method provided by the present application is shown;

[0046] Figure 2 The following is a schematic diagram showing the interaction between the user end and the server end in a data processing scenario provided by the present application;

[0047] Figure 3 This figure shows the interaction diagram of the server provided by this application in the data processing scenario;

[0048] Figure 4 A schematic structural diagram of an embodiment of a data processing device provided by the present application is shown;

[0049] Figure 5 A schematic structural diagram of a computing device provided by the present application is shown. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0051] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to being different types.

[0052] The technical solutions of the embodiments of the present application can be applied to the fields of database management and data query, such as query construction of data analysis systems and self-service data exploration by non-technical users, and other application scenarios.

[0053] As described in the background, traditional data query methods often rely on technicians manually writing query statements or using fixed query templates to access data. However, this approach suffers from low access efficiency. For non-technical personnel, self-service database data exploration remains a high barrier to entry due to their lack of necessary technical knowledge, resulting in low database utilization efficiency. Therefore, to bridge the gap between human language and database query languages, facilitating database queries without the need for manual query writing, technologies that convert text to query statements have emerged. For example, Text-to-SQL (Text2SQL) technology converts text into SQL (Structured Query Language) statements. Although technologies that utilize large language models for text-to-query conversion have emerged in recent years, these technologies often require large amounts of training data and expensive computing resources for model fine-tuning, making their implementation and maintenance costs very high.

[0054] In order to overcome the limitations of the above-mentioned scheme, the inventors proposed a new solution, which can achieve efficient conversion of text into query statements without additional fine-tuning of the existing large model. It can also enable non-technical personnel to easily realize self-service data exploration of the database, thereby not only reducing the computational cost of converting text to query statements, but also lowering the technical threshold, and improving data utilization and query efficiency.

[0055] The technical solution of the embodiment of the present application includes: obtaining a target query text; recalling at least one sample pair from a sample collection, the sample pair comprising the sample query text and a sample query statement corresponding to the sample query text; determining database description information that matches the target query text; and directly generating a target query statement corresponding to the target query text using an existing large model based on the database description information, the at least one sample pair, and the target query text. This process, by obtaining the target query text, accurately captures the query requirements, providing a foundation for subsequent query statement construction. By recalling at least one sample pair from the sample set, the existing sample pairs can be used as context information to guide the generation of target query statements, thereby increasing diversity and coverage while ensuring relevance, making the recall results more comprehensive, ensuring the quality of the recalled sample pairs while taking into account the recall efficiency; by determining the database description information that matches the target query text, it is ensured that the large model can understand the database structure to improve the accuracy of the generated target query statement; based on the database description information, at least one sample pair, and the target query text, the existing large model is directly used to generate the target query statement corresponding to the target query text. By combining the sample pairs, the target query text and the database description information, the large model is used to generate accurate and effective query statements. This not only fully utilizes the powerful generation capabilities of the large model and ensures that the generated query statements are both accurate and efficient, but more importantly, this method avoids the expensive fine-tuning process of the large model and greatly reduces the computational cost and technical threshold.

[0056] The technical solution of the embodiment of the present application not only simplifies the process of converting text into query statements and lowers the technical threshold, so that users without database expertise can easily obtain complex and accurate query statements, so as to subsequently obtain corresponding query results based on the query statements, but also significantly improves the flexibility and accuracy of the query process and enhances the user experience.

[0057] It should be noted that the embodiments of this application may involve the use of user data. In actual applications, user-specific personal data can be used in the scheme described herein within the scope permitted by applicable laws and regulations, subject to the requirements of applicable laws and regulations of the country where the user is located (for example, with the user's explicit consent, effective notification to the user, etc.).

[0058] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0059] It should be noted that the technical solution of the embodiment of the present application is applicable to a network virtual environment. The users described generally refer to "virtual users". Real users can register user accounts on the server through registration to obtain user identities in the network environment.

[0060] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0061] Figure 1 This is a flowchart of an embodiment of a data processing method provided in an embodiment of the present application. In a practical application, the technical solution of the embodiment of the present application can be applied to a system architecture consisting of a user terminal and a server terminal. The user terminal can interact with the server terminal via a network to receive or send messages, etc. The technical solution of this embodiment can be executed by the server terminal.

[0062] The user end can be a browser, an app (application), a web application such as an H5 (HyperText Markup Language 5) application, a lightweight application (also known as a mini-program), or a cloud application. The user end can be deployed in electronic devices that support functions such as information browsing, such as personal mobile terminals (such as mobile phones and tablets), personal computers, desktop computers, smart speakers, and smart watches.

[0063] The server side can include servers that provide various services, such as query generation and query execution. For example, after receiving a target query from a client, the server side can invoke a large model to generate a corresponding query statement, and then invoke a database to execute the query statement and obtain the query results. Furthermore, the server side can notify the client side of the query results; alternatively, the server side can directly notify the client side of the query statement for further operation or confirmation by the user of the client side.

[0064] The server side can be implemented as a distributed server cluster consisting of multiple servers or as a single server. The server can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with integrated artificial intelligence technology.

[0065] Figure 1 The method of the illustrated embodiment may include the following steps:

[0066] 101: Get the target query text.

[0067] The target query text obtained may be a target query text submitted by the user. Obtaining the target query text submitted by the user is the starting point of the entire data processing process. Optionally, the server may obtain a user-provided query request sent by the client. The query request may include the target query text, which represents the query intent that the user wishes to express in natural language, for example, "Search for products with sales exceeding 1 million in the first quarter of this year."

[0068] In combination with the above description, it can be seen that the user end can be implemented as a browser, an application, or a web application such as an H5 application, or a light application or a cloud application, etc. Therefore, optionally, the method of submitting the target query text by the user may include but is not limited to: the query text entered by the user through a web application, an application, a light application, a cloud application or any application that can receive user input and send a request to the server side.

[0069] The user may directly provide the target query text. In addition, in order to further improve the user experience, obtaining the target query text may include: obtaining the target query voice submitted by the user, recognizing the target query voice to obtain the corresponding target query text.

[0070] Furthermore, to facilitate user-provided target query text, multiple predefined query templates can be provided to the user terminal. The user can select a template that best meets their needs from these predefined query templates and then modify it to meet their query requirements to generate a target query text. Alternatively, the server can analyze the user's previously submitted query texts, identify multiple query texts generated by the user's intent, and send the multiple query texts to the user terminal, allowing the user to select a target query text from the multiple query texts based on their needs.

[0071] Of course, in addition to obtaining the target query text provided by the user, the target query text may also be obtained by other means, which is not limited in the embodiments of the present application.

[0072] It should be noted that after receiving the target query text submitted by the user, the server can perform preliminary validation on the input target query text to ensure that it is correctly formatted and contains no illegal characters. In addition to this validation function, the server can also provide a smart prompt function. This function predicts the user's intent based on the partial text entered and provides possible content options to help the user quickly determine the target query text. Furthermore, the smart prompt function can also provide clear error messages to guide the user to correct the input if the target query text entered by the user does not conform to the expected format.

[0073] As described above, obtaining the target query text can be a comprehensive process covering multiple aspects such as user interaction and data processing. This design is intended to ensure that users can submit query requests conveniently and prepare for the subsequent acquisition of the query statement corresponding to the target query text.

[0074] 102: Recall at least one sample pair from the sample set.

[0075] The sample set includes multiple sample pairs, each of which includes a sample query text and a sample query statement corresponding to the sample query text. The sample pairs can serve as examples of input and output for the large model, allowing the large model to learn the conversion rules between the sample query texts and the sample query statements corresponding to the sample query texts in the recalled sample pairs, so as to subsequently more accurately generate the target query statement corresponding to the target query text. The query statement is implemented using a database query language, such as an SQL statement, and can refer to a specific instruction or code snippet constructed using a query statement to obtain specific information from a database.

[0076] Optionally, the sample set may be constructed by professional technicians pre-writing representative sample query texts and their corresponding sample query statements, etc., which is not limited in this application.

[0077] For example, a database containing book information can have professional technicians pre-compile representative sample pairs to construct a sample set. This sample set contains the query texts that users may ask about the database and the query statements corresponding to these query texts. The following is an example of multiple sample pairs that this sample set may contain:

[0078] Sample pair 1:

[0079] Sample query text: "Find all books published in 2023";

[0080] Sample query: SELECT * FROM books WHERE publication_year = 2023;

[0081] Sample pair 2:

[0082] Sample query text: "List all works by Jane Smith";

[0083] Sample query: SELECT * FROM books WHERE author = 'Jane Smith';

[0084] Sample pair 3:

[0085] Sample query text: "Show novels published within the last year";

[0086] Sample query: SELECT * FROM books WHERE publication_year >= YEAR(CURRENT_DATE) - 1 AND genre = 'Novel'.

[0087] After the target query text is obtained, at least one sample pair may be recalled from the sample set. The recall may be implemented in a variety of ways, which will be described in detail in the following embodiments.

[0088] 103: Determine database description information that matches the target query text.

[0089] In this step, the database description information that matches the target query text can be determined so that an accurate target query statement can be generated subsequently.

[0090] Among them, the database description information may include table description information of the data tables in the database and field description information of the fields in the data tables, etc. The table description information may include, for example, the table name, included fields, foreign keys, primary keys, table comment information, table relationship information, etc. In addition, it may also include index information, views and / or constraints and other information.

[0091] Among them, table relationship information can refer to the relationship between a data table and other data tables stored in the database; table relationship information can include, for example, table association relationships, table lineage relationships, field lineage relationships, lineage path information, etc. Table association relationships refer to the connection between different data tables, such as the association between a record in one data table and a record in another data table. Association relationships are usually established by associating a foreign key in one data table with a primary key in another data table. For example, there is a customer ID (identification) field in the order table, which references the primary key ID in the customer table; table lineage relationships refer to lineage relationships at the table granularity, indicating the source data table and target data table involved in the data flow process; field lineage relationships refer to lineage relationships at the field granularity, indicating the fields in the source data table and the fields in the target data table involved in the data flow process; lineage path information can refer to the data tables involved in the data flow process, as well as the flow order, path depth, etc. In addition, table relationship information may also include inclusion relationships, transformation relationships, dependency relationships, aggregation relationships and / or derivation relationships of tables or fields. An inclusion relationship may refer to a field or record in a data table being part of a field or record in another data table. For example, an order table contains information about an order details table, and each record in the order details table corresponds to a specific order. A transformation relationship may refer to the change that occurs in the process of data moving from one data table or field to another. A dependency relationship describes a situation where a field or data table depends on another field or data table. This dependency may be logical or formed during data processing. An aggregation relationship describes how to aggregate data from multiple data tables into one data table. For example, a sales overview table may aggregate data from multiple sales details tables. A derivation relationship refers to the calculation of data in a field or data table based on data in other fields or data tables.

[0092] In an optional manner, the database description information may adopt the database Scheme (data model information), or of course, may also be some specified type of description data collected or generated based on data tables and field data in the database.

[0093] Database description information can help the large model understand the data structure, clarify the data table where the field is located and its data type, reduce ambiguity, and process complex queries and construct correct query statements based on the association and blood relationship between tables.

[0094] Among them, all database description information can be used as database description information matching the target query text; of course, the database description information matching the target query text can also be determined based on intent recognition in the target query text, which will be introduced in detail in the following embodiments.

[0095] 104: Generate a target query statement corresponding to the target query text using the large model according to the database description information, the at least one sample pair, and the target query text.

[0096] A large model refers to a machine learning model with a large number of parameters and a complex structure. It can process massive amounts of data and complete various complex tasks, such as natural language processing, computer vision, and speech recognition. It is an AI (Artificial Intelligence) model. For example, a large model can be implemented using a large language model (LLM) or a multimodal large model (MLM). For example, models such as GPT-3 (Generative Pre-Trained Transformer -3, a third-generation generative pre-trained model), GPT-4 (Generative Pre-Trained Transformer-4, a fourth-generation generative pre-trained model), BERT (Bidirectional Encoder Representation from Transformers, a bidirectional encoder model based on Transformers), and Turing NLG (Turing Natural Language Generation) can be used. This application does not limit this. Such large models can perform well in various natural language processing tasks, such as text generation, text translation, and question-answering systems. Of course, in addition to large language models, there are many other types of large models.

[0097] Optionally, based on the database description information, at least one sample pair, and the target query text, using the big model to generate a target query statement corresponding to the target query text may include: generating prompt information according to a prompt template based on the database description information, the target query text and at least one sample pair; inputting the prompt information into the big model to use the big model to perform context learning from at least one sample pair, and generate a query statement corresponding to the target query text based on the database description information.

[0098] Prompts are a form of input used to prompt or guide the model to produce expected output. Prompts are natural language inputs, similar to commands or instructions, that let the model know what to do.

[0099] In an embodiment of the present application, in order to facilitate the generation of prompt information, a prompt template can be pre-configured, and database description information, at least one sample pair, and target query text can be added to the prompt template to generate corresponding prompt information.

[0100] Among them, the prompt template usually contains the following parts:

[0101] Database description information: presented in an easy-to-understand format, for example, it can be the example of “database description information” provided in the above embodiment.

[0102] Sample pair: This example shows how to convert natural language text into a query statement. For example, a sample pair might be: sample query text: "Find all books published in 2023" and sample query statement: SELECT * FROM books WHERE publication_year = 2023.

[0103] Target query text: The query question asked by the user. For example, the query question is: "What is the sales volume of A in 2023?"

[0104] Furthermore, after the prompt information is input into the big model, the big model learns how to convert natural language text into query statements based on the provided sample pairs, and then generates a target query statement based on the provided prompt information, and the query statement can extract the specific information required by the user from the database.

[0105] For example, the large model generates the following target query statement: SELECT product, year, quantity FROMsales WHERE product = 'A' AND year = 2023;

[0106] In this embodiment, by learning sample pairs, the large model can accurately capture user intent and generate correct query statements. By learning database description information, the large model can generate appropriate query statements based on different database structures to ensure that the generated query statements can correctly access the database.

[0107] In the technical solution provided in the embodiment of the present application, by determining the database description information that matches the target query text, it is ensured that the big model can understand the database structure to improve the accuracy of the generated target query statement; based on the database description information, at least one sample pair, and the target query text, the existing big model is directly used to generate the target query statement corresponding to the target query text. By combining the sample pairs, the target query text and the database description information, the big model is used to generate accurate and effective query statements. This not only makes full use of the powerful generation capabilities of the big model, ensuring that the generated query statements are both accurate and efficient, but more importantly, this method avoids the expensive fine-tuning process of the big model, greatly reducing the computing cost and technical threshold.

[0108] In some possible implementations, recalling at least one sample pair from a sample set may be performed by randomly selecting at least one sample pair from the sample set. Furthermore, to improve accuracy, some other possible implementations are provided below to facilitate understanding of how to recall at least one sample pair from a sample set.

[0109] In some embodiments, recalling at least one sample pair from the sample set may include: recalling at least one sample pair from the sample set that meets a relevance requirement with the target query text.

[0110] The following possible implementation methods may be included for "recalling at least one sample pair that meets the relevance requirement with the target query text":

[0111] As a possible implementation method, at least one sample pair whose similarity with the target query text satisfies a similarity condition may be recalled from the sample collection.

[0112] The similarity condition may be, for example, selecting a certain number of sample pairs in descending order of similarity; or the similarity condition may be that the similarity is greater than a similarity threshold, etc.

[0113] In practical applications, for example, the K-Nearest Neighbors (KNN) algorithm can be used to recall K sample pairs from a sample set that meet similarity requirements with the target query text.

[0114] Among them, KNN is a distance-based similarity measurement method used to find a certain number of sample pairs that are most similar to a given target query text.

[0115] In this implementation, the sample query text and its corresponding sample query statement can first be converted into vector form, typically using word embedding (such as Word2Vec, BERT, etc.) or other text representation techniques. Secondly, the target query text can also be converted into vector form. By calculating the similarity between the vector corresponding to the target query text and the vector corresponding to each sample query text in the sample set (for example, cosine similarity, Euclidean distance, etc.), the sample pair corresponding to at least one sample query text that meets the similarity condition with the target query text is selected as the recall result.

[0116] As another possible implementation method, the sample set may be clustered to obtain multiple sample combinations; and at least one sample pair whose similarity to the target query text meets a similarity condition is recalled from each of the multiple sample combinations.

[0117] The sample set can be clustered using a clustering algorithm. Clustering algorithms, such as K-means clustering (K-means), density-based spatial clustering of applications with noise (DBSCAN), and hierarchical clustering (HC), can be used to divide the sample set into multiple clusters, so that multiple sample pairs within the same cluster have a high similarity, while sample pairs between different clusters have a low similarity.

[0118] The embodiment of the present application is explained by taking the clustering algorithm including the K-means clustering algorithm as an example. First, the number of clusters k needs to be determined in advance. That is, in this implementation method, first, the sample set is divided into k clusters by the clustering algorithm, so that the sample pairs in each cluster have a high similarity. Secondly, for each cluster, the similarity between the sample query text and the target query text in all sample pairs in the cluster is calculated in turn. If the similarity between a sample pair in the cluster and the target query text is higher than the similarity threshold, the sample pair is recalled or a predetermined number of sample pairs are selected for recall in descending order of similarity. The final recall result includes the sample pairs in all clusters whose similarity with the target query text meets the similarity condition.

[0119] The clustering algorithm can be used to divide the sample set into multiple sample sets, and sample pairs can be recalled from each sample set to ensure the diversity of sample pairs. In addition, each sample set can be recalled according to similarity to ensure the relevance of sample pairs.

[0120] As another possible implementation method, in order to further ensure that at least one sample pair takes into account both relevance and diversity, unrecalled sample pairs that are relevant to the target query text and irrelevant to the recalled sample pairs can be recalled from the sample set to obtain multiple sample pairs.

[0121] Among them, relevance can be measured by similarity, and irrelevance can be measured by difference. Therefore, as an optional method, multiple sample pairs can be obtained as follows: from the sample set, unrecalled sample pairs that are relevant to the target query text and irrelevant to the recalled sample pairs are recalled to obtain multiple sample pairs, which can include: from the sample set, unrecalled sample pairs whose similarity to the target query text meets the similarity condition and whose difference from the recalled sample pairs meets the difference condition, to obtain multiple sample pairs.

[0122] This process aims to ensure that the recalled sample pairs are both relevant to the target query text and different from the recalled sample pairs, so as to improve the diversity and relevance of the recalled sample pairs.

[0123] In this implementation, similarity conditions are used to ensure that the recalled sample pairs are sufficiently similar to the target query text, and difference conditions are used to ensure that the newly recalled sample pairs are sufficiently different from the previously recalled sample pairs.

[0124] Similarity can be achieved, for example, by calculating cosine similarity or Euclidean distance. The similarity condition can be, for example, the maximum similarity, and the difference can be obtained, for example, from the similarity calculation result. For example, the similarity is cosine similarity s, and the difference can be 1-s, etc. For example, the difference can also be achieved through Euclidean distance. The larger the Euclidean distance, the greater the difference. The difference condition can be, for example, the maximum difference.

[0125] As another alternative, multiple sample pairs can be obtained as follows:

[0126] For each unrecalled sample pair in the sample set, calculate the first similarity between the target query text and the sample query text of the sample pair, as well as the maximum second similarity with the sample query texts of the recalled sample pairs. Find the sample pair that maximizes the first similarity and minimizes the maximum second similarity for recall. This step can be repeated until a certain number of sample pairs are obtained.

[0127] As another alternative, multiple sample pairs may be obtained as follows:

[0128] For each unrecalled sample pair in the sample set, calculate the first similarity between the target query text and the sample query text of the sample pair, as well as the maximum second similarity with the sample query texts of the recalled sample pairs. Calculate the difference between the first similarity and the maximum second similarity, and select the sample pair with the largest similarity difference for recall. This step can be repeated until a certain number of sample pairs are obtained or the similarity difference falls below a certain threshold.

[0129] The following example illustrates the options listed above:

[0130] Assume that the data is obtained as follows:

[0131] Target query text Q: "Query the salaries of all employees";

[0132] The sample set R contains the following sample pairs:

[0133] D1: "Query the names and positions of all employees" and the corresponding query statement;

[0134] D2: "Query the salaries and bonuses of all employees" and the corresponding query statement;

[0135] D3: "Query the department numbers of all employees" and the corresponding query statement;

[0136] Assume that the sample pair D1 has been recalled, and the first similarity between the target query text Q and the sample query text in the recalled sample pair D1, the first similarity between the target query text Q and the sample query text in the unrecalled sample pair (D2, D3), and the second similarity between the sample query text in the unrecalled sample pair (D2, D3) and the sample query text in the recalled sample pair D1 are calculated as follows:

[0137] (D1,Q)=0.5, indicating that the first similarity between the recalled sample pair D1 and the target query Q is 0.5;

[0138] (D2,Q)=0.8, indicating that the first similarity between the unrecalled sample pair D2 and the target query Q is 0.8;

[0139] (D3,Q)=0.2, indicating that the first similarity between the unrecalled sample pair D3 and the target query Q is 0.2;

[0140] (D2, D1) = 0.4, indicating that the second similarity between the unrecalled sample pair D2 and the recalled sample pair D1 is 0.4;

[0141] (D3, D1) = 0.3, indicating that the second similarity between the unrecalled sample pair D3 and the recalled sample pair D1 is 0.3;

[0142] In the above-mentioned optional scheme of "finding the sample pair with the maximum first similarity and the minimum maximum second similarity for recall", it is necessary to find the sample pair with the maximum first similarity and the minimum maximum second similarity among the unrecalled sample pairs (D2, D3). From the above data, it can be determined that the first similarity between the unrecalled sample pair D2 and the target query Q is 0.8, and the second similarity between the unrecalled sample pair D2 and the recalled sample pair D1 is 0.4 (it should be noted that if the recalled sample pairs are not limited to D1, it is also necessary to calculate the second similarities between other recalled sample pairs and the unrecalled sample pair D2, so as to determine the maximum second similarity between the unrecalled sample pair D2 and the recalled sample pairs); the first similarity between the unrecalled sample pair D3 and the target query Q is 0.2, and the second similarity between the unrecalled sample pair D3 and the recalled sample pair D1 is 0.3.

[0143] By comparing the first similarity data between the unrecalled sample pair D2 and the target query Q with the first similarity data between the unrecalled sample pair D3 and the target query Q, it can be determined that the first similarity of the unrecalled sample pair D2 is much greater than the first similarity of the unrecalled sample pair D3, while its maximum second similarity is only slightly smaller than the maximum second similarity of the unrecalled sample pair D3. Therefore, it can be determined that the next recalled sample pair is D2.

[0144] In the above optional solution of "calculating the similarity difference between the first similarity and the largest second similarity, and selecting the sample pair with the largest similarity difference for recall", it can be determined from the above data that for the unrecalled sample pair D2, its first similarity with the target query Q is 0.8, and its second similarity with the recalled sample pair D1 is 0.4; for the unrecalled sample pair D3, its first similarity with the target query Q is 0.2, and its second similarity with the recalled sample pair D1 is 0.3. Therefore, for each unrecalled sample pair (D2, D3), the corresponding similarity difference is calculated as follows:

[0145] Similarity difference (D2) = 0.8 − 0.4 = 0.4;

[0146] Similarity difference (D3) = 0.2 − 0.3 = − 0.1;

[0147] Therefore, the next recalled sample pair can be determined to be D2.

[0148] Optionally, an embodiment of the present application can use the following recall formula to implement, for any unrecalled sample pair in the sample set, calculation of the first similarity between the target query text and the sample query text in the unrecalled sample pair, and the maximum second similarity with the sample query text in the recalled sample pair, and subtracting the maximum second similarity from the first similarity to obtain the similarity difference and recall the unrecalled sample pair with the largest similarity difference.

[0149] The recall formula can be specifically as follows:

[0150] ;

[0151] Where Di represents the i-th sample pair that has not been recalled, Q represents the target query text, R represents the sample set of the unrecalled sample pairs, which can be a sample set whose similarity with the target query text meets the similarity condition, and S represents the sample set of the recalled sample pairs. It is represented as the similarity score between Di and Q, that is, the first similarity, and Dj is represented as the jth sample pair recalled in S. It is expressed as the similarity score between Di and the jth sample pair recalled in S, that is, the second similarity, Expressed as a weight parameter to balance similarity and differences The contribution between.

[0152] The purpose of this formula is to select a sample pair Di from the sample set R of unrecalled sample pairs so that the similarity between the sample pair Di and the target query text Q is as high as possible, while the similarity with the sample set S of recalled sample pairs is as low as possible.

[0153] In the above formula, Expressed as the first similarity between Di and the target query text Q, multiplied by the weight factor ; It is expressed as the maximum second similarity between Di and all sample pairs in the sample set S of the recalled sample pairs, multiplied by the weight factor (1- ). It is expressed as the similarity difference between the first similarity between Di and the target query text Q and the maximum second similarity between Di and all sample pairs in the sample set S of the recalled sample pairs.

[0154] This formula can be used to select the next sample pair from the unrecalled sample set to increase the diversity and relevance of the recall results. Specifically:

[0155] This formula is used to find the sample pair with the largest similarity difference, that is, by maximizing the first similarity , to ensure that the similarity between the newly recalled sample pairs and the target query text Q is as high as possible, while minimizing the maximum second similarity , to ensure that the similarity between the newly recalled sample pairs and the recalled sample pairs S is as low as possible, so as to ensure that the similarity between the recalled sample pairs and the target query text meets the similarity condition while ensuring the diversity and uniqueness of the recalled sample pairs.

[0156] Here's a specific example:

[0157] Assume there is a target query text Q and a sample set S of recalled sample pairs. It is hoped to select a sample pair Di from the sample set R of unrecalled sample pairs. The most appropriate Di is selected by using the above recall formula:

[0158] First, for each Di∈R: calculate ; For each Dj∈S, calculate And select the maximum value;

[0159] Second, use the weighting factor To balance similarity and differences , to select the expression The biggest Di.

[0160] In this way, we can ensure that the selected sample pairs are highly relevant to the target query text and different from the recalled sample pairs.

[0161] It should be noted that by using one or a combination of the above-mentioned possible implementation methods, diverse sample pairs relevant to the target query text can be selected from the sample collection. This ensures that the recalled sample pairs are not only relevant to the target query text but also sufficiently different from each other, thereby increasing the diversity of the sample pairs. Furthermore, as can be seen from the above detailed description, recalling sample pairs from a sample collection is a complex process involving the selection of multiple techniques (such as K-means). The choice of these techniques depends on the characteristics of the specific application scenario and can be determined based on actual needs.

[0162] In some embodiments, determining the database description information that matches the target query text includes: identifying entities and attributes involved in the target query text; and searching for the matched database description information based on the entities and attributes.

[0163] Among them, the method of identifying the entities and attributes involved in the target query text can use Natural Language Processing (NLP) technology, such as Named Entity Recognition (NER), dependency syntax analysis, etc., to identify the entities and attributes in the target query text. In an embodiment of the present application, an entity may refer to a key object mentioned in the target query text. For example, in the target query text "What is the sales volume of A in 23 years", "A" is an entity, where A may refer to a product of a certain brand. An entity can generally correspond to a record in a database table or the data table itself. For example, in a book database, "book", "author" and "publisher" can all be regarded as entities.

[0164] Attributes, on the other hand, can refer to characteristics associated with an entity. For example, in the target query "What was the sales volume of item A in 2023?", "Time (2023)" and "Sales Volume" are attributes associated with entity "A." Attributes typically correspond to columns, or fields, in a database table.

[0165] Furthermore, after identifying the entities and attributes involved in the target query text, the hit database description information can be queried based on the entities and attributes.

[0166] This can be done by associating entities with database table names and attributes with database field names. For example, suppose there's a database table named "A" that contains A's sales data, including columns like inventory quantity and time, and fields like "year" and "quantity." Entity "A" can be associated with the "A_clothing_sales" table, and the attributes "time" and "sales volume" can be associated with the "year" and "quantity" fields in the "A" table. This allows you to determine the matching table description based on the table name and the matching field description based on the field name. The matching table description and field description can then serve as the database description for the hit.

[0167] Here's a specific example:

[0168] Assume that the input target query text is: "What is the sales volume of A in 2023?" After identifying the table name "A_clothing_sales" and the attributes "year" and "quantity" fields involved in the target query text, the matching table description information is determined based on the table name and the matching field description information is determined based on the field name, resulting in the following database description information:

[0169] <Table name: "A_clothing_sales" [Product sales details]

[0170] Fields: 'year' [year] [int], 'product_name' [product name] [varchar(50)], 'quantity' [sales] [int], 'price' [price] [float], 'color' [color] [varchar(50)], 'size' [size] [varchar(10)]

[0171] Primary Key: 'year' and 'product_name'

[0172] Foreign key: 'quantity' references the product inventory table

[0173] Note: 'year' indicates the year, 'product_name' indicates the product name, 'quantity' indicates the sales volume in that year, 'price' indicates the price of each product, 'color' indicates the color of the product, and 'size' indicates the size of the product.

[0174] Relationship: 'year' and 'product_name' form a composite primary key, 'quantity' is a foreign key that references the inventory table in another table

[0175] Field description information:

[0176] 'year': year [int], indicating the year

[0177] 'product_name': Product name [varchar(50)], indicating the product name

[0178] 'quantity': sales volume [int], indicating the sales volume for the year

[0179] 'price': price [float], indicating the price of each item

[0180] 'color': color [varchar(50)], indicating the color of the product

[0181] 'size': size [varchar(10)], indicating the size of the product

[0182] Table description information:

[0183] A_clothing_sales: A product sales details table, including year, product name, sales volume, price, color, size, etc.

[0184] Primary key: Consists of year and product name

[0185] Foreign key: Sales volume references product inventory table

[0186] Bloodline relationship: Year and product name together constitute the primary key, and sales volume serves as a foreign key to reference the product inventory table. By obtaining the aforementioned database description information and generating a query statement based on the database description information, sample pairs, and target query objects, it is ensured that the generated query statement not only complies with database requirements but also meets the need for accurate execution of the target query text.

[0187] It's important to note that in addition to identifying entities and attributes in the target query text, we can also further identify operations (such as sorting and filtering) in the target query text. Operations are actions performed on entities and attributes, typically to obtain specific information or results. Operations can correspond to database operations such as SELECT, WHERE, ORDER BY, and GROUP BY SQL commands. The large model can generate the corresponding target query statement based on the operations in the target query text.

[0188] The following is a specific example of a complete data processing process:

[0189] When the target query text "What is the sales volume of product A in 2023?" is input, at least one sample pair is recalled from the sample set, and after the database description information that matches the target query text is determined, the following prompt information is generated based on the database description information, the at least one sample pair, and the target query text:

[0190] Given the following sample pairs and database description information, please use the sample pairs to generate the SQL statement corresponding to the query text.

[0191] Sample query text: "What is the sales volume of product A in each quarter in 2023?"

[0192] Sample query: SELECT EXTRACT(QUARTER FROM sale_date) AS quarter, SUM(quantity) AS total_sales

[0193] FROM sales

[0194] WHERE YEAR(sale_date) = 2023 AND product_code = 'A'

[0195] GROUP BY quarter;

[0196] Sample query text: "What is the average sales volume of product A in each month in 2023?"

[0197] Sample query: SELECT EXTRACT(MONTH FROM sale_date) AS month, AVG(quantity) AS avg_sales

[0198] FROM sales

[0199] WHERE YEAR(sale_date) = 2023 AND product_code = 'A'

[0200] GROUP BY month;

[0201] Sample query text: "By what percentage did the sales volume of product A increase in 2023 compared to 2022?"

[0202] Sample query statement: SELECT

[0203] ((SUM(sales_2023.quantity) - SUM(sales_2022.quantity)) / SUM(sales_2022.quantity)) * 100 AS growth_percentage

[0204] FROM (

[0205] SELECT SUM(quantity) AS quantity

[0206] FROM sales

[0207] WHERE YEAR(sale_date) = 2023 AND product_code = 'A'

[0208] ) AS sales_2023, (

[0210] SELECT SUM(quantity) AS quantity

[0211] FROM sales

[0212] WHERE YEAR(sale_date) = 2022 AND product_code = 'A'

[0213] ) AS sales_2022;

[0214] Database description information:

[0215] <Table name: "A_clothing_sales" [Product sales details]

[0216] Fields: 'year' [year] [int], 'product_name' [product name] [varchar(50)], 'quantity' [sales] [int], 'price' [price] [float], 'color' [color] [varchar(50)], 'size' [size] [varchar(10)]

[0217] Primary Key: 'year' and 'product_name'

[0218] Foreign key: 'quantity' references the product inventory table

[0219] Note: 'year' indicates the year, 'product_name' indicates the product name, 'quantity' indicates the sales volume in that year, 'price' indicates the price of each product, 'color' indicates the color of the product, and 'size' indicates the size of the product.

[0220] Relationship: 'year' and 'product_name' form a composite primary key, 'quantity' is a foreign key that references the inventory table in another table

[0221] Field description information:

[0222] 'year': year [int], indicating the year

[0223] 'product_name': Product name [varchar(50)], indicating the product name

[0224] 'quantity': sales volume [int], indicating the sales volume for the year

[0225] 'price': price [float], indicating the price of each item

[0226] 'color': color [varchar(50)], indicating the color of the product

[0227] 'size': size [varchar(10)], indicating the size of the product

[0228] Table description information:

[0229] A_clothing_sales: A product sales details table, including year, product name, sales volume, price, color, size, etc.

[0230] Primary key: Consists of year and product name

[0231] Foreign key: Sales volume references product inventory table

[0232] Blood relationship: Year and product name together constitute the primary key, and sales volume is used as a foreign key to reference the product inventory table.

[0233] Target query text: What is the sales volume of A in 2023?

[0234] }

[0235] After the above prompt information is input into the large model, the large model will output the following content:

[0236] "Target query statement: SELECT product, year, quantity FROM sales WHERE product= 'A' AND year = 2023;"

[0237] In some embodiments, the method may further include: adding the target query text and the target query sentence as a sample pair to the sample set.

[0238] In this embodiment, after the target query statement for the target query text is generated by the large model, the server can also add this pair of query text and query statement as a new sample pair to the sample collection. Adding new sample pairs can help the model better understand and process different query scenarios, especially when facing new database structures or query requirements.

[0239] In some embodiments, after generating the target query statement, the method further includes: executing the target query statement to obtain a query result, and notifying the user of the query result.

[0240] In this embodiment, the server not only generates target query statements for the target query text by calling the large model, but also further calls the database to execute these query statements to obtain query results and send the query results to the user on the client side.

[0241] Here is a specific example:

[0242] Suppose you are building an application that helps users query sales data.

[0243] An example may include the following specific steps:

[0244] Get target query text: The user submitted a target query text "What is the sales volume of Apple in the second quarter of 2023?"

[0245] Generate a target query statement: The server generates a target query statement corresponding to the target query text based on the filtered sample pairs (for example, "first quarter pear sales" corresponds to "SELECT * FROM sales WHERE product='pear' AND quarter=1 AND year=2023"), database description information, and target query text: SELECT * FROM sales WHERE product='apple' AND quarter=2 AND year=2023;

[0246] Execute the target query statement: The server chooses to execute this query statement. Specifically, the server connects to the database containing sales data and executes the above query statement. After the query is completed, the server obtains the specific sales data of Apple in the second quarter of 2023 from the database and sends the specific sales data as the query result (for example, "The sales volume of Apple in the second quarter of 2023 is 10,000"). Send it to the user.

[0247] In some embodiments, after generating the target query statement, the method further includes: notifying the user of the target query statement.

[0248] In this embodiment, if the query instruction is not executed, the server may also directly send the generated query statement to the client, so that the user can decide whether to execute it.

[0249] Here is a specific example:

[0250] If the user only needs to view the query statement and does not want to execute the query immediately, the specific steps include the following:

[0251] Get target query text: The user submitted a target query text "What is the sales volume of Apple in the second quarter of 2023?"

[0252] Generate a target query statement: The server generates a target query statement corresponding to the target query text based on the filtered sample pairs (for example, "first quarter pear sales" corresponds to "SELECT * FROM sales WHERE product='pear' AND quarter=1 AND year=2023"), database description information, and target query text: SELECT * FROM sales WHERE product='apple' AND quarter=2 AND year=2023;

[0253] Feedback target query statement: The server sends the generated query statement to the user for confirmation or the user can choose to execute the query statement in a suitable environment.

[0254] These two implementations provide flexibility, allowing users to choose whether to execute queries immediately based on their needs. The first method is suitable for scenarios where fast results are required, while the second method is more suitable for situations where users prefer to control query execution. In this way, the server can better meet the needs of different users.

[0255] Figure 2 This is an interactive diagram of a data processing scenario provided in an actual application of an embodiment of the present application, such as Figure 2As shown in the figure, client 1 and server 2 implement accurate database query for the "target query text" through a series of interactions. The following is a detailed description of the specific interaction process:

[0256] Step 21: The user enters a target query text through the user terminal, for example, "Query the sales of all products in the second quarter of 2023."

[0257] Step 22: Server 2 obtains the target query text sent by client 1. Client 1 sends the target query text to server 2.

[0258] Step 23: The server 2 recalls at least one sample pair from the sample set.

[0259] The server 2 recalls the unrecalled sample pairs that are relevant to the target query text and irrelevant to the recalled sample pairs from the sample set through the Maximum Marginal Relevance (MMR) algorithm to obtain multiple sample pairs, where each sample pair includes a sample object sample query text and its corresponding database query statement sample query statement.

[0260] Step 24: Server 2 determines the database description information that matches the target query text.

[0261] Server 2 analyzes the target query text and determines the matching database description information, including table names, field names, etc., so that the correct database query statement can be generated later.

[0262] Step 25: The server 2 calls the large model to generate a target query statement corresponding to the target query text according to the database description information, at least one sample pair, and the target query text.

[0263] The server 2 uses the above information to call the pre-trained large model, which can generate corresponding database query statements based on the input target query text, sample pairs and database description information.

[0264] The generated target query statement should be able to accurately express the user's query intention and be executable in the actual database.

[0265] Step 26: Server 2 calls the database to execute the target query statement to obtain the query result.

[0266] Step 27: The server 2 notifies the query result to the user 1. Optionally, the target query statement may also be notified to the user 1, which is not limited in this application.

[0267] In this step, the server can choose to directly execute the generated target query statement and obtain the query results from the database. Alternatively, the server can choose not to execute the query and instead send the generated target query statement to the client, allowing the user to decide whether to execute the query. If the query is executed, the server sends the query results back to the client, which then displays the query results to the user. If the query is not executed, the server sends the generated target query statement to the client, which can then display it to the user, who can then decide whether to execute the query as needed.

[0268] in, Figure 3 This is a diagram of the interaction between server 2 and steps 22 to 26. Figure 3 As shown, the server receives the target query text "Question" sent by client 1 (i.e., step 22). Based on the target query text "Question", it retrieves sample pairs "Reference" that meet similar requirements from the sample collection "SQL Examples" to serve as reference templates for subsequently constructing the target query statement (i.e., step 23). The server determines the database description information that matches the target query text from the database schema "Schema" (i.e., step 24). Based on the database description information, at least one sample pair, and the target query text, it generates a prompt message "Prompt" (i.e., step 25). Finally, the server inputs the prompt message "Prompt" into the large model "LLMGenerate SQL" to generate the target query statement "SQL" corresponding to the target query text through the large model (i.e., step 26).

[0269] In an embodiment of the present application, by determining the database description information that matches the target query text, it is ensured that the big model can understand the database structure to improve the accuracy of the generated target query statement; based on the database description information, at least one sample pair, and the target query text, the existing big model is directly used to generate the target query statement corresponding to the target query text. By combining the sample pairs, the target query text and the database description information, the big model is used to generate accurate and effective query statements. This not only makes full use of the powerful generation capabilities of the big model, ensuring that the generated query statements are both accurate and efficient, but more importantly, this method avoids the expensive fine-tuning process of the big model, greatly reducing the computing cost and technical threshold.

[0270] Figure 4 This is a structural diagram of an embodiment of a data processing device provided by this application, such as Figure 4 As shown, the device includes:

[0271] An acquisition module 41 is used to acquire a target query text;

[0272] A recall module 42 is configured to recall at least one sample pair from a sample set, the sample pair comprising a sample query text and a sample query statement corresponding to the sample query text;

[0273] A determination module 43 is used to determine the database description information that matches the target query text;

[0274] The generating module 44 is configured to generate a target query statement corresponding to the target query text using a large model according to the database description information, the at least one sample pair, and the target query text.

[0275] Optionally, in the embodiment of the present application, the recall module 42 is specifically configured to recall at least one sample pair from the sample set that meets a relevance requirement with the target query text.

[0276] Optionally, in an embodiment of the present application, the recall module 42 is specifically configured to recall, from the sample set, unrecalled sample pairs that are relevant to the target query text and unrelated to the recalled sample pairs, to obtain a plurality of sample pairs.

[0277] Optionally, in an embodiment of the present application, the recall module 42 is specifically used to recall from the sample set unrecalled sample pairs whose similarity with the target query text meets the similarity condition and whose difference with the recalled sample pairs meets the difference condition, so as to obtain multiple sample pairs.

[0278] Optionally, in the embodiment of the present application, the recall module 42 is specifically configured to calculate, for any unrecalled sample pair in the sample set, a first similarity between the target query text and the sample query text in the unrecalled sample pair, and a maximum second similarity with the sample query text in the recalled sample pair, and subtract the maximum second similarity from the first similarity to obtain a similarity difference;

[0279] The unrecalled sample pairs with the largest similarity difference are recalled.

[0280] Optionally, in an embodiment of the present application, the recall module 42 is specifically used to calculate, for any unrecalled sample pair in the sample set, a first similarity between the target query text and the sample query text in the unrecalled sample pair, as well as a maximum second similarity with the sample query text in the recalled sample pair, and to find a sample pair for recall that maximizes the first similarity and minimizes the maximum second similarity.

[0281] Optionally, in an embodiment of the present application, the determination module 43 is specifically configured to identify entities and attributes involved in the target query text; and query the hit database description information based on the entities and the attributes.

[0282] Optionally, in an embodiment of the present application, the generation module 44 is specifically used to generate prompt information according to a prompt template based on the database description information, the target query text and the at least one sample pair; input the prompt information into a large model to use the large model to perform context learning from the at least one sample pair, and generate a query statement corresponding to the target query text based on the database description information.

[0283] Optionally, in the embodiment of the present application, the apparatus further includes: a processing module 45;

[0284] The processing module 45 is configured to add the target query text and the target query sentence as a sample pair to the sample set.

[0285] Optionally, in an embodiment of the present application, the recall module 42 is specifically configured to cluster the sample set to obtain multiple sample combinations; and recall at least one sample pair from each of the multiple sample combinations whose similarity to the target query text meets a similarity condition.

[0286] Optionally, in the embodiment of the present application, the acquisition module 41 is specifically used to acquire the target query text submitted by the user;

[0287] Optionally, in the embodiment of the present application, the apparatus further includes: a sending module 46;

[0288] The sending module 46 is configured to execute the target query statement to obtain a query result and notify the user of the query result; or notify the user of the target query statement.

[0289] Optionally, in the embodiment of the present application, the acquisition module 41 is specifically configured to acquire a target query voice submitted by a user; and recognize the target query voice to acquire a corresponding target query text.

[0290] Figure 4 The label processing device can perform Figure 1 The implementation principle and technical effects of the label processing method described in the embodiment are not described in detail. The specific manner in which each module and unit performs operations in the label processing device in the above embodiment has been described in detail in the embodiment of the method, and will not be elaborated here.

[0291] The present application also provides a computing device, such as Figure 5 As shown, the computing device may include a storage component 51 and a processing component 52;

[0292] The storage component 51 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component to obtain a target query text; recall at least one sample pair from a sample set, wherein the sample pair includes a sample query text and a sample query statement corresponding to the sample query text; determine database description information that matches the target query text; and generate a target query statement corresponding to the target query text using a large model based on the database description information, the at least one sample pair, and the target query text.

[0293] Of course, a computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0294] The input / output interface provides an interface between the processing component 52 and peripheral interface modules, which may be output devices, input devices, etc. The communication component is configured to facilitate wired or wireless communication between the computing device and other devices.

[0295] The processing component 52 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0296] The storage component is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0297] The display component may be an electroluminescent (EL) element, a liquid crystal display or a micro display having a similar structure, or a direct retinal display or a similar laser scanning display.

[0298] It should be noted that the above computing device implements Figure 1 In the case of the data processing method shown, it can be a physical device or an elastic computing host provided by a cloud computing platform, etc. It can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device.

[0299] The present application also provides a computer-readable storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The computer-readable medium may be included in the computing device described in the above embodiment, or may exist independently without being incorporated into the computing device.

[0300] The embodiment of the present application also provides a computer program product, which includes a computer program carried on a computer-readable storage medium, and when the computer program is executed by a computer, it can achieve the above-mentioned Figure 1 In such an embodiment, the computer program may be downloaded and installed from a network, and / or installed from a removable medium. When the computer program is executed by a processor, various functions defined in the system of the present application are performed.

[0301] It should be noted that the embodiments of this application may involve the use of user data. In actual applications, user-specific personal data can be used in the scheme described herein within the scope permitted by applicable laws and regulations, subject to the requirements of applicable laws and regulations of the country where the user is located (for example, with the user's explicit consent, effective notification to the user, etc.).

[0302] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0303] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0304] Through the description of the above embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0305] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method, characterized in that: include: Get the target query text; Recalling at least one sample pair from the sample set, the sample pair comprising a sample query text and a sample query statement corresponding to the sample query text; Determining database description information that matches the target query text; Generate prompt information based on the database description information, the target query text and the at least one sample pair according to a prompt template; Inputting the prompt information into a large model, using the large model to perform context learning from the at least one sample pair, and generating a query statement corresponding to the target query text based on the database description information; The step of recalling at least one sample pair from the sample set includes: Recalling, from the sample set, unrecalled sample pairs that are relevant to the target query text and unrelated to the recalled sample pairs to obtain a plurality of sample pairs; recalling, from the sample set, unrecalled sample pairs that are relevant to the target query text and unrelated to the recalled sample pairs to obtain a plurality of sample pairs includes: For any unrecalled sample pair in the sample set, calculate the first similarity between the target query text and the sample query text in the unrecalled sample pair, and the maximum second similarity between the sample query text in the unrecalled sample pair and the sample query text in the recalled sample pair, and subtract the maximum second similarity from the first similarity to obtain a similarity difference; recall the unrecalled sample pair with the maximum similarity difference; or, For any unrecalled sample pair in the sample set, calculate the first similarity between the target query text and the sample query text in the unrecalled sample pair, and the maximum second similarity between the sample query text in the unrecalled sample pair and the sample query text in the recalled sample pair, and find the sample pair that maximizes the first similarity and minimizes the maximum second similarity for recall.

2. The method according to claim 1, characterized in that The database description information for determining the target query text match includes: Identify entities and attributes involved in the target query text; According to the entity and the attribute, the hit database description information is queried.

3. The method according to claim 1, characterized in that Also includes: The target query text and the target query sentence are taken as a sample pair and added to the sample set.

4. The method according to claim 1, wherein The recalling of at least one sample pair from the sample set comprises: Performing clustering processing on the sample set to obtain multiple sample combinations; At least one sample pair whose similarity to the target query text meets a similarity condition is recalled from the multiple sample combinations.

5. The method according to claim 1, wherein The acquiring of the target query text includes: Obtain the target query text submitted by the user; After generating the target query statement, the method further includes: Execute the target query statement to obtain a query result, and notify the user of the query result; or notify the user of the target query statement.

6. The method according to any one of claims 1 to 5, characterized in that The acquiring of the target query text includes: Obtain the target query voice submitted by the user; The target query speech is recognized to obtain a corresponding target query text.

7. A computing device, characterized in that It comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement the data processing method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by the processing component, the data processing method according to any one of claims 1 to 6 is implemented.

9. A computer program product, characterized in that The method comprises a computer program / instruction, wherein when the computer program / instruction is executed by a processing component, the data processing method according to any one of claims 1 to 6 is implemented.

Images