Data processing method, device, equipment and storage medium

Abstract

The present invention discloses a data processing method, apparatus, device, and storage medium. The method comprises: performing unsupervised automatic clustering on all training samples in a pre-constructed training sample set to obtain at least two categories of training sample subsets; and automatically performing online correction and labeling on all error samples in each of the training sample subsets based on a pre-created error sample correction model to obtain corresponding corrected samples. This embodiment performs unsupervised automatic clustering on all training samples in a pre-created training sample set to obtain at least two categories of training sample subsets, and automatically performs online correction and labeling on all error samples in each training sample subset based on a pre-created error sample correction model to obtain corresponding corrected samples, thereby achieving the effect of online automatic correction and labeling of error samples, thereby improving the correction efficiency of error samples.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of data processing, and in particular to a data processing method, apparatus, device, and storage medium. Background Art

[0002] NL2SQL converts natural language statements into corresponding SQL statements, allowing users unfamiliar with databases to quickly find the data they need. The effectiveness of the NL2SQL model depends on the comprehensiveness and completeness of the sample, and the initial sample collection has limitations.

[0003] Over time, changes in the external data environment can cause the NL2SQL model to experience a decline in question-answering accuracy. Current solutions to this problem include manually correcting erroneous samples and training and updating the model based on them; and automatically capturing positive samples through scripts, using only positive samples for model training and updating, ignoring erroneous samples. However, manually correcting erroneous samples is labor-intensive, requiring significant human and material resources, and is inefficient. Automatically capturing samples through scripts is highly efficient and can collect positive samples, but it cannot automatically analyze and label erroneous samples. Furthermore, without analyzing and correcting erroneous samples, relying solely on positive samples results in ineffective model updates. Summary of the Invention

[0004] In view of this, the present invention provides a data processing method, apparatus, device and storage medium, which achieve the effect of online automatic correction and labeling of erroneous samples.

[0005] In a first aspect, an embodiment of the present invention provides a data processing method, including:

[0006] Perform unsupervised automatic clustering on all training samples in the pre-constructed training sample set to obtain a subset of training samples with at least two categories;

[0007] Based on the pre-created error sample correction model, all error samples in each training sample subset are automatically corrected and labeled online to obtain corresponding corrected samples.

[0008] In a second aspect, an embodiment of the present invention further provides a data processing device, including:

[0009] A clustering module is used to perform unsupervised automatic clustering on all training samples in the pre-constructed training sample set to obtain training sample subsets of at least two categories;

[0010] The correction module is used to automatically perform online correction and annotation on all erroneous samples in each of the training sample subsets based on a pre-created erroneous sample correction model to obtain corresponding corrected samples.

[0011] In a third aspect, an embodiment of the present invention further provides an electronic device, the device comprising: a memory, and one or more processors;

[0012] a memory for storing one or more programs;

[0013] When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method as described in any of the above embodiments.

[0014] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data processing method as described in any of the above embodiments.

[0015] The embodiment of the present invention performs unsupervised automatic clustering on all training samples in a pre-created training sample set to obtain at least two categories of training sample subsets, and automatically performs online correction and labeling on all erroneous samples in each training sample subset based on a pre-created erroneous sample correction model to obtain corresponding corrected samples, thereby achieving the effect of online automatic correction and labeling of erroneous samples, thereby improving the correction efficiency of erroneous samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of a data processing method provided by an embodiment of the present invention;

[0017] Figure 2 is a flow chart of another data processing method provided by an embodiment of the present invention;

[0018] Figure 3 is a flow chart of another data processing method provided by an embodiment of the present invention;

[0019] Figure 4 is a flow chart of another data processing method provided by an embodiment of the present invention;

[0020] Figure 5 is a structural block diagram of a data processing device provided by an embodiment of the present invention;

[0021] Figure 6 The figure is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0023] In one embodiment, Figure 1 This is a flow chart of a data processing method provided by an embodiment of the present invention. This embodiment is applicable to the case where an error sample is automatically corrected online. This embodiment can be executed by an electronic device. The electronic device can be a terminal device with data processing capabilities such as a computer, a laptop, or an iPad. Figure 1 As shown, this embodiment includes the following steps:

[0024] S110 , performing unsupervised automatic clustering on all training samples in the pre-constructed training sample set to obtain training sample subsets of at least two categories.

[0025] The training sample set refers to the set containing all training samples; the training sample subset refers to multiple training sample subsets obtained by classifying the training samples. A training sample subset can be understood as a subset of the training sample set, and each training sample subset corresponds to a category. The classification of categories is related to the domain of the samples. For example, assuming the training samples are in the field of electricity data, the categories are classified according to the electricity data. In actual operation, the categories can be divided according to different dimensions. For example, the categories can be divided according to total electricity consumption, average electricity consumption, and electricity consumption objects. Assume that the categories include: company electricity consumption data, household electricity consumption data, store electricity consumption data, etc. Alternatively, the categories may include: daily total electricity consumption, weekly total electricity consumption, monthly total electricity consumption, annual total electricity consumption, etc. Alternatively, the categories may include: daily average electricity consumption, weekly average electricity consumption, monthly average electricity consumption, annual average electricity consumption, etc.

[0026] This means that after obtaining training samples, they are stored in a corresponding training sample set for use in training the NL2SQL model. It should be noted that clustering can be understood as dividing the training sample set into different classes (or clusters) according to specific criteria, so that the similarities between sample data within the same cluster are as large as possible, while the differences between sample data in different clusters are also as large as possible.

[0027] Unsupervised learning refers to the fact that each training sample in the training set is unlabeled, meaning that each training sample does not have a label. This can be understood as automatically clustering all training samples in the training set after obtaining them and before labeling them to obtain subsets of training samples with at least two categories.

[0028] S120 , based on a pre-created error sample correction model, automatically perform online correction and annotation on all error samples in each training sample subset to obtain corresponding correction samples.

[0029] The error sample correction model is used to correct the error sample to obtain a corrected sample. It can be understood that the corrected sample refers to the corrected sample obtained by correcting the error sample. In an embodiment, the implementation process of the error sample correction model includes: inputting the natural language query statement of the error sample into the error sample correction model, parsing and correcting the erroneous SQL statement corresponding to the natural language query statement through the error sample correction model, and outputting the corresponding correct SQL statement as the corrected sample.

[0030] It should be noted that the error sample correction model is used to construct the model input and output sample pairs. The model backbone network structure and initial weight parameters adopt the corresponding structure and parameters of pre-trained models such as end-to-end BERT, GPT-3, mBART, and are obtained by migration training.

[0031] The technical solution of this embodiment performs unsupervised automatic clustering on all training samples in a pre-created training sample set to obtain at least two categories of training sample subsets, and based on a pre-created error sample correction model, automatically performs online correction and labeling on all error samples in each training sample subset to obtain corresponding corrected samples, thereby achieving the effect of online automatic correction and labeling of error samples, thereby improving the correction efficiency of error samples.

[0032] In one embodiment, Figure 2 This is a flow chart of another data processing method provided by an embodiment of the present invention. This embodiment further illustrates the data processing process based on the above embodiment. Figure 2 As shown, the data processing method in this embodiment includes the following steps:

[0033] S210: Input all received natural language query statements into the pre-created original NL2SQL model, and output corresponding SQL statements.

[0034] It should be noted that the implementation process of the NL2SQL model includes: for natural language query statements, using word segmentation, part-of-speech tagging, entity recognition, dependency syntax, etc. to perform syntactic analysis on the sentence, and then using various methods such as rules, word vectors, language models, and deep learning to fill in the semantic slots, specifically including query fields, aggregation functions, filter conditions, grouping fields, etc., and then generating the corresponding SQL query statement based on this filled-in information. For example, assuming that the natural language query statement entered by the user is "What is the total amount of electricity sold by Company A in the past month?", the original NL2SQL model converts the natural language query statement into a corresponding SQL statement (for example, the time is the past month, the object is Company A, and the query content is: total electricity sales), and this SQL statement is a query statement.

[0035] S220: Combine each natural language query statement with the corresponding SQL statement to obtain a corresponding training sample.

[0036] In this embodiment, each natural language query statement is bound to a corresponding SQL statement to obtain a corresponding training sample. It can be understood that each training sample includes a natural language query statement and an SQL statement. In actual operation, to facilitate querying training samples, each training sample can be numbered, that is, each training sample corresponds to a sample number, and each sample number is associated with a corresponding natural language query statement and SQL statement. This allows the other two parameters to be quickly queried using one of the sample number, natural language query statement, and SQL statement.

[0037] S230: Combine all training samples to obtain a corresponding training sample set.

[0038] In the embodiment, the process of combining all training samples can be understood as the process of integrating and storing the training samples.

[0039] It can be understood that after each training sample is obtained, it is stored in the corresponding training sample set, so that the corresponding training sample can be quickly found in the training sample set. Of course, in order to easily distinguish each training sample in the training sample set, the sample number of each training sample in the training sample set is different. It can be understood that the sample number of each training sample is a unique identifier.

[0040] S240: Convert each training sample in the pre-constructed training sample set into a corresponding word vector.

[0041] Word embedding is a technique used to convert words into corresponding dense vectors. Similar words have similar word vectors. It should be noted that each training sample consists of a natural language query and a corresponding SQL statement. This means that word embedding technology is used to convert each word in the training sample into its corresponding word vector.

[0042] S250 , calling a partitioning clustering method to automatically cluster the word vectors to obtain at least two categories of training sample subsets.

[0043] In an embodiment, the process of automatically clustering word vectors by calling a partitioning clustering method includes: converting the natural language query statement input by the user into a corresponding word vector based on word2vec technology; then calling a partitioning clustering method (bi-kmeans) to cluster the word vector, that is, based on the principle of minimizing SSE (Sum of Squared Error, which represents the sum of squares of the distance between the clustered clusters and the cluster center of the cluster, the smaller the SSE, the better the clustering effect), first all data points are regarded as a cluster, then the cluster is divided into two, and then one of the clusters is selected to continue dividing, and the selection of the cluster depends on whether the division can reduce the SSE value to the greatest extent.

[0044] In the embodiment, a partitioning clustering method is used to automatically cluster word vectors, that is, all data points are divided into two, that is, training sample subsets of at least two categories are obtained. It should be noted that the similarity between training samples in the same training sample subset is as large as possible, and the difference between training samples in different training sample subsets is also as large as possible.

[0045] The technical solution of this embodiment, based on the above embodiment, automatically clusters the word vectors corresponding to all training samples through a partitioning clustering method to obtain at least two categories of training sample subsets, so that when the training samples in different training sample subsets are corrected and labeled online, the differences between different training samples can be more intuitively reflected.

[0046] In one embodiment, Figure 3 This is a flow chart of another data processing method provided by an embodiment of the present invention. This embodiment further illustrates the data processing process based on the above embodiment. Figure 3 As shown, the data processing method in this embodiment includes the following steps:

[0047] S310: Receive user evaluation feedback information for each training sample.

[0048] User feedback refers to user evaluations of the NL2SQL model's conversion results. In practice, after a user enters a natural language query, the NL2SQL model converts the query into a corresponding SQL statement, uses the SQL statement to query a pre-created database, and provides feedback to the user. This user feedback can be used to determine the NL2SQL model's conversion effectiveness.

[0049] In the embodiment, the user evaluation feedback information may be represented in a variety of ways. For example, it may be represented by a language description (i.e., a sentence of language description), by a rating (e.g., a full score of 10, and different scores are used to represent the user's evaluation), or by the number of stars (e.g., a positive review is rated 5 stars, and different numbers of stars are used to represent different user reviews). Of course, in actual operation, other representation methods may also be used to describe the user evaluation feedback information, and this is not limited to this.

[0050] S320: Determine the sample label of the corresponding training sample according to the user evaluation feedback information.

[0051] Among them, the sample label is used to characterize different training samples. In one embodiment, the sample label includes one of the following: correct sample, error sample, and unanswered sample. It can be understood that the sample label includes: answered sample and unanswered sample. At the same time, both correct samples and error samples are answered samples. Among them, correct samples refer to samples whose feedback results correspond to the natural language query statement; error samples refer to samples whose feedback results do not correspond to the natural language query statement; and unanswered samples refer to results that do not correspond to the natural language query statement.

[0052] S330: Perform unsupervised automatic clustering on all training samples in the pre-constructed training sample set to obtain training sample subsets of at least two categories.

[0053] S340: Determine the number of erroneous samples in each training sample subset.

[0054] The number of error samples refers to the number of error samples in each training sample subset. The number of error samples is less than or equal to the total number of training samples in the corresponding training sample subset. In an embodiment, after performing unsupervised automatic clustering on all training samples, at least two categories of training sample subsets are obtained, and then the number of error samples in each training sample subset is determined based on user evaluation feedback information for each training sample.

[0055] S350: Determine the proportion of error samples in the corresponding training sample subset according to the number of error samples and the total number of training samples in the corresponding training sample subset.

[0056] The error sample ratio refers to the proportion of error samples in the corresponding training sample subset. In an embodiment, the ratio of the number of error samples to the total number of training samples in the corresponding training sample subset is used as the error sample ratio in the training sample subset.

[0057] S360: Based on the pre-created error sample correction model, all error samples in each training sample subset are automatically corrected and labeled online to obtain corresponding corrected samples.

[0058] S370: Determine the number of corresponding corrected samples to be extracted based on the proportion of erroneous samples in each training sample subset.

[0059] The number of samples to be extracted refers to the number of corrected samples to be extracted from each training sample subset. It should be noted that the higher the proportion of error samples in a training sample subset, the larger the number of corrected samples to be extracted. For example, assuming that the proportion of error samples in training sample subset 1 is 20% and the proportion of error samples in training sample subset 2 is 30%, the number of corrected samples extracted from training sample subset 2 is greater than the number of corrected samples extracted from training sample subset 1.

[0060] S380: Add the extracted number of corrected samples to the new training sample set.

[0061] In an embodiment, the extracted corrected samples are added to the original training sample set to obtain a new training sample set, so as to ensure that the categories with a high proportion of erroneous samples have a high proportion in the new training sample set.

[0062] S390: Retrain the original NL2SQL model according to the new training sample set to obtain a new NL2SQL model.

[0063] In the embodiment, the original NL2SQL model is retrained based on the new training sample set to obtain a new NL2SQL model, thereby realizing self-learning of the NL2SQL model.

[0064] The technical solution of this embodiment, based on the above embodiment, constructs a new training sample set by determining the proportion of erroneous samples in the training sample subset corresponding to each category, and trains based on the new training sample set to obtain a new NL2SQL model. When the question-answering accuracy of the original NL2SQL model decreases over time, the NL2SQL model is automatically updated, thereby improving the question-answering accuracy and realizing self-learning of the model.

[0065] In one embodiment, Figure 4This is a flowchart of another data processing method provided by an embodiment of the present invention. This embodiment is a preferred embodiment, which illustrates the specific process of data processing. This embodiment creates an original NL2SQL model and deploys the original NL2SQL model for online application; online sample reflow, automatically collects question and answer samples, and classifies them according to user evaluation feedback information; for all samples, automatically clusters and stores them based on unsupervised learning; for error samples, trains an end-to-end model (i.e., an error sample correction model) to automatically correct and annotate the error samples, wherein the model input of the error sample correction model is the user's natural language query input and the error parsed SQL statement, and outputs the corrected SQL statement.

[0066] like Figure 4 As shown, the data processing method in this embodiment includes the following steps:

[0067] S410: Create an original NL2SQL model.

[0068] In an embodiment, an NL2SQL model is created in advance as an original NL2SQL model, and the original NL2SQL model is deployed in an online application.

[0069] S420: Generate a training sample set.

[0070] In an embodiment, natural language query statements input by users are collected and recorded in the background, and are input into the NL2SQL model to generate corresponding SQL statements, and the SQL statements are used as training samples.

[0071] S430: Determine sample labels corresponding to training samples according to user evaluation feedback information.

[0072] In an embodiment, the training samples are labeled (e.g., correct samples, incorrect samples, other categories, etc.) based on user evaluation feedback information (e.g., "good answer", "wrong answer", "unanswered"), and the training samples and corresponding sample labels are stored.

[0073] S440: Perform unsupervised automatic clustering on all training samples to obtain training sample subsets of at least two categories.

[0074] S450: Determine the proportion of erroneous samples in each training sample subset.

[0075] In an embodiment, the proportion of erroneous samples in each category is counted. Each training sample subset corresponds to a category, that is, the process of determining the proportion of erroneous samples in each training sample subset is the process of determining the proportion of erroneous samples in each category.

[0076] S460: Create an error sample correction model, and perform online correction and labeling on the error samples.

[0077] S470: Construct a new training sample set.

[0078] The new training sample set refers to a new NL2SQL training sample set.

[0079] S480: Retrain the original NL2SQL model based on the new training sample set to obtain a new NL2SQL model.

[0080] In this embodiment, based on the proportion of error samples in each category, corrected error samples are extracted and added to a new sample set to ensure that categories with high error rates have a high proportion in the new sample set. A new NL2SQL model is then retrained based on the new training set to achieve self-learning of the model.

[0081] The technical solution implemented in this paper constructs a new training set by correcting and labeling erroneous samples online, combining sample clustering and calculating the proportion of erroneous samples in each category, and trains a new NL2SQL model. This allows the model to be automatically updated to improve accuracy as the question-answering accuracy of the NL2SQL model decreases over time, thereby achieving self-learning of the model.

[0082] In one embodiment, Figure 5 This is a structural block diagram of a data processing device provided by an embodiment of the present invention. The device is suitable for automatically correcting error samples online. The device can be implemented by hardware / software and can be configured in a server to implement a data processing method in an embodiment of the present invention. Figure 5 As shown, the device includes: a clustering module 510 and a correction module 520.

[0083] The clustering module 510 is configured to perform unsupervised automatic clustering on all training samples in the pre-constructed training sample set to obtain at least two categories of training sample subsets;

[0084] The correction module 520 is used to automatically perform online correction and annotation on all erroneous samples in each of the training sample subsets based on a pre-created erroneous sample correction model to obtain corresponding corrected samples.

[0085] The technical solution of this embodiment performs unsupervised automatic clustering on all training samples in a pre-created training sample set to obtain at least two categories of training sample subsets, and based on a pre-created error sample correction model, automatically performs online correction and labeling on all error samples in each training sample subset to obtain corresponding corrected samples, thereby achieving the effect of online automatic correction and labeling of error samples, thereby improving the correction efficiency of error samples.

[0086] In one embodiment, the data processing apparatus further includes:

[0087] An output module, configured to input all received natural language query statements into a pre-created original NL2SQL model and output corresponding SQL statements before performing unsupervised automatic clustering on all training samples in the pre-created training sample set;

[0088] A first combining module is used to combine each of the natural language query statements with the corresponding SQL statement to obtain a corresponding training sample;

[0089] The second combination module is used to combine all training samples to obtain a corresponding training sample set.

[0090] In one embodiment, the data processing apparatus further includes:

[0091] A receiving module, configured to receive user evaluation feedback information of each training sample before performing unsupervised automatic clustering on all training samples in the pre-constructed training sample set;

[0092] The first determining module is used to determine the sample label of the corresponding training sample according to the user evaluation feedback information.

[0093] In one embodiment, the data processing apparatus further includes:

[0094] A second determination module is configured to determine the number of erroneous samples in each of the training sample subsets after performing unsupervised automatic clustering on all training samples in the pre-constructed training sample set to obtain at least two categories of training sample subsets;

[0095] The third determining module is used to determine the proportion of error samples in the corresponding training sample subset according to the number of error samples and the total number of training samples in the corresponding training sample subset.

[0096] In one embodiment, the data processing apparatus further includes:

[0097] A fourth determination module is configured to automatically perform online correction and annotation on all error samples in each of the training sample subsets based on the pre-created error sample correction model, and after obtaining corresponding corrected samples, determine the number of corresponding corrected samples to be extracted based on the proportion of error samples in each of the training sample subsets;

[0098] An adding module, configured to add the extracted number of corrected samples to a new training sample set;

[0099] The training module is used to retrain the original NL2SQL model according to the new training sample set to obtain a new NL2SQL model.

[0100] In one embodiment, the clustering module includes:

[0101] A conversion unit, used to convert each training sample in the pre-built training sample set into a corresponding word vector;

[0102] The clustering unit is used to call a partitioning clustering method to automatically cluster the word vectors to obtain training sample subsets of at least two categories.

[0103] In one embodiment, the sample label includes one of the following: a correct sample, an incorrect sample, or an unanswered sample.

[0104] The above-mentioned data processing device can execute the data processing method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the data processing method.

[0105] In one embodiment, Figure 6 1 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. The device in the embodiment of the present invention is described by taking a computer as an example. Figure 6 As shown, the computer provided by the embodiment of the present invention includes: a processor 610, a memory 620, an input device 630 and an output device 640. The processor 610 in the computer can be one or more, Figure 6 Taking a processor 610 as an example, the processor 610, memory 620, input device 630 and output device 640 in the computer can be connected via a bus or other means. Figure 6 The bus connection is taken as an example.

[0106] The memory 620 in the computer serves as a computer-readable storage medium and can be used to store one or more programs. The programs can be software programs, computer executable programs, and modules, such as program instructions / modules corresponding to the embodiments of the present invention or the provided data processing methods (for example, modules in a data processing device, including: clustering module 510 and correction module 520). The processor 610 executes the various functional applications and data processing of the computer by running the software programs, instructions, and modules stored in the memory 620, thereby implementing the data processing method in the above-mentioned method embodiment.

[0107] The memory 620 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the device, etc. In addition, the memory 620 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some examples, the memory 620 may further include a memory remotely located relative to the processor 610, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0108] The input device 630 may be used to receive digital or character information input by the user to generate key signal input related to user settings and function control of the terminal device. The output device 640 may include a display device such as a display screen.

[0109] Moreover, when one or more programs included in the above-mentioned cloud server are executed by one or more processors 610, the program performs the following operations: performing unsupervised automatic clustering on all training samples in the pre-constructed training sample set to obtain at least two categories of training sample subsets; based on the pre-created error sample correction model, automatically performing online correction and annotation on all error samples in each of the training sample subsets to obtain corresponding corrected samples.

[0110] In one embodiment, the embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data processing method provided by the embodiment of the present invention, the method comprising: performing unsupervised automatic clustering on all training samples in a pre-constructed training sample set to obtain training sample subsets of at least two categories; based on a pre-created error sample correction model, automatically performing online correction and labeling on all error samples in each of the training sample subsets to obtain corresponding corrected samples.

[0111] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0112] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0113] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0114] The computer program code for performing the operations of the present invention can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0115] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the concept of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data processing method, characterized in that: include: Perform unsupervised automatic clustering on all training samples in the pre-constructed training sample set to obtain a subset of training samples with at least two categories; Determining the number of erroneous samples in each of the training sample subsets; Determining a proportion of the error samples in the corresponding training sample subset based on the number of error samples and the total number of training samples in the corresponding training sample subset; Based on the pre-created error sample correction model, all error samples in each training sample subset are automatically corrected and labeled online to obtain corresponding corrected samples; Determine the number of corrected samples to be extracted based on the proportion of erroneous samples in each of the training sample subsets; the higher the proportion of erroneous samples in the training sample subset, the larger the number of corrected samples to be extracted; the number of samples to be extracted is the number of corrected samples to be extracted from each training sample subset; Adding the extracted number of corrected samples to a new training sample set; The original NL2SQL model is retrained according to the new training sample set to obtain a new NL2SQL model.

2. The method according to claim 1, characterized in that Before performing unsupervised automatic clustering on all training samples in the pre-constructed training sample set, the method further includes: Input all received natural language query statements into the pre-created original NL2SQL model and output the corresponding SQL statements; Combining each of the natural language query statements with the corresponding SQL statement to obtain a corresponding training sample; All training samples are combined to obtain the corresponding training sample set.

3. The method according to claim 1, characterized in that Before performing unsupervised automatic clustering on all training samples in the pre-constructed training sample set, the method further includes: Receive user evaluation feedback information for each training sample; Determine the sample label of the corresponding training sample according to the user evaluation feedback information.

4. The method according to any one of claims 1 to 3, characterized in that: The method performs unsupervised automatic clustering on all training samples in the pre-constructed training sample set to obtain at least two categories of training sample subsets, including: Convert each training sample in the pre-built training sample set into a corresponding word vector; The word vectors are automatically clustered using a partitioning clustering method to obtain training sample subsets of at least two categories.

5. The method according to claim 3, characterized in that The sample label includes one of the following: a correct sample, an incorrect sample, or an unanswered sample.

6. A data processing device, characterized in that: include: A clustering module is used to perform unsupervised automatic clustering on all training samples in the pre-constructed training sample set to obtain training sample subsets of at least two categories; A second determining module is used to determine the number of erroneous samples in each of the training sample subsets; A third determining module is configured to determine a proportion of error samples in the corresponding training sample subset based on the number of error samples and the total number of training samples in the corresponding training sample subset; A correction module is used to automatically perform online correction and annotation on all erroneous samples in each of the training sample subsets based on a pre-created erroneous sample correction model to obtain corresponding corrected samples; A fourth determination module is configured to determine the number of corrected samples to be extracted based on the proportion of erroneous samples in each of the training sample subsets; the higher the proportion of erroneous samples in the training sample subset, the larger the number of corrected samples to be extracted; the number of corrected samples to be extracted from each training sample subset; An adding module, configured to add the extracted number of corrected samples to a new training sample set; The training module is used to retrain the original NL2SQL model according to the new training sample set to obtain a new NL2SQL model.

7. An electronic device, characterized in that: include: memory, and one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the data processing method according to any one of claims 1 to 5 is implemented.

Images