SQL metadata processing method based on large model, SQL statement generation method, electronic equipment and data medium table

Abstract

The invention discloses an SQL metadata processing method based on a large model, an SQL statement generation method, electronic equipment and a data medium table, and belongs to the technical field of semantic recognition, analysis, retrieval and fusion. In the steps of the method, whether triggering operation needing metadata maintenance exists or not is detected, corresponding metadata processing is carried out according to the triggering operation, and the metadata maintenance efficiency is improved. And uploading a metadata processing result to a corresponding metadata knowledge base, calling a large language model to generate metadata knowledge points corresponding to the slices, converting the metadata knowledge points into corresponding vectors through an embedded model, and storing the vectors in a vector database. According to the method, the existing metadata in the data platform is fully utilized, the metadata maintenance efficiency is improved, the accuracy and consistency of the metadata are enhanced, and the availability of the metadata is improved, so that a developer can perform analysis and development by depending on the updated latest metadata when performing data operation.

Description

Technical Field

[0001] The present invention relates to a SQL metadata processing method, a SQL statement generating method, an electronic device and a data middle platform based on a large model, and belongs to the technical field of semantic recognition, analysis, retrieval and fusion. Background Art

[0002] Currently, when users use the data center for data development, they need to write SQL to query and analyze data, which requires users to be familiar with SQL syntax and the data table structure in the database. Users need to modify SQL statements repeatedly during the query and analysis process until they find data that meets their needs.

[0003] When users write SQL to query and analyze data, they need to observe metadata, observe the SQL execution results, and modify the SQL statements repeatedly to finally achieve the goal of querying and analyzing data, which is relatively inefficient. The existing data center already has the metadata required for writing SQL, but due to the large amount of metadata, it is not convenient for users to search according to their needs, and the metadata utilization rate in the data center is not high.

[0004] There is a lack of accumulation of commonly used SQL in the data center, and the experience of writing SQL has not been effectively used in subsequent data development work. It can no longer meet people's requirements and urgently needs to be improved. Summary of the invention

[0005] The purpose of the present invention is to provide a SQL metadata processing method, SQL statement generation method, electronic device and data middle platform based on a large model, so as to solve the shortcomings of the prior art in view of the problems that the existing metadata is difficult to maintain and analyze, and the SQL statement query is difficult and not intelligent enough.

[0006] The present invention provides the following scheme:

[0007] A SQL metadata processing method based on a large model is applied to metadata processing in the data center, including:

[0008] Obtain the metadata of the data center and detect whether there are trigger operations that require metadata maintenance. The trigger operations include: adding a new table, changing a table, or deleting a table.

[0009] If there is a trigger operation corresponding to metadata maintenance, the corresponding metadata processing is performed according to the trigger operation, and the metadata processing result is uploaded to the corresponding metadata knowledge base. The metadata processing includes: adding new table content, changing metadata in the table, or deleting table operations;

[0010] The large language model is called to generate metadata knowledge points corresponding to the slices, and the metadata knowledge points are converted into corresponding vectors through the embedded model and stored in the vector database.

[0011] Furthermore, the step of obtaining metadata of the data center and detecting whether there is a trigger operation requiring metadata maintenance further includes:

[0012] The metadata obtained from the data center includes: table structure, field attributes, indexes, and constraints. Check whether the table structure, field attributes, indexes, and constraints have trigger operations that require metadata maintenance.

[0013] Track the change history of the data center by setting triggers in the data center to detect the corresponding trigger operations, or: determine whether the trigger operation has occurred by performing log analysis on the data center.

[0014] Furthermore, if there is a trigger operation corresponding to metadata maintenance, the corresponding metadata processing is performed according to the trigger operation, and the metadata processing result is uploaded to the corresponding metadata knowledge base, and the metadata processing includes: adding new table content, changing metadata in the table, or deleting table operations, further including:

[0015] The metadata processing of the newly added table content includes: supplementing table description, supplementing field description, supplementing the relationship between tables, supplementing value space, supplementing primary key, supplementing foreign key relationship, and adding new documents;

[0016] Generate a new table content file from the new table content after metadata processing, upload the new table content to the metadata knowledge base to form a new table document, upload the entire content of the new table document as a slice to the metadata knowledge base, and / or:

[0017] In the metadata processing process of changing the metadata in the table, it includes: changing the table description, changing the field description, changing the relationship between tables, changing the value space, updating the primary key, updating the foreign key relationship, and updating the document;

[0018] Generate an update table content document from the metadata in the change table after metadata processing, upload the entire content of the update table content document as a slice to the metadata knowledge base, and / or:

[0019] The metadata processing of the table deletion operation includes: changing metadata, deleting table documents, deleting related table documents, and adding related table documents;

[0020] The delete table operation after metadata processing generates a delete table operation document, and the entire content of the delete table operation document is uploaded to the metadata knowledge base as a slice.

[0021] Furthermore, the calling of the large language model to generate metadata knowledge points corresponding to the slices, converting the metadata knowledge points into corresponding vectors through the embedded model and storing them in the vector database further includes:

[0022] Receiving a data query request, parsing the data query request using natural language processing technology, and extracting query intent and key information;

[0023] The large language model is used to understand the semantics of the data query request, and the corresponding SQL statement is constructed according to the semantic analysis results obtained from the query intent and key information, specifically:

[0024] Retrieve metadata corresponding to the data query request in the metadata knowledge base, put the metadata in the prompt word, input the prompt word containing the metadata into the large language model, and generate corresponding semantic parsing results;

[0025] According to the execution SQL statement corresponding to the semantic analysis result, the execution SQL statement is converted into natural language for description and output.

[0026] Furthermore, it also includes:

[0027] Call the large language model to generate knowledge points for adding table content, changing metadata in the table, or deleting table operations corresponding to the slice;

[0028] Using the embedded model, knowledge points of adding new table content, changing metadata in a table, or deleting table operations are converted into corresponding vectors, and the vectors are stored in a vector database.

[0029] The uploading of the entire content of the newly added table document as a slice to the metadata knowledge base is specifically as follows:

[0030] Call the large language model to generate the new table content knowledge points corresponding to the slice, and generate the triple information of the slice content;

[0031] Analyze metadata content and identify corresponding tables and fields in the metadata knowledge base;

[0032] Generate a content overview of the table and its fields. The content overview includes all the information stored in the table and all the fields of all the tables.

[0033] Construct triple information, by identifying entities, relations and objects in metadata, and construct triple information according to the structure of "subject, relation, object".

[0034] Furthermore, the use of a large language model to perform semantic understanding on data query requests is specifically as follows:

[0035] The following formula is used in the process of using a large language model to understand the semantics of data query requests:

[0036]

[0037] Where: E = Embed(Q), E represents the conversion of the data query request Q into a word vector set in a high-dimensional space, R represents the output of the encoder, A represents the output of the attention mechanism, S represents the result of the semantic similarity calculation of the query statement, g is the integration function, θ g Represents the parameters of the integration function g.

[0038] A method for generating SQL statements based on a large model, including the SQL metadata processing method, further comprising:

[0039] The data center obtains the question and submits the question to the question-answering knowledge base for retrieval, specifically: full-text retrieval and / or semantic retrieval, where:

[0040] In full-text retrieval, the questions are segmented, stop words are removed or stems are extracted. In semantic retrieval, the questions are converted into vectors through embedded models.

[0041] Determine whether there is an answer corresponding to the question in the question-answering knowledge base. If the answer corresponding to the question is retrieved, execute the data query step, otherwise execute the metadata retrieval step;

[0042] In the metadata retrieval step, the question is converted into a vector and queried in a vector database;

[0043] Generate corresponding SQL statements using a large language model based on the query results in the vector database;

[0044] According to the received user's selection of SQL statements, the corresponding SQL statements are saved in the question-answer knowledge base, and the SQL statements are vectorized and indexed.

[0045] Furthermore, the data center obtains the question and submits the question to the question-answering knowledge base for retrieval, specifically:

[0046] The question is converted into a vector through an embedded model, and the vector is queried in a vector database of a metadata knowledge base to obtain a corresponding slice knowledge point and obtain a metadata slice according to the slice knowledge point;

[0047] Generate a corresponding SQL statement using a large language model according to the metadata slice in the vector database, specifically: retrieve metadata corresponding to the question in the question-answering knowledge base, put the metadata in a prompt word, input the prompt word containing the metadata into the large language model, and use the large language model to generate a corresponding question semantic analysis result;

[0048] The step of segmenting the question, removing stop words or extracting stems during full-text retrieval, and converting the question into a vector through an embedded model during semantic retrieval further includes:

[0049] In the process of converting the intersection problem into a vector through the embedded model, the features of the SQL statement are extracted, and the features include the keywords of the SQL statement, the data table name, the field name and the operator;

[0050] The extracted features are mapped to a high-dimensional space to form a feature vector, where the feature vector is represented by V = (v 1 , v 2 , ..., ν n ), v n Represents the weight value of the nth feature;

[0051] Normalize the eigenvector, the normalization formula is: ∥V∥ represents the Euclidean norm of vector V;

[0052] The complexity of the eigenvector is calculated to determine the complexity of the eigenvector. The complexity calculation formula is C = α·log(||V′||)+β·D(V′)+γ·E(V′) where:

[0053] α represents the logarithmic weight of the normalized vector norm, β represents the diversity weight of the vector distribution, D(V′) represents the standard deviation in the feature vector V′, γ represents the proportional weight of zero-weight features in the feature vector, and E(V′) represents the proportion of zero-weight features in the feature vector V′.

[0054] An electronic device comprises: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the method.

[0055] A data middle platform, the data middle platform includes the electronic device mentioned above.

[0056] Compared with the prior art, the present invention has the following advantages:

[0057] The present invention can analyze the existing metadata, detect whether there is a corresponding trigger operation, perform the corresponding metadata processing operation according to the trigger operation, maintain and update the metadata by adding new table content, changing the metadata in the table, or deleting the table operation, and convert the metadata knowledge points into vectors and store them in the vector database for calling at any time when querying SQL statements. The present invention makes full use of the existing metadata in the data center, and improves the efficiency of metadata maintenance, enhances the accuracy and consistency of metadata, and improves the availability of metadata by performing automated and intelligent data processing on metadata, so that developers can rely on the latest updated metadata for analysis and development when performing data operations.

[0058] The present invention also provides a method for generating SQL statements for SQL queries, which enables users to ask questions using natural language and the data middle-end system to return SQL statements, thereby lowering the threshold for writing SQL statements and improving writing efficiency. Commonly used SQL statements are stored in a knowledge base, improving the effectiveness of data development work and lowering the technical threshold, so that non-professional users or users who are unfamiliar with SQL can directly ask queries using natural language without having to master the SQL language, making database queries more popular and easier to use.

[0059] The present invention improves the efficiency of writing SQL statements. Users do not need to write SQL statements manually, but can directly use automatically generated SQL statements, thereby reducing query preparation time and improving work efficiency. By storing commonly used SQL statements, developers are provided with a direct calling function, which reduces repetitive work, improves the overall effect of data development, and enhances user experience. Users can interact with the SQL database more intuitively and naturally, making the query of the SQL database more friendly and intuitive. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1 It is a flowchart of the SQL metadata processing method based on the big model.

[0062] Figure 1A It is a flow chart of the optimization technical solution from step S11 to step S15.

[0063] Figure 1B It is a flow chart of the optimization technical solution from step S21 to step S23.

[0064] Figure 1C It is a flow chart of the optimization technical solution from step S31 to step S33.

[0065] Figure 2 This is the architecture diagram of the SQL metadata processing system based on the big model.

[0066] Figure 3 It is a flowchart of the SQL statement generation method based on the large model.

[0067] Figure 4 This is the architecture diagram of the SQL statement generation system based on the big model.

[0068] Figure 5 It is a structural diagram of the components of the data center.

[0069] Figure 6 It is a structural diagram of an electronic device. DETAILED DESCRIPTION

[0070] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] like Figure 1 As shown, the SQL metadata processing method based on the big model is applied to the SQL metadata processing system based on the big model, including:

[0072] Step S1, obtaining metadata of the data center, and detecting whether there is a trigger operation that requires metadata maintenance, wherein the trigger operation includes: adding a new table, changing a table, or deleting a table;

[0073] In step S1, metadata has corresponding specific content, which can be information describing the structure, attributes, relationships and storage characteristics of metadata, including metadata providing context for data to help systems and users understand the meaning, source, structure and purpose of data, such as table structure metadata, data relationship metadata, metadata dictionary parameters, data quality metadata, data security and compliance metadata, etc. Exemplarily, in the specific content of the above metadata, table structure metadata includes table name, field information, key information, index information, data relationship metadata includes relationship between tables and view definition, view dependency table, metadata dictionary parameters include business terms and parameter data, data quality metadata includes data validity rules and data quality assessment, data security and compliance metadata includes access control metadata and data sensitivity tags. In step S1, by comprehensively acquiring and analyzing the metadata of the data center, detecting and responding to the triggering operation of metadata, ensuring that the changes of metadata are updated and maintained in a timely manner, the prerequisites for automated metadata detection and detection are realized, and the efficiency and accuracy of metadata management and database maintenance are improved.

[0074] Step S2: If there is a trigger operation corresponding to metadata maintenance, perform corresponding metadata processing according to the trigger operation, and upload the metadata processing result to the corresponding metadata knowledge base. The metadata processing includes: adding new table content, changing metadata in the table, or deleting table operations.

[0075] Step S3, calling the large language model to generate metadata knowledge points corresponding to the slice, converting the metadata knowledge points into corresponding vectors through the embedded model and storing them in the vector database. The large language model in step S3 can be a large model with the ability to process and understand complex data, and a model with the ability to convert unstructured data into structured or semi-structured data, such as: GPT, Transformer-XL, RoBERTa, ALBERT (A Lite BERT), T5 (Text-to-Text Transfer Transformer), XLNet (eXtreme Language Model), knowledge graph embedding model, neural network model or autoencoder (an unsupervised learning model for feature extraction and data compression), etc. The above large model can process a large amount of data and extract useful information from it to generate vectors that can be understood by the machine. The large model can be pre-trained on relevant data sets and then fine-tuned for specific tasks to generate the best metadata-related knowledge points and their vectors.

[0076] The technical solution provided by steps S1 to S3 realizes the intelligent management and maintenance of metadata, and stores the processing results in vector form to facilitate subsequent efficient query and application, realizes automated metadata maintenance, reduces manual intervention, and improves the efficiency and accuracy of metadata maintenance. By real-time trigger operation monitoring of metadata that needs to be maintained, corresponding metadata processing operations can be performed, making maintenance operations more targeted, and automatically performing operations such as adding table content, changing metadata in a table, or deleting a table to ensure timely updating and accuracy of metadata. The processed metadata results are uploaded to the metadata knowledge base, providing the data center with the latest metadata information to support further analysis and application of the data. The metadata knowledge points generated by the large model are converted into vectors and stored in the vector database, which can improve the query efficiency of SQL statements and make metadata-based queries faster, especially when processing large-scale data. The query response speed can be significantly improved, and the consistency and reliability of metadata in the data center can be ensured, providing a guarantee for the accuracy of the data, so that the data center can better serve data analysis and decision support.

[0077] like Figure 1A As shown, in step S1, the metadata of the data center is obtained, and whether there is a trigger operation that requires metadata maintenance is detected, further comprising:

[0078] Step S11, the metadata obtained from the data center includes: table structure, field attributes, indexes, constraints, and checks whether the table structure, field attributes, indexes, and constraints have trigger operations that require metadata maintenance.

[0079] Step S12, tracking the change history of the data center, detecting the corresponding trigger operation by setting a trigger in the data center, or: determining whether a trigger operation has occurred by performing log analysis on the data center.

[0080] The optimization technical solution provided in step S11 to step S12 realizes comprehensive monitoring and management of metadata of the data center. In the process of executing step S11 to step S12, by acquiring and analyzing metadata and tracking change history, it is used to detect and respond to metadata changes that may need maintenance, collect corresponding database object information, such as table structure, field attributes, indexes and constraints, etc., and track the change history of the data center through trigger or log analysis, so as to identify trigger operations such as adding a new table, changing a table or deleting a table.

[0081] The optimization technical solution provided in steps S11 to S12 can monitor changes in the data center in real time, quickly respond to metadata change requirements, detect corresponding trigger operations in real time, automatically track and detect metadata changes, improve the efficiency and accuracy of metadata changes and management, ensure the consistency and accuracy of metadata through timely updating and maintenance of metadata, improve the quality of metadata, provide a reliable data foundation for subsequent data analysis and decision-making based on data analysis, and improve the maintenance efficiency of the data center and the quality of data management through timely updating, automation and refined metadata management.

[0082] like Figure 1B As shown, in step S2, if there is a trigger operation corresponding to metadata maintenance, the corresponding metadata processing is performed according to the trigger operation, and the metadata processing result is uploaded to the corresponding metadata knowledge base. The metadata processing includes: adding table content, changing metadata in the table, or deleting table operations, and further includes:

[0083] Step S21, in the metadata processing process of the newly added table content, includes: supplementing the table description, supplementing the field description, supplementing the association relationship between tables, supplementing the value space, supplementing the primary key, supplementing the foreign key relationship, and adding a new document;

[0084] Generate a new table content file from the new table content after metadata processing, upload the new table content file to the metadata knowledge base to form a new table document, upload the entire content of the new table document as a slice to the metadata knowledge base, and / or:

[0085] Step S22, in the metadata processing process of changing the metadata in the table, includes: changing the table description, changing the field description, changing the association between tables, changing the value space, updating the primary key, updating the foreign key relationship, and updating the document;

[0086] Generate an update table content document from the metadata in the change table after metadata processing, first delete the existing metadata document of the table, and then upload the entire content of the update table content document as a slice to the metadata knowledge base, and / or:

[0087] Step S23, in the metadata processing process of the table deletion operation, includes: changing metadata, deleting table documents, deleting related table documents, and adding related table documents;

[0088] The delete table operation after metadata processing generates a delete table operation document, and the entire content of the delete table operation document is uploaded to the metadata knowledge base as a slice.

[0089] The optimization technical solution provided by steps S21 to S23 involves metadata processing for adding, changing, and deleting table operations. The optimization technical solution of steps S21 to S23 ensures the integrity, accuracy, and timely updating of metadata, and integrates the processing results into the metadata knowledge base for easy management and query. For example, by supplementing and updating table descriptions, field descriptions, associations, etc., the integrity of metadata is ensured; by updating primary keys, foreign key relationships, and value spaces, data consistency is maintained, and the associations between tables are clarified, which helps to understand data structures and data flows; by uploading processing results to the metadata knowledge base, the metadata in the metadata knowledge base is ensured to be the latest, and continuous updating of metadata is supported; uploading metadata knowledge points as a slice can improve query efficiency, especially when a large amount of related metadata needs to be accessed quickly, which improves the efficiency and quality of metadata management, ensures data consistency and integrity, and also supports data governance and compliance requirements.

[0090] For example, the optimization technical solution of step S21 to step S23 may be further improved:

[0091] Call the large language model to generate knowledge points for adding table content, changing metadata in the table, or deleting table operations corresponding to the slice;

[0092] Using the embedded model, knowledge points of adding new table content, changing metadata in a table, or deleting table operations are converted into corresponding vectors, and the vectors are stored in a vector database.

[0093] The uploading of the entire content of the newly added table document as a slice to the metadata knowledge base is specifically as follows:

[0094] Call the large language model to generate the new table content knowledge points corresponding to the slice, and generate the triple information of the slice content;

[0095] Analyze metadata content and identify corresponding tables and fields in the metadata knowledge base;

[0096] Generate a content overview of the table and its fields. The content overview includes all the information stored in the table and all the fields of all the tables.

[0097] Construct triple information, by identifying entities, relations and objects in metadata, and construct triple information according to the structure of "subject, relation, object".

[0098] The optimization technical solution from step S21 to step S23 realizes automated metadata management and knowledge point vectorization processing, enhances data consistency and accuracy, and by building a knowledge base containing metadata knowledge points, can better manage and utilize data resources, help improve the efficiency and quality of data management, and help improve the development efficiency of developers and shorten the development cycle. Through the vectorization and storage of metadata knowledge points, it supports more complex data analysis and query, such as semantic-based search and knowledge graph completion.

[0099] In the optimization technical solution of step S21 to step S23, "embedded model" refers to a machine learning model, which can convert input data (such as text, images, audio, etc.) into a fixed-length numerical vector, which is called an embedding vector. The embedding vector captures the high-level features and semantic information of the input data, allowing the model to represent and compare complex data structures in a low-dimensional space. In the field of natural language processing (NLP), embedded models are generally used to convert text data (such as words, phrases or documents) into continuous vector representations, through which the semantics and contextual information of the original data can be retained. Embedded models in the prior art include: Word2Vec, GloVe (Global Vectors for Word Representation), etc.

[0100] like Figure 1C As shown, in step S3, calling the large language model to generate metadata knowledge points corresponding to the slice, calling the large language model to generate metadata knowledge points corresponding to the slice, converting the metadata knowledge points into corresponding vectors through the embedded model and storing the vectors in the vector database, further includes:

[0101] Step S31, receiving a data query request, parsing the data query request using natural language processing technology, and extracting query intent and key information;

[0102] Step S32, using the large language model to perform semantic understanding on the data query request, and constructing a corresponding SQL statement based on the semantic analysis results obtained from the query intent and key information, specifically: retrieving the metadata corresponding to the data query request in the metadata knowledge base, placing the metadata in the prompt word, inputting the prompt word containing the metadata into the large language model, and generating the corresponding semantic analysis results.

[0103] In step S32, a large language model or its variants may be used to semantically understand the query request, such as ChatGPT, ERNIE, ChatGLM, etc., including selecting the correct database table, field, and constructing necessary query conditions in the query intent and key information.

[0104] Step S33: according to the execution SQL statement corresponding to the semantic analysis result, convert the execution SQL statement into natural language for description and output.

[0105] The optimization technical solution provided in step S31 to step S33 realizes the automatic conversion from natural language query request to structured query language, improves the convenience and efficiency of data query, and realizes seamless connection between natural language and database query for users. Using a large language model to realize the conversion from natural language to SQL statement can accurately analyze the user's query intention and key information, conduct in-depth semantic understanding of the query request, and build the correct SQL statement based on this, including selecting appropriate database tables and fields, and building necessary query conditions. After executing the SQL statement, the query results are converted back to natural language description, so that non-technical users can also easily understand the query results, lower the technical threshold, improve the accuracy and efficiency of the query, and enable users to interact with the database in natural language, thereby improving the user experience and expanding the application scenarios of database queries.

[0106] Exemplarily, in the optimization technical solution provided in step S31 to step S33, a large language model is used to perform semantic understanding on the data query request, specifically:

[0107] The following formula is used in the process of using a large language model to understand the semantics of data query requests:

[0108]

[0109] Where: E = Embed(Q), E represents the conversion of the data query request Q into a word vector set in a high-dimensional space;

[0110] R represents the output of the encoder. The role of R is to further process E to capture the contextual information and long-distance dependencies in the query sentence. The encoder used can be a Transformer-based structure, such as GPT or other variants, and so on.

[0111] A = Attention(R), where A represents the output of the attention mechanism. The attention mechanism focuses on the key parts of the query and assigns different weights to each word or phrase to highlight the information most relevant to the query intent.

[0112] S represents the calculation result of the semantic similarity between the query statement R and the domain knowledge base Z. The calculation formula of S is: S = SemanticSimilarity(R,Z). The calculation result of the semantic similarity is used to evaluate the similarity between the query statement and the relevant knowledge to ensure that the generated SQL statement can accurately reflect the user's query intention.

[0113] g is an integration function, which combines the outputs of all the above components into a unified representation vector Q. The integration function can be a weighted function or a neural network function, which is used to integrate the parameters such as E, R, A, S, etc. in the formula.

[0114] θ g represents the parameters of the integration function g, θ g The parameters are learned through training data to optimize the representation of query intent and key information.

[0115] The optimization technical solution provided in steps S31 to S33 uses a large language model to perform in-depth semantic understanding and processing of data query requests in the form of natural language. By forming a unified representation vector Q for natural language and its corresponding vectors, it can accurately understand and convert the user's query intention, and finally generate an SQL statement that meets the user's needs. The understanding process also includes context information and long-distance dependencies, which improves the accuracy of SQL statement queries. The attention mechanism is applied to enable the key parts of the query statement to be identified and focused on during the SQL statement query and generation process, and to ensure that the generated SQL statement meets the user's business needs by calculating the semantic similarity between the query statement and the relevant domain knowledge base. After the query, the SQL statement can be automatically generated, which reduces manual intervention and improves query efficiency. The parameter θg of the integration function g is obtained through training data learning, and the self-learning function is used to more accurately capture the user's true query intention and generate query results that better meet user needs.

[0116] like Figure 2 As shown, the SQL metadata processing system based on the big model is used to implement the SQL metadata processing method based on the big model, including:

[0117] The data center metadata acquisition module is used to obtain the metadata of the data center and detect whether there is a trigger operation that requires metadata maintenance. The trigger operation includes: adding a new table, changing a table, or deleting a table;

[0118] The metadata maintenance trigger operation module is used to perform corresponding metadata processing according to the trigger operation if there is a trigger operation corresponding to metadata maintenance, and upload the metadata processing result to the corresponding metadata knowledge base. The metadata processing includes: adding new table content, changing metadata in the table, or deleting table operations;

[0119] The metadata knowledge point conversion vector module is used to call the large language model to generate metadata knowledge points corresponding to the slice, convert the metadata knowledge points into corresponding vectors through the embedded model and store them in the vector database.

[0120] like Figure 3 As shown, the present invention further provides a method for generating SQL statements based on a large model. The method for generating SQL statements based on a large model includes the SQL metadata processing method described in any one of the embodiments of the present invention. Exemplarily, based on steps S1 to S3, the method further includes:

[0121] Step S4: The data center obtains the question and submits the question to the question-answering knowledge base for retrieval, specifically: full-text retrieval and / or semantic retrieval are used for retrieval, wherein:

[0122] During full-text search, the question is segmented, stop words are removed or stems are extracted. When entering semantic search, the question is converted into a vector through an embedded model.

[0123] Step S5, determining whether there is an answer corresponding to the question in the question-answering knowledge base, if the answer corresponding to the question is retrieved, executing the data query step, otherwise executing the metadata retrieval step;

[0124] Step S6, in the metadata retrieval step, converting the question into a vector and querying it in a vector database;

[0125] Step S7, using the large language model to generate a corresponding SQL statement according to the query result in the vector database;

[0126] Step S8, based on the received user's selection of SQL statements, the corresponding SQL statements are saved in the question-answer knowledge base, and the SQL statements are vectorized and indexed.

[0127] The technical solution provided in steps S4 to S8 combines question-answer knowledge base retrieval, vector database query and large model generation technology to achieve automatic conversion from natural language questions to SQL statements, improving the efficiency and usability of database queries. Through full-text retrieval and semantic retrieval technology, users' natural language questions can be understood, which improves the interactive friendliness of the system. The use of intelligent retrieval of word segmentation, removal of stop words or extraction of stems and vector conversion of semantic retrieval improves the accuracy and efficiency of retrieval. The corresponding SQL statements can be automatically generated according to the user's questions, reducing the workload of users in writing SQL statements and lowering the technical threshold. Users can interact with the database directly through natural language without having to master complex SQL language, which improves the user experience.

[0128] In step S5, the data center obtains the question and submits the question to the question-answering knowledge base for retrieval, specifically:

[0129] Step S51, converting the question into a vector through an embedded model, querying the vector in the vector database of the metadata knowledge base through the vector, obtaining the corresponding slice knowledge point and obtaining the metadata slice according to the slice knowledge point.

[0130] Step S52, using the large language model to generate corresponding SQL statements based on the metadata slices in the vector database, specifically: retrieving metadata corresponding to the question in the question-and-answer knowledge base, placing the metadata in prompt words, inputting the prompt words containing the metadata into the large language model, and using the large language model to generate corresponding semantic analysis results of the question.

[0131] The optimization technology provided in step S51 to step S52 utilizes the combination of embedded model and big model to realize efficient conversion from natural language question to structured query language (SQL). First, the natural language question of the user is converted into a corresponding vector through the embedded model, and then the query is performed in the vector database of the metadata knowledge base to retrieve the metadata slice most relevant to the question. Finally, the big language model is used to generate the corresponding SQL statement from the metadata slice, so as to intelligently respond to the user's query request.

[0132] The optimization technical solution provided by steps S51 to S52 can query the vector database by asking the vector corresponding to the question, quickly locate the relevant metadata slices, and reduce the redundant steps in the traditional retrieval method. The embedded model can capture the semantic information in the question, improve the accuracy of question understanding, and ensure the high relevance of the query results. In addition, a large language model is used to intelligently generate SQL statements based on metadata slices, reducing the workload of manually writing SQL statements. Users can submit corresponding questions with only a small amount of SQL knowledge, which lowers the technical threshold for SQL application, improves user experience, reduces possible human errors, optimizes the metadata query process, and improves the query capability based on natural language, providing powerful SQL statement query support for the data middle platform.

[0133] Furthermore, in step S5, the question is segmented, stop words are removed or stems are extracted during full-text search, and the question is converted into a vector through an embedded model during semantic search, further comprising:

[0134] In the process of converting the intersection problem into a vector through the embedded model, the features of the SQL statement are extracted, and the features include the keywords of the SQL statement, the data table name, the field name and the operator;

[0135] The extracted features are mapped to a high-dimensional space to form a feature vector, where the feature vector is represented by V = (v 1 , v 2 , ..., vn ), v n Represents the weight value of the nth feature;

[0136] Normalize the eigenvector, the normalization formula is: ∥V∥ represents the Euclidean norm of vector V;

[0137] The complexity of the eigenvector is calculated to determine the complexity of the eigenvector. The complexity calculation formula is C = α·log(||V′||)+β·D(V′)+γ·E(V′) where:

[0138] α represents the logarithmic weight of the normalized vector norm, which is used to adjust the influence of the normalized vector norm on complexity; β represents the diversity weight of the vector distribution, which is used to adjust the influence of the diversity of distribution of different features in the vector on complexity; D(V′) represents the standard deviation of the weights of each feature in the feature vector V′, and D(V′) is used to measure the distribution range of the feature weights; γ represents the proportional weight of the zero-weight feature in the feature vector, and the role of γ is to adjust the influence of the zero-weight feature on complexity; E(V′) represents the proportion of zero-weight features in the feature vector V′, and E(V′) is used to measure the proportion of unactivated features in the feature vector.

[0139] In the optimization technical solution of step S5, the data query request in natural language form is converted into a structured SQL statement through a large language model, and the converted feature vector is normalized and complexity calculated to ensure the accuracy and effectiveness of the generated SQL statement. The optimization technical solution of step S5 first uses an embedded model to extract the key features of the SQL statement, which include keywords, data table names, field names, operators, etc., and then maps these features to a high-dimensional space to form a feature vector. Next, the dimensional influence between different features is reduced through normalization, and the complexity of the feature vector is evaluated through complexity calculation, thereby ensuring the accuracy and efficiency of the generated SQL statement. In particular, by adding the calculation of the complexity of SQL statements, the complexity of SQL statements can be evaluated by calculating the complexity of feature vectors, so that appropriate adjustments can be made when generating SQL statements to ensure the accuracy and executability of the statements, thereby improving the quality and efficiency of SQL statement generation, making the generated SQL statements more in line with the user's true intentions, and at the same time reducing the number of user questions, improving the interaction efficiency between the system and users, and achieving efficient and accurate conversion from natural language to SQL statements, improving the intelligence level of data query, and providing users with a more convenient and accurate SQL statement query and generation experience.

[0140] like Figure 4The SQL statement generation system based on the big model shown is used to implement the SQL statement generation method based on the big model shown in the embodiment of the present invention, including:

[0141] The data center question acquisition module is used to obtain questions from the data center and submit the questions to the question-answering knowledge base for retrieval, specifically: full-text retrieval and / or semantic retrieval, where:

[0142] During full-text search, the question is segmented, stop words are removed or stems are extracted. When entering semantic search, the question is converted into a vector through an embedded model.

[0143] The question answer retrieval module is used to determine whether there is an answer corresponding to the question in the question and answer knowledge base. If the answer corresponding to the question is retrieved, the data query step is executed, otherwise the metadata retrieval step is executed;

[0144] In the metadata retrieval step, the question is converted into a vector and queried in a vector database;

[0145] The SQL statement generation module is used to generate corresponding SQL statements using a large language model based on the query results in the vector database; based on the user's selection of SQL statements received, the corresponding SQL statements are saved in the question-and-answer knowledge base, and the SQL statements are vectorized and indexed.

[0146] like Figure 5 As shown, the present invention also provides a data middle platform, which includes the SQL metadata processing system based on the big model, and / or: the SQL statement generation system based on the big model, and the SQL metadata processing system and the SQL statement generation system are applied in the data middle platform. The data middle platform in the embodiment of the present invention includes: a big language model, a knowledge base and other modules, wherein the knowledge base includes a metadata knowledge base and a question-and-answer knowledge base, and other modules include the following sub-modules: metadata management, data collection, master data, data quality, data development, data standards, data security, and data services.

[0147] The data middle platform provided based on the above embodiment can implement the following implementation methods in some specific scenarios:

[0148] 1. Overview:

[0149] The data center of this embodiment uses the big language model technology to generate SQL statements that meet the expectations of natural language questions raised by users and query data. Improve the efficiency of user data query. The existing data center includes metadata management, data collection, master data, data quality, data development, data standards, data security and data services. Compared with the existing technical solutions, the present invention adds a knowledge base and a big language model. The knowledge base logically includes a metadata knowledge base and a question-and-answer knowledge base, and technically consists of an embedded model and a vector database.

[0150] (I) Process of maintaining metadata knowledge base:

[0151] There are three types of metadata changes: 1. Adding a new table; 2. Changing metadata within a table; 3. Deleting a table.

[0152] The metadata management module adds processing for these three situations.

[0153] 1. Add a new table and process it as follows:

[0154] The first step is to supplement the table description: If the existing metadata table annotations are incomplete, record the table annotations. Supplement the table description, using natural language to describe what data the table stores and describe the usage scenarios of this data. Example: "The basic personnel information table stores the common basic information of students, faculty and staff."

[0155] Step 2: Supplement field description: If the existing field annotation is incomplete, supplement the field annotation. Use natural language description, for example: "Name refers to the Chinese name officially registered with the public security household registration management department and officially recorded in the personnel file."

[0156] The third step is to supplement the relationship between tables: If the fields in this table are associated with fields in other tables, supplement the relationship and use natural language to describe it. For example: "table1.col1 is associated with table2.col1".

[0157] Step 4: Supplement the value space. If the data elements of the fields in the table have a value format, range, and specification, supplement the value space description of the field. Use natural language description, for example: "t_staff.region is the personnel birthplace code. The personnel birthplace code uses the administrative division code of the People's Republic of China. The personnel birthplace code is stored in the table t_regioncode, t_regioncode.id is the administrative division code, and t_regioncode.name is the administrative division name".

[0158] Step 5: Add the primary key. If the table has a primary key, add the description of the primary key. Use natural language description, for example: "t_staff.id staff code is the primary key".

[0159] Step 6: Add foreign key relationships. If the table has foreign keys, add foreign key descriptions. Use natural language descriptions, for example: "t_student.orgid, the department where the student is located, is the foreign key of the student table t_student, and the foreign key value is t_organization.id".

[0160] Step 7: Add a new document. Obtain the following information from the metadata management module to form the metadata file for this table, including:

[0161] CREATE TABLE statement: This includes the field name, field type, length, whether it is empty, comments, and:

[0162] Table description, field description, relationships between tables, value space, primary key, and foreign key.

[0163] Upload the table metadata file to the metadata knowledge base to form a document. The document is not segmented, and all the contents are treated as one slice.

[0164] Generate knowledge points for the slice. Based on the slice content, call the large language model to generate knowledge points for the slice. The methods for generating knowledge points include: (1) generating questions with the slice content as answers; (2) generating an overview of the slice; (3) generating triple information for the slice content as knowledge points.

[0165] Example of question knowledge point: "What is the primary key of the staff table t_staff?".

[0166] Example of a knowledge point overview: "This document describes in detail the database table structure and field meanings of student information and grade data, including the fields, field associations, field value meanings, and data examples of the student information table and the student grade table. This information makes it easy to query, count, and analyze student information."

[0167] An example of a triplet of information knowledge point: (t_student, stores data, information about all students in the school).

[0168] The knowledge points generated above can effectively improve the recall rate of metadata slices.

[0169] The problem of generating slice content, the prompt words are as follows:

[0170] #Character settings:

[0171] As a database metadata expert, your task is to ask questions about the metadata {{meta}}. Do not output your thinking process and explanation, just output the answer.

[0172] #Capability Description:

[0173] You have the ability to analyze metadata.

[0174] The first step is to change the table description: If the table annotation changes, record the new annotation of the table. Update the table description, using natural language to describe what data the table stores and the usage scenarios of this data. Example: "The basic information table of personnel stores the common basic information of students, faculty and staff."

[0175] Step 2: Change the field description: If the table field changes, update the field annotation. Use natural language description, for example: "Name refers to the Chinese name officially registered with the public security household registration management department and officially recorded in the personnel file."

[0176] Step 3: Change the association between tables: If the association between the fields in this table and other tables changes, update the association. Use natural language to describe, for example: "table1.col1 is associated with table2.col1".

[0177] Step 4: Change the value space: If the format, range, and specification of the data element value of the field in the table changes, update the field value space description. Use natural language description, for example: "t_staff.region is the personnel birthplace code. The personnel birthplace code uses the administrative division code of the People's Republic of China. The personnel birthplace code is stored in the table t_regioncode, and t_regioncode.id is the administrative division code.

[0178] code, t_regioncode.name is the administrative region name".

[0179] Step 5: Update the primary key: If the primary key of the table changes, update the primary key description of the table. Use natural language description, for example: "t_staff.id staff code is the primary key".

[0180] Step 6: Update foreign key relationships: If the foreign key of the table changes, update the foreign key description of the table. Use natural language description, for example: "t_student.orgid, the department where the student is located, is the foreign key of the student table t_student, and the foreign key value is t_organization.id".

[0181] Step 7, update the document: obtain the following information from the metadata management module to form the metadata file of the table.

[0182] CREATE TABLE statement, which includes field name, field type, length, whether it is empty, comments, as well as: table description, field description, relationship between tables, value space, primary key, and foreign key.

[0183] First, delete the existing metadata document of the table, and then upload the new metadata file of the table to the metadata knowledge base to form a document. The document is not split, and all the contents are taken as one slice.

[0184] 3. Delete the table and proceed as follows:

[0185] The first step is to change the metadata. In the metadata management module, adjust the associations and foreign key relationships between tables by deleting the tables.

[0186] The second step is to delete the table document. Find the knowledge base document code of the table in the metadata management module and delete the existing metadata document of the table in the knowledge base.

[0187] The third step is to delete the associated table documents. In the metadata management module, find all the associated tables and the associated table knowledge base document codes corresponding to the table, and delete the metadata documents that already exist for these tables in the knowledge base.

[0188] Step 4: Add a new related table document.

[0189] The following information is obtained from the metadata management module to form an association table metadata file.

[0190] CREATE TABLE statement. It includes field name, field type, length, whether it is empty, comments, as well as: table description, field description, relationship between tables, value space, primary key, and foreign key.

[0191] Upload the new metadata file of the association table to the metadata knowledge base to form a document. The document is not segmented, and all the contents are treated as one slice.

[0192] (III) The process of answering questions is as follows:

[0193] The first step is to search for questions: users submit natural language questions to the data center. The data center submits the questions to the question-answering knowledge base for retrieval. The retrieval uses full-text retrieval and semantic retrieval.

[0194] Full-text search uses an inverted index strategy for search recall. The user's question is segmented, stop words are removed, and stems are extracted. According to the words in the question, the inverted list of the question-answering knowledge base is searched to find the list of document IDs related to it. The document ID list is sorted using a relevance evaluation algorithm (such as TF-IDF).

[0195] Semantic retrieval uses the question-answering knowledge base to query SQL statements. First, the user's question is converted into a vector through the embedded model, and then queried in the vector database by vector. The vector database returns sorted SQL statements. The number of returned results and similarity distance can be set to control the number of returned results.

[0196] Determine whether the question-answering knowledge base has retrieved the answer. If so, execute the second step, otherwise execute the third step.

[0197] The second step is to query data: the data center provides an interface for querying data. This interface uses the SQL retrieved in the first step as an input parameter, queries the database, and returns the data and its metadata. The metadata describes the data. The data center returns the metadata and data to the user, and the process ends.

[0198] The third step is to retrieve metadata: first convert the user's question into a vector through the embedded model, and query the vector in the vector database of the metadata knowledge base. The vector database returns the sorted slice knowledge points. The number of returned results and the similarity distance are set to control the number of returned results. If no metadata is retrieved, execute the fourth step. If metadata is retrieved, execute the fifth step.

[0199] Step 4: Write SQL: The user writes SQL based on the question and query data results and executes step 6.

[0200] Step 5: Generate SQL using the big model: Use the big model to generate SQL based on the questions raised by the user and the metadata corresponding to the questions.

[0201] Step 6, query data: The data center provides an interface for querying data. This interface uses the SQL generated in steps 4 and 5 as input parameters, queries the database, and returns data and its metadata. Metadata describes the data. The user determines whether the SQL is correct based on the query data results. If the SQL is inaccurate, execute step 4. If the SQL is correct, the user chooses whether to save it to the question-and-answer knowledge base. If it is saved to the question-and-answer knowledge base, execute step 7. If it is not saved to the question-and-answer knowledge base, the process ends.

[0202] Step 7: Save to the question-answering knowledge base: vectorization of user questions. The user's questions are converted into vectors through the embedded model, and the vectors and SQL are stored in the vector database of the question-answering knowledge base.

[0203] The process ends.

[0204] like Figure 6 As shown, the present invention, based on the SQL metadata processing method and SQL statement generation method based on the big model, also provides corresponding electronic equipment:

[0205] An electronic device comprises: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of a SQL metadata processing method and an SQL statement generation method based on a large model.

[0206] In the description of the present invention, unless otherwise specified, the large model refers to the large language model.

[0207] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0208] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A SQL metadata processing method based on a large model, applied to metadata processing in a data center, characterized in that: include: Obtain the metadata of the data center and detect whether there are trigger operations that require metadata maintenance. The trigger operations include: adding a new table, changing a table, or deleting a table. If there is a trigger operation corresponding to metadata maintenance, the corresponding metadata processing is performed according to the trigger operation, and the metadata processing result is uploaded to the corresponding metadata knowledge base. The metadata processing includes: adding new table content, changing metadata in the table, or deleting table operations; The large language model is called to generate metadata knowledge points corresponding to the slices, and the metadata knowledge points are converted into corresponding vectors through the embedded model and stored in the vector database.

2. The SQL metadata processing method based on a large model according to claim 1 is characterized in that: The step of obtaining metadata of the data center and detecting whether there is a trigger operation that requires metadata maintenance further includes: The metadata obtained from the data center includes: table structure, field attributes, indexes, and constraints. Check whether the table structure, field attributes, indexes, and constraints have trigger operations that require metadata maintenance. Track the change history of the data center by setting triggers in the data center to detect the corresponding trigger operations, or: determine whether the trigger operation has occurred by performing log analysis on the data center.

3. The SQL metadata processing method based on a large model according to claim 1 is characterized in that: If there is a trigger operation corresponding to metadata maintenance, then the corresponding metadata processing is performed according to the trigger operation, and the metadata processing result is uploaded to the corresponding metadata knowledge base. The metadata processing includes: adding table content, changing metadata in the table, or deleting table operations, and further includes: The metadata processing of the newly added table content includes: supplementing table description, supplementing field description, supplementing the relationship between tables, supplementing value space, supplementing primary key, supplementing foreign key relationship, and adding new documents; Generate a new table content file from the new table content after metadata processing, upload the new table content to the metadata knowledge base to form a new table document, upload the entire content of the new table document as a slice to the metadata knowledge base, and / or: In the metadata processing process of changing the metadata in the table, it includes: changing the table description, changing the field description, changing the relationship between tables, changing the value space, updating the primary key, updating the foreign key relationship, and updating the document; Generate an update table content document from the metadata in the change table after metadata processing, upload the entire content of the update table content document as a slice to the metadata knowledge base, and / or: The metadata processing of the table deletion operation includes: changing metadata, deleting table documents, deleting related table documents, and adding related table documents; The delete table operation after metadata processing generates a delete table operation document, and the entire content of the delete table operation document is uploaded to the metadata knowledge base as a slice.

4. The SQL metadata processing method based on a large model according to claim 1 is characterized in that: The calling of the large language model to generate metadata knowledge points corresponding to the slice, converting the metadata knowledge points into corresponding vectors through the embedded model and storing them in the vector database further includes: Receiving a data query request, parsing the data query request using natural language processing technology, and extracting query intent and key information; The large language model is used to understand the semantics of the data query request, and the corresponding SQL statement is constructed according to the semantic analysis results obtained from the query intent and key information, specifically: Retrieve metadata corresponding to the data query request in the metadata knowledge base, put the metadata in the prompt word, input the prompt word containing the metadata into the large language model, and generate corresponding semantic parsing results; According to the execution SQL statement corresponding to the semantic analysis result, the execution SQL statement is converted into natural language for description and output.

5. The SQL metadata processing method based on a large model according to claim 3 is characterized in that: Also includes: Call the large language model to generate knowledge points for adding table content, changing metadata in the table, or deleting table operations corresponding to the slice; Using the embedded model, knowledge points of adding new table content, changing metadata in a table, or deleting table operations are converted into corresponding vectors, and the vectors are stored in a vector database. The uploading of the entire content of the newly added table document as a slice to the metadata knowledge base is specifically as follows: Call the large language model to generate the new table content knowledge points corresponding to the slice, and generate the triple information of the slice content; Analyze metadata content and identify corresponding tables and fields in the metadata knowledge base; Generate a content overview of the table and its fields. The content overview includes all the information stored in the table and all the fields of all the tables. Construct triple information by identifying entities, relations, and objects in metadata and constructing triple information according to the "subject, relation, object" structure.

6. The SQL metadata processing method based on a large model according to claim 4 is characterized in that: The use of a large language model to perform semantic understanding on data query requests is specifically as follows: The following formula is used in the process of using a large language model to understand the semantics of data query requests: Where: E = Embed(Q), E represents the conversion of the data query request Q into a word vector set in a high-dimensional space, R represents the output of the encoder, A represents the output of the attention mechanism, S represents the result of the semantic similarity calculation of the query statement, g is the integration function, θ g Represents the parameters of the integration function g.

7. A method for generating SQL statements based on a large model, characterized in that: The SQL metadata processing method according to any one of claims 1 to 6, further comprising: The data center obtains the question and submits the question to the question-answering knowledge base for retrieval, specifically: full-text retrieval and / or semantic retrieval, where: In full-text retrieval, the questions are segmented, stop words are removed or stems are extracted. In semantic retrieval, the questions are converted into vectors through embedded models. Determine whether there is an answer corresponding to the question in the question-answering knowledge base. If the answer corresponding to the question is retrieved, execute the data query step, otherwise execute the metadata retrieval step; In the metadata retrieval step, the question is converted into a vector and queried in a vector database; Generate corresponding SQL statements using a large language model based on the query results in the vector database; According to the received user's selection of SQL statements, the corresponding SQL statements are saved in the question-answer knowledge base, and the SQL statements are vectorized and indexed.

8. The method for generating SQL statements based on a large model according to claim 7, characterized in that: The data center obtains the question and submits the question to the question-answering knowledge base for retrieval, specifically: The question is converted into a vector through an embedded model, and the vector is queried in a vector database of a metadata knowledge base to obtain a corresponding slice knowledge point and obtain a metadata slice according to the slice knowledge point; Generate a corresponding SQL statement using a large language model according to the metadata slice in the vector database, specifically: retrieve metadata corresponding to the question in the question-answering knowledge base, put the metadata in a prompt word, input the prompt word containing the metadata into the large language model, and generate a corresponding question semantic analysis result using the large language model; The step of segmenting the question, removing stop words or extracting stems during full-text retrieval, and converting the question into a vector through an embedded model during semantic retrieval further includes: In the process of converting the intersection problem into a vector through the embedded model, the features of the SQL statement are extracted, and the features include the keywords of the SQL statement, the data table name, the field name and the operator; The extracted features are mapped to a high-dimensional space to form a feature vector, where the feature vector is represented by V = (v1, v2, ..., v n ), v n Represents the weight value of the nth feature; Normalize the eigenvector, the normalization formula is: ||V|| represents the Euclidean norm of vector V; The complexity of the eigenvector is calculated to determine the complexity of the eigenvector. The complexity calculation formula is C = α·log(||V′||)+β·D(V′)+γ·E(V′), where: α represents the logarithmic weight of the normalized vector norm, β represents the diversity weight of the vector distribution, D(V′) represents the standard deviation in the feature vector V′, γ represents the proportional weight of zero-weight features in the feature vector, and E(V′) represents the proportion of zero-weight features in the feature vector V′.

9. An electronic device, characterized in that: include: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the method described in any one of claims 1 to 8.

10. A data middle platform, characterized in that: The data middle platform includes the electronic device described in claim 9.

Images