Rail transit industry large model NL2SQL capability improvement method and system based on hybrid cloud multimode and RAG framework
Through the rail transit industry large model based on hybrid cloud multimode and RAG framework, combined with pre-trained language models and knowledge graphs, the problem of data processing in the rail transit industry is solved, and efficient natural language to SQL conversion is achieved, which significantly improves the Query efficiency and user experience.
Patent Information
- Application Number
- CN202411156213.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-15
AI Technical Summary
Data in the rail transit industry is highly professional, complex and real-time. The existing natural language to SQL (NL2SQL) technologies are difficult to meet the needs of high intelligence and adaptability, especially in industry terminology understanding, complex query construction and real-time Data processing.
Using a large model of the rail transit industry based on hybrid cloud multimode and RAG frameworks, the resource allocation and load balancing are achieved through hybrid cloud multimode technology, and combined with the NL2SQL engine enhanced by the RAG framework, pre-trained language model and knowledge graph to conduct natural language query. The conversion of SQL statements uses industry-specific data set fine-tuning mechanism and query execution and feedback learning mechanism to continuously optimize system performance and model parameters.
It significantly improves the accuracy and efficiency of SQL statement conversion of natural language query, reduces the complexity of SQL queries, meets the needs of efficient information extraction in the big data era, and is especially suitable for non-professional users, which significantly improves query efficiency.
Smart Images

Figure CN118964580A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of natural language processing technology, and specifically to a method and system for improving the NL2SQL capabilities of a large model in the rail transit industry based on a hybrid cloud multi-mode and RAG framework. Background Art
[0002] With the rapid development of artificial intelligence technology, natural language processing (NLP) and machine learning are increasingly being used in all walks of life. Especially in the rail transit industry, with the surge in data volume and increasing complexity, traditional data processing and query methods can no longer meet the growing business needs. In this context, the natural language to SQL (NL2SQL) technology came into being, aiming to convert users' natural language queries into structured SQL queries through natural language understanding (NLU) technology, thereby achieving direct interaction with the database.
[0003] However, the data in the rail transit industry is highly professional, complex and real-time, which poses higher challenges to NL2SQL technology. For example, the understanding of industry terminology, the construction of complex queries, and the processing of real-time data all require the NL2SQL system to have a higher level of intelligence and adaptability. Summary of the invention
[0004] The purpose of the present invention is to provide a method and system for improving the NL2SQL capabilities of a large model in the rail transit industry based on hybrid cloud multi-mode and RAG framework to solve the problems raised in the above background technology.
[0005] To achieve the above object, the present invention provides the following technical solution: a method for improving the NL2SQL capability of a large model in the rail transit industry based on hybrid cloud multi-mode and RAG framework, the method comprising the following steps:
[0006] Use hybrid cloud multi-mode technology architecture to achieve optimal resource allocation and load balancing;
[0007] The NL2SQL engine enhanced by the RAG framework is used, combined with pre-trained language models and knowledge graphs, to improve the accuracy and efficiency of converting natural language queries to SQL statements;
[0008] Through the industry-specific data set fine-tuning mechanism, continuous data collection, annotation and model fine-tuning are carried out to adapt the system to the specific needs of the rail transit industry;
[0009] Execute the generated SQL statements and collect execution results and user feedback for continuous optimization of system performance and adjustment of model parameters.
[0010] Preferably, the hybrid cloud multi-mode technology architecture includes a multi-cloud management platform and an automated resource allocation system to achieve unified management and monitoring across cloud services.
[0011] Preferably, the NL2SQL engine enhanced by the RAG framework includes a retrieval enhancement module, a context encoder, a pre-trained language model and a sequence-to-sequence model.
[0012] Preferably, the industry-specific dataset fine-tuning mechanism includes a data collection and preprocessing module, a data annotation and quality control module, a model selection and transfer learning module, and a model fine-tuning and optimization module.
[0013] Preferably, the natural language processing model uses the Hairuo model and the fine-tuned nl2sql Hairuo model to enhance the understanding of rail transit industry terminology and query logic;
[0014] The query execution and feedback learning mechanism can perform adaptive learning based on the execution results and continuously optimize system performance and model parameters.
[0015] A rail transit industry large model NL2SQL capability enhancement system based on hybrid cloud multi-mode and RAG framework, the system comprising:
[0016] Multi-channel vectorized matching engine provides powerful data processing and analysis capabilities for the NL2SQL system in the rail transit industry;
[0017] Deep understanding and pattern representation, enhanced by the NL2SQL engine of the RAG framework, to improve the accuracy and efficiency of natural language processing (NLP) and structured query language (SQL) generation;
[0018] Industry-specific dataset fine-tuning mechanism.
[0019] Preferably, the multi-path vectorized matching engine includes:
[0020] Multi-cloud environment integration, containerization and orchestration, microservices architecture, edge computing, data lake and data warehouse, multi-cloud management platform, and security and compliance.
[0021] Preferably, deep understanding and pattern representation include:
[0022] Retrieval-augmented generation, knowledge graph integration, context encoders, sequence-to-sequence Seq2Seq models, transfer learning and fine-tuning, model optimization techniques, and end-to-end training and evaluation.
[0023] Preferably, the industry-specific dataset fine-tuning mechanism includes:
[0024] Data collection and preprocessing, deep learning and transfer learning, incremental learning and active learning, multi-task learning, model optimization and compression, and knowledge graph fusion.
[0025] Compared with the prior art, the present invention has the following beneficial effects:
[0026] The proposed method and system for improving the NL2SQL capability of the rail transit industry based on hybrid cloud multi-mode and RAG framework integrates multi-cloud management, automated resource allocation, advanced NLP technology, knowledge graph, graph neural network and deep learning algorithm to achieve high-accuracy conversion from natural language query to SQL statement. The present invention meets the demand for efficient information extraction in the era of big data, is particularly suitable for non-professional users, and can significantly reduce the complexity of SQL queries and improve query efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 The present invention is a system flow chart;
[0028] Figure 2 The present invention is a multi-path vectorized matching engine flow chart;
[0029] Figure 3 Provide a deep understanding and pattern representation flow chart for the present invention;
[0030] Figure 4 This is a flow chart of the industry-specific dataset fine-tuning mechanism of the present invention. DETAILED DESCRIPTION
[0031] In order to make the purpose and technical solution of the present invention clearly and completely described, and the advantages more clearly understood, the embodiments of the present invention are further described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the present invention, rather than all of the embodiments, and are only used to explain the embodiments of the present invention, and are not used to limit the embodiments of the present invention. All other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0032] In the first embodiment, the present invention provides a technical solution: a method for improving the NL2SQL capability of a large model in the rail transit industry based on a hybrid cloud multi-mode and RAG framework, the method comprising the following steps:
[0033] The hybrid cloud multi-mode technology architecture is used to achieve optimal resource allocation and load balancing; the hybrid cloud multi-mode technology architecture includes a multi-cloud management platform and an automated resource allocation system to achieve unified management and monitoring across cloud services.
[0034] The NL2SQL engine enhanced by the RAG framework is combined with a pre-trained language model and knowledge graph to improve the accuracy and efficiency of converting natural language queries to SQL statements; the NL2SQL engine enhanced by the RAG framework includes a retrieval enhancement module, a context encoder, a pre-trained language model, and a sequence-to-sequence model.
[0035] Through the industry-specific dataset fine-tuning mechanism, data collection, labeling and model fine-tuning are continuously carried out to adapt the system to the specific needs of the rail transit industry; the industry-specific dataset fine-tuning mechanism includes data collection and preprocessing module, data labeling and quality control module, model selection and transfer learning module, model fine-tuning and optimization module; the natural language processing model adopts the Hairuo model and the fine-tuned nl2sql Hairuo model to enhance the understanding of rail transit industry terminology and query logic.
[0036] Execute the generated SQL statements, collect execution results and user feedback for continuous optimization of system performance and adjustment of model parameters. The query execution and feedback learning mechanism can perform adaptive learning based on the execution results and continuously optimize system performance and model parameters.
[0037] Embodiment 2, refer to the attached Figure 1 As shown, based on the first embodiment, a large-model NL2SQL capability enhancement system for the rail transit industry based on hybrid cloud multi-mode and RAG framework is proposed. The intelligent NL2SQL query generation system consists of the following three main technical modules:
[0038] Module 1: Multi-way vectorized matching engine
[0039] The hybrid cloud multi-mode technology architecture is the core of this invention, which aims to provide powerful data processing and analysis capabilities for the NL2SQL system in the rail transit industry. The following are the technical details of the architecture:
[0040] Multi-cloud environment integration:
[0041] Integrate resources from multiple cloud service providers (such as AWS, Azure, and Google Cloud) to achieve optimal allocation of computing and storage resources.
[0042] Private cloud is deployed in local data centers to ensure data security and compliance.
[0043] Containerization and Orchestration:
[0044] Use Docker container technology to encapsulate applications to ensure environment consistency and rapid deployment.
[0045] Use Kubernetes for container orchestration to achieve automatic expansion, load balancing, and fault recovery of services.
[0046] Microservices Architecture:
[0047] The system is designed using a microservice architecture, with each microservice responsible for a specific function, such as data collection, processing, and analysis.
[0048] Microservices interact with each other through lightweight communication protocols such as gRPC or REST.
[0049] Edge computing:
[0050] Deploy edge computing nodes at rail transit stations to realize localized data processing and analysis and reduce latency.
[0051] Edge nodes work together with the central cloud to form a distributed computing architecture.
[0052] Data Lakes and Data Warehouses:
[0053] Use data lake technology to store unstructured and semi-structured data such as IoT sensor data, log files, etc.
[0054] Data warehouses are used to store structured data and support complex queries and analysis.
[0055] Multi-cloud management platform:
[0056] Achieve unified management and monitoring across cloud services, optimize resource usage, and reduce costs.
[0057] Automatic resource allocation, dynamically adjust resource allocation according to system load.
[0058] Security and Compliance:
[0059] Implement an end-to-end encryption strategy to protect data during transmission and storage.
[0060] Follow industry standards and regulations, such as GDPR, ISO 27001, etc. to ensure compliance in data processing.
[0061] Through the hybrid cloud multi-mode technology architecture, the NL2SQL system of the present invention can provide the rail transit industry with a highly flexible, scalable, secure and efficient technology platform to meet the industry's high standards for data processing and analysis.
[0062] Module 2: Deep Understanding and Pattern Representation
[0063] The NL2SQL engine enhanced by the RAG framework combines multiple cutting-edge technologies to improve the accuracy and efficiency of natural language processing (NLP) and structured query language (SQL) generation.
[0064] Retrieval Enhancement Generation (RAG):
[0065] Using the RAG framework, the retrieval mechanism and the sequence generation model are combined to enhance the context understanding ability of the generation model by retrieving relevant information.
[0066] Apply vector indexing techniques, such as FAISS or Annoy, to quickly retrieve database structure and historical query information related to user queries.
[0067] Knowledge graph integration:
[0068] Build a knowledge graph in the field of rail transit, integrate industry terms, concepts, entity relationships, etc., and store them in the form of a graph database.
[0069] The retrieval module is enhanced through the knowledge graph to provide a deeper understanding of professional terms and concepts.
[0070] Context Encoder:
[0071] Develop a context encoder to combine user queries and retrieved information to generate rich context vectors.
[0072] The context encoder uses a multi-head attention mechanism to capture the multi-dimensional semantic features of the query.
[0073] Sequence to Sequence (Seq2Seq) Model:
[0074] Use advanced Seq2Seq models, such as T5 or BART, to convert natural language queries into SQL statements.
[0075] Customize the decoder of the Seq2Seq model to support the specific syntax structure of the SQL language.
[0076] Transfer learning and fine-tuning:
[0077] Implement transfer learning strategies to migrate models pre-trained on large-scale datasets to industry-specific datasets.
[0078] Through fine-tuning, the model can adapt to the language habits and query patterns of the rail transit industry.
[0079] Model optimization techniques:
[0080] Apply techniques such as model pruning, quantization, and knowledge distillation to reduce the computational complexity of the model and improve the inference speed.
[0081] Model parallel technology and pipeline parallel technology are used to further improve the model processing speed.
[0082] End-to-end training and evaluation:
[0083] Implement an end-to-end training process, from data preprocessing, model training to fine-tuning and evaluation.
[0084] Use automated machine learning (AutoML) technology to automatically adjust model parameters and find the optimal configuration.
[0085] The NL2SQL engine enhanced by the RAG framework can not only accurately understand complex natural language queries, but also efficiently generate semantically compliant SQL statements. It is particularly suitable for industries such as rail transit that have extremely high requirements for accuracy and professionalism.
[0086] Module 3: Industry-specific dataset fine-tuning mechanism
[0087] The industry-specific dataset fine-tuning mechanism is a key module for customizing the NL2SQL system for industries such as rail transit.
[0088] Data collection and preprocessing:
[0089] Use data collection tools to collect data from rail transit industry databases, logs, user interactions and other sources.
[0090] Apply natural language processing technology to clean, denoise and format data to prepare for high-quality data annotation.
[0091] Deep Learning and Transfer Learning:
[0092] Use transfer learning techniques to migrate models pre-trained on large-scale general datasets to industry-specific datasets.
[0093] Leverage deep learning frameworks (e.g., TensorFlow, PyTorch) to fine-tune the model to learn industry-specific language patterns.
[0094] Incremental learning and active learning:
[0095] An integrated incremental learning mechanism enables the model to continuously learn from newly annotated data without having to retrain from scratch.
[0096] By applying active learning strategies, the model intelligently selects the most valuable data samples for annotation, improving learning efficiency.
[0097] Multi-task learning:
[0098] A multi-task learning framework is adopted to enable the model to simultaneously learn multiple related tasks such as entity recognition, intent recognition, and SQL generation during fine-tuning.
[0099] Model optimization and compression:
[0100] Apply model optimization techniques such as model pruning and quantization to reduce model size and improve inference speed.
[0101] Model distillation is a technique used to transfer knowledge from large and complex models into smaller models suitable for resource-constrained environments.
[0102] Knowledge graph fusion:
[0103] Combining industry knowledge graphs with fine-tuning mechanisms provides the model with rich background knowledge and enhances contextual understanding capabilities.
[0104] A / B testing and model evaluation:
[0105] Conduct A / B testing to compare the effects of different fine-tuning strategies and select the optimal model.
[0106] The model performance is quantitatively evaluated using indicators such as precision, recall, and F1 score.
[0107] Through the industry-specific data set fine-tuning mechanism, the NL2SQL system of the present invention can provide highly customized and accurate natural language query services for industries such as rail transit, greatly improving user experience and business efficiency.
[0108] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A method for improving the NL2SQL capability of a large model in the rail transit industry based on hybrid cloud multi-mode and RAG framework, characterized by: The method comprises the following steps: Use hybrid cloud multi-mode technology architecture to achieve optimal resource allocation and load balancing; The NL2SQL engine enhanced by the RAG framework is used, combined with pre-trained language models and knowledge graphs, to improve the accuracy and efficiency of converting natural language queries to SQL statements; Through the industry-specific data set fine-tuning mechanism, continuous data collection, annotation and model fine-tuning are carried out to adapt the system to the specific needs of the rail transit industry; Execute the generated SQL statements and collect execution results and user feedback for continuous optimization of system performance and adjustment of model parameters.
2. The method for improving the NL2SQL capability of a large model in the rail transit industry based on hybrid cloud multi-mode and RAG framework according to claim 1 is characterized by: The hybrid cloud multi-mode technology architecture includes a multi-cloud management platform and an automated resource allocation system to achieve unified management and monitoring across cloud services.
3. The method for improving the NL2SQL capability of a large model in the rail transit industry based on hybrid cloud multi-mode and RAG framework according to claim 1 is characterized by: The NL2SQL engine enhanced by the RAG framework includes a retrieval enhancement module, a context encoder, a pre-trained language model, and a sequence-to-sequence model.
4. The method for improving the NL2SQL capability of a large model in the rail transit industry based on hybrid cloud multi-mode and RAG framework according to claim 1 is characterized by: The industry-specific dataset fine-tuning mechanism includes data collection and preprocessing module, data labeling and quality control module, model selection and transfer learning module, and model fine-tuning and optimization module.
5. According to claim 1, the method for improving the NL2SQL capability of a large model in the rail transit industry based on hybrid cloud multi-mode and RAG framework is characterized by: The natural language processing model uses the Hairuo model and the fine-tuned nl2sql Hairuo model to enhance the understanding of rail transit industry terminology and query logic; The query execution and feedback learning mechanism can perform adaptive learning based on the execution results and continuously optimize system performance and model parameters.
6. A rail transit industry large model NL2SQL capability improvement system based on hybrid cloud multi-mode and RAG framework according to any one of claims 1-5, characterized in that: The system comprises: Multi-channel vectorized matching engine provides powerful data processing and analysis capabilities for the NL2SQL system in the rail transit industry; Deep understanding and pattern representation, enhanced by the NL2SQL engine of the RAG framework, to improve the accuracy and efficiency of natural language processing (NLP) and structured query language (SQL) generation; Fine-tuning mechanism for industry-specific datasets.
7. The rail transit industry large model NL2SQL capability enhancement system based on hybrid cloud multi-mode and RAG framework according to claim 6 is characterized by: The multi-way vectorized matching engine includes: Multi-cloud environment integration, containerization and orchestration, microservices architecture, edge computing, data lake and data warehouse, multi-cloud management platform, and security and compliance.
8. The rail transit industry large model NL2SQL capability enhancement system based on hybrid cloud multi-mode and RAG framework according to claim 6 is characterized by: Deep understanding and pattern representation include: Retrieval-augmented generation, knowledge graph integration, context encoders, sequence-to-sequence Seq2Seq models, transfer learning and fine-tuning, model optimization techniques, and end-to-end training and evaluation.
9. The rail transit industry large model NL2SQL capability enhancement system based on hybrid cloud multi-mode and RAG framework according to claim 6 is characterized by: The industry-specific dataset fine-tuning mechanisms include: Data collection and preprocessing, deep learning and transfer learning, incremental learning and active learning, multi-task learning, model optimization and compression, and knowledge graph fusion.
Citation Information
Patent Citations
Marketing business support method and system, electronic equipment and storage medium
CN118170955A
Method for converting full-link natural language into structured query language
CN118227653A
Database query statement generation method and device, equipment and storage medium
CN118227655A
Data processing method, device and equipment based on large language model
CN118349661A
Management and control method and system based on intelligent sewage treatment cloud platform
CN118429139A
Cited By
Urban rail transit intelligent operation management method and device
CN119784304A
Nervous system closed-loop regulation and control system based on transcranial opto-acoustic synergy
CN119925839A
NL2SQL model query enhancement method and device based on reinforcement learning and medium
CN120256467A
Dynamic RAG-NL2SQL system based on real-time data blood map
CN121278003A
Airworthiness document review method based on dynamic active learning and knowledge graph agent
CN121542425A
