Natural language query method and device, electronic equipment and storage medium
By performing word segmentation processing and feature extraction on natural language query data, combined with embedding model and distillation algorithm, the high deployment cost and frequent training problems caused by large language model dependence are solved, and efficient and accurate natural language query data processing is achieved.
Patent Information
- Application Number
- CN202411882614.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-19
- Publication Date
- 2025-05-06
AI Technical Summary
When querying data in natural language, the existing technology relies on large language models, resulting in high deployment costs for privatized scenarios, and the data needs to be frequently retrained when LLM is updated, which has data security and hallucination problems.
By performing word segmentation processing on the input query language, extracting time features and merging it, high-dimensional vectors are obtained using the embedded model, and vector data is processed in combination with distillation algorithm and self-attention mechanism, the dependence on large language models is reduced and query efficiency and accuracy are improved.
It reduces the dependence on large language models when querying data in natural language, reduces the cost of privatization, reduces the training burden during LLM updates, and improves the controllability and accuracy of query results.
Smart Images
Figure CN119938827A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of natural language processing technology, and in particular to a natural language query method, device, electronic device and storage medium. Background Art
[0002] With the rapid development of technology, many open source frameworks and closed source products have basically realized simple natural language query data and display.
[0003] At present, without considering the data quality, if you want to communicate with data through natural language interaction, the simplest and most common way is to rely on a large language model (LLM). However, the performance and reasoning ability of LLM are directly related to the accuracy of the results, and the reasoning ability of LLM is proportional to its own parameter quantity. Deploying LLM in private scenarios brings considerable cost issues, and using cloud services also has data security issues. In addition, if you always hope that the ability of LLM can bring better results, it is most effective to keep LLM at the latest version. However, according to the traditional method, you need to perform SFT, RLHF or even partial pre-training on the new model every time. This process consumes a lot of computing power and time. If you encounter a different data set or architecture from the previous version when updating LLM, you may need to re-prepare the training data or pre-training set and training framework, otherwise the hallucination problem will increase significantly. Summary of the invention
[0004] The main purpose of the present invention is to provide a natural language query method, device, electronic device and storage medium, which can reduce the dependence on large language models and the deployment cost of private scenarios when querying natural language data.
[0005] To achieve the above object, the present application provides a natural language query method, the method comprising: Perform word segmentation on the input query language, extract time features and merge them to obtain a time feature word set; For words other than the time feature word set, an embedding model is used to obtain multiple high-dimensional vectors; Processing the multiple high-dimensional vectors based on a preset processing algorithm to obtain first text data; Based on the first text data and the time feature word set, the corresponding vectors are obtained and recalled in sequence, and the recall index and dimension results are recorded to obtain vector data; Processing the vector data using a distillation algorithm and a self-attention mechanism to obtain second text data; Using the query condition determined based on the second text data, querying in the large language model to obtain structured results; Based on the structured result, a target query statement is generated at the semantic layer, and query result data is obtained according to the target query statement.
[0006] Optionally, the using a distillation algorithm and a self-attention mechanism to process the vector data to obtain the second text data includes: The self-attention mechanism is used to find the attention weight of each word in the whole sentence and sort them from large to small, and then the adaptive weighted sum is used to filter out the n words with the highest weight.
[0007] Optionally, the following formula is used in the distillation algorithm: ; Where T is the temperature parameter, Z i With Z j Respectively represent the last result and the new input is the loss value obtained.
[0008] Optionally, the query condition determined based on the second text data is used to query in a large language model to obtain a structured result, including: Determining the query condition according to the weight of the keywords in the second text data in the round of conversation; The query condition and the prompt word are provided to the large language model to obtain the structured result.
[0009] Optionally, after generating a target query statement at a semantic layer based on the structured result and obtaining query result data according to the target query statement, the method further includes: Generate a chart based on the query result data and return to the front-end page.
[0010] Optionally, the front-end page also outputs generation rules for the chart, including but not limited to one or more of the following: Statistics time, indicators, dimensions, filter conditions, analysis and sorting.
[0011] Optionally, the method further includes: When the data volume is less than a threshold or specific DEMO data, the RAG model is used to realize the conversion from natural language to the structured result.
[0012] Another aspect of the present application provides a natural language query device, comprising: The word segmentation processing module is used to perform word segmentation processing on the input query language, extract time features and merge them to obtain a time feature word set; An embedding module, used for obtaining multiple high-dimensional vectors using an embedding model for words other than the time feature word set; A processing module, used for processing the multiple high-dimensional vectors based on a preset processing algorithm to obtain first text data; The processing module is further used to obtain the corresponding vectors based on the first text data and the time feature word set, perform recall in sequence, and record the recall index and dimension results to obtain vector data; The processing module is further used to process the vector data using a distillation algorithm and a self-attention mechanism to obtain second text data; A query module, configured to query a large language model using a query condition determined based on the second text data to obtain a structured result; A generation module is used to generate a target query statement at a semantic layer based on the structured result, and obtain query result data according to the target query statement.
[0013] On the other hand, the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes each step in the natural language query method as described in the first aspect.
[0014] On the other hand, the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes each step in the natural language query method as described in the first aspect.
[0015] The present application provides a natural language query method, device, electronic device and storage medium, which perform word segmentation processing on the input query language, extract time features and merge them to obtain a time feature word set; for words other than the time feature word set, use an embedding model to obtain multiple high-dimensional vectors; process the multiple high-dimensional vectors based on a preset processing algorithm to obtain first text data; based on the first text data and the time feature word set, obtain the corresponding vectors and recall them in sequence, and record the recalled indicators and dimensional results to obtain vector data; use a distillation algorithm and a self-attention mechanism to process the vector data to obtain second text data; use a query condition determined based on the second text data to query in a large language model to obtain a structured result; based on the structured result, generate a target query statement at the semantic layer, and obtain query result data according to the target query statement; the dependence on the large language model and the deployment cost of private scenarios when querying natural language data can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0017] in: Figure 1 A flowchart of a natural language query method provided in an embodiment of the present application; Figure 2 A schematic diagram of a query result page provided in an embodiment of the present application; Figure 3 A schematic diagram of a natural language query process architecture provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of a natural language query device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0018] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.
[0019] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.
[0020] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0021] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.
[0022] See also Figure 1 , is a flow chart of a natural language query method provided in an embodiment of the present application. Figure 1 As shown, the method includes: 101. Perform word segmentation on the input query language, extract and merge time features, and obtain a time feature word set.
[0023] The method in the embodiment of the present application can be executed by a natural language query device, and can be specifically executed on an electronic device, such as a terminal device.
[0024] In the present application embodiment Figure 1 The method shown mainly describes the actual application steps. The embodiments of the present application also involve early data preprocessing and deployment steps, which will be described later.
[0025] Specifically, in the embodiment of the present application, a user can input a natural language query through a terminal device, and the input query language can be segmented.
[0026] The word segmentation process mentioned in the embodiments of the present application is one of the key preprocessing steps in natural language processing (NLP), which involves breaking down text into combinations of words or sub-words so that the machine can better understand and analyze the text.
[0027] For example, if the query language entered by the user is: "How is Zhang XX's performance in June this year compared with that in March? In addition, how is the receivables of the customers he is responsible for?", the result of word segmentation processing can be expressed as: [How does Zhang XX's performance in June this year compare to that in March? "In addition, how is the receivables situation of the customers he is responsible for? "] In natural language processing, extracting time features is a key step, which can help the model understand and use the time information in the text. In the embodiments of the present application, tools can be used to parse the time information in the text, such as extracting time features based on open source tools (such as spaCy, Time-NLP) and merging the results obtained, which are not limited in the embodiments of the present application.
[0028] Taking the query language input above as an example, the result of extracting time features and merging them can be expressed as: [“June this year”, “March”] 102. For words other than the above-mentioned time feature word set, an embedding model is used to obtain multiple high-dimensional vectors.
[0029] Here, the text data is converted into a numerical vector so that the machine learning model can process it. High-dimensional vectors can capture rich semantic information, and each dimension can represent a specific semantic feature, such as gender, tense, emotion, etc., which helps the model understand the deeper meaning of the text. By using high-dimensional vectors, the model can cover a wide range of semantic relationships and contextual dependencies in the language, thereby improving the ability to understand and generate text data.
[0030] Specifically, the remaining words can be transformed into multiple tensors of 768 dimensions * the number of words using the embedding model. Tensors are used to represent and process text data. Through the word embedding algorithm, the vocabulary can be mapped to a continuous vector space, so that semantically similar words are close to each other in the vector space.
[0031] 103. Process the multiple high-dimensional vectors based on a preset processing algorithm to obtain first text data.
[0032] The preset processing algorithms in the embodiments of the present application can be selected and adjusted as needed to perform text analysis, for example, algorithms in the field of text mining and information retrieval can be used.
[0033] The LDA (Latent Dirichlet Allocation) mentioned in the embodiments of the present application is a topic model used to discover hidden topic information from a document collection. It is an unsupervised machine learning algorithm that can be used for tasks such as document classification, topic discovery, and document summarization.
[0034] The TF-IDF (Term Frequency-Inverse Document Frequency) mentioned in the embodiments of this application is a statistical method used to evaluate the importance of a word to a document in a document collection. It is a feature extraction technology widely used in information retrieval and text mining.
[0035] In the embodiment of the present application, the above two methods can be used in combination to improve the effect of text analysis, which is not limited in the embodiment of the present application.
[0036] Specifically, in the embodiment of the present application, the core algorithm mainly uses the algorithm of the attention mechanism: the self-attention mechanism is used to find the attention weight of each word in the whole sentence and sort them from large to small, and then the adaptive weighted sum is used to filter out the n words with the highest weight (words greater than the weight value).
[0037] Self-attention formula:
[0038] Adaptive weighted summation formula:
[0039] Taking the above input as an example, the first text data displayed after the simulation result is denormalized is: salesperson, Zhang XX, supplier, customer, customer abbreviation, customer classification, accounts receivable, accounts payable, order number, sales target, actual sales and other indicators and dimensional information.
[0040] The self-attention mechanism mentioned in the embodiments of this application allows the model to dynamically focus on the information of different parts of the sequence when processing sequence data. This mechanism has been widely used in the Transformer model, which captures the dependencies between elements by calculating the attention score of each element in the sequence to other elements.
[0041] The distillation algorithm mentioned in the embodiments of this application is a model compression technology that transfers the knowledge of a large, complex model (teacher model) to a smaller, simpler model (student model). In NLP, this usually involves using the output or intermediate representation of the teacher model as the target of student model training, so that the student model learns the behavior of the teacher model.
[0042] In the embodiment of the present application, when these two techniques are used in combination, the self-attention mechanism can be used as part of the teacher model to generate rich intermediate representations. These representations can then be used as soft targets in the distillation process to help the student model learn the attention pattern of the teacher model. For example, the student model can be trained by distilling the self-attention weights of the teacher model so that it can produce a similar attention distribution when processing the same input.
[0043] Through self-attention distillation, the student model can learn how the teacher model focuses on the key parts of the input sequence, which is crucial to improving the performance of the student model on various NLP tasks. In addition, this method can also help the student model capture complex language patterns and long-distance dependencies while maintaining a small model size.
[0044] 104. Based on the first text data and the time feature word set, the corresponding vectors are obtained and recalled in sequence, and the indicators and dimension results of the recall are recorded to obtain vector data.
[0045] The corresponding vectors can be obtained according to the result obtained in step 103 and recalled in sequence, and the recalled index and dimension results (vector data) can be recorded.
[0046] 105. Use the distillation algorithm and the self-attention mechanism to process the above vector data to obtain the second text data.
[0047] The recurrent neural network (RNN) constructed by the self-attention mechanism can be gradually distilled to obtain the most obvious value of each weight change, and finally record, compare and extract the result with the highest weight.
[0048] The recurrent neural network (RNN) mentioned in the embodiments of the present application is a neural network architecture for processing sequence data. Unlike traditional feedforward neural networks, RNN can process sequence data of any length and can utilize the dependencies between time steps in the sequence.
[0049] Specifically, taking the aforementioned input language as an example, after word segmentation, the words with the highest weights obtained through the self-attention mechanism and weighted sum are: "Zhang XX", "performance", "compared to", "he", and "customer". The results obtained at this time will have certain noise. The similarity between the obtained results and the results retrieved from the vector database can be calculated and added to the RNN. At this time, the results and the results of the new words are added to the RNN again. In this process, the distillation technology is used to amplify the loss, and finally the most suitable query object can be obtained.
[0050] The number of RNN layers used at this time is the same as the length of the loop list. The input parameter of the first layer is the similarity between the first word in the list and the result retrieved from the vector database. Each subsequent layer uses the result of the previous layer plus the new similarity as the input parameter. Before the result of each layer is passed to the next layer, the distillation algorithm is used to amplify its loss value until the end.
[0051] Further optionally, the following formula is used in the above distillation algorithm: ; Where T is the temperature parameter, Z i With Z j Respectively represent the last result and the new input is the loss value obtained.
[0052] Taking the above input language as an example, the second text data displayed after the simulation result is denormalized is: salesperson, Zhang XX, sales, customer, accounts receivable.
[0053] 106. Use the query condition determined based on the second text data to query in the large language model to obtain a structured result.
[0054] The Large Language Model (LLM) mentioned in the embodiments of this application is a type of neural network model based on deep learning, which plays an important role in the field of natural language processing. LLM learns the statistical laws of language by pre-training on a large amount of text data, and can generate new texts similar to human-written texts. These models can perform a variety of tasks, including but not limited to text summarization, translation, sentiment analysis, etc.
[0055] In an optional implementation, the above step 106 includes: Determining the query condition according to the weight of the keywords in the second text data in the conversation; The query conditions and prompt words are provided to the large language model to obtain the structured results.
[0056] Specifically, you can give the question and the prompt words to LLM, and you will get the structured result: {"limit":None,"order_by":"DESC","asalysis":"contrast"} The prompt words can be set as needed, and general prompt words can be selected. For example, LLM is required to extract the sorting requirements, aggregation requirements, and analysis requirements in the question and return them in JSON format.
[0057] Taking the aforementioned input language as an example, for further example, using the conditional query interface according to the result of step 105, it can be obtained that the condition that Zhang XX is a salesperson is also a condition for a warehouse manager, but because the warehouse manager has a lower weight in this round of dialogue, the final query condition is: the salesperson is Zhang XX.
[0058] In an optional implementation, the above method further includes: When the amount of data is less than the threshold or specific DEMO data, the RAG model is used to achieve the conversion from natural language to the above structured results.
[0059] 107. Based on the structured result, a target query statement is generated at the semantic layer, and query result data is obtained according to the target query statement.
[0060] Specifically, the serialized results can be combined into SQL statements after combining all conditions and user permissions at the semantic layer, and then the corresponding data can be obtained through the underlying data query layer. This SQL statement can be executed by the database to retrieve the required data, so the generated SQL statement is sent to the database management system (DBMS), which executes this statement and returns the query results. These results can then be further processed by the application or presented directly to the user.
[0061] Serialization is the process of converting a data structure or object state into a format that can be stored or transmitted, such as JSON, XML, or a string. Here, serialization involves converting query conditions, user input, or other data into a format that can be understood by the database or further processed.
[0062] The semantic layer in the embodiment of the present application is an abstract layer that provides an advanced way to express data queries without directly writing complex SQL statements. This layer is usually used to separate business logic from database operations, so that non-technical users can also query data through a more friendly interface.
[0063] Optionally, in terms of current natural language query data, you can manually build a semantic layer, manually annotate, manually build graph relationships, write scripts, exhaustively enumerate SQL statements, and associate graphs. At the RAG (enhanced retrieval generation, without using algorithms) level, you can also directly use RAG to achieve the function of natural language query data when the data volume is small or for specific DEMO data.
[0064] The advantage of RAG technology is that it can dynamically retrieve information from external knowledge sources and use this information to enhance the model's answers, thereby significantly improving the accuracy and relevance of the response. This approach effectively solves the problem of hallucinations in LLMs, that is, the model generates information that does not match reality. In addition, by introducing external knowledge, RAG enhances the model's ability to understand and answer specific questions, making the generated text content richer, more accurate, and in line with user needs.
[0065] In the embodiment of the present application, based on the preset underlying data model, combined with the classification of indicators and dimensions during data preprocessing, a relationship map from the perspective of any indicator or any dimension can be formed. With this relationship map and data structure, the precise indicators and dimensions obtained by the NLP engine can be directly used by the corresponding system's own search engine, which can easily achieve the transition from natural language to data acquisition and display.
[0066] In an optional implementation, after the above step 107, the above method further includes: Generate a chart based on the above query result data and return to the front-end page.
[0067] In the embodiment of the present application, the front-end page of the system can render the returned results into a graph and display it to the user, so that the user can clearly view the query results.
[0068] Further optionally, the front-end page also outputs the generation rules of the chart, including but not limited to one or more of the following: Statistics time, indicators, dimensions, filter conditions, analysis and sorting.
[0069] For specific data queries, charts are generated according to specific generation rules, so they can be explained to show the relevant data and information more clearly.
[0070] For example, you can refer to Figure 2 A schematic diagram of a query result page is shown in FIG. The query information entered by the user is: How is the monthly performance of employee A this year? Through the method in this application, the final result returned is as follows: Figure 2 shown.
[0071] The text information returned describes the chart generation rules, including statistical time, indicators (sales unit price, sales quantity, sales target, sales target completion rate, sales amount), dimensions (salesperson, monthly), screening conditions (salesperson, employee A), analysis and sorting (not covered here); and the corresponding statistical charts are given ( Figure 2 The data of each dimension within the statistical time is displayed, and users can perform further operations on this basis, such as querying related data or information, or continuing to enter other questions.
[0072] To more clearly illustrate the method in the embodiments of the present application, see Figure 3 , Figure 3 A schematic diagram of a natural language query process architecture provided in an embodiment of the present application. Figure 3 The specific process steps involved are described as follows: The first step is to determine the data preprocessing requirements: relevant staff and business personnel confirm the requirements document. Based on the requirements document, an indicator system is built and a knowledge graph is automatically generated as the basic text framework.
[0073] The second step is vectorization processing: vectorize the extracted data to prepare for subsequent calculations.
[0074] The third step is feature extraction and algorithm selection: select specific algorithms and lightweight open source models (such as LDA, TF-IDF, self-attention, RAG+ distillation algorithm, spaCy, Rerank model, etc.) to process data and extract the main features. According to the task requirements, select appropriate algorithm parameters to optimize the model effect.
[0075] The fourth step is normalization and aggregation: the problem is converted into aggregation and sorting relationships that can be handled by the large model through normalization operations.
[0076] Step 5: Use PROMPT to provide preliminary model input: Use common prompt words and combine the model to calculate the aggregation and ranking relationships for specific problems.
[0077] Step 6: Initially return the results: self-attention returns the vectorized data of the subject, object, and condition in the question. Record the weight value of the recall result to form a preliminary output.
[0078] Step 7: Multiple return results and calculations: The returned results are weighted through the self-attention mechanism, and distilled repeatedly to obtain the final optimized answer. If necessary, use the RAG retrieval algorithm for feedback enhancement to extract fine-grained information in the results for further optimization.
[0079] Step 8: Semantic layer processing: pass the calculation results to the semantic layer for storage or query to ensure that every question can be answered.
[0080] Step 9: Final result aggregation and sorting: Aggregate and sort the results through LLM (such as large language model). Return structured query objects based on weights.
[0081] Step 10: Get data: Extract relevant data from the data warehouse / data lake / database.
[0082] Step 11: Chart rendering and display: Render the returned data in charts and display them to users intuitively.
[0083] Before executing the application process, preliminary preparation is required, mainly data preprocessing steps, which may include: 1. Investigate business needs and analyze business processes, quantify and qualitatively determine business needs, and deeply explore business application scenarios and core needs.
[0084] 2. Strictly follow the indicator system specifications, clarify the indicator classification and constraint naming methods, make each indicator self-explanatory, and construct its conditions and dimension relationships according to the indicator system, and set descriptive information for specific nouns.
[0085] 3. Build a map based on the indicator system.
[0086] 4. Verify data, create a semantic layer and associate data warehouses, data lakes, and databases.
[0087] 6. Vectorized semantic layer.
[0088] The natural language query method in the embodiment of the present application has the following effects: 1. Reduce the dependence on LLM: The NLP engine in the embodiment of the present application reduces the dependence on LLM performance, making it easy to upgrade / replace LLM through iterative replacement like a plug-in. It mainly relies on algorithms to filter out clear requirements. LLM does not require particularly strong reasoning capabilities, so large models below 14B can be used for assistance, instead of the 72B parameter-level LLM required to start in traditional ChatBI.
[0089] 2. Controllable output: The NLP engine in the embodiment of the present application rejects the black box in principle, and its results are completely controllable, and each step of the internal execution logic can be traced. This is mainly because the core calculation and reasoning process are implemented by algorithms, rather than directly using LLM for reasoning. This makes it easy to monitor the calculation results of each step and to fine-tune the calculation method for a specific environment, which is completely different from the traditional black box state of the entire reasoning process when relying on LLM reasoning.
[0090] 3. Reduce deployment costs: Since the NLP engine in the embodiment of the present application does not rely on LLM in the private deployment scenario, even if a small parameter LLM is used, the final accuracy will not be affected. In extreme cases, CPU deployment can even be used.
[0091] Based on the description of the foregoing method embodiment, the embodiment of the present application also provides a natural language query device.
[0092] Figure 4 A schematic diagram of the structure of a natural language query device provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the natural language query device 400 includes: The word segmentation processing module 410 is used to perform word segmentation processing on the input query language, extract time features and merge them to obtain a time feature word set; An embedding module 420, for obtaining multiple high-dimensional vectors using an embedding model for words other than the above-mentioned time feature word set; A processing module 430 is used to process the above-mentioned multiple high-dimensional vectors based on a preset processing algorithm to obtain first text data; The processing module 430 is further used to obtain the corresponding vectors based on the first text data and the time feature word set, perform recall in sequence, and record the recall index and dimension results to obtain vector data; The processing module 430 is further used to process the vector data using a distillation algorithm and a self-attention mechanism to obtain second text data; A query module 440, configured to query the large language model using the query condition determined based on the second text data to obtain a structured result; The generating module 450 is used to generate a target query statement at the semantic layer based on the structured result, and obtain query result data according to the target query statement.
[0093] It can be understood that the relevant contents of each module in the above device have been described in detail in the aforementioned method embodiment, and the details can be referred to the contents in the method embodiment; that is, a natural language query device provided in the present application can be executed as follows Figure 1 or Figure 3 Any steps in the illustrated embodiment will not be described in detail here.
[0094] In one embodiment of the present application, an electronic device is also provided. The electronic device may include a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the execution Figure 1 or Figure 3 Any step in the method embodiment shown. The electronic device may also include an input / output device, etc. In a specific implementation, the electronic device may be a terminal device, etc.
[0095] In one embodiment, a computer-readable storage medium is further provided. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes any step in the above method embodiment.
[0096] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0097] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0098] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A natural language query method, characterized in that: The method comprises: Perform word segmentation on the input query language, extract time features and merge them to obtain a time feature word set; For words other than the time feature word set, an embedding model is used to obtain multiple high-dimensional vectors; Processing the multiple high-dimensional vectors based on a preset processing algorithm to obtain first text data; Based on the first text data and the time feature word set, the corresponding vectors are obtained and recalled in sequence, and the recall index and dimension results are recorded to obtain vector data; Processing the vector data using a distillation algorithm and a self-attention mechanism to obtain second text data; Using the query condition determined based on the second text data, querying in the large language model to obtain structured results; Based on the structured result, a target query statement is generated at the semantic layer, and query result data is obtained according to the target query statement.
2. The natural language query method according to claim 1, characterized in that: The step of processing the vector data using a distillation algorithm and a self-attention mechanism to obtain second text data includes: The self-attention mechanism is used to find the attention weight of each word in the whole sentence and sort them from large to small, and then the adaptive weighted sum is used to filter out the n words with the highest weight.
3. The natural language query method according to claim 2, characterized in that: The following formula is used in the distillation algorithm: ； Where T is the temperature parameter, Z i With Z j Respectively represent the last result and the new input is the loss value obtained.
4. The natural language query method according to claim 3, characterized in that: The query condition determined based on the second text data is used to query in a large language model to obtain a structured result, including: Determining the query condition according to the weight of the keywords in the second text data in the round of conversation; The query condition and the prompt word are provided to the large language model to obtain the structured result.
5. The natural language query method according to claim 1, characterized in that: After generating a target query statement at the semantic layer based on the structured result and obtaining query result data according to the target query statement, the method further includes: Generate a chart based on the query result data and return to the front-end page.
6. The natural language query method according to claim 5, characterized in that: The front-end page also outputs the generation rules of the chart, including but not limited to one or more of the following: Statistics time, indicators, dimensions, filter conditions, analysis and sorting.
7. The natural language query method according to claim 1, characterized in that: The method further comprises: When the data volume is less than a threshold or specific DEMO data, the RAG model is used to realize the conversion from natural language to the structured result.
8. A natural language query device, characterized in that: include: The word segmentation processing module is used to perform word segmentation processing on the input query language, extract time features and merge them to obtain a time feature word set; An embedding module, used for obtaining multiple high-dimensional vectors using an embedding model for words other than the time feature word set; A processing module, used for processing the multiple high-dimensional vectors based on a preset processing algorithm to obtain first text data; The processing module is further used to obtain the corresponding vectors based on the first text data and the time feature word set, perform recall in sequence, and record the recall index and dimension results to obtain vector data; The processing module is further used to process the vector data using a distillation algorithm and a self-attention mechanism to obtain second text data; A query module, configured to query a large language model using a query condition determined based on the second text data to obtain a structured result; A generation module is used to generate a target query statement at a semantic layer based on the structured result, and obtain query result data according to the target query statement.
9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.