A natural language query method and system for multidimensional database

Abstract

The present invention discloses a natural language query method and system for a multidimensional database. The method comprises the following steps: collecting a user's natural language question and extracting data required for a multidimensional expression; constructing a prompt word template; using a retrieval enhancement technology to perform similarity calculation on vectors of external knowledge documents required for the multidimensional expression to obtain the external knowledge documents corresponding to the vectors; constructing a multidimensional database with a tree structure according to dimensions and metrics, and then performing dimension filtering on the multidimensional database to obtain a filtered dimensional tree structure string; using a retrieval enhancement technology to calculate the similarity between vector data in the vector database and the user's natural language question to obtain a question-multidimensional expression sample corresponding to the vector; inputting the above content as prompt words into a large model to generate a multidimensional expression; performing syntax and hallucination checking on the multidimensional expression to obtain an executable multidimensional expression; and performing reverse parsing to obtain the rows, columns, pages, and conditions queried in the multidimensional expression to complete the natural language query.

Description

Technical Field

[0001] The present invention relates to the technical field of database and natural language processing, and in particular to a natural language query method and system for a multidimensional database. Background Art

[0002] Multidimensional databases, due to their efficient data storage and query capabilities, are widely used in fields such as business intelligence (BI), data warehousing, and decision support systems (DSS). However, similar to the query language of relational databases, the query language of multidimensional databases usually requires professional development, which creates a certain barrier to entry for ordinary users. By implementing a natural language query interface, the difficulty of using multidimensional databases can be significantly reduced. In the process of developing natural language query capabilities for multidimensional databases, the following technologies played a key role:

[0003] Large Language Model (LLM) generation technology: This technology plays a core role in the generation of Multidimensional Expressions (MDX). The most critical step is to accurately process natural language questions posed by users and parse complex language requirements into multidimensional expressions. To achieve this goal, the model must possess the following capabilities: First, it must have strong natural language understanding capabilities to accurately capture the core intent, contextual semantics, and implicit information in user questions; second, it must be able to convert the understanding results into structured expressions that conform to the multidimensional database query logic, ensuring the logical correctness and executable feasibility of the generated multidimensional expressions. Furthermore, the model must flexibly adapt to the user's different expression methods and be able to generate customized query results based on specific instructions. This capability requires not only advanced natural language processing technology but also in-depth modeling and application capabilities of multidimensional database syntax and operational logic, thereby achieving a seamless transition from natural language to precise multidimensional expression generation.

[0004] Retrieval-augmented generation (RAG) technology plays a crucial role in the generation of Multidimensional Expressions (MDX). User questions often contain specialized vocabulary or complex expressions, which can prevent large language models from accurately understanding their intent. Furthermore, directly inputting all knowledge documents can affect the accuracy of generated results due to information redundancy. RAG technology provides an efficient solution to this problem. It converts user questions into fixed-length vectors and performs semantic matching with knowledge documents in a vector database to retrieve the documents most relevant to the user's question. These documents provide clear context for the user's question, enhancing the model's understanding. Furthermore, RAG technology can also be applied to the example search process for generating multidimensional expressions. By locating multiple multidimensional expression examples that are semantically closest to the user's question, RAG provides large language models with more precise contextual information, enabling the generation of multidimensional expressions that better meet user needs. This technology combines the advantages of retrieval and generation, significantly improving the professionalism and accuracy of generated results, and is particularly valuable when dealing with complex problems.

[0005] Chain of Thought (CoT) technology plays a key role in the generation of multidimensional expressions. If the user does not explicitly specify the workflow of the large language model, the model may generate results in a random manner, lacking a systematic analysis step. In this case, the generated multidimensional expression may contain omissions or errors, making it difficult to meet the user's actual needs. Chain of Thought technology clearly specifies the reasoning method of the large language model, guiding the model to gradually decompose the problem and analyze the details, ensuring that each step is logically verified. It can help the model focus on the key points in the generation process, thereby reducing randomness and potential error risks. By introducing chain of thought, the large language model not only improves its reasoning capabilities, but also generates more accurate and logically rigorous multidimensional expressions, providing users with higher-quality solutions. Summary of the Invention

[0006] The present invention provides a natural language query method for a multidimensional database, the method comprising:

[0007] Step S1. Collecting the user's natural language questions, and extracting the data required for the multidimensional expression according to the natural language questions;

[0008] Step S2. Constructing a prompt word template, wherein the prompt word template includes roles, metadata, user questions, output formats, samples, and workflows;

[0009] Step S3. Using retrieval enhancement technology to perform similarity calculation on the external knowledge documents in the data required by the multidimensional expression, screening a preset number of vectors, and obtaining the external knowledge documents corresponding to the vectors;

[0010] Step S4. Constructing a tree-structured multidimensional database based on the dimensions and metrics in the multidimensional expression, and then performing dimension filtering on the multidimensional database to obtain a filtered dimension tree-structured string;

[0011] Step S5. Using retrieval enhancement technology, calculate the similarity between the vector data in the vector database and the user's natural language question, filter a preset number of vectors, and obtain question-multidimensional expression samples corresponding to the vectors;

[0012] Step S6. Input the user's natural language question, external knowledge document, filtered dimension tree structure string, metric tree structure string, and question-multidimensional expression sample into the prompt word template to obtain a complete prompt word, input the complete prompt word into the large language model, and generate an initial multidimensional expression;

[0013] Step S7. Repeat step S using the large language model and the temperature parameter to generate several initial multidimensional expressions, form candidate multidimensional expressions based on the several initial multidimensional expressions, perform syntax checking on the candidate multidimensional expressions, screen out executable multidimensional expressions, and group the selected executable multidimensional expressions according to their execution results.

[0014] Step S8: After reverse parsing the executable multidimensional expression, the rows, columns, pages and conditions of the query in the multidimensional expression are obtained, and the natural language query is completed.

[0015] Optionally, in step S3, using retrieval enhancement technology to perform similarity calculation on external knowledge documents in the data required by the multidimensional expression, screening a preset number of vectors, and obtaining the content of the external knowledge documents corresponding to the vectors specifically includes:

[0016] Encode the external knowledge in the data required by the multidimensional expression into an external knowledge document using an encoding model;

[0017] Encode the user's natural language question into a user question vector using an encoding model;

[0018] The cosine similarity between the external knowledge document and the user question vector is calculated, and the cosine similarity results are sorted in descending order, and a preset number of vectors are selected to obtain the external knowledge document corresponding to the screening vector.

[0019] Optionally, in step S4, a multidimensional database with a tree structure is constructed according to the dimensions and measures in the multidimensional expression, and then dimension filtering is performed on the multidimensional database to obtain the content of the filtered dimension tree structure string, which specifically includes:

[0020] The user's natural language questions, external knowledge documents, and dimensional tree-structured strings in the multidimensional database are filled into prompt words and then input into the large language model to generate filtering results. Specifically:

[0021] The prompt word sequence of the prompt word is transformed into a vector representation through the embedding layer and position encoding to obtain the representation matrix of the prompt word sequence;

[0022] Based on the representation matrix, a self-attention mechanism is used to calculate the weights of the word units in the representation matrix;

[0023] Through the multi-head attention mechanism, different contextual information is captured from multiple subspaces to obtain the attention representation of each word;

[0024] After the attention representation of each word is input into the feedforward network for processing, the final representation of each word is obtained through the residual connection and normalization layer;

[0025] Input the final representation into the decoder to generate the probability distribution of the next word;

[0026] Use autoregressive method to gradually expand and generate the next word unit, and finally get the filtering result.

[0027] Optionally, the calculation content of inputting the final representation of the word unit into the decoder to generate the probability distribution of the next word unit includes:

[0028] P(y t |y <t ,X)=Softmax(h t W out +b out )

[0029] Among them, y t The output token generated for the current time step t, y ＜t is the output word sequence before time step t, X is the input word sequence, h t is the hidden state of the decoder at time step t, W out and b out are the output layer parameters.

[0030] The present invention also provides a natural language query system for a multidimensional database, the system comprising:

[0031] A data collection module is used to collect natural language questions from users and extract data required for multidimensional expressions based on the natural language questions;

[0032] Creating a prompt word template for constructing a prompt word template, wherein the prompt word template includes a role, metadata, a user question, an output format, a sample, and a workflow;

[0033] A document retrieval module is used to use retrieval enhancement technology to perform similarity calculation on external knowledge documents in the data required by the multidimensional expression, screen a preset number of vectors, and obtain external knowledge documents corresponding to the vectors;

[0034] A dimension filtering module is used to construct a multidimensional database with a tree structure according to the dimensions and measures in the multidimensional expression, and then perform dimension filtering on the multidimensional database to obtain a filtered dimension tree structure string;

[0035] A multidimensional expression sample retrieval module is used to calculate the similarity between vector data in the vector database and the user's natural language question using retrieval enhancement technology, filter a preset number of vectors, and obtain the question-multidimensional expression sample corresponding to the vector;

[0036] A multidimensional expression generation module is configured to input the user's natural language question, the external knowledge document, the filtered dimension tree structure string, the metric tree structure string, and the question-multidimensional expression sample into a prompt word template to obtain a complete prompt word, input the complete prompt word into a large language model, and generate an initial multidimensional expression;

[0037] a multidimensional expression checking module, configured to repeatedly execute step S6 using the large language model and the temperature parameter to generate a plurality of initial multidimensional expressions, form candidate multidimensional expressions based on the plurality of initial multidimensional expressions, perform syntax checking on the candidate multidimensional expressions, screen out executable multidimensional expressions, and group the selected executable multidimensional expressions according to execution results;

[0038] The natural language query module is used to reversely parse the executable multidimensional expression to obtain the rows, columns, pages and conditions of the query in the multidimensional expression, and complete the natural language query.

[0039] Optionally, the workflow of the document retrieval module specifically includes:

[0040] Encode the external knowledge in the data required by the multidimensional expression into an external knowledge document using an encoding model;

[0041] Encode the user's natural language question into a user question vector using an encoding model;

[0042] The cosine similarity between the external knowledge document and the user question vector is calculated, and the cosine similarity results are sorted in descending order, and a preset number of vectors are selected to obtain the external knowledge document corresponding to the screening vector.

[0043] Optionally, the workflow of the dimension filtering module specifically includes:

[0044] The user's natural language questions, external knowledge documents, and dimensional tree-structured strings in the multidimensional database are filled into prompt words and then input into the large language model to generate filtering results. Specifically:

[0045] The prompt word sequence of the prompt word is transformed into a vector representation through the embedding layer and position encoding to obtain the representation matrix of the prompt word sequence;

[0046] Based on the representation matrix, a self-attention mechanism is used to calculate the weights of the word units in the representation matrix;

[0047] Through the multi-head attention mechanism, different contextual information is captured from multiple subspaces to obtain the attention representation of each word;

[0048] After the attention representation of each word is input into the feedforward network for processing, the final representation of each word is obtained through the residual connection and normalization layer;

[0049] Input the final representation into the decoder to generate the probability distribution of the next word;

[0050] Use autoregressive method to gradually expand and generate the next word unit, and finally get the filtering result.

[0051] Optionally, the calculation content of inputting the output word-unit into the decoder to generate the probability distribution of the next word-unit includes:

[0052] P(y t |y <t ,X)=Softmax(h t W out +b out )

[0053] Among them, y t The output token generated for the current time step t, y ＜t is the output word sequence before time step t, X is the input word sequence, h t is the hidden state of the decoder at time step t, W out and b out are the output layer parameters.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] This paper discloses a multidimensional expression generation architecture for generating multidimensional expressions (MDX) that meet multidimensional database query requirements from natural language questions. This architecture covers the key links in the generation process, from data preparation to expression generation and verification, ensuring the accuracy and consistency of the output results. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 This is a method step diagram of a natural language query method for a multidimensional database according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0059] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0060] Example 1

[0061] A natural language query method for multidimensional databases, such as Figure 1 As shown, the method includes:

[0062] Step S1: Collect the user's natural language questions, and extract the data required for the multidimensional expression according to the natural language questions.

[0063] The required data includes: (1) Natural language questions: The natural language questions that users need to convert. (2) Measures and dimensions: The measures and dimensions of the cube corresponding to the user's questions. (3) External knowledge documents: These can be corporate documents or professional terminology documents. The documents are stored in the form of sentences or paragraphs and are used to explain and improve professional terminology that may appear in user questions. (4) Question-multidimensional expression examples: Examples are stored in the form of key-value pairs and are used to demonstrate the multidimensional expression during the generation process.

[0064] First, the present embodiment is further explained using the following example:

[0065] User question: Query the total profit of longan produced in Seattle in August 2023.

[0066] Cube: Fruit sales cube.

[0067] Metrics: gross profit, gross margin, collections, costs.

[0068] Dimensions: time dimension, origin dimension, product dimension, and customer dimension.

[0069] Knowledge document: 1. Seattle is located in Washington, USA; 2. Longan is another name for dried longan; 3. Sunshine Rose is a type of grape; 4. Chaoyang District is located in Beijing, China.

[0070] Question - Multidimensional Expressions Example:

[0071] 1. Get the total sales of all product categories in the "Sales Amount" dimension: SELECT {[Measures].[SalesAmount]} ON COLUMNS, [Product].[Category].MEMBERS ON ROWS FROM [Sales];

[0072] 2. Calculate the total sales amount for each product category in each quarter: SELECT {[Measures].[SalesAmount]} ON COLUMNS,[Time].[Quarter].MEMBERS ON ROWS FROM[Sales] WHERE ([Product].[Category].[All Products]);

[0073] 3. Return the top five product categories with the highest sales amount: SELECT TopCount([Product].[Category].MEMBERS,5,[Measures].[Sales Amount])ON ROWS,{[Measures].[Sales Amount]}ON COLUMNS FROM[Sales].

[0074] 4. Query the sales amount in 2023: SELECT {[Measures].[Sales Amount]} ON COLUMNS,[Product].[Category].MEMBERS ON ROWS FROM[Sales] WHERE ([Time].[Year].

[2023] ).

[0075] Step S2: Constructing a prompt word template, wherein the prompt word template includes roles, metadata, user questions, output formats, samples, and workflows.

[0076] The prompt word template contains multiple insertion slots for filling in the actual generated data. Specifically, the prompt word template mainly includes the following contents: (1) Generation role (Role): The role played by the large language model (LLM) in the generation process, helping the large model to generate more accurate and professional content. (2) Metadata (Metadata): The dimensions and measurements of the cube. (3) User questions: The questions that the user needs to convert. (4): Output format: The description of the output format to ensure the consistency of the large language model output. (5) Sample: Serves as a demonstration in the large language model generation process. (6) Workflow: The chain of thought (CoT) specifies the reasoning process of the large language model.

[0077] A simple example of a prompt word template:

[0078] ##Role

[0079] You are a multidimensional database expert focused on extracting the relevant top-level dimensions from user questions.

[0080] Metadata

[0081] -Dimensions:

[0082] {dimensions}

[0083] ##Question

[0084] {question}

[0085] ##Output

[0086] - Output format:

[0087] {"dimensions":["dimension1","dimension2",..]}

[0088] ##Workflow

[0089] 1. Extract key information related to the dimension from the user's question, considering keywords and descriptions that may indicate the dimension.

[0090] 2. Based on the dimension information provided in the metadata, find all the dimensions mentioned in the question to ensure comprehensive coverage.

[0091] 3. After reasoning according to the above process, select the highest level of the involved dimensions from {dimList} and return it in the format specified in Output.

[0092] Step S3: Use retrieval enhancement technology to perform similarity calculation on the external knowledge documents in the data required by the multidimensional expression, screen a preset number of vectors, and obtain the external knowledge documents corresponding to the vectors.

[0093] Retrieval-Augmented Generation (RAG) is used to extract relevant documents for the user's question. This step mainly uses RAG, a technology that can perform semantic matching retrieval. The specific operation process is as follows:

[0094] Step 3-1: Use the encoding model to encode external knowledge into a vector. The specific process is as follows:

[0095] Using the set {m1,m2,m3,...,m i} represents an external knowledge document, where m i Represents a document.

[0096] The encoding function E(·) is applied to each piece of text, converting it into an encoded representation:

[0097]

[0098] The result after encoding is: {y1,y2,y3,...,y i};

[0099] Step 3-2, encode the user's question into a vector z, and then compare the encoded user question vector with the document vector {y1,y2,y3,...,y i}Calculate the cosine similarity and get {c1,c2,c3,...,c i}, where the calculation formula for cosine similarity is:

[0100]

[0101] Step 3-3, find the three vectors with the largest cosine similarity, and then get their corresponding texts.

[0102] The following are the screening results corresponding to the questions in step S1:

[0103] 1. Seattle is located in Washington, USA;

[0104] 2. Longan is another name for dried longan;

[0105] 3. Sunshine Rose is a type of grape.

[0106] Step S4: construct a multidimensional database with a tree structure according to the dimensions and measures in the multidimensional expression, and then perform dimension filtering on the multidimensional database to obtain a filtered dimension tree structure character string.

[0107] In a multidimensional database, dimensions and measures are composed of a tree structure. In order to load dimensions and measures into prompt words, they need to be converted into corresponding tree structure strings. The structure can be expressed in the following format:

[0108] Dimension: Dimension(member1(..),member2(..),..);

[0109] Measure: Measure(member1(..),member2(..),..);

[0110] Since the depth and width of the tree structure of dimensions and measures are uncertain, the tree structure string may be too long. Therefore, it is necessary to filter the dimensions to ensure that the generation of multidimensional expressions does not fail due to the prompt word being too long. The specific process of dimension filtering is as follows:

[0111] Step 4-1: Load the following content into the prompt word slot of dimension filtering:

[0112] The question entered by the user; the three question-related documents found in step 3; the dimension tree structure string constructed in step 4.

[0113] Step 4-2: Submit the filled prompt words to the large language model to generate filtering results. The generation process of the large language model is as follows:

[0114] Input representation: Assume that the input prompt word sequence is: X = {x1, x2, ..., x n}, where xi represents the i-th token in the input sequence. The input token is transformed into a vector representation through the embedding layer and positional encoding:

[0115] E i =Embed(x i )+PosEnc(i)

[0116] The representation matrix of the input sequence is: E=[E1,E2,...,E n ].

[0117] Self-Attention Mechanism: The self-attention mechanism calculates the weight of each word in the input sequence. The query (Q), key (K), and value (V) vectors are obtained by linear transformation:

[0118] Q=EW Q , K=EW K , V=EW v ;

[0119] Among them, W Q , W K , WV is a trainable parameter matrix. The attention weight is calculated by dot product attention:

[0120]

[0121] Among them, d k Is the dimension of the key vector, used to scale the dot product. Multi-Head Attention Mechanism: Multi-Head Attention captures information from different subspaces through parallel computation of multiple attention heads:

[0122] MultiHead(Q,K,V)=Concat(head1,...,head h )W O

[0123] Each attention head is calculated as:

[0124] head i =Attention(Q i , K i , V i )

[0125] Where W0 is the output transformation matrix.

[0126] Feed-Forward Network (FFN): The attention output of each token is processed through a feed-forward neural network:

[0127] FFN(x)=ReLU(xW1+b1)W2+b2

[0128] Among them, W1, W2, b1, b2 are trainable parameters.

[0129] Layer Normalization & Residual Connection: Add residual connection and layer normalization to each layer:

[0130] Output=LayerNorm(x+SubLayer(x))

[0131] Decoding and Generation: For generation tasks, the decoder uses the previous output token as input and generates a probability distribution for the next token:

[0132] P(y t |y<t ,X)=Softmax(h t W out +b out )

[0133] Among them, y t The output token generated for the current time step t, y ＜t is the output word sequence before time step t, X is the input word sequence, h t is the hidden state of the decoder at time step t, W out and b out are the output layer parameters.

[0134] Recursive generation process: The generated sequence is expanded in an autoregressive manner:

[0135] y t+1 =argmax(P(y t |y ＜t ,X))

[0136] Until the end word [EOS] is generated.

[0137] In step 4-3, since the results generated by the large language model may contain dimension sub-levels or hallucinations, that is, incorrect dimensions, the filtered dimensions need to be post-processed. This involves extracting the highest level of the dimension and checking whether the dimension name matches the actual dimension.

[0138] The following are the results of the dimension filtering in step S1:

[0139] Time dimension, origin dimension, and product dimension.

[0140] Step S5. Use retrieval enhancement technology to calculate the similarity between the vector data in the vector database and the user's natural language question, filter a preset number of vectors, and obtain question-multidimensional expression samples corresponding to the vectors.

[0141] The search enhancement technology is used to find the three most similar questions corresponding to the user's question in the vector database and obtain their corresponding multidimensional expressions. The specific process is as follows:

[0142] Step 5-1, use the encoding model to encode the example question into a vector {x1,x2,x3,...,x n}.

[0143] Step 5-2, encode the user's question into a vector z, and then compare the encoded user question vector with the document vector {y1,y2,y3,...,y i}Calculate the cosine similarity and get {c1,c2,c3,...,ci}.

[0144] In step 5-3, find the three vectors with the largest cosine similarity, and then obtain their corresponding problem - multidimensional expression example.

[0145] The following are the results of the multidimensional expression filter in step S1:

[0146] 1. Get the total sales of all product categories in the "Sales Amount" dimension: SELECT {[Measures].[SalesAmount]} ON COLUMNS, [Product].[Category].MEMBERS ON ROWS FROM [Sales];

[0147] 2. Calculate the total sales amount for each product category in each quarter: SELECT {[Measures].[SalesAmount]} ON COLUMNS,[Time].[Quarter].MEMBERS ON ROWS FROM[Sales] WHERE ([Product].[Category].[All Products]);

[0148] 3. Query the sales amount in 2023: SELECT {[Measures].[Sales Amount]} ON COLUMNS,[Product].[Category].MEMBERS ON ROWS FROM[Sales] WHERE ([Time].[Year].

[2023] ).

[0149] Step S6. Input the user's natural language question, external knowledge document, filtered dimension tree structure string, metric tree structure string and question-multidimensional expression sample into the prompt word template to obtain a complete prompt word, input the complete prompt word into the large language model, and generate an initial multidimensional expression.

[0150] After the six preparatory steps above, start generating multidimensional expressions. The specific steps are as follows:

[0151] In step 6-1, enter the following content into the prompt slot for generating the multidimensional expression:

[0152] 1) User input issues.

[0153] 2) Documents related to the three issues found in step 3.

[0154] 3) The dimension tree structure string corresponding to the dimension filtered out in step 5.

[0155] 4) Measure the tree structure string.

[0156] 5) The three questions obtained in step 6 - multidimensional expression examples.

[0157] In step 6-2, the prompt word is submitted to the large language model to generate the corresponding result. The detailed generation process of the large language model has been given in step 4-2 and will not be repeated here.

[0158] Step 6-3. Fill the dimensions of the generated multidimensional expression, that is, fill the filtered dimensions with default values ​​into the WHERE clause of the multidimensional expression.

[0159] Through the above sub-steps, a preliminary multidimensional expression has been generated.

[0160] The following is the multidimensional expression corresponding to the problem in S1:

[0161] SELECT {[Measurement].[Gross Profit]} ON COLUMNS, {[Time].

[2023] .[Third Quarter].[August], [Origin].[United States].[Washington].[Seattle], [Product].[Grade A Fruit].[Imported Longan]} ON ROWS FROM [Fruit Sales] WHERE ([Customer].[Customer]); where [Customer] represents all contents of the customer dimension.

[0162] Step S7. Repeat step S6 using the large language model and the temperature parameter to generate several initial multidimensional expressions, form candidate multidimensional expressions based on the several initial multidimensional expressions, perform syntax check on the candidate multidimensional expressions, filter out executable multidimensional expressions, and group the filtered executable multidimensional expressions according to their execution results.

[0163] Specifically, in this embodiment, two strategies are used to generate multidimensional expressions: one uses the same large model at a higher temperature parameter to generate multiple initial multidimensional expressions; the other uses different large models at lower temperature parameters to generate multiple initial multidimensional expressions. Regarding the temperature parameter, in this embodiment, a larger value for the parameter results in a greater randomness in the large model generation. After grouping, a multidimensional expression is selected from the group with the most group members as the final result.

[0164] The generated multidimensional expression may have syntax problems or hallucinations in the tree structure. Therefore, it is necessary to perform syntax and hallucination checks on the generated multidimensional expression to ensure that the syntax is correct and that the dimensions and measures conform to the metadata of the multidimensional database that needs to be queried.

[0165] Because the large language model is a probabilistic model, its generated results are subject to significant randomness. This randomness is controlled by the large language model's temperature parameter (a value range of 0-1). When the temperature is 0, the large language model adopts a greedy strategy, always selecting the word with the highest probability as the prediction value. As the temperature approaches 1, the large language model randomly selects a high-probability prediction item with increasing randomness. To improve the accuracy of the multidimensional expressions generated by the large language model, one or a combination of the following two strategies is used to generate multiple results: The large language model generates multiple results at a higher temperature. Multiple different large language models are used to generate multiple results corresponding to user questions.

[0166] A consistency algorithm is used to select the optimal solution from the multiple generated multidimensional expressions. The consistency algorithm follows these steps: All generated multidimensional expressions are subjected to the syntax and hallucination checks described in step 8, filtering out those with syntax or hallucination issues and retaining executable multidimensional expressions. All filtered multidimensional expressions from the previous step are executed, and the results are grouped into groups, such as Group A and Group B.

[0167] Select one result from the group with the largest number of members.

[0168] Step S8. Reverse-parse the executable multidimensional expression to obtain the rows, columns, pages, and conditions of the query, completing the natural language query. After the multidimensional expression is successfully generated, reverse-parse it to extract the rows, columns, pages, and conditions of the query. The extracted content is then transmitted to the front-end, where it is processed and formed into a worksheet.

[0169] Example 2

[0170] A natural language query system for a multidimensional database, the system comprising:

[0171] The data collection module is used to collect natural language questions from users and extract data required for multidimensional expressions based on the natural language questions.

[0172] The required data includes: (1) Natural language questions: The natural language questions that users need to convert. (2) Measures and dimensions: The measures and dimensions of the cube corresponding to the user's questions. (3) External knowledge documents: These can be corporate documents or professional terminology documents. The documents are stored in the form of sentences or paragraphs and are used to explain and improve professional terminology that may appear in user questions. (4) Question-multidimensional expression examples: Examples are stored in the form of key-value pairs and are used to demonstrate the multidimensional expression during the generation process.

[0173] A prompt word template is created for constructing the prompt word template, wherein the prompt word template includes a role, metadata, a user question, an output format, a sample, and a workflow.

[0174] The prompt word template contains multiple insertion slots for filling in the actual generated data. Specifically, the prompt word template mainly includes the following contents: (1) Generation role (Role): The role played by the large language model (LLM) in the generation process, helping the large model to generate more accurate and professional content. (2) Metadata (Metadata): The dimensions and measurements of the cube. (3) User questions: The questions that the user needs to convert. (4): Output format: The description of the output format to ensure the consistency of the large language model output. (5) Sample: Serves as a demonstration in the large language model generation process. (6) Workflow: The chain of thought (CoT) specifies the reasoning process of the large language model.

[0175] The document retrieval module is used to use retrieval enhancement technology to perform similarity calculation on the external knowledge documents in the data required by the multidimensional expression, screen a preset number of vectors, and obtain the external knowledge documents corresponding to the vectors.

[0176] Retrieval-Augmented Generation (RAG) is used to extract relevant documents for the user's question. This step mainly uses RAG, a technology that can perform semantic matching retrieval. The specific operation process is as follows:

[0177] Use the encoding model to encode external knowledge into a vector. The specific process is as follows:

[0178] Using the set {m1,m2,m3,...,m i} represents an external knowledge document, where m i Represents a document.

[0179] The encoding function E(·) is applied to each piece of text, converting it into an encoded representation:

[0180]

[0181] The result after encoding is: {y1,y2,y3,...,y i};

[0182] Encode the user's question into a vector z, and then compare the encoded user question vector with the document vector {y1,y2,y3,...,y i}Calculate the cosine similarity and get {c1,c2,c3,...,ci}, where the calculation formula for cosine similarity is:

[0183]

[0184] Find the three vectors with the largest cosine similarity and then get their corresponding text.

[0185] The dimension filtering module is used to construct a multidimensional database with a tree structure according to the dimensions and measures in the multidimensional expression, and then perform dimension filtering on the multidimensional database to obtain a filtered dimension tree structure string.

[0186] In a multidimensional database, dimensions and measures are composed of a tree structure. In order to load dimensions and measures into prompt words, they need to be converted into corresponding tree structure strings. The structure can be expressed in the following format:

[0187] Dimension: Dimension(member1(..),member2(..),..); Measure: Measure(member1(..),member2(..),..);

[0188] Since the depth and width of the tree structure of dimensions and measures are uncertain, the tree structure string may be too long. Therefore, it is necessary to filter the dimensions to ensure that the generation of multidimensional expressions does not fail due to the prompt word being too long. The specific process of dimension filtering is as follows:

[0189] In the dimension filter prompt slot, add the following content:

[0190] The question entered by the user; the three question-related documents found in step 3; the dimension tree structure string constructed in step 4.

[0191] Submit the filled prompt words to the large language model to generate the filtering results. The generation process of the large language model is as follows:

[0192] Input representation: Assume that the input prompt word sequence is: X = {x1, x2, ..., x n}, where x i Represents the i-th token in the input sequence. The input token is transformed into a vector representation through the embedding layer and positional encoding:

[0193] E i =Embed(x i )+PosEnc(i)

[0194] The representation matrix of the input sequence is: E = [E1, E2, ..., E n ].

[0195] Self-Attention Mechanism: The self-attention mechanism calculates the weight of each word in the input sequence. The query (Q), key (K), and value (V) vectors are obtained by linear transformation:

[0196] Q=EW Q , K=EW K , V=EW Ｖ ;

[0197] Among them, W Q , W K , WV is a trainable parameter matrix. The attention weight is calculated by dot product attention:

[0198]

[0199] Among them, d k Is the dimension of the key vector, used to scale the dot product. Multi-Head Attention Mechanism: Multi-Head Attention captures information from different subspaces through parallel computation of multiple attention heads:

[0200] MultiHead(Q,K,V)=Concat(head1,...,head h )W O

[0201] Each attention head is calculated as:

[0202] head i =Attention(Q i , K i , V i )

[0203] Where W0 is the output transformation matrix.

[0204] Feed-Forward Network (FFN): The attention output of each token is processed through a feed-forward neural network:

[0205] FFN(x)=ReLU(xW1+b1)W2+b2

[0206] Among them, W1, W2, b1, b2 are trainable parameters.

[0207] Layer Normalization & Residual Connection: Add residual connection and layer normalization to each layer:

[0208] Output=LayerNorm(x+SubLayer(x))

[0209] Decoding and Generation: For generation tasks, the decoder uses the previous output token as input and generates a probability distribution for the next token:

[0210] P(y t |y ＜t ,X)=Softmax(h t W out +b out )

[0211] Among them, y t The output token generated for the current time step t, y ＜t is the output word sequence before time step t, X is the input word sequence, h t is the hidden state of the decoder at time step t, W out and b out are the output layer parameters.

[0212] Recursive generation process: The generated sequence is expanded in an autoregressive manner:

[0213] y t+1 =argmax(P(y t |y <t , X))

[0214] Until the end word [EOS] is generated.

[0215] Because the results generated by large language models may contain sub-level dimensions or hallucinations, that is, incorrect dimensions, it is necessary to post-process the filtered dimensions, that is, extract the highest level of the dimension and check whether the dimension name matches the actual dimension.

[0216] A multidimensional expression sample retrieval module is used to calculate the similarity between vector data in the vector database and the user's natural language question using retrieval enhancement technology, filter a preset number of vectors, and obtain the question-multidimensional expression sample corresponding to the vector;

[0217] The search enhancement technology is used to find the three most similar questions corresponding to the user's question in the vector database and obtain their corresponding multidimensional expressions. The specific process is as follows:

[0218] Use the encoding model to encode the example question into a vector {x1, x2, x3, ..., x n}.

[0219] Encode the user's question into a vector z, and then compare the encoded user question vector with the document vector {y1, y2, y3, ..., y i}Calculate the cosine similarity and get {c1, c2, c3, ..., c i}.

[0220] Find the three vectors with the largest cosine similarity and then get their corresponding problem - multidimensional expression example.

[0221] The multidimensional expression generation module is used to input the user's natural language question, external knowledge document, filtered dimension tree structure string, measure tree structure string and question-multidimensional expression sample into the prompt word template to obtain a complete prompt word, input the complete prompt word into the large language model, and generate an initial multidimensional expression.

[0222] After the six preparatory steps above, start generating multidimensional expressions. The specific steps are as follows:

[0223] In the prompt slot for generating multidimensional expressions, enter the following:

[0224] 1) User input issues.

[0225] 2) Documents related to the three issues found in step 3.

[0226] 3) The dimension tree structure string corresponding to the dimension filtered out in step 5.

[0227] 4) Measure the tree structure string.

[0228] 5) The three questions obtained in step 6 - multidimensional expression examples.

[0229] Submit the prompt word to the large language model to generate the corresponding result. The detailed generation process of the large language model has been given in the previous article and will not be repeated here.

[0230] Perform dimension filling on the generated multidimensional expression, that is, fill the filtered dimensions with default values ​​into the WHERE clause of the multidimensional expression.

[0231] Through the above sub-steps, a preliminary multidimensional expression has been generated.

[0232] The multidimensional expression checking module is configured to repeatedly execute the multidimensional expression generating module using the large language model and the temperature parameter to generate a number of initial multidimensional expressions, form candidate multidimensional expressions based on the initial multidimensional expressions, perform syntax checking on the candidate multidimensional expressions, screen out executable multidimensional expressions, and group the screened executable multidimensional expressions according to their execution results.

[0233] Specifically, in this embodiment, two strategies are used to generate multidimensional expressions: one uses the same large model at a higher temperature parameter to generate multiple initial multidimensional expressions; the other uses different large models at lower temperature parameters to generate multiple initial multidimensional expressions. Regarding the temperature parameter, in this embodiment, a larger value for the parameter results in a greater randomness in the large model generation. After grouping, a multidimensional expression is selected from the group with the most group members as the final result.

[0234] The generated multidimensional expression may have syntax problems or hallucinations in the tree structure. Therefore, it is necessary to perform syntax and hallucination checks on the generated multidimensional expression to ensure that the syntax is correct and that the dimensions and measures conform to the metadata of the multidimensional database that needs to be queried.

[0235] Because the large language model is a probabilistic model, its generated results are subject to significant randomness. This randomness is controlled by the large language model's temperature parameter (a value range of 0-1). When the temperature is 0, the large language model adopts a greedy strategy, always selecting the word with the highest probability as the prediction value. As the temperature approaches 1, the large language model randomly selects a high-probability prediction item with increasing randomness. To improve the accuracy of the multidimensional expressions generated by the large language model, one or a combination of the following two strategies is used to generate multiple results: The large language model generates multiple results at a higher temperature. Multiple different large language models are used to generate multiple results corresponding to user questions.

[0236] A consistency algorithm is used to select the optimal solution from the multiple generated multidimensional expressions. The consistency algorithm follows these steps: All generated multidimensional expressions are subjected to the syntax and hallucination checks described in step 8, filtering out those with syntax or hallucination issues and retaining executable multidimensional expressions. All filtered multidimensional expressions from the previous step are executed, and the results are grouped into groups, such as Group A and Group B.

[0237] Select one result from the group with the largest number of members.

[0238] The natural language query module is used to reversely parse the executable multidimensional expression to obtain the rows, columns, pages and conditions of the query in the multidimensional expression, and complete the natural language query.

[0239] After the multidimensional expression is successfully generated, reverse parsing is performed to extract the rows, columns, pages and conditions queried in the multidimensional expression, and then the extracted content is passed to the front end to form a worksheet after processing.

[0240] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A natural language query method for a multidimensional database, characterized in that: The method comprises: Step S1. Collecting the user's natural language questions, and extracting the data required for the multidimensional expression according to the natural language questions; Step S2. Constructing a prompt word template, wherein the prompt word template includes roles, metadata, user questions, output formats, samples, and workflows; Step S3. Using retrieval enhancement technology to perform similarity calculation on the external knowledge documents in the data required by the multidimensional expression, screening a preset number of vectors, and obtaining the external knowledge documents corresponding to the vectors; Step S4. Constructing a tree-structured multidimensional database based on the dimensions and metrics in the multidimensional expression, and then performing dimension filtering on the multidimensional database to obtain a filtered dimension tree structure string and a filtered metric tree structure string; Step S5. Using retrieval enhancement technology, calculate the similarity between the vector data in the vector database and the user's natural language question, filter a preset number of vectors, and obtain question-multidimensional expression samples corresponding to the vectors; Step S6. Input the user's natural language question, the external knowledge document, the filtered dimension tree structure string, the filtered metric tree structure string, and the question-multidimensional expression sample into the prompt word template to obtain a complete prompt word, input the complete prompt word into the large language model, and generate an initial multidimensional expression; Step S7. Repeat step S6 using the large language model and the temperature parameter to generate several initial multidimensional expressions, form candidate multidimensional expressions based on the several initial multidimensional expressions, perform syntax checking on the candidate multidimensional expressions, screen out executable multidimensional expressions, and group the selected executable multidimensional expressions according to their execution results. Step S8: After reverse parsing the executable multidimensional expression, the rows, columns, pages and conditions of the query in the multidimensional expression are obtained, and the natural language query is completed.

2. The natural language query method for a multidimensional database according to claim 1, characterized in that: In step S3, the search enhancement technology is used to calculate the similarity of the external knowledge documents in the data required by the multidimensional expression, and a preset number of vectors are screened to obtain the content of the external knowledge documents corresponding to the vectors, specifically including: Encode the external knowledge in the data required by the multidimensional expression into an external knowledge document using an encoding model; Encode the user's natural language question into a user question vector using an encoding model; The cosine similarity between the external knowledge document and the user question vector is calculated, and the cosine similarity results are sorted in descending order, and a preset number of vectors are selected to obtain the external knowledge document corresponding to the screening vector.

3. The natural language query method for a multidimensional database according to claim 1, characterized in that: In step S4, a multidimensional database with a tree structure is constructed based on the dimensions and metrics in the multidimensional expression, and then the multidimensional database is dimensionally filtered to obtain the filtered dimensional tree structure string, which specifically includes: The user's natural language questions, external knowledge documents, and dimensional tree-structured strings in the multidimensional database are filled into prompt words and then input into the large language model to generate filtering results. Specifically: The prompt word sequence of the prompt word is transformed into a vector representation through the embedding layer and position encoding to obtain the representation matrix of the prompt word sequence; Based on the representation matrix, a self-attention mechanism is used to calculate the weights of the word units in the representation matrix; Through the multi-head attention mechanism, different contextual information is captured from multiple subspaces to obtain the attention representation of each word; After the attention representation of each word is input into the feedforward network for processing, the final representation of each word is obtained through the residual connection and normalization layer; Input the final representation into the decoder to generate the probability distribution of the next word; Use autoregressive method to gradually expand and generate the next word unit, and finally get the filtering result.

4. The natural language query method for a multidimensional database according to claim 3, characterized in that: The calculation content of inputting the final representation of the word into the decoder to generate the probability distribution of the next word includes: P(y t |y ＜t ,X)=Softmax(h t W out +b out ) Among them, y t The output token generated for the current time step t, y ＜t is the output word sequence before time step t, X is the input word sequence, h t is the hidden state of the decoder at time step t, W out and b out are the output layer parameters.

5. A natural language query system for a multidimensional database, the query system being configured to use the query method according to any one of claims 1 to 4, characterized in that: The system comprises: A data collection module is used to collect natural language questions from users and extract data required for multidimensional expressions based on the natural language questions; Creating a prompt word template for constructing a prompt word template, wherein the prompt word template includes a role, metadata, a user question, an output format, a sample, and a workflow; A document retrieval module is used to use retrieval enhancement technology to perform similarity calculation on external knowledge documents in the data required by the multidimensional expression, screen a preset number of vectors, and obtain external knowledge documents corresponding to the vectors; A dimension filtering module is used to construct a multidimensional database with a tree structure according to the dimensions and measures in the multidimensional expression, and then perform dimension filtering on the multidimensional database to obtain a filtered dimension tree structure string and a filtered measure tree structure string; A multidimensional expression sample retrieval module is used to calculate the similarity between vector data in the vector database and the user's natural language question using retrieval enhancement technology, filter a preset number of vectors, and obtain the question-multidimensional expression sample corresponding to the vector; A multidimensional expression generation module is configured to input the user's natural language question, the external knowledge document, the filtered dimension tree structure string, the filtered metric tree structure string, and the question-multidimensional expression sample into a prompt word template to obtain a complete prompt word, input the complete prompt word into a large language model, and generate an initial multidimensional expression; a multidimensional expression checking module, configured to repeatedly execute step S6 using the large language model and the temperature parameter to generate a plurality of initial multidimensional expressions, form candidate multidimensional expressions based on the plurality of initial multidimensional expressions, perform syntax checking on the candidate multidimensional expressions, screen out executable multidimensional expressions, and group the selected executable multidimensional expressions according to execution results; The natural language query module is used to reversely parse the executable multidimensional expression to obtain the rows, columns, pages and conditions of the query in the multidimensional expression, and complete the natural language query.

6. The natural language query system for a multidimensional database according to claim 5, characterized in that: The workflow of the document retrieval module specifically includes: Encode the external knowledge in the data required by the multidimensional expression into an external knowledge document using an encoding model; Encode the user's natural language question into a user question vector using an encoding model; The cosine similarity between the external knowledge document and the user question vector is calculated, and the cosine similarity results are sorted in descending order, and a preset number of vectors are selected to obtain the external knowledge document corresponding to the screening vector.

7. The natural language query system for a multidimensional database according to claim 5, characterized in that: The workflow of the dimension filtering module specifically includes: The user's natural language questions, external knowledge documents, and dimensional tree-structured strings in the multidimensional database are filled into prompt words and then input into the large language model to generate filtering results. Specifically: The prompt word sequence of the prompt word is transformed into a vector representation through the embedding layer and position encoding to obtain the representation matrix of the prompt word sequence; Based on the representation matrix, a self-attention mechanism is used to calculate the weights of the word units in the representation matrix; Through the multi-head attention mechanism, different contextual information is captured from multiple subspaces to obtain the attention representation of each word; After the attention representation of each word is input into the feedforward network for processing, the final representation of each word is obtained through the residual connection and normalization layer; Input the final representation into the decoder to generate the probability distribution of the next word; Use autoregressive method to gradually expand and generate the next word unit, and finally get the filtering result.

8. The natural language query system for a multidimensional database according to claim 7, characterized in that: The calculation content of the probability distribution of the next word generated by inputting the final representation into the decoder includes: P(y t |y ＜t ,X)=Softmax(h t W out +b out ) Among them, y t The output token generated for the current time step t, y ＜t is the output word sequence before time step t, X is the input word sequence, h t is the hidden state of the decoder at time step t, W out and b out are the output layer parameters.

Images