Natural language query method and system for multi-dimensional database

Abstract

The invention discloses a natural language query method and system for a multi-dimensional database. The method comprises the following steps: collecting a natural language problem of a user and extracting data required by a multi-dimensional expression; constructing a cue word template; performing similarity calculation on vectors of the external knowledge documents required by the multi-dimensional expression by using a retrieval enhancement technology to obtain the external knowledge documents corresponding to the vectors; constructing a multi-dimensional database of a tree structure according to the dimensionality and the measurement, and then performing dimensionality filtering on the multi-dimensional database to obtain a filtered dimensionality tree structure character string; calculating the similarity between the vector data of the vector database and the natural language problem of the user by using a retrieval enhancement technology to obtain a problem-multi-dimensional expression pattern sample corresponding to the vector; inputting the content as a cue word into a large model to generate a multi-dimensional expression; performing grammar and illusion inspection on the multi-dimensional expression to obtain an executable multi-dimensional expression; and performing reverse analysis to obtain queried rows, columns, pages and conditions in the multi-dimensional expression, and completing natural language query.

Description

Technical Field

[0001] The present invention relates to the technical field of database and natural language processing, and in particular to a natural language query method and system for a multidimensional database. Background Art

[0002] Multidimensional databases are widely used in business intelligence (BI), data warehouses, and decision support systems (DSS) due to their efficient data storage and query capabilities. However, similar to the query language of relational databases, the query language of multidimensional databases usually needs to be written by professionals, which creates a certain usage threshold for ordinary users. By implementing a natural language query interface, the difficulty of using multidimensional databases can be significantly reduced. In the process of developing the natural language query function of multidimensional databases, the following technologies played a key role:

[0003] Large Language Model (LLM) Generation Technology: This technology plays a core role in the process of generating multidimensional expressions (MDX). The most critical step is to accurately process the natural language questions raised by users and parse complex language requirements into multidimensional expressions. To achieve this goal, the model needs to have the following capabilities: first, it must have strong natural language understanding capabilities and be able to accurately capture the core intent, contextual semantics, and implicit information in user questions; second, it must be able to convert the results of understanding into structured expressions that conform to the query logic of multidimensional databases, ensuring that the generated multidimensional expressions are logically correct and feasible to execute. In addition, the model must be able to flexibly adapt to different user expressions and be able to generate customized query results based on specific instructions. This capability requires not only advanced natural language processing technology, but also in-depth modeling and application capabilities for multidimensional database syntax and operation logic, so as to achieve seamless connection from natural language to precise multidimensional expression generation.

[0004] Retrieval-augmented generation (RAG) technology plays an important role in the generation of multidimensional expressions (MDX). User questions often contain professional vocabulary or complex expressions, which may cause the large language model to be unable to accurately understand their intentions. At the same time, if all knowledge documents are directly passed in, the accuracy of the generated results may be affected by information redundancy. To solve this problem, RAG technology provides an efficient solution. It converts user questions into vectors of fixed length and semantically matches them with knowledge documents in the vector database to retrieve the documents most relevant to the user's questions. These documents can provide clear background support for user questions and improve the model's understanding ability. In addition, retrieval-augmented generation technology can also be applied to the example search process of generating multidimensional expressions. By locating multiple multidimensional expression examples that are closest to the semantics of user questions, retrieval-augmented generation can provide more accurate context information for the large language model, thereby generating multidimensional expressions that are more in line with user needs. This technology combines the advantages of retrieval and generation, greatly improving the professionalism and accuracy of the generated results, especially showing significant value when dealing with complex problems.

[0005] Chain of Thought (CoT) technology plays a key role in the generation of multidimensional expressions. If the user does not clearly specify the workflow of the large language model, the model may generate results in a random manner, lacking systematic analysis steps. In this case, the generated multidimensional expression may be omitted or erroneous, making it difficult to meet the actual needs of users. Chain of Thought technology guides the model to gradually decompose the problem and analyze the details by clearly specifying the reasoning method of the large language model, ensuring that each step is logically verified. It can help the model focus on the key points in the generation process, thereby reducing randomness and potential error risks. By introducing the chain of thought, the large language model can not only improve the reasoning ability, but also generate more accurate and logically rigorous multidimensional expressions, providing users with higher quality solutions. Summary of the invention

[0006] The present invention provides a natural language query method for a multidimensional database, the method comprising:

[0007] Step S1. Collecting natural language questions from users, and extracting data required for multidimensional expressions according to the natural language questions;

[0008] Step S2. construct a prompt word template, wherein the prompt word template includes roles, metadata, user questions, output formats, samples and workflows;

[0009] Step S3. Use the search enhancement technology to calculate the similarity of the external knowledge documents in the data required by the multidimensional expression, screen a preset number of vectors, and obtain the external knowledge documents corresponding to the vectors;

[0010] Step S4. construct a multidimensional database with a tree structure according to the dimensions and measures in the multidimensional expression, and then perform dimension filtering on the multidimensional database to obtain a filtered dimension tree structure string;

[0011] Step S5. Use the search enhancement technology to calculate the similarity between the vector data in the vector database and the user's natural language question, filter a preset number of vectors, and obtain the question-multidimensional expression sample corresponding to the vector;

[0012] Step S6. Input the user's natural language question, external knowledge document, filtered dimension tree structure string, metric tree structure string and question-multidimensional expression sample into the prompt word template to obtain a complete prompt word, input the complete prompt word into the large language model, and generate an initial multidimensional expression;

[0013] Step S7. Repeat step S using the large language model and the temperature parameter to generate a number of initial multidimensional expressions, form candidate multidimensional expressions based on the initial multidimensional expressions, perform syntax check on the candidate multidimensional expressions, filter out executable multidimensional expressions, and group according to the execution results of the filtered executable multidimensional expressions;

[0014] Step S8: After reverse parsing the executable multidimensional expression, the rows, columns, pages and conditions of the query in the multidimensional expression are obtained, and the natural language query is completed.

[0015] Optionally, in step S3, using the search enhancement technology to perform similarity calculation on the external knowledge documents in the data required by the multidimensional expression, screening a preset number of vectors, and obtaining the content of the external knowledge documents corresponding to the vectors specifically includes:

[0016] Encode the external knowledge in the data required by the multidimensional expression into an external knowledge document by using the encoding model;

[0017] Encode the user's natural language question into a user question vector using an encoding model;

[0018] The cosine similarity between the external knowledge document and the user question vector is calculated, and the cosine similarity results are sorted in descending order, and a preset number of vectors are selected to obtain the external knowledge document corresponding to the screening vector.

[0019] Optionally, in step S4, a multidimensional database with a tree structure is constructed according to the dimensions and measures in the multidimensional expression, and then the multidimensional database is dimensionally filtered to obtain the content of the filtered dimensional tree structure string, which specifically includes:

[0020] The user's natural language questions, external knowledge documents, and dimensional tree-structured strings in the multidimensional database are filled into prompt words and then input into the large language model to generate filtering results, specifically:

[0021] The prompt word sequence of the prompt word is transformed into a vector representation through an embedding layer and position encoding to obtain a representation matrix of the prompt word sequence;

[0022] Based on the representation matrix, a self-attention mechanism is used to calculate the weights of the word units in the representation matrix;

[0023] Through the multi-head attention mechanism, different context information is captured from multiple subspaces to obtain the attention representation of each word.

[0024] After the attention representation of each word is input into the feedforward network for processing, the final representation of each word is obtained through residual connection and normalization layer;

[0025] Input the final representation into a decoder to generate a probability distribution of the next word unit;

[0026] Use autoregressive method to gradually expand and generate the next word unit, and finally get the filtering result.

[0027] Optionally, the calculation content of inputting the final representation of the word unit into the decoder to generate the probability distribution of the next word unit includes:

[0028] P(y t |y <t , X) = Softmax(h t W out +b out )

[0029] Among them, y t The output token generated for the current time step t, y ＜t is the output word sequence before time step t, X is the input word sequence, h t is the hidden state of the decoder at time step t, W out and b out are the output layer parameters.

[0030] The present invention also provides a natural language query system for a multidimensional database, the system comprising:

[0031] A data collection module, used to collect natural language questions from users and extract data required for multidimensional expressions based on the natural language questions;

[0032] Creating a prompt word template, used to construct the prompt word template, wherein the prompt word template includes roles, metadata, user questions, output formats, samples, and workflows;

[0033] A document retrieval module, used to use retrieval enhancement technology to perform similarity calculation on the external knowledge documents in the data required by the multidimensional expression, screen a preset number of vectors, and obtain the external knowledge documents corresponding to the vectors;

[0034] A dimension filtering module is used to construct a multidimensional database with a tree structure according to the dimensions and measures in the multidimensional expression, and then perform dimension filtering on the multidimensional database to obtain a filtered dimension tree structure string;

[0035] A multidimensional expression sample retrieval module is used to calculate the similarity between the vector data in the vector database and the user's natural language question using retrieval enhancement technology, filter a preset number of vectors, and obtain the question-multidimensional expression sample corresponding to the vector;

[0036] A multidimensional expression generation module is used to input the user's natural language question, the external knowledge document, the filtered dimension tree structure string, the metric tree structure string and the question-multidimensional expression sample into the prompt word template to obtain a complete prompt word, input the complete prompt word into the large language model, and generate an initial multidimensional expression;

[0037] a multidimensional expression checking module, configured to repeatedly execute step S6 using the large language model and the temperature parameter to generate a number of initial multidimensional expressions, form candidate multidimensional expressions based on the initial multidimensional expressions, perform syntax checking on the candidate multidimensional expressions, filter out executable multidimensional expressions, and group according to execution results of the filtered executable multidimensional expressions;

[0038] The natural language query module is used to reversely parse the executable multidimensional expression to obtain the rows, columns, pages and conditions queried in the multidimensional expression, and complete the natural language query.

[0039] Optionally, the workflow of the document retrieval module specifically includes:

[0040] Encode the external knowledge in the data required by the multidimensional expression into an external knowledge document by using the encoding model;

[0041] Encode the user's natural language question into a user question vector using an encoding model;

[0042] The cosine similarity between the external knowledge document and the user question vector is calculated, and the cosine similarity results are sorted in descending order, and a preset number of vectors are selected to obtain the external knowledge document corresponding to the screening vector.

[0043] Optionally, the workflow of the dimension filtering module specifically includes:

[0044] The user's natural language questions, external knowledge documents, and dimensional tree-structured strings in the multidimensional database are filled into prompt words and then input into the large language model to generate filtering results, specifically:

[0045] The prompt word sequence of the prompt word is transformed into a vector representation through an embedding layer and position encoding to obtain a representation matrix of the prompt word sequence;

[0046] Based on the representation matrix, a self-attention mechanism is used to calculate the weights of the word units in the representation matrix;

[0047] Through the multi-head attention mechanism, different context information is captured from multiple subspaces to obtain the attention representation of each word.

[0048] After the attention representation of each word is input into the feedforward network for processing, the final representation of each word is obtained through residual connection and normalization layer;

[0049] Input the final representation into a decoder to generate a probability distribution of the next word unit;

[0050] Use autoregressive method to gradually expand and generate the next word unit, and finally get the filtering result.

[0051] Optionally, the calculation content of inputting the output word unit into the decoder to generate the probability distribution of the next word unit includes:

[0052] P(y t |y <t , X) = Softmax(h t W out +b out )

[0053] Among them, y t The output token generated for the current time step t, y ＜t is the output word sequence before time step t, X is the input word sequence, h t is the hidden state of the decoder at time step t, W out and b out are the output layer parameters.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] The present invention discloses a multidimensional expression generation architecture, which is used to generate multidimensional expressions (MDX) that meet the multidimensional database query requirements from natural language questions. The architecture covers the key links in the generation process from data preparation to expression generation and verification, ensuring the accuracy and consistency of the output results. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0057] Figure 1 This is a method step diagram of a natural language query method for a multidimensional database according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0060] Embodiment 1

[0061] A natural language query method for multidimensional databases, such as Figure 1 As shown, the method includes:

[0062] Step S1: Collect natural language questions from users, and extract data required for multidimensional expressions based on the natural language questions.

[0063] The required data includes: (1) Natural language questions: natural language questions that users need to convert. (2) Measures and dimensions: measures and dimensions of the cube corresponding to the user's questions. (3) External knowledge documents: can be enterprise documents or professional terminology documents. Documents are stored in the form of sentences or paragraphs and are used to explain and improve professional terms that may appear in user questions. (4) Question-multidimensional expression examples: examples are stored in the form of key-value pairs and are used to demonstrate the generation process of multidimensional expressions.

[0064] First, the present embodiment is further explained using the following example:

[0065] User question: Query the total profit of longan produced in Seattle in August 2023.

[0066] Cube: Fruit sales cube.

[0067] Metrics: gross profit, gross margin, collections, costs.

[0068] Dimensions: time dimension, origin dimension, product dimension, and customer dimension.

[0069] Knowledge document: 1. Seattle is located in Washington, USA; 2. Longan is another name for longan; 3. Sunshine Rose is a kind of grape; 4. Chaoyang District is located in Beijing, China.

[0070] Question - Multidimensional Expressions Example:

[0071] 1. Get the total sales of all product categories in the "Sales Amount" dimension: SELECT {[Measures].[SalesAmount]} ON COLUMNS, [Product].[Category].MEMBERS ON ROWS FROM [Sales];

[0072] 2. Calculate the total sales amount of each product category in each quarter: SELECT {[Measures].[SalesAmount]} ON COLUMNS,[Time].[Quarter].MEMBERS ON ROWS FROM[Sales] WHERE ([Product].[Category].[All Products]);

[0073] 3. Return the top five product categories with the highest sales amount: SELECT TopCount([Product].[Category].MEMBERS,5,[Measures].[Sales Amount])ON ROWS,{[Measures].[SalesAmount]}ON COLUMNS FROM[Sales].

[0074] 4. Query the sales amount in 2023: SELECT {[Measures].[Sales Amount]} ON COLUMNS,[Product].[Category].MEMBERS ON ROWS FROM[Sales] WHERE ([Time].[Year].

[2023] ).

[0075] Step S2: construct a prompt word template, wherein the prompt word template includes roles, metadata, user questions, output formats, samples and workflows.

[0076] The prompt word template contains multiple insertion slots for filling in the actual generated data. Specifically, the prompt word template mainly includes the following contents: (1) Generation role (Role): The role played by the large language model (LLM) in the generation process, helping the large model to generate more accurate and professional content. (2) Metadata (Metadata): The dimensions and measurements of the cube. (3) User questions: The questions that users need to convert. (4): Output format: The description of the output format to ensure the consistency of the large language model output. (5) Sample: Plays a demonstration role in the large language model generation process. (6) Workflow: The chain of thought (CoT), which specifies the reasoning process of the large language model.

[0077] A simple example of a prompt word template:

[0078] ##Role

[0079] You are a multidimensional database expert focused on extracting the relevant top-level dimensions from user questions.

[0080] ##Metadata

[0081] -Dimensions:

[0082] {dimensions}

[0083] ##Question

[0084] {question}

[0085] ##Output

[0086] - Output Format:

[0087] {"dimensions":["dimension1","dimension2",..]}

[0088] ##Workflow

[0089] 1. Extract key information related to the dimension from the user's question, considering keywords and descriptions that may indicate the dimension.

[0090] 2. Based on the dimension information provided in the Metadata, find all the dimensions mentioned in the question to ensure comprehensive coverage.

[0091] 3. After reasoning according to the above process, select the highest level of the involved dimensions from {dimList} and return it in the format specified in Output.

[0092] Step S3: Use retrieval enhancement technology to calculate the similarity of external knowledge documents in the data required by the multidimensional expression, screen a preset number of vectors, and obtain external knowledge documents corresponding to the vectors.

[0093] Retrieval-Augmented Generation (RAG) is used to extract relevant documents of the user's question. In this step, retrieval-augmented generation is mainly used, which is a technology that can perform semantic matching retrieval. The specific operation process is as follows:

[0094] Step 3-1: Use the encoding model to encode external knowledge into a vector. The specific process is as follows:

[0095] Using the set {m1,m2,m3,...,m i} represents an external knowledge document, where m i Represents a document.

[0096] The encoding function E(·) acts on each piece of text and converts it into an encoded representation:

[0097]

[0098] The result after encoding is: {y1,y2,y3,...,y i};

[0099] Step 3-2, encode the user's question into a vector z, and then compare the encoded user question vector with the document vector {y1,y2,y3,...,y i}Calculate the cosine similarity and get {c1,c2,c3,...,c i}, where the calculation formula for cosine similarity is:

[0100]

[0101] Step 3-3, find the three vectors with the largest cosine similarity, and then get their corresponding texts.

[0102] The following are the screening results corresponding to the questions in step S1:

[0103] 1. Seattle is located in Washington, USA;

[0104] 2. Longan is another name for longan;

[0105] 3.Sunflower Rose is a kind of grape.

[0106] Step S4: construct a multidimensional database with a tree structure according to the dimensions and measures in the multidimensional expression, and then perform dimension filtering on the multidimensional database to obtain a filtered dimensional tree structure character string.

[0107] In a multidimensional database, dimensions and measures are composed of a tree structure. In order to load dimensions and measures into prompt words, they need to be converted into corresponding tree structure strings. The structure can be expressed in the following format:

[0108] Dimension: Dimension(member1(..),member2(..),..);

[0109] Measure: Measure(member1(..),member2(..),..);

[0110] Since the depth and width of the tree structure of dimensions and measures are uncertain, the tree structure string may be too long. Therefore, it is necessary to filter the dimensions to ensure that the generation of multidimensional expressions will not fail due to the prompt word being too long. The specific process of dimension filtering is as follows:

[0111] Step 4-1, load the following content into the prompt word slot of dimension filtering:

[0112] The question entered by the user; the three question-related documents found in step 3; the dimension tree structure string constructed in step 4.

[0113] Step 4-2: Submit the filled prompt words to the large language model to generate filtering results. The generation process of the large language model is as follows:

[0114] Input representation: Assume that the input prompt word sequence is: X = {x1, x2, ..., x n}, where xi represents the ith token in the input sequence. The input token is transformed into a vector representation through the embedding layer and positional encoding:

[0115] E i =Embed(x i )+PosEnc(i)

[0116] The representation matrix of the input sequence is: E = [E1, E2, ..., E n ].

[0117] Self-Attention Mechanism: The self-attention mechanism is used to calculate the weight of each word in the input sequence. The query (Q), key (K), and value (V) vectors are obtained by linear transformation:

[0118] Q=EW Q , K=EW K , V=EW v ;

[0119] Among them, W Q , W K , WV is a trainable parameter matrix. The attention weight is calculated by dot product attention:

[0120]

[0121] Among them, d k is the dimension of the key vector, used to scale the dot product. Multi-Head Attention: Multi-Head Attention captures information from different subspaces through parallel computation of multiple attention heads:

[0122] MultiHead(Q,K,V)=Concat(head1,...,head h )W O

[0123] Each attention head is calculated as:

[0124] head i =Attention(Q i , K i , V i )

[0125] Where W0 is the output transformation matrix.

[0126] Feed-Forward Network (FFN): The attention output of each token is processed through a feed-forward neural network:

[0127] FFN(x)=ReLU(xW1+b1)W2+b2

[0128] Among them, W1, W2, b1, b2 are trainable parameters.

[0129] Layer Normalization & Residual Connection: Add residual connection and layer normalization to each layer:

[0130] Output=LayerNorm(x+SubLayer(x))

[0131] Decoding and Generation: For generation tasks, the decoder uses the previous output token as input and generates a probability distribution for the next token:

[0132] P(y t |y<t , X) = Softmax(h t W out +b out )

[0133] Among them, y t The output token generated for the current time step t, y ＜t is the output word sequence before time step t, X is the input word sequence, h t is the hidden state of the decoder at time step t, W out and b out are the output layer parameters.

[0134] Recursive generation process: The generated sequence is expanded in an autoregressive manner:

[0135] y t+1 =argmax(P(y t |y ＜t ,X))

[0136] Until the end word [EOS] is generated.

[0137] In step 4-3, since the results generated by the large language model may contain sub-levels of dimensions or hallucinations, i.e. incorrect dimensions, it is necessary to post-process the filtered dimensions, i.e. extract the highest level of the dimensions and check whether the dimension names are consistent with the actual dimensions.

[0138] The following are the results of the dimension filtering in step S1:

[0139] Time dimension, origin dimension, and product dimension.

[0140] Step S5. Use the retrieval enhancement technology to calculate the similarity between the vector data in the vector database and the user's natural language question, filter a preset number of vectors, and obtain the question-multidimensional expression sample corresponding to the vector.

[0141] Through the retrieval enhancement technology, the three most similar questions corresponding to the user's question in the vector database are found, and their corresponding multidimensional expressions are obtained. The specific process is as follows:

[0142] Step 5-1, use the encoding model to encode the example question into a vector {x1,x2,x3,...,x n}.

[0143] Step 5-2, encode the user's question into a vector z, and then compare the encoded user question vector with the document vector {y1,y2,y3,...,y i}Calculate the cosine similarity and get {c1,c2,c3,...,ci}.

[0144] Step 5-3, find the three vectors with the largest cosine similarity, and then get their corresponding problem - multidimensional expression example.

[0145] The following are the results of the problem in step S1 - Multidimensional Expressions Filter:

[0146] 1. Get the total sales of all product categories in the "Sales Amount" dimension: SELECT {[Measures].[SalesAmount]} ON COLUMNS, [Product].[Category].MEMBERS ON ROWS FROM [Sales];

[0147] 2. Calculate the total sales amount of each product category in each quarter: SELECT {[Measures].[SalesAmount]} ON COLUMNS,[Time].[Quarter].MEMBERS ON ROWS FROM[Sales] WHERE ([Product].[Category].[All Products]);

[0148] 3. Query the sales amount in 2023: SELECT {[Measures].[Sales Amount]} ON COLUMNS,[Product].[Category].MEMBERS ON ROWS FROM[Sales] WHERE ([Time].[Year].

[2023] ).

[0149] Step S6. Input the user's natural language question, external knowledge document, filtered dimension tree structure string, metric tree structure string and question-multidimensional expression sample into the prompt word template to obtain a complete prompt word, input the complete prompt word into the large language model, and generate an initial multidimensional expression.

[0150] After the above six preparation steps, start generating multidimensional expressions. The specific generation steps are as follows:

[0151] Step 6-1, load the following content into the prompt word slot used to generate the multidimensional expression:

[0152] 1) User input question.

[0153] 2) Documents related to the three issues found in step 3.

[0154] 3) The dimension tree structure string corresponding to the dimension filtered out in step 5.

[0155] 4) Metric tree structure string.

[0156] 5) The three questions obtained in step 6 - multidimensional expression examples.

[0157] Step 6-2: Submit the prompt word to the large language model to generate the corresponding result. The detailed generation process of the large language model has been given in step 4-2 and will not be repeated here.

[0158] Step 6-3. Fill the dimensions of the generated multidimensional expression, that is, fill the filtered dimensions in the form of default values ​​into the WHERE clause of the multidimensional expression.

[0159] Through the above sub-steps, a preliminary multidimensional expression has been generated.

[0160] The following is the multidimensional expression corresponding to the problem in S1:

[0161] SELECT{[Measurement].[Gross Profit]}ON COLUMNS,{[Time].

[2023] .[Third Quarter].[August],[Origin].[United States].[Washington].[Seattle],[Product].[Grade A Fruit].[Imported Longan]}ON ROWS FROM[Fruit Sales]WHERE([Customer].[Customer]); where [Customer] represents all contents of the customer dimension.

[0162] Step S7. Repeat step S6 using the large language model and the temperature parameter to generate several initial multidimensional expressions, form candidate multidimensional expressions based on the several initial multidimensional expressions, perform syntax check on the candidate multidimensional expressions, filter out executable multidimensional expressions, and group according to the execution results of the filtered executable multidimensional expressions.

[0163] Specifically, in this embodiment, there are two strategies to obtain multidimensional expressions. One is to use the same large model to generate multiple initial multidimensional expressions under a higher temperature parameter, and the other is to use different large models to generate multiple initial multidimensional expressions under a lower temperature parameter. For the temperature parameter, in this embodiment, the larger the value of the parameter, the stronger the randomness of the large model generation. After grouping, a multidimensional expression is selected from the group with the most group members as the final result.

[0164] The generated multidimensional expression may have syntax problems or hallucinations in the tree structure of the multidimensional expression. Therefore, it is necessary to perform syntax checks and hallucination checks on the generated multidimensional expression to ensure that the syntax is correct and that the dimensions and measures conform to the metadata of the multidimensional database that needs to be queried.

[0165] Since the large language model is a probabilistic model, its generated results will have a large degree of randomness, which is controlled by the temperature parameter of the large language model (temperature, ranging from 0 to 1). When the temperature is 0, the large model adopts a greedy strategy and selects the word with the highest probability as the prediction value each time. When the temperature is closer to 1, the large language model will randomly select an item from the high-probability prediction values ​​with higher randomness. In order to improve the correctness of the multidimensional expression generated by the large language model, one of the following two strategies or a combination of them will be used to generate multiple results: The large language model generates multiple results at a higher temperature. Use multiple different large language models to generate multiple results corresponding to user questions.

[0166] Among the multiple multidimensional expressions generated, the consistency algorithm is used to select the optimal solution. The specific steps of the consistency algorithm are as follows: Perform the syntax check and hallucination check in step 8 on all generated multidimensional expressions, filter out the multidimensional expressions with syntax or hallucination problems, and leave the executable multidimensional expressions. Execute all the multidimensional expressions that have been filtered in the previous step, and group the execution results of the multidimensional expressions, such as group A, group B, etc.

[0167] Select one of the groups with the largest number of members as the result.

[0168] Step S8. After reverse parsing the executable multidimensional expression, the rows, columns, pages and conditions of the query in the multidimensional expression are obtained to complete the natural language query. After the multidimensional expression is successfully generated, reverse parsing is performed to extract the rows, columns, pages and conditions of the query in the multidimensional expression, and then the extracted content is transmitted to the front end, and a worksheet is formed after processing.

[0169] Embodiment 2

[0170] A natural language query system for a multidimensional database, the system comprising:

[0171] The data collection module is used to collect natural language questions from users and extract data required for multidimensional expressions based on the natural language questions.

[0172] The required data includes: (1) Natural language questions: natural language questions that users need to convert. (2) Measures and dimensions: measures and dimensions of the cube corresponding to the user's questions. (3) External knowledge documents: can be enterprise documents or professional terminology documents. Documents are stored in the form of sentences or paragraphs and are used to explain and improve professional terms that may appear in user questions. (4) Question-multidimensional expression examples: examples are stored in the form of key-value pairs and are used to demonstrate the generation process of multidimensional expressions.

[0173] A prompt word template is created to construct the prompt word template, wherein the prompt word template includes roles, metadata, user questions, output formats, samples, and workflows.

[0174] The prompt word template contains multiple insertion slots for filling in the actual generated data. Specifically, the prompt word template mainly includes the following contents: (1) Generation role (Role): The role played by the large language model (LLM) in the generation process, helping the large model to generate more accurate and professional content. (2) Metadata (Metadata): The dimensions and measurements of the cube. (3) User questions: The questions that users need to convert. (4): Output format: The description of the output format to ensure the consistency of the large language model output. (5) Sample: Plays a demonstration role in the large language model generation process. (6) Workflow: The chain of thought (CoT), which specifies the reasoning process of the large language model.

[0175] The document retrieval module is used to use the retrieval enhancement technology to perform similarity calculation on the external knowledge documents in the data required by the multidimensional expression, screen a preset number of vectors, and obtain the external knowledge documents corresponding to the vectors.

[0176] Retrieval-Augmented Generation (RAG) is used to extract relevant documents of the user's question. In this step, retrieval-augmented generation is mainly used, which is a technology that can perform semantic matching retrieval. The specific operation process is as follows:

[0177] Use the encoding model to encode external knowledge into a vector. The specific process is as follows:

[0178] Using the set {m1,m2,m3,...,m i} represents an external knowledge document, where m i Represents a document.

[0179] The encoding function E(·) is applied to each piece of text to convert it into an encoded representation:

[0180]

[0181] The result after encoding is: {y1,y2,y3,...,y i};

[0182] Encode the user's question into a vector z, and then compare the encoded user question vector with the document vector {y1,y2,y3,...,y i}Calculate the cosine similarity and get {c1,c2,c3,...,ci}, where the calculation formula for cosine similarity is:

[0183]

[0184] Find the three vectors with the largest cosine similarity, and then get their corresponding text.

[0185] The dimension filtering module is used to construct a multidimensional database with a tree structure according to the dimensions and measures in the multidimensional expression, and then perform dimension filtering on the multidimensional database to obtain a filtered dimension tree structure string.

[0186] In a multidimensional database, dimensions and measures are composed of a tree structure. In order to load dimensions and measures into prompt words, they need to be converted into corresponding tree structure strings. The structure can be expressed in the following format:

[0187] Dimension: Dimension(Member1(..),Member2(..),..); Measurement: Measurement(Member1(..),Member2(..),..);

[0188] Since the depth and width of the tree structure of dimensions and measures are uncertain, the tree structure string may be too long. Therefore, it is necessary to filter the dimensions to ensure that the generation of multidimensional expressions will not fail due to the prompt word being too long. The specific process of dimension filtering is as follows:

[0189] In the dimension filter prompt slot, add the following content:

[0190] The question entered by the user; the three question-related documents found in step 3; the dimension tree structure string constructed in step 4.

[0191] Submit the filled prompt words to the large language model to generate filtering results. The generation process of the large language model is as follows:

[0192] Input representation: Assume that the input prompt word sequence is: X = {x1, x2, ..., x n}, where x i Represents the i-th token in the input sequence. The input token is transformed into a vector representation through the embedding layer and positional encoding:

[0193] E i =Embed(x i )+PosEnc(i)

[0194] The representation matrix of the input sequence is: E = [E1, E2, ..., E n ].

[0195] Self-Attention Mechanism: The weight of each word in the input sequence is calculated through the self-attention mechanism. The query (Q), key (K) and value (V) vectors are obtained by linear transformation:

[0196] Q=EW Q , K=EW K , V=EW Ｖ ;

[0197] Among them, W Q , W K , WV is a trainable parameter matrix. The attention weight is calculated by dot product attention:

[0198]

[0199] Among them, d k is the dimension of the key vector, used to scale the dot product. Multi-Head Attention: Multi-Head Attention captures information from different subspaces through parallel computation of multiple attention heads:

[0200] MultiHead(Q,K,V)=Concat(head1,...,head h )W O

[0201] Each attention head is calculated as:

[0202] head i =Attention(Q i , K i , V i )

[0203] Where W0 is the output transformation matrix.

[0204] Feed-Forward Network (FFN): The attention output of each token is processed through a feed-forward neural network:

[0205] FFN(x)=ReLU(xW1+b1)W2+b2

[0206] Among them, W1, W2, b1, b2 are trainable parameters.

[0207] Layer Normalization & Residual Connection: Add residual connection and layer normalization to each layer:

[0208] Output=LayerNorm(x+SubLayer(x))

[0209] Decoding and Generation: For generation tasks, the decoder uses the previous output token as input and generates a probability distribution for the next token:

[0210] P(y t |y ＜t ,X)=Softmax(h t W out +b out )

[0211] Among them, y t The output token generated for the current time step t, y ＜t is the output word sequence before time step t, X is the input word sequence, h t is the hidden state of the decoder at time step t, W out and b out are the output layer parameters.

[0212] Recursive generation process: The generated sequence is expanded in an autoregressive manner:

[0213] y t+1 =argmax(P(y t |y <t , X))

[0214] Until the end word [EOS] is generated.

[0215] Since the results generated by the large language model may contain sub-levels of dimensions or hallucinations, that is, incorrect dimensions, the filtered dimensions need to be post-processed, that is, extracting the highest level of the dimensions and checking whether the dimension names are consistent with the actual dimensions.

[0216] A multidimensional expression sample retrieval module is used to calculate the similarity between the vector data in the vector database and the user's natural language question using retrieval enhancement technology, filter a preset number of vectors, and obtain the question-multidimensional expression sample corresponding to the vector;

[0217] Through the retrieval enhancement technology, the three most similar questions corresponding to the user's question in the vector database are found, and their corresponding multidimensional expressions are obtained. The specific process is as follows:

[0218] Use the encoding model to encode the example question into a vector {x1, x2, x3, ..., x n}.

[0219] The user's question is encoded into a vector z, and then the encoded user question vector is compared with the document vectors {y1, y2, y3, ..., y i}Calculate the cosine similarity and get {c1, c2, c3, ..., c i}.

[0220] Find the three vectors with the largest cosine similarity and then get their corresponding questions - multidimensional expression examples.

[0221] The multidimensional expression generation module is used to input the user's natural language question, external knowledge document, filtered dimension tree structure string, measure tree structure string and question-multidimensional expression sample into the prompt word template to obtain a complete prompt word, input the complete prompt word into the large language model, and generate an initial multidimensional expression.

[0222] After the above six preparation steps, start generating multidimensional expressions. The specific generation steps are as follows:

[0223] In the prompt word slot for generating multidimensional expressions, add the following:

[0224] 1) User input question.

[0225] 2) Documents related to the three issues found in step 3.

[0226] 3) The dimension tree structure string corresponding to the dimension filtered out in step 5.

[0227] 4) Metric tree structure string.

[0228] 5) The three questions obtained in step 6 - multidimensional expression examples.

[0229] Submit the prompt word to the large language model to generate the corresponding result. The detailed generation process of the large language model has been given before and will not be repeated here.

[0230] Fill the generated multidimensional expression with dimensions, that is, fill the filtered dimensions with default values ​​into the WHERE clause of the multidimensional expression.

[0231] Through the above sub-steps, a preliminary multidimensional expression has been generated.

[0232] The multidimensional expression checking module is used to repeatedly execute the multidimensional expression generating module using the large language model and the temperature parameter to generate a number of initial multidimensional expressions, form candidate multidimensional expressions based on the initial multidimensional expressions, perform syntax checking on the candidate multidimensional expressions, filter out executable multidimensional expressions, and group according to the execution results of the filtered executable multidimensional expressions.

[0233] Specifically, in this embodiment, there are two strategies to obtain multidimensional expressions. One is to use the same large model to generate multiple initial multidimensional expressions under a higher temperature parameter, and the other is to use different large models to generate multiple initial multidimensional expressions under a lower temperature parameter. For the temperature parameter, in this embodiment, the larger the value of the parameter, the stronger the randomness of the large model generation. After grouping, a multidimensional expression is selected from the group with the most group members as the final result.

[0234] The generated multidimensional expression may have syntax problems or hallucinations in the tree structure of the multidimensional expression. Therefore, it is necessary to perform syntax checks and hallucination checks on the generated multidimensional expression to ensure that the syntax is correct and that the dimensions and measures conform to the metadata of the multidimensional database that needs to be queried.

[0235] Since the large language model is a probabilistic model, its generated results will have a large degree of randomness, which is controlled by the temperature parameter of the large language model (temperature, ranging from 0 to 1). When the temperature is 0, the large model adopts a greedy strategy and selects the word with the highest probability as the prediction value each time. When the temperature is closer to 1, the large language model will randomly select an item from the high-probability prediction values ​​with higher randomness. In order to improve the correctness of the multidimensional expression generated by the large language model, one of the following two strategies or a combination of them will be used to generate multiple results: The large language model generates multiple results at a higher temperature. Use multiple different large language models to generate multiple results corresponding to user questions.

[0236] Among the multiple multidimensional expressions generated, the consistency algorithm is used to select the optimal solution. The specific steps of the consistency algorithm are as follows: Perform the syntax check and hallucination check in step 8 on all generated multidimensional expressions, filter out the multidimensional expressions with syntax or hallucination problems, and leave the executable multidimensional expressions. Execute all the multidimensional expressions that have been filtered in the previous step, and group the execution results of the multidimensional expressions, such as group A, group B, etc.

[0237] Select one of the groups with the largest number of members as the result.

[0238] The natural language query module is used to reversely parse the executable multidimensional expression to obtain the rows, columns, pages and conditions queried in the multidimensional expression, and complete the natural language query.

[0239] After the multidimensional expression is successfully generated, reverse parsing is performed to extract the rows, columns, pages and conditions queried in the multidimensional expression, and then the extracted content is passed to the front end to form a worksheet after processing.

[0240] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A natural language query method for a multidimensional database, characterized in that: The method comprises: Step S1. Collecting natural language questions from users, and extracting data required for multidimensional expressions according to the natural language questions; Step S2. construct a prompt word template, wherein the prompt word template includes roles, metadata, user questions, output formats, samples and workflows; Step S3. Use the search enhancement technology to calculate the similarity of the external knowledge documents in the data required by the multidimensional expression, screen a preset number of vectors, and obtain the external knowledge documents corresponding to the vectors; Step S4. construct a multidimensional database with a tree structure according to the dimensions and measures in the multidimensional expression, and then perform dimension filtering on the multidimensional database to obtain a filtered dimension tree structure string; Step S5. Use the search enhancement technology to calculate the similarity between the vector data in the vector database and the user's natural language question, filter a preset number of vectors, and obtain the question-multidimensional expression sample corresponding to the vector; Step S6. Input the user's natural language question, the external knowledge document, the filtered dimension tree structure string, the metric tree structure string and the question-multidimensional expression sample into the prompt word template to obtain a complete prompt word, input the complete prompt word into the large language model, and generate an initial multidimensional expression; Step S7. Repeat step S6 using the large language model and the temperature parameter to generate a number of initial multidimensional expressions, form candidate multidimensional expressions based on the initial multidimensional expressions, perform syntax check on the candidate multidimensional expressions, filter out executable multidimensional expressions, and group according to the execution results of the filtered executable multidimensional expressions; Step S8: After reverse parsing the executable multidimensional expression, the rows, columns, pages and conditions of the query in the multidimensional expression are obtained, and the natural language query is completed.

2. The natural language query method for a multidimensional database according to claim 1, characterized in that: In step S3, using the search enhancement technology to calculate the similarity of the external knowledge documents in the data required by the multidimensional expression, screening a preset number of vectors, and obtaining the content of the external knowledge documents corresponding to the vectors specifically includes: Encode the external knowledge in the data required by the multidimensional expression into an external knowledge document by using the encoding model; Encode the user's natural language question into a user question vector using an encoding model; The cosine similarity between the external knowledge document and the user question vector is calculated, and the cosine similarity results are sorted in descending order, and a preset number of vectors are selected to obtain the external knowledge document corresponding to the screening vector.

3. The natural language query method for a multidimensional database according to claim 1, characterized in that: In step S4, a multidimensional database with a tree structure is constructed according to the dimensions and measures in the multidimensional expression, and then the multidimensional database is dimensionally filtered to obtain the content of the filtered dimensional tree structure string, which specifically includes: The user's natural language questions, external knowledge documents, and dimensional tree-structured strings in the multidimensional database are filled into prompt words and then input into the large language model to generate filtering results, specifically: The prompt word sequence of the prompt word is transformed into a vector representation through an embedding layer and position encoding to obtain a representation matrix of the prompt word sequence; Based on the representation matrix, a self-attention mechanism is used to calculate the weights of the word units in the representation matrix; Through the multi-head attention mechanism, different context information is captured from multiple subspaces to obtain the attention representation of each word. After the attention representation of each word is input into the feedforward network for processing, the final representation of each word is obtained through residual connection and normalization layer; Input the final representation into a decoder to generate a probability distribution of the next word unit; Use autoregressive method to gradually expand and generate the next word unit, and finally get the filtering result.

4. The natural language query method for a multidimensional database according to claim 3, characterized in that: The calculation contents of inputting the final representation of the word unit into the decoder to generate the probability distribution of the next word unit include: P(y t |y <t ,X)=Softmax(h t W out +b out ) Among them, y t The output token generated for the current time step t, y ＜t is the output word sequence before time step t, X is the input word sequence, h t is the hidden state of the decoder at time step t, W out and b out are the output layer parameters.

5. A natural language query system for a multidimensional database, the query system being used to use the query method according to any one of claims 1 to 4, characterized in that: The system comprises: A data collection module, used to collect natural language questions from users and extract data required for multidimensional expressions based on the natural language questions; Creating a prompt word template, used to construct the prompt word template, wherein the prompt word template includes roles, metadata, user questions, output formats, samples, and workflows; A document retrieval module, used to use retrieval enhancement technology to perform similarity calculation on the external knowledge documents in the data required by the multidimensional expression, screen a preset number of vectors, and obtain the external knowledge documents corresponding to the vectors; A dimension filtering module is used to construct a multidimensional database with a tree structure according to the dimensions and measures in the multidimensional expression, and then perform dimension filtering on the multidimensional database to obtain a filtered dimension tree structure string; A multidimensional expression sample retrieval module is used to calculate the similarity between the vector data in the vector database and the user's natural language question using retrieval enhancement technology, filter a preset number of vectors, and obtain the question-multidimensional expression sample corresponding to the vector; A multidimensional expression generation module is used to input the user's natural language question, the external knowledge document, the filtered dimension tree structure string, the metric tree structure string and the question-multidimensional expression sample into the prompt word template to obtain a complete prompt word, input the complete prompt word into the large language model, and generate an initial multidimensional expression; a multidimensional expression checking module, configured to repeatedly execute step S6 using the large language model and the temperature parameter to generate a number of initial multidimensional expressions, form candidate multidimensional expressions based on the initial multidimensional expressions, perform syntax checking on the candidate multidimensional expressions, filter out executable multidimensional expressions, and group according to execution results of the filtered executable multidimensional expressions; The natural language query module is used to reversely parse the executable multidimensional expression to obtain the rows, columns, pages and conditions queried in the multidimensional expression, and complete the natural language query.

6. The natural language query system for a multidimensional database according to claim 5, characterized in that: The workflow of the document retrieval module specifically includes: Encode the external knowledge in the data required by the multidimensional expression into an external knowledge document by using the encoding model; Encode the user's natural language question into a user question vector using an encoding model; The cosine similarity between the external knowledge document and the user question vector is calculated, and the cosine similarity results are sorted in descending order, and a preset number of vectors are selected to obtain the external knowledge document corresponding to the screening vector.

7. The natural language query system for a multidimensional database according to claim 5, characterized in that: The workflow of the dimension filtering module specifically includes: The user's natural language questions, external knowledge documents, and dimensional tree-structured strings in the multidimensional database are filled into prompt words and then input into the large language model to generate filtering results, specifically: The prompt word sequence of the prompt word is transformed into a vector representation through an embedding layer and position encoding to obtain a representation matrix of the prompt word sequence; Based on the representation matrix, a self-attention mechanism is used to calculate the weights of the word units in the representation matrix; Through the multi-head attention mechanism, different context information is captured from multiple subspaces to obtain the attention representation of each word. After the attention representation of each word is input into the feedforward network for processing, the final representation of each word is obtained through residual connection and normalization layer; Input the final representation into a decoder to generate a probability distribution of the next word unit; Use autoregressive method to gradually expand and generate the next word unit, and finally get the filtering result.

8. The natural language query system for a multidimensional database according to claim 7, characterized in that: The calculation contents of the probability distribution of the output word unit input into the decoder to generate the next word unit include: P(y t |y ＜t ,X)=Softmax(h t W out +b out ) Among them, y t The output token generated for the current time step t, y ＜t is the output word sequence before time step t, X is the input word sequence, h t is the hidden state of the decoder at time step t, W out and b out are the output layer parameters.

Images