Data query method and device based on text processing model

By combining the constellation data model and text processing model, identifying and processing query requirements for multi-dimensional operation fields, complex query problems in the existing technology are solved, and more accurate and flexible data analysis support is achieved.

CN118708704BActive Publication Date: 2024-11-19GUANGZHOU SMART SOFTWARE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411194768.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-11-19
Estimated Expiration
2044-08-29

AI Technical Summary

Technical Problem

The prior art is difficult to effectively handle complex queries, especially those involving multi-dimensional operation fields, which leads to the query results not meeting expectations and limits the depth and breadth of data analysis.

Method used

Using a data query method based on text processing model, the trained text processing model is integrated through the constellation data model, the candidate basic fields and multi-dimensional operation fields in the user's natural query statement are identified, task processing instructions are constructed to accurately understand the query intention, and from the database Extract business data from.

Benefits of technology

It significantly improves the accuracy and flexibility of data queries, supports deeper data analysis and business insights, lowers user operation thresholds, and optimizes user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118708704B_ABST
    Figure CN118708704B_ABST
Patent Text Reader

Abstract

The present application relates to a data query method and device based on a text processing model. When a user inputs a query statement in natural language, the system can quickly filter out relevant candidate basic fields and candidate multidimensional operation fields from the constellation data model. Subsequently, these candidate basic fields, candidate multidimensional operation fields, natural query statements, and preset task execution texts are combined to construct accurate task processing instructions. This instruction not only guides the text processing model on how to interpret the user's query intent, but also ensures that query requirements involving multidimensional operation fields can be effectively identified and processed, query information that meets complex query requirements can be accurately output, and business data can be accurately extracted from the database, thereby supporting users to conduct deeper data analysis and business insights. Through the method and device of the embodiments of the present application, the processing capabilities of complex operation fields are significantly enhanced, so that data queries are no longer limited to the traditional simple query scope.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of data warehouse, and in particular to a data query method and device based on a text processing model. Background Art

[0002] With the rapid development of big data technology, data query has become a key means for enterprises to gain insights and make decisions. However, the traditional SQL (or other specific query languages) query method has a high technical threshold, which limits the data exploration ability of people with non-technical backgrounds. In recent years, with the rapid development of natural language processing technology, especially the widespread application of large language models represented by ChatGPT, it has brought revolutionary changes to data query, allowing users to propose query requirements in the form of natural language and implement queries.

[0003] However, existing technical solutions mainly focus on querying basic data in the database. The main technical means is to segment the natural query statements input by users, match the basic fields in the database and determine them as candidate basic fields, and then use the understanding of the large language model to determine the basic fields involved in the query and construct the query statement. However, when processing complex queries, especially those involving multi-dimensional operation fields, there are still huge challenges.

[0004] As an advanced feature in data analysis, multidimensional operation fields are generated through complex calculation logic and basic data of multiple dimensions. Their query requirements require the model to not only understand the literal meaning of natural language, but also to deeply understand the business logic and data analysis requirements behind them. In addition, these multidimensional operation fields are usually customized by users according to specific needs, so they are flexible and changeable in different industries and application scenarios. Although the current large language model can understand simple natural language queries to a certain extent, it is often difficult to accurately capture the query intent for complex queries involving multidimensional operation fields, resulting in query results that do not meet expectations, which seriously restricts the depth and breadth of data analysis.

[0005] Therefore, how to expand the scope of application of the existing technology so that it can support queries involving multi-dimensional operation fields has become an urgent problem to be solved. Summary of the invention

[0006] Based on this, the purpose of this application is to provide a data query method based on a text processing model, which can effectively identify and process query requirements involving multi-dimensional operation fields, thereby supporting users to conduct deeper data analysis and business insights.

[0007] The data query method based on the text processing model provided in the embodiment of the present application includes the following steps:

[0008] Get the natural query statement entered by the user;

[0009] Acquire a constellation data model; determine, according to the natural query statement, several relevant basic fields from the constellation data model as candidate basic fields; wherein the constellation data model records several data tables, basic fields contained in each data table, and multi-dimensional operation fields bound to at least some of the basic fields;

[0010] If the candidate basic field is bound to a multi-dimensional operation field, the bound multi-dimensional operation field is determined as a candidate multi-dimensional operation field;

[0011] Constructing a task processing instruction according to the candidate basic fields, the candidate multidimensional operation fields, the natural query statement, and a preset task execution text; wherein the task execution text is used to prompt the text processing model to output query information that conforms to the query intent of the natural query statement, and the query information is determined according to the candidate basic fields and the candidate multidimensional operation fields;

[0012] Inputting the task processing instruction into a trained text processing model to obtain output query information;

[0013] According to the query information and the constellation data model, corresponding service data is queried from a database corresponding to the constellation data model.

[0014] The present application also provides a data query device based on a text processing model, including:

[0015] A query statement acquisition module is used to acquire natural query statements input by users;

[0016] A candidate basic field determination module is used to obtain a constellation data model; determine a number of related basic fields from the constellation data model as candidate basic fields according to the natural query statement; wherein the constellation data model records a number of data tables, basic fields contained in each data table, and multi-dimensional operation fields bound to at least some of the basic fields;

[0017] A candidate multidimensional operation field determination module, configured to determine the bound multidimensional operation field as a candidate multidimensional operation field if the candidate basic field is bound with the multidimensional operation field;

[0018] A task processing instruction construction module is used to construct a task processing instruction according to the candidate basic fields, the candidate multidimensional operation fields, the natural query statement and a preset task execution text; wherein the task execution text is used to prompt the text processing model to output query information that meets the query intent of the natural query statement, and the query information is determined according to the candidate basic fields and the candidate multidimensional operation fields;

[0019] A query information acquisition module, used to input the task processing instruction into the trained text processing model to obtain output query information;

[0020] The service data query module is used to query and obtain corresponding service data from a database corresponding to the constellation data model according to the query information and the constellation data model.

[0021] The data query method and device based on the text processing model proposed in the embodiment of the present application realize the in-depth understanding of the user's natural query statement and output accurate query information by integrating the constellation data model with the trained text processing model. Among them, the constellation data model, as the cornerstone of structured data, not only carries the detailed information of the data table and the basic fields, but also additionally associates the multidimensional operation field and its complex calculation logic for the basic field. When the user enters the query statement in the form of natural language, the system can quickly filter out the relevant candidate basic fields and candidate multidimensional operation fields from the constellation data model. Subsequently, combining these candidate basic fields, candidate multidimensional operation fields, natural query statements and preset task execution texts, the system constructs an accurate task processing instruction. This instruction not only guides the text processing model on how to interpret the user's query intention, but also ensures that the query requirements involving multidimensional operation fields can be effectively identified and processed, and the query information that meets the complex query requirements can be accurately output, and the business data can be accurately extracted from the database, thereby supporting users to conduct deeper data analysis and business insights. Through the method and device of the embodiment of the present application, not only direct query of the basic fields of the database is supported, but also the processing capability of complex operation fields is significantly enhanced, so that data query is no longer limited to the traditional simple query scope. This innovative technology not only simplifies the data query process and lowers the user operation threshold, but also greatly improves the accuracy and flexibility of the query, provides users with more comprehensive and in-depth data analysis support, and significantly optimizes the overall user experience.

[0022] For better understanding and implementation, the present application is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flowchart of a data query method based on a text processing model according to an embodiment of the present application;

[0024] Figure 2 A schematic diagram of the steps of determining relevant candidate basic fields according to a natural query statement in an embodiment of the present application;

[0025] Figure 3 A schematic diagram of the steps of performing keyword recognition on a natural query sentence in an embodiment of the present application;

[0026] Figure 4A schematic diagram of the steps of the query condition identification module identifying the time condition in the embodiment of the present application;

[0027] Figure 5 A schematic diagram of a data query device based on a text processing model according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present application clearer, the embodiments of the present application will be further described in detail below in conjunction with the accompanying drawings. Wherein, when the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements.

[0029] It should be clear that the implementation methods described in the embodiments described below do not represent all implementation methods consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0030] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application are also intended to include plural forms, unless the context clearly indicates other meanings. In addition, in the description of this application, unless otherwise specified, "multiple" refers to two or more. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone; the character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0031] It should be understood that, although the terms first, second, third, etc. may be used in the present application to describe various information, such information should not be limited to these terms, and these terms are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances. Depending on the context, the words "if" / "if" used in the present application can be interpreted as "at the time of" or "when" or "in response to determination".

[0032] In recent years, with the rapid development of natural language processing technology, especially the widespread application of large language models represented by ChatGPT, revolutionary changes have been brought about in data query, enabling users to raise query requirements and implement queries in the form of natural language.

[0033] However, existing technical solutions mainly focus on querying basic data in the database. The main technical means is to segment the natural query statements input by users, match the basic fields in the database and determine them as candidate basic fields, and then use the understanding of the large language model to determine the basic fields involved in the query and construct the query statement. However, when processing complex queries, especially those involving multi-dimensional operation fields, there are still huge challenges.

[0034] As an advanced feature in data analysis, multidimensional operation fields are generated through complex calculation logic and basic data of multiple dimensions. Their query requirements require the model to not only understand the literal meaning of natural language, but also to deeply understand the business logic and data analysis requirements behind them. In addition, these multidimensional operation fields are usually customized by users according to specific needs, so they are flexible and changeable in different industries and application scenarios. Although the current large language model can understand simple natural language queries to a certain extent, it is often difficult to accurately capture the query intent for complex queries involving multidimensional operation fields, resulting in query results that do not meet expectations, which seriously restricts the depth and breadth of data analysis.

[0035] Therefore, how to expand the scope of application of the existing technology so that it can support queries involving multi-dimensional operation fields has become an urgent problem to be solved.

[0036] In this regard, the embodiment of the present application provides a data query method based on a text processing model. By constructing a constellation data model, the model not only records data tables and basic fields, but also associates multidimensional operation fields and their calculation logic, providing a structured basis for complex queries. After receiving the natural query statement input by the user, the system first uses the constellation data model to determine the relevant candidate basic fields, and further identifies the multidimensional operation fields bound to these basic fields as candidate multidimensional operation fields. Subsequently, in combination with the candidate basic fields, the candidate multidimensional operation fields, the natural query statement and the preset task execution text, a task processing instruction that can guide the text processing model to accurately understand the query intent is constructed. Finally, the instruction is executed by the trained text processing model, and the query information that meets the user's query intent is output, and the corresponding business data is retrieved from the database accordingly. It can effectively identify and process query requirements involving multidimensional operation fields, thereby supporting users to conduct deeper data analysis and business insights.

[0037] Please refer to Figure 1 The data query method based on the text processing model provided in the embodiment of the present application includes the following steps:

[0038] S101: Obtaining a natural query sentence input by a user;

[0039] S102: Acquire a constellation data model; determine, according to the natural query statement, several relevant basic fields from the constellation data model as candidate basic fields; wherein the constellation data model records several data tables, basic fields contained in each data table, and multi-dimensional operation fields bound to at least some of the basic fields;

[0040] S103: If the candidate basic field is bound to a multi-dimensional operation field, the bound multi-dimensional operation field is determined as a candidate multi-dimensional operation field;

[0041] S104: constructing a task processing instruction according to the candidate basic fields, the candidate multidimensional operation fields, the natural query statement, and a preset task execution text; wherein the task execution text is used to prompt the text processing model to output query information that meets the query intent of the natural query statement, and the query information is determined according to the candidate basic fields and the candidate multidimensional operation fields;

[0042] S105: inputting the task processing instruction into the trained text processing model to obtain output query information;

[0043] S106: According to the query information and the constellation data model, query and obtain corresponding service data from a database corresponding to the constellation data model.

[0044] The data query method based on the text processing model proposed in the embodiment of the present application realizes an in-depth understanding of the user's natural query statement and outputs accurate query information by integrating the constellation data model with the trained text processing model. Among them, the constellation data model, as the cornerstone of structured data, not only carries the detailed information of the data table and basic fields, but also additionally associates the multidimensional operation field and its complex calculation logic with the basic field. When the user enters the query statement in the form of natural language, the system can quickly filter out the relevant candidate basic fields and candidate multidimensional operation fields from the constellation data model. Subsequently, combining these candidate basic fields, candidate multidimensional operation fields, natural query statements and preset task execution texts, the system constructs an accurate task processing instruction. This instruction not only guides the text processing model on how to interpret the user's query intention, but also ensures that the query requirements involving multidimensional operation fields can be effectively identified and processed, and the query information that meets the complex query requirements can be accurately output, and the business data can be accurately extracted from the database, thereby supporting users to conduct deeper data analysis and business insights. Through the method of the embodiment of the present application, not only direct query of the basic fields of the database is supported, but also the processing capability of complex operation fields (such as MDX fields) is significantly enhanced, so that data query is no longer limited to the traditional simple query scope. This innovative technology not only simplifies the data query process and lowers the user operation threshold, but also greatly improves the accuracy and flexibility of the query, provides users with more comprehensive and in-depth data analysis support, and significantly optimizes the overall user experience.

[0045] The data query method based on the text processing model in the embodiment of the present application is executed by a computer. Hereinafter, the text processing model in the embodiment of the present application is first described, and then each step is described in detail.

[0046] In the embodiment of the present application, the text processing model is responsible for parsing the natural query sentence input by the user, understanding its query intent, and generating corresponding query information based on these intents so as to subsequently retrieve data from the database. The text processing model is usually a complex machine learning model, especially a natural language processing (NLP) model based on deep learning.

[0047] In an embodiment of the present application, the text processing model may be based on one or more pre-trained language models, such as BERT, GPT, etc. (these models have been trained on a large amount of text data and can capture the deep semantic and syntactic features of the language), and is fine-tuned to adapt to the specific tasks in the present application (i.e., query intent understanding and query information generation). This usually involves further training the pre-trained language model on an annotated data set containing query statements and corresponding query information, so that it can better understand the query intent and generate accurate query information. Of course, the text processing model may also be an existing large language model, and as long as it has the processing capabilities of the specific tasks of the present application, it can be applied in the technical solution of the present application, and the embodiments of the present application are not limited to this.

[0048] In one embodiment, the trained text processing model is obtained by:

[0049] Step S301, obtaining a pre-trained first text processing model; the first text processing model includes an encoder and / or a decoder, and the encoder and the decoder respectively include an attention layer and a fully connected layer connected to each other;

[0050] Step S302, obtaining a training data set; the training data set includes a plurality of training samples, each training sample includes a first multidimensional query statement, candidate basic fields and candidate multidimensional operation fields related to the first multidimensional query statement, the task execution text, and query tag information corresponding to the first multidimensional query statement, the query tag information includes a multidimensional operation field and a basic field bound to the multidimensional operation field;

[0051] Step S303, obtaining an initialized low-rank matrix, embedding the low-rank matrix in a weight matrix of a fully connected layer or an attention layer of an encoder of the first text processing model, or embedding the low-rank matrix in a weight matrix of a fully connected layer or an attention layer of a decoder of the first text processing model;

[0052] Step S304, input the training data set into the first text processing model, obtain the query information corresponding to each generated training sample; calculate the loss value between the query information of each training sample and the corresponding query label information; according to the loss value, adjust the weight of the low-rank matrix until the loss value is less than a preset loss threshold or the total number of weight adjustments reaches a preset number of adjustments, and obtain the trained text processing model; or, step S305, input the training data set into the first text processing model; obtain the query information corresponding to each training sample generated within the preset time at preset intervals; calculate the loss value between the query information of each training sample generated within the preset time and the corresponding query label information, and cache the current first text processing model and the corresponding loss value; according to the loss value, adjust the weight of the low-rank matrix until the total number of weight adjustments reaches the preset number of adjustments, and determine the first text processing model with the smallest loss value as the trained text processing model from among the cached first text processing models and the corresponding loss values.

[0053] Among them, step S301, obtains the existing text processing model that has been pre-trained with a large amount of data as a basis. These models usually have powerful language understanding and generation capabilities. The model includes an encoder and / or a decoder, that is, the text processing model of this embodiment can be an encoder-only model, a decoder-only model, or an encoder-decoder model. These three types of models can be applied to the technical solution of this application. Among them, in the encoder or decoder, an attention layer and a fully connected layer are respectively included. The attention layer and the fully connected layer are interconnected, which can effectively process text data and realize the conversion from input to output.

[0054] Step S302, obtain a data set for training the model. These data sets should contain rich training samples. Each training sample is a different first multidimensional query statement (a query statement based on natural language expression and involving multidimensional operation field query), and a candidate basic field and a candidate multidimensional operation field corresponding to the first multidimensional query statement in the sample. In addition, it also includes data such as task execution text. These data will also construct a task processing instruction according to the steps in the method of this application and input it into the text processing model for training. In addition, each training sample also includes query label information corresponding to the multidimensional query statement. These query label information are the target query information of the first multidimensional query statement, that is, the accurate query information expected to be generated, so as to serve as a label during model training, which is used for the model to calculate the loss value and adjust the parameters according to the loss value. Among them, since the training samples used in this embodiment are all query statements involving multidimensional queries, the query label information obviously also includes a multidimensional operation field and a basic field bound to the multidimensional operation field. In one embodiment, the query label information as a label is the target query information of the first multidimensional query statement in a preset report query request format, so that when fine-tuning the training model, the training model can organize the query information according to the preset report query request format and then output it. Correspondingly, the preset loss function comprehensively evaluates the accuracy of the query information and the accuracy of the format and calculates the loss value, thereby more accurately adjusting the weights of the low-rank matrix so that the fine-tuned trained text processing model meets the accuracy requirements of the query information and the report query request format.

[0055] Step S303, optimize the weight matrix of the model by embedding a low-rank matrix to reduce the number of parameters and computational complexity of the model, and at the same time may improve the generalization ability of the model. Specifically, the initialized low-rank matrix is ​​embedded in the weight matrix of the fully connected layer or attention layer of the encoder or decoder. This embedding method can be flexibly selected and adjusted according to the specific structure and performance requirements of the model. Regarding the specific structure and principle of the low-rank matrix, it can be roughly understood that it is based on the original parameter weight matrix of the model, and two low-rank column matrices A and row matrices B are determined, and these two matrices are embedded in the model as a low-rank matrix in the form of a product, so that in the data training process, without changing the original structure and original weight matrix of the model, the weight of the low-rank matrix can be adjusted to achieve the adjustment of the overall superimposed weight of the model (the weight of the low-rank matrix is ​​superimposed). After the training is completed, the weight of the low-rank matrix is ​​merged into the model to obtain a fine-tuned model. Please refer to the LoRA fine-tuning principle in this field for details. If there are any inaccuracies in the description here, please refer to the LoRA fine-tuning principle in this field, and no further description will be given here.

[0056] Step S304 proposes a training strategy, in which the training data set is input into the first text processing model embedded in the low-rank matrix. The model outputs the corresponding query information based on the input first multidimensional query statement, candidate fields and other information. Then, a preset loss function is used to calculate the loss value between the query information and the query label information of each training sample. According to the loss value, the weight of the low-rank matrix is ​​adjusted by the back propagation algorithm until the loss value is less than the preset loss threshold or the total number of weight adjustments reaches the preset number of adjustments. This process continuously adjusts the weight of the low-rank matrix and iterates continuously until the model converges, and the generated query information is as close as possible to the actual query label information, thereby obtaining a trained text processing model.

[0057] Step S305 is an alternative to step 304. Compared with step S304, it adopts a more flexible training strategy, that is, the model performance is evaluated every preset time, and the current model and loss value are cached, and finally the model with the smallest loss value is selected from the cache as the final model. Specifically, in the step of inputting the training data set into the first text processing model after embedding the low-rank matrix, the training data set can be divided into multiple small batches of data sets and input into the first text processing model in sequence; or it can be divided into multiple batches of data sets, and each time a number of training samples are randomly obtained from the training data set (self-help resampling method or sampling without replacement method) and input into the first text processing model. After receiving the input training samples, the first text processing model starts to calculate and output the query information corresponding to the training samples. This embodiment obtains query information corresponding to each training sample generated within the preset time at preset intervals, uses a preset loss function to calculate the loss value between the query information of each training sample generated within the preset time and the corresponding query label information, and caches the current first text processing model and the corresponding loss value; according to the loss value, adjusts the weight of the low-rank matrix to update the first text processing model; continues to input training samples into the updated first text processing model, and again obtains query information corresponding to each training sample generated within the preset time at preset intervals, calculates the loss value, and caches the current first text processing model and the corresponding loss value; until the total number of weight adjustments reaches the preset number of adjustments, the first text processing model with the smallest loss value is determined as the trained text processing model from among the cached first text processing models and the corresponding loss values.

[0058] The text processing model of this embodiment is fine-tuned on the basis of a pre-trained text processing model. Specifically, the LoRA fine-tuning method can be adopted. By embedding a low-rank matrix in the weight matrix of the model, the number of parameters and computational complexity that need to be adjusted in the model training process are reduced. Based on rich natural query statements, this embodiment determines the candidate basic fields and candidate multidimensional operation fields of natural query statements, thereby constructing a rich training data set for training, so that the model can accurately understand multidimensional query statements, candidate fields and other information, and generate query information that meets expectations. The final trained text processing model can show higher accuracy and efficiency when processing similar tasks, improving the performance of the overall system and user experience.

[0059] In one embodiment, the candidate basic fields and candidate multidimensional operation fields related to the first multidimensional query statement in step S302 are: a first basic field related to the first multidimensional query statement and a first multidimensional operation field bound to the first basic field, determined from the constellation data model;

[0060] And / or, in step S302, the candidate basic fields and candidate multidimensional operation fields related to the first multidimensional query statement are: the second basic fields and the second multidimensional operation fields related to the first multidimensional query statement, which are determined from the user-defined multidimensional operation fields and the custom operation information corresponding to the customized multidimensional operation fields.

[0061] In this embodiment, two schemes are described that can be used to construct the training data set of the above embodiment to train the original first text processing model. Specifically, first, candidate basic fields and candidate multidimensional operation fields can be selected from the constellation data model, and first multidimensional query statements of various situations can be constructed based on this. These multidimensional query statements will be used to train the first text processing model. Therefore, various variant forms should be used as much as possible to enrich the scenarios covered by the training set, and the overall training sample set should involve various basic fields and multidimensional operation fields of the constellation data model as much as possible, so that the trained text processing model can cope with various basic fields and multidimensional operation fields involved in the constellation data model. Second, determine from the user-defined multidimensional operation field, the system checks whether the user has defined a new multidimensional operation field and its corresponding operation definition information. These user-defined multidimensional operation fields may not have been bound to the constellation data model, so they also need to be paid attention to, and the corresponding training data set is constructed for training, so that the model can understand and recognize the newly added customized multidimensional operation fields. Among them, the construction idea of ​​the training data set is the same as the first point above, and will not be repeated here.

[0062] This embodiment determines the relevant candidate basic fields and candidate multidimensional operation fields of the first multidimensional query statement in the training data set through the multidimensional operation fields that have been bound to the constellation data model and the newly added user-defined multidimensional operation fields, that is, it can be to collect various natural statements involving multidimensional queries, and identify the relevant candidate basic fields and candidate multidimensional operation fields based on the constellation data model and / or the user-defined multidimensional operation fields, or from another perspective, it can be said that various first multidimensional query statements prepared for training are constructed based on the constellation data model and / or these user-defined multidimensional operation fields. This embodiment optimizes the understanding ability of the text processing model in terms of queries involving multidimensional operation fields, and can support complex data analysis and query requirements.

[0063] In one embodiment, the method further comprises the steps of:

[0064] In response to a user's update operation on the constellation data model, acquiring update information of the constellation data model;

[0065] If the update information indicates that the user has bound a custom multidimensional operation field in the constellation data model, obtaining the custom multidimensional operation field and custom operation information corresponding to the custom multidimensional operation field;

[0066] According to the custom multi-dimensional operation field and the custom operation information corresponding to the custom multi-dimensional operation field, construct a training data set in the form of the training data set of step S302;

[0067] According to step S303 and step S304, or according to the training process of step S303 and step S305, new training is added to the trained text processing model.

[0068] In this embodiment, in response to the addition of a new multidimensional operation field bound to the constellation data model, the text processing model is supplemented with fine-tuning training by constructing a corresponding training data set, so that the text processing model can recognize the newly defined multidimensional operation field. The custom operation information is used to determine the operation definition information of the custom multidimensional operation field when the MDX query engine queries.

[0069] In one embodiment, after the step of step S304 until the loss value is less than a preset loss threshold or the total number of weight adjustments reaches a preset number of adjustments, the step further includes:

[0070] Step S401, obtaining a verification data set; the verification data set includes a plurality of verification samples, each verification sample includes a second multidimensional query statement, candidate basic fields and candidate multidimensional operation fields related to the second multidimensional query statement, the task execution text, and query tag information corresponding to the second multidimensional query statement;

[0071] Step S402, inputting the verification data set into the first text processing model to generate query information corresponding to each verification sample;

[0072] Step S403, evaluating the accuracy of the model output according to the query information of each verification sample and the corresponding query label information;

[0073] Step S404: if the accuracy is less than a preset accuracy threshold, the task execution text is updated, and the training data set is reconstructed based on the updated task execution text;

[0074] Step S405, use the reconstructed training data set to re-initialize the low-rank matrix weights and re-adjust the low-rank matrix weights until the accuracy of the verification data set on the first text processing model is not less than the preset accuracy threshold or the total number of weight adjustments reaches the preset maximum number of adjustments, thereby obtaining the trained text processing model.

[0075] This embodiment introduces the acquisition of a validation data set to evaluate the performance of the current model. Specifically, the validation data set includes multiple validation samples, each of which contains all the information required to evaluate the performance of the model, wherein the second multidimensional query statement, the relevant candidate basic fields and the candidate multidimensional operation fields, and the task execution text are used to construct a task processing instruction to input the model to obtain the output result of the model and verify the accuracy of the model output result through the corresponding query tag information. If the calculated accuracy is less than the preset accuracy threshold, it means that the performance of the current model has not yet met the requirements and needs to be optimized. The optimization method is to change the task execution text by updating the task execution text, which can be specifically to adjust the task description information, task instructions or context information in the task execution text to guide the model to better understand the query intent. And based on the updated task execution text, the training data set is reconstructed. Specifically, the training data set of the above embodiment can be continued, and only the task execution text therein is replaced with a new task execution text. Using the reconstructed training data set, the initialization and adjustment process of the low-rank matrix weight is restarted, specifically referring to the above embodiment. This process will be repeated until the accuracy of the model on the validation data set is not less than the preset accuracy threshold, or the total number of weight adjustments reaches the preset maximum number of adjustments.

[0076] This embodiment achieves dynamic adjustment and optimization of the model training process by introducing a validation data set and a model performance evaluation mechanism. When the performance of the model still does not meet the requirements after preliminary training, the model can be trained on data closer to actual applications by updating the task execution text and rebuilding the training data set, thereby improving its generalization ability and accuracy. This iterative training and optimization process not only improves the performance of the model, but also enhances the robustness and adaptability of the model, so that the final text processing model can better meet the needs of actual applications.

[0077] In one embodiment, when the total number of weight adjustments reaches a preset number of adjustments, step S305 determines the first text processing model with the smallest loss value from the cached first text processing models, and after this step, for the first text processing model with the smallest loss value, the process from step S401 to step S405 is performed until the accuracy of the verification data set on the first text processing model is not less than the preset accuracy threshold or the total number of weight adjustments reaches the preset maximum number of adjustments, and the first text processing model is determined as the trained text processing model. For the specific description of this embodiment, please refer to the above embodiment, which will not be repeated here.

[0078] In step S101, a natural query sentence input by a user is obtained.

[0079] In this embodiment, the user can input the query requirements in the form of natural language through the user query interface provided by the system (such as a web page, mobile application or command line interface). For example, the user may input "query the names and sales of the top 10 products with the highest sales in the first quarter of 2023", or for another example, the user may input "query the total sales and year-on-year growth rate of each sales region in the first quarter of this year". This step is the starting point of the data query process and provides the necessary input for subsequent steps.

[0080] For step S102, a constellation data model is obtained; and several relevant basic fields are determined from the constellation data model as candidate basic fields according to the natural query statement; wherein the constellation data model records several data tables, basic fields contained in each data table, and multi-dimensional operation fields bound to at least some of the basic fields.

[0081] Among them, the constellation data model is an abstract representation used to describe the database structure. It is a data logic layer established in advance based on various data tables stored in the data warehouse. It records several data tables, the basic fields contained in each data table, and the graph relationship of each data table. In addition, it also records the binding relationship between at least some basic fields and multi-dimensional operation fields. In other words, the constellation data model does not record the specific business data of the fields of each data table, but only records the relevant information used to generate query statements. This information mainly includes the table name of the data table, the field information of the data table, and the multi-dimensional operation field information bound to the basic field. In addition, the graph relationship between each data table is established in the constellation data model, and the graph relationship at least includes a directed graph relationship.

[0082] Among them, the basic field is generally a field that records the original business data in the database. More specifically, the basic field is generally divided into a metric field and a dimension field. The metric field generally refers to the field in the fact table, and the dimension field generally refers to the field in the dimension table. Among them, the fact table and the dimension table are concepts in data modeling in the field of data warehouses. At present, the main data models involved in the field of data warehouses are star data models, snowflake data models generated on the basis of star data models (which can actually be considered as a type of star data model) and constellation data models. In the star data model, it includes a data table located in the center and other data tables connected to the center data table, wherein the center data table is the fact table, and the other data tables connected to it are dimension tables.

[0083] Among them, the multidimensional operation field, also known as the MDX operation field, is different from the basic field. It is not a native field recorded in the database, but a field obtained by definition. The data queried by it is obtained by performing complex operations (such as complex calculations) based on the data of the basic field in the database. Specifically, multidimensional operation fields for various complex operations or various data analysis indicators can be defined according to the needs of data analysis. In one embodiment, the multidimensional operation field can be "same period", "previous period", "year-on-year", "month-on-month", "global proportion", "group proportion", "global ranking" and "group ranking", etc. Obviously, the query of the multidimensional operation field involves the calculation or comparison of the data of the basic field in multiple dimensions. Unlike the query of the basic field, it only needs to obtain the data corresponding to the basic field from the database, but requires more complex analysis and processing involving multiple dimensions and / or multiple basic data. In addition, according to the needs of data analysis, users can also customize more multidimensional operation fields and corresponding calculation formulas. The query of these multidimensional operation fields often contains important information of business logic and data analysis, and is an indispensable part of deep data exploration, but it is also flexible and changeable in different industries and application scenarios, and is often difficult to be directly understood and recognized by existing large language models.

[0084] In the embodiment of the present application, at least some of the basic fields of the constellation data model are bound to multidimensional operation fields. This binding is generally established in advance by the user according to the query requirements, so that when the user enters natural language to query, the required data analysis results can be accurately and quickly queried. This binding mechanism allows any basic field to be associated with one or more multidimensional operation fields. Different multidimensional operation fields define different specific operations that can be performed on the business data represented by the basic fields to which they are bound, greatly enhancing the flexibility and dynamism of data processing.

[0085] This step first loads or accesses the constellation data model, and then selects basic fields related to the query content, such as "product name" and "sales", as candidate basic fields based on the natural query statement entered by the user. These candidate basic fields are the column names that may be needed when constructing the query statement later.

[0086] In one embodiment, the step of obtaining the natural query statement input by the user in step S101 further includes:

[0087] Get the constellation data model identifier selected by the user;

[0088] The step of obtaining the constellation data model in step S102 includes:

[0089] The corresponding constellation data model is acquired according to the constellation data model identifier.

[0090] In this embodiment, when the user enters a natural query statement in the query interface, the user can also select the target data warehouse to be queried, so that the system obtains the constellation data model identifier corresponding to the target data warehouse. In step S102, the system obtains the data of the corresponding constellation data model according to the constellation data model identifier, that is, the structural modeling data of the target data warehouse selected by the user. Through this embodiment, it means that the user can specify to query in different target data warehouses. These different data warehouses are established based on different business scenarios or different user subjects. Therefore, the data that can be queried is also different. Accordingly, the user expresses appropriate natural query statements according to the query purpose and the selected data warehouse to ensure that the desired business data can be queried in the correct data warehouse.

[0091] In summary, this embodiment introduces the constellation data model identifier selected by the user as part of the query condition, and accurately obtains the corresponding constellation data model based on it, thereby further refining and personalizing the natural query statement, ensuring that the subsequent processing process is based on the user's real intention and preference, thereby providing query results or services that are more in line with user needs. This technical improvement not only improves the convenience and satisfaction of user queries, but also enhances the system's ability to process complex query requests, making the final query results or services more accurate and comprehensive, and providing users with a richer and more interesting constellation-related content experience.

[0092] Please refer to Figure 2 In one embodiment, the step of determining a number of related basic fields as candidate basic fields from the constellation data model according to the natural query statement in step S102 includes:

[0093] Step S1021, performing keyword recognition on the natural query sentence to obtain a number of keywords;

[0094] Step S1022, vectorizing each of the keywords and each basic field in the constellation data model; calculating the vector similarity between any of the keywords and any of the basic fields;

[0095] Step S1023, for any of the keywords, determining a first preset number of basic fields that are ranked high in similarity to the keyword as relevant basic fields of the keyword;

[0096] Step S1024, ranking the relevant basic fields of each keyword by similarity according to the relevant basic fields of each keyword and their corresponding similarities;

[0097] Step S1025 : Determine a second preset number of related basic fields with top similarity rankings as candidate basic fields.

[0098] The first preset number and the second preset number can be set according to specific circumstances. In this embodiment, the first preset number is set to 8, and the second preset number can be set to 40.

[0099] This embodiment describes in detail how to filter out the basic fields related to the natural query statement input by the user from the constellation data model as the candidate basic fields. First, in step S1021, the natural query statement is identified by keywords, and a number of keywords that can represent the user's query intention are extracted using natural language processing technology (such as word segmentation, part-of-speech tagging, etc.). These keywords are the basis for subsequent analysis and are crucial for accurately understanding user needs. Then, in step S1022, the identified keywords and each basic field in the constellation data model are converted into a vector representation in a high-dimensional space using vectorization technology. Vectorization is a common method for processing text data. It can convert text information into a numerical form that is easy for a computer to process, which is convenient for subsequent similarity calculation. Then, the vector similarity between each keyword and each basic field is calculated. Similarity calculation is a key step in evaluating the proximity of two vectors in a high-dimensional space, and its result can reflect the correlation between the keyword and the basic field. Furthermore, in step S1023, for each keyword, the first preset number of basic fields with a high similarity ranking are selected as the relevant basic fields of the keyword. Through this step, the basic fields most relevant to each keyword can be screened out, reducing the amount of data to be processed subsequently and improving the efficiency and accuracy of the query. Step S1024 performs a global similarity ranking on these basic fields according to the relevant basic fields of each keyword and their corresponding similarities. Through the global similarity ranking, the query requirements of multiple keywords can be comprehensively considered to ensure that the candidate basic fields finally selected can fully reflect the user's query intention, thereby improving the comprehensiveness and accuracy of the query results. Finally, in step S1025, the second preset number of relevant basic fields with the highest similarity ranking are determined as candidate basic fields. Through this step, a limited number of candidate basic fields are finally determined, which are most likely to meet the user's query requirements. This not only reduces the complexity and computational complexity of subsequent query processing, but also improves the response speed and user satisfaction of the query.

[0100] In summary, this embodiment achieves efficient and accurate conversion from natural query statements to candidate basic fields through steps such as keyword recognition, vectorization processing, similarity calculation, relevant basic field screening and global ranking, providing strong support for subsequent query processing.

[0101] In one embodiment, the step of ranking the related basic fields of each keyword by similarity according to the related basic fields of each keyword and their corresponding similarities in step S1024 includes:

[0102] Step S10241: If any two of the keywords have the same related basic fields and correspond to different similarities, the related basic field with the highest similarity among the same related basic fields is retained, and the remaining related basic fields are removed.

[0103] This embodiment further refines the specific implementation method of step S1024. Specifically, when it is found during the processing that any two or more keywords have the same relevant basic fields, but these same basic fields correspond to different similarities, a specific processing strategy is adopted. First, the system needs to identify which basic fields are considered relevant by multiple keywords, that is, there are repeated basic fields. For each basic field shared by multiple keywords, the system needs to compare the similarity between these keywords and the basic field. After comparing the similarities, the system will choose to retain the basic field with the highest similarity (with the corresponding keyword), and remove other (same) basic fields (that is, they will not participate in the global ranking).

[0104] For example, the natural query sentence input by the user is "What is the contract amount of the financial industry this year?" Assume that the determined keywords include "this year", "finance", "industry" and "contract amount". Among them, the related basic fields determined by the keyword "finance" and the corresponding similarities include "industry category" (0.84), "industry subcategory" (0.82), "contract amount" (0.77), etc.; and the related basic fields determined by the keyword "contract amount" and the corresponding similarities include "contract amount" (0.98), "average contract amount" (0.88), "number of contracts" (0.75), etc. Then, it can be found that the keywords "finance" and "contract amount" have the same related basic field "contract amount", but the corresponding similarities of the two are 0.77 and 0.98 respectively. Therefore, when all related basic fields are ranked globally, the related basic field with the greatest similarity is retained, that is, the "contract amount" with a similarity of 0.98 is included in the global similarity ranking, while the "contract amount" with a similarity of 0.77 is removed and does not participate in the global similarity ranking.

[0105] Finally, in the global similarity ranking of related basic fields, for two or more identical related basic fields (corresponding to different keywords), only the related basic field with the highest similarity will be retained to participate in the ranking. Therefore, two identical basic fields (these two identical basic fields correspond to different keywords with different similarities) will not appear in the global ranking.

[0106] This embodiment limits the sorting rules for the similarity of the relevant basic fields of keywords. For the case where different keywords correspond to the same relevant basic fields, by retaining the same relevant basic fields with the highest similarity and removing other same relevant basic fields with lower similarities, it is ensured that the second preset number of candidate basic fields finally determined contain as many different basic fields as possible, effectively reducing the redundant information in the candidate basic fields and expanding the scope of the candidate basic fields selected. On the whole, the candidate basic fields cover as many basic fields as possible, which is as comprehensive and relatively accurate as possible, laying a better foundation for the subsequent text processing model processing and improving the accuracy, efficiency and user experience of the final query.

[0107] Please refer to Figure 3 In one embodiment, the step of performing keyword recognition on the natural query statement to obtain a plurality of keywords in step S1021 includes:

[0108] Step S10211, segmenting the natural query sentence based on a preset segmenter to obtain a segmentation result;

[0109] Step S10212: Based on the pre-trained entity recognition model, entity recognition is performed on the natural query sentence and the word segmentation result to obtain a first entity recognition result and a second entity recognition result respectively;

[0110] Step S10213, obtaining a plurality of keywords according to the first entity recognition result and the second entity recognition result.

[0111] Among them, the word segmenter is a basic tool in natural language processing, and its main function is to divide continuous text strings into independent vocabulary units (tokens). There are many ways to implement the word segmenter, including rule-based word segmentation, statistical-based word segmentation, and hybrid methods. Rule-based word segmentation relies on a preset dictionary and rule set, and segments the text by matching the entries and rules in the dictionary. The statistical-based word segmentation uses statistical models (such as hidden Markov models, conditional random fields, etc.) to predict the boundaries of words in the text. The hybrid method combines the advantages of rules and statistics to improve the accuracy and efficiency of word segmentation. In practical applications, the choice of word segmenter depends on the specific language characteristics and task requirements. In this embodiment, a suitable word segmenter can be selected according to the specific scenario.

[0112] The entity recognition model is an important component in natural language processing, which is used to identify entities with specific meanings from text. These entities can be named entities (such as names of people, places, names of institutions, etc.), or other entities with specific meanings (such as time, date, numbers, etc.). Entity recognition models are usually built based on machine learning or deep learning technology, and learn the characteristics and patterns of entities in text by training a large amount of labeled data. During the training process, the model learns how to distinguish between entities and non-entity parts in the text, and identifies the types and boundaries of entities. This embodiment accurately identifies entities in the text through the entity recognition model, and can better determine the candidate basic fields related to natural query statements.

[0113] This embodiment describes in detail how to perform keyword recognition on natural query statements. This process aims to extract keywords that are crucial to subsequent query processing from natural language texts to more accurately determine relevant candidate basic fields. First, in step S10211, a preset word segmenter is used to perform word segmentation on the natural query statement. Through word segmentation, complex query statements can be decomposed into more easily processed vocabulary sequences. Then, in step S10212, a pre-trained entity recognition model is used to perform dual entity recognition on the natural query statement and its word segmentation results. By performing entity recognition on the original query statement and the word segmentation result respectively, the key information in the query statement can be captured more comprehensively and the accuracy of keyword recognition can be improved. The first entity recognition result is based on the overall understanding of the original query statement, while the second entity recognition result is more focused on the vocabulary unit after word segmentation. The two complement each other and together constitute an in-depth understanding of the query statement. Finally, in step S10213, several keywords are extracted based on the comprehensive analysis of the first entity recognition result and the second entity recognition result. These keywords include not only the explicit named entities in the query, but also some words with specific meanings or importance, which together constitute the core content of the query and provide key guidance for subsequent data retrieval and processing. In one embodiment, the entities in the first entity recognition result and the second entity recognition result are combined to obtain all the keywords of the natural query.

[0114] In summary, this embodiment achieves accurate recognition of keywords in natural query sentences by introducing word segmentation and dual entity recognition. This process not only improves the accuracy and comprehensiveness of keyword recognition, thereby ensuring accurate and comprehensive determination of candidate basic fields, but also provides strong support for subsequent query information generation and data retrieval.

[0115] For step S103, if the candidate basic field is bound with a multi-dimensional operation field, the bound multi-dimensional operation field is determined as a candidate multi-dimensional operation field.

[0116] After determining the candidate basic fields, this step further checks whether these basic fields are bound to multidimensional operation fields. If there are basic fields in the candidate basic fields that are bound to multidimensional operation fields, these multidimensional operation fields are also determined as candidate multidimensional operation fields for use in subsequent query processing.

[0117] For step S104, a task processing instruction is constructed according to the candidate basic fields, the candidate multidimensional operation fields, the natural query statement and the preset task execution text; wherein the task execution text is used to prompt the text processing model to output query information that conforms to the query intent of the natural query statement, and the query information is determined according to the candidate basic fields and the candidate multidimensional operation fields.

[0118] In this step, a task processing instruction is constructed using the candidate basic fields, the candidate multidimensional operation fields, the original natural query statement, and the preset task execution text. This task processing instruction is designed to guide the text processing model to select accurate basic fields and multidimensional operation fields from the candidate basic fields and the candidate multidimensional operation fields according to the intent of the natural query statement, and generate query information that meets the query intent.

[0119] Among them, the task execution text is used to require or prompt the task requirements of the text processing model, including specific requirements and format instructions for the model output, and may also include task-related scene information or context information for the model to learn and understand the task to be processed, etc. In the embodiment of the present application, different sub-texts can be set in the task execution text according to the needs of the query, respectively used to require or prompt the text processing model for the corresponding task requirements. For example, the task execution text may include a first task execution text, a second task execution text, a third task execution text... and so on, and different sub-texts contain specific requirements for model processing and output according to the task requirements, etc.

[0120] In one embodiment, the task execution text includes a first task execution text; the first task execution text is used to prompt the query information output by the text processing model to include a multidimensional operation field and a third basic field bound to the multidimensional operation field, the third basic field is determined from the candidate basic fields, and the multidimensional operation field is determined from the candidate multidimensional operation fields; the multidimensional operation field is used to specify multidimensional operations on business data of the third basic field.

[0121] The first task execution text is generally a command text, which is used to request or prompt the text processing model what task to perform, and which details the specific rules that the text processing model should follow when generating query information. Alternatively, the first task execution text can also be an instruction set in a specific format, which also details the specific rules that the text processing model should follow when generating query information. The first task execution text can be determined in what form according to the requirements of the text processing model for task processing instructions.

[0122] In this embodiment, the first task execution text prompts the text processing model, and the output query information includes a multidimensional operation field and a third basic field bound to the multidimensional operation field. This will guide the text processing model to deeply understand whether the natural query statement involves a query of the multidimensional operation field, and identify the multidimensional operation field and its corresponding basic field. Among them, the text processing model is specifically instructed to determine the third basic field from the candidate basic fields, and to determine the multidimensional operation field from the candidate multidimensional operation fields, guiding the text processing model to focus on these candidate fields. Since these candidate fields are all fields related to the user's query intent determined according to the constellation data model, therefore, it actually gives the text processing model enough information to enable it to accurately lock the query intent of the natural query statement and accurately determine the basic fields and / or multidimensional operation fields involved in this query.

[0123] It should be noted that those skilled in the art understand that the task execution text prompts what the text processing model should do, which does not mean that the text processing model must do it. For example, if the text processing model understands the natural query statement and believes that the query does not involve multidimensional operation fields, it may not output the multidimensional operation fields to avoid violating the core task of the task execution text prompt, that is, outputting query information that meets the query intent.

[0124] In one embodiment, the first task execution text is further used to prompt that the third basic field bound to the multidimensional operation field is determined from the candidate basic fields bound to the multidimensional operation field. As described in the embodiment of the present application, any basic field in the constellation data model can be bound to one or more multidimensional operation fields. Similarly, the same multidimensional operation field may be bound to multiple basic fields. Therefore, when determining the candidate basic fields and determining the candidate multidimensional operation fields based on the candidate basic fields, the binding relationship between the basic fields and the multidimensional operation fields can also be recorded, and these binding relationships can be used to generate task processing instructions. When the text processing model recognizes that the query intent involves a multidimensional operation field, the third basic field can be determined based on the candidate basic fields bound to this multidimensional operation field.

[0125] In one embodiment, the task execution text also includes a second task execution text; the second task execution text is used to prompt that the query information output by the text processing model includes a first query condition, and the first query condition is determined according to the candidate basic field; the first query condition is used to limit the conditions that the business data queried by the third basic field needs to meet.

[0126] In this embodiment, the second task execution text prompts the text processing model to include the first query condition in the output query information, and constructs it based on the candidate basic fields to further limit the scope of the query results and ensure that the final query results meet the limited conditions.

[0127] In one embodiment, the constellation data model further records the data type of each basic field, and different data types correspond to different candidate conditional operators; the first query condition includes a fourth basic field and a conditional operator; the task execution text also includes a fourth task execution text; the fourth task execution text is used to prompt a text processing model, the fourth basic field in the first query condition is determined from the candidate basic fields, and the conditional operator in the first query condition is determined from a set of candidate conditional operators;

[0128] After the step of determining a number of related basic fields as candidate basic fields from the constellation data model according to the natural query statement in step S102, the following steps are also included:

[0129] Step S1026, determining a candidate conditional operator for each of the candidate basic fields according to the data type of each of the candidate basic fields;

[0130] Step S1027, obtaining the candidate conditional operator set according to the candidate conditional operators of each of the candidate basic fields;

[0131] The step of constructing a task processing instruction according to the candidate basic fields, the candidate multi-dimensional operation fields, the natural query statement and the preset task execution text in step S104 includes:

[0132] Step S1041 , constructing a task processing instruction according to the candidate basic fields, the candidate multi-dimensional operation fields, the candidate conditional operator set, the natural query statement and the preset task execution text.

[0133] In this embodiment, special attention is paid to the impact of differences in data types on query conditions, and the construction process of the task execution text is adjusted accordingly to improve the flexibility and accuracy of user operations.

[0134] In step S1026, the system first reads the data type information of each candidate basic field recorded in the constellation data model. Since different data types support different query condition operations (such as equal to, greater than, less than, contains, etc.), the system will select the condition operator suitable for the data type from the predefined condition operator set according to the specific data type of each candidate basic field to form a candidate condition operator list for the candidate basic field. For example, for numeric fields, candidate condition operators may include "=", ">", "<", ">=", "<=", etc.; and for string type fields, they may include "=", "LIKE" (fuzzy matching), etc.

[0135] Step S1027 is a summary of the results of the previous step. The system traverses the candidate conditional operator lists of all candidate basic fields, removes duplicates, and finally forms a set containing all unique candidate conditional operators, namely, the candidate conditional operator set. This set will be used in subsequent steps to help the text processing model select a suitable operator from the candidate conditional operator set when determining the conditional operator in the first query condition.

[0136] In the process of constructing the task processing instruction, step S1041 not only considers the candidate basic fields, candidate multidimensional operation fields and natural query statements, but also adds the consideration of the candidate conditional operator set. At the same time, a new guidance content is introduced into the task execution text - the fourth task execution text. This text is specifically used to prompt the text processing model to select the fourth basic field from the candidate basic fields and select the appropriate conditional operator from the candidate conditional operator set, so as to obtain the first query condition.

[0137] In summary, this embodiment significantly enhances the flexibility and accuracy of data query processing by introducing detailed management of candidate basic field data types and precise matching of candidate conditional operators. It not only allows users or the system to automatically select appropriate query conditional operators based on data types, but also further improves the efficiency and accuracy of query information output by the text processing model through prompts of task execution text, which helps reduce query errors caused by data type mismatch or misuse of conditional operators, thereby improving the reliability and stability of the entire data processing process.

[0138] In one embodiment, the fourth task execution text is further used to prompt a text processing model, and the conditional operator in the first query condition is determined from a subset of candidate conditional operators corresponding to the fourth basic field;

[0139] The step of obtaining the candidate conditional operator set according to the candidate conditional operators of each of the candidate basic fields in step S1027 includes:

[0140] According to the candidate conditional operators of each of the candidate basic fields, a subset of candidate conditional operators corresponding to each of the candidate basic fields is determined; according to each of the candidate conditional operator subsets and a binding relationship between each of the candidate conditional operator subsets and the corresponding candidate basic field, the candidate conditional operator set is obtained.

[0141] The text processing model of this embodiment further provides a clearer prompt. When determining the first query condition, it is necessary to determine a condition operator from the corresponding candidate condition operator subset according to the determined fourth basic field, thereby further ensuring the accuracy of the first query condition.

[0142] For step S105, the task processing instruction is input into the trained text processing model to obtain output query information.

[0143] In this step, the constructed task processing instructions are sent to the trained text processing model. The model analyzes the intent of the natural query statement according to the requirements in the instruction, combines the candidate basic fields and the candidate multi-dimensional operation fields, and generates query information that meets the query requirements / intent. The query information is usually expressed in a structured format (such as JSON), which contains the specific field names and possible query logic used for database query.

[0144] For step S106, according to the query information and the constellation data model, corresponding service data is queried from a database corresponding to the constellation data model.

[0145] This step builds a complete database query statement based on the query information output by the text processing model and the constellation data model. This query statement is sent to the database for execution to retrieve business data that meets the user's query intent. This business data may include data from basic fields, calculation results of multi-dimensional operation fields, etc. This data can then be returned to the user or used for further analysis and processing.

[0146] In one embodiment, the task execution text includes a third task execution text, and the third task execution text is used to prompt the text processing model to output the query information according to a preset report query request format;

[0147] The step of querying and obtaining corresponding service data from a database corresponding to the constellation data model according to the query information and the constellation data model in step S106 includes:

[0148] Step S1061, using the query information as a report query request, and inputting the report query request and the constellation data model into an MDX query engine; obtaining corresponding business data from a database corresponding to the constellation data model through the MDX query engine; wherein the MDX query engine is an engine for parsing report query requests and performing data queries based on the parsing results and the constellation data model.

[0149] Among them, the MDX (Multidimensional Expressions) query engine is a component specially designed to process complex report queries. It can understand and parse query requests written in the MDX query language, and retrieve corresponding business data from the database based on these requests and the definition of the constellation data model (multidimensional data structure model, commonly used in OLAP systems).

[0150] In this embodiment, for the understanding of the MDX query engine, specific reference can be made to the working principle of the Mondrian engine (an open source online analytical processing engine developed based on Java). In the MDX query engine, it is necessary to combine the data model description information and the MDX statement for querying. The MDX statement is a query statement with a similar grammatical structure to the SQL statement. In the SQL statement or the MDX statement, the general format is: select query field from query data source; when there is a field that needs to be defined, the query statement also includes "with field operation definition" before the select field; when there is a query restriction condition that needs to be limited, the query statement also includes "where query restriction condition" after the from field. Among them, the query data source generally specifies the target data table of the target database that the user wants to query, so it generally fills in the table name of the data table. In some cases, it can also fill in another query statement (the query statement actually determines a set of data, and this set of data can also be considered to constitute a data table).

[0151] Specifically, the data model description information is generally a Schema XML (Extensible Markup Language) file, which describes the database table description information, measurement description information, dimension description information, and multidimensional operation field (MDX operation field) description information (including calculated member description information and / or named set description information) of a data model cube; the query data source of the MDX statement is determined to be the model name defined in the data model description information (generally speaking, the Schema XML defines the model name as cube, which corresponds to from cube in the MDX statement). In the MDX query engine, for a multidimensional operation field (MDX operation field), corresponding operation definition information (such as a calculation formula) is required to perform the corresponding operation. Specifically, the definition of the multidimensional operation field can be defined in the with part of the MDX statement, or in the data model description information (Schema XML file).

[0152] In this embodiment, the MDX query engine is a multidimensional query engine obtained by secondary development based on a general multidimensional analysis engine such as the Mondrian engine, that is, it not only executes queries according to MDX statements and data model description information like a general multidimensional analysis engine, but also generates MDX statements and data model description information before that. For details, please refer to the prior art description, and also refer to the description involved in the subsequent embodiments.

[0153] In this embodiment, the function of the task execution text is further expanded, and a third task execution text is introduced. The text has a specific function - to guide the text processing model to organize and output the query information according to the preset report query request format. Specifically, the third task execution text can set prompt texts one by one according to the preset report query request format, so as to require the text processing model to adjust the format of the query information according to the prompt requirements one by one, so as to finally output the query information that conforms to the report query request format. This design is intended to ensure that the format of the query information matches the expected input format of the backend data system (especially the database system that supports MDX queries), thereby improving the accuracy and efficiency of data queries.

[0154] Specifically, when performing data query, the query information output by the text processing model is directly submitted to the MDX query engine as a report query request. The MDX query engine receives the report query request, parses the target field of the user's query and the query restriction conditions involved, and then uses the constellation data model as a navigation map to perform query operations in the database and finally returns the business data that meets the conditions.

[0155] In summary, this application significantly enhances the interaction between the text processing model and the backend data system by introducing the third task execution text and defining the implementation process of step S106 in detail. Specifically, it ensures that the format of the query information is highly consistent with the input requirements of the MDX query engine, so that based on the existing technical architecture of the MDX query engine, it is only necessary to directly input the query information as a general report query request into the MDX query engine to perform the query.

[0156] In one embodiment, the MDX query engine in step S1061 obtains the corresponding business data by querying in the following manner:

[0157] Step S10611, parsing the report query request according to a preset parsing method to obtain a target field for query; determining a basic field according to the target field;

[0158] Step S10612, determining a plurality of star data models including the basic fields from the constellation data model; wherein each of the star data models respectively records a fact table, a dimension table, and the basic fields respectively included in each data table;

[0159] Step S10613, if the target field includes a multidimensional operation field, obtain corresponding operation definition information according to the multidimensional operation field; generate data model description information according to each of the star data models and the operation definition information; generate an MDX statement according to the report query request and the operation definition information; query the query result of the target field from the database corresponding to the constellation data model according to the MDX statement and the data model description information; determine that the query result of the target field is the business data obtained by the query;

[0160] Step S10614, if the target field does not include a multidimensional operation field, generate data model description information according to each of the star data models; generate an MDX statement according to the report query request; according to the MDX statement and the data model description information, query the query result of the target field from the database corresponding to the constellation data model; determine that the query result of the target field is the business data obtained by the query.

[0161] Among them, in step S10611, the report query request is parsed to obtain the target field that the user wants to query. Specifically, in different query requirements, the target field obtained by parsing is generally different. In this embodiment, the target field obtained by parsing the report query request includes basic fields and / or multi-dimensional operation fields. Generally speaking, the target field queried by the report query request may include only basic fields (meaning that only basic data in the database is queried), or only multi-dimensional operation fields (meaning that only a certain data analysis result or a certain specified multi-dimensional data operation is queried, such as the year-on-year growth rate, without querying specific basic data), and may also include both basic fields and multi-dimensional operation fields (meaning that both basic data and data obtained by specified multi-dimensional data operations are queried). Among them, if the target field of the query includes a multidimensional operation field, the multidimensional operation field will also be bound to at least one basic field - that is, the multidimensional operation field actually specifies the operation of the bound basic field. However, it should be noted that in this case, the multidimensional operation field is the target field of the query, but the basic field bound to it is not the target field of the query. For example, the query request queries the "sales volume" "same period growth rate", where "sales volume" is the target field of the query, and "sales volume" is just the basic field referenced by the target field. The final query result should be the query result of "sales volume" and does not include the data of the "sales volume" field (unless the basic field "sales volume" is also the target field of the user's query). This step determines the basic field related to the query based on the target field (same as above, not necessarily the target field of the query), so that in step S10612, the star data model related to this query can be determined based on the basic field. Specifically, if the target field of the query is a basic field, then it is the basic field related to the query; if the target field of the query is a multidimensional operation field, then the basic field bound to it is the basic field related to the query. In some special cases, there is another type of aggregate field. This type of field is associated with a basic field. The associated basic field is the basic field related to the query. For example, for the aggregate field "average sales volume", the "sales volume" field is the basic field related to the query.

[0162] In step S10612, the star data model is determined from the constellation data model. Generally speaking, in a complex constellation data model, if different data tables are centered, different star data models can be determined. A data table can be used as a fact table in a star data model and as a dimension table in other star data models.

[0163] In this step, the main goal of determining the star data model from the constellation data model is to obtain the minimum available star data model related to the query. The minimum available star data model should only contain data tables related to the query, and the data tables should only contain basic fields determined in step S10611, wherein the data tables related to the query are divided into several cases: for dimension tables, they are considered to be data tables related to the query only when they contain the basic fields determined in step S10611 or are bound to query-related restriction information; there is no limitation for fact tables. Generally speaking, a fact table is a necessary data table in a star data model, even if it does not contain basic fields. There are many ways to determine the minimum available star data model, and specific reference may be made to the prior art in the field, which will not be elaborated here.

[0164] In step S10613, the operation definition information is used to define the multidimensional operation mode corresponding to the multidimensional operation field, which can also be said to define what kind of operation is performed on the data of the basic field bound to the multidimensional operation field. Take an example of querying in an existing report query interface (including various query components) for explanation, assuming that the predefined multidimensional operation field is "sales volume" and the user selects the "sales volume" metric field during the query, and selects the parameter of its "time" dimension field as 2024, and selects the "sales volume" multidimensional operation field, submits the query, then, when the backend processes the query, it obtains the corresponding definition formula according to the "sales volume", such as "sales volume = (current data - previous data) / previous data", and substitutes the specific basic field (metric field and / or dimension field) and related parameters into the definition formula to obtain the operation definition information of the "sales volume" of the query, such as "sales volume in 2024 - sales volume in 2023) / sales volume in 2023". Of course, this example is only for illustrative purposes to facilitate understanding of the multidimensional operation fields and their operation definition information. The specific implementation details can be set according to the situation. How each of the star data models generates data model description information and how the report query request generates an MDX statement are prior arts in the art. Please refer to the prior arts for details and will not be elaborated here.

[0165] In step S10614, the database includes data of various data tables corresponding to the constellation data model. That is, the database is actually the basis for modeling the constellation data model. In one embodiment, the database is preferably an OLAP (On-Line Analytical Processing) database.

[0166] In this embodiment, the basic field is determined according to the target field queried by the user, and several star data models containing the basic field are determined. If the target field includes a multidimensional operation field, the corresponding operation definition information is obtained according to the multidimensional operation field through the MDX query engine; data model description information is generated according to each of the star data models and the operation definition information; MDX statements are generated according to the report query request and the operation definition information; and query results of the target field are obtained from the database according to the MDX statements and the data model description information. If the target field does not include a multidimensional operation field, data model description information is directly generated according to each of the star data models, and MDX statements are directly generated according to the report query request; the subsequent steps are similar according to the MDX statements and the data model description information. Through this embodiment, the MDX query engine can accurately parse the report query request, generate data model description information and MDX statements, and thus, in combination with the data model description information and the MDX statements, it can efficiently query the query results of the target field from the database, and determine that the query results of the target field are the business data obtained by the query.

[0167] Furthermore, considering that existing text processing models have certain limitations in processing complex logic and contextual understanding, they are still unable to cope with complex query conditions, such as time range screening, sorting requirements, quantity restrictions, etc., and are difficult to accurately parse and convert into effective data query statements. In addition, the data models in different industries and application scenarios are complex and diverse, the associations between data tables, the data types and constraints of fields are different, and the complex query conditions involved in users' queries may also vary, further increasing the difficulty of the text processing model to accurately generate query statements. In this regard, there is an urgent need for a data query method that can effectively cope with complex query conditions and fully utilize the advantages of text processing models. The following embodiments of the present application further provide a solution.

[0168] In one embodiment, after the step of obtaining the natural query statement input by the user in step S101, the following steps are included:

[0169] Step S1011, inputting the natural query statement into a preset query condition recognition module to obtain a second query condition; the query condition recognition module is used to recognize a query condition of a specified type;

[0170] Before the step S103 of determining the bound multidimensional operation field as a candidate multidimensional operation field if the candidate basic field is bound to the multidimensional operation field, the method includes the following steps:

[0171] Step S1031, removing the basic fields related to the query condition of the specified type from the candidate basic fields;

[0172] Before the step of using the query information as a report query request in step S1061, the following steps are included:

[0173] The second query condition is merged into the query information according to the report query request format.

[0174] The query condition recognition module is a trained or configured module that can recognize a specified type of query condition in a natural query statement, and can specifically be a computer program. In this embodiment, the query condition recognition module recognizes these conditions as the second query condition to provide necessary query constraints for subsequent steps.

[0175] It is understandable that the second query condition and the first query condition are essentially limiting conditions for the query results. The difference is that in the embodiment of the present application, the second query condition is determined by the query condition recognition module, while the first query condition can be directly determined by the text processing model. In other words, the first query condition is a condition that the text processing model has the ability to accurately identify, so it does not need to be identified by the query condition recognition model or by a custom recognition rule. The second query condition of this embodiment meets the characteristics of the above-mentioned complex query condition and needs to be identified by the query condition recognition module.

[0176] The specified type of query condition, in this embodiment, refers to the query condition that is difficult for the text processing model to accurately understand and identify. In layman's terms, it refers to complex query conditions, such as time conditions, quantity restriction conditions, sorting conditions, etc. that are currently found to be generally unable to be accurately identified in various text processing models. These can be set as the specified type of query conditions. For example, time conditions ("first quarter of 2023"), sorting conditions ("sorted in descending order by sales") or quantity restriction conditions ("top 10").

[0177] It should be noted that in the embodiments of the present application, the complexity of the complex query conditions refers to the text processing model - if a certain type of query condition cannot be accurately identified in a certain text processing model but can be accurately identified in another text processing model, then the text processing model for the former is a complex query condition, but the latter may not be considered a complex query condition (correspondingly, this type of query condition can be used as the first query condition to allow the latter text processing model to directly identify it, without being identified through the query condition identification module, and without setting corresponding identification rules in the query condition identification module).

[0178] Generally speaking, data models in different industries and application scenarios are complex and diverse, and the relationships between data tables, field data types and constraints are different. Therefore, the complex query conditions involved may also vary. Therefore, the query conditions of the specified type can be determined according to the specific scenario.

[0179] In this embodiment, it is defined that after receiving the natural query sentence input by the user in step S101, it is passed to the preset query condition recognition module to identify the second query condition. In addition, since the text processing model cannot accurately process complex query conditions (query conditions of a specified type), in order to avoid it trying to understand and output these conditions (these outputs are likely to be wrong and interfere with the final query), it is defined that before step S103, those basic fields in the candidate basic fields that are directly related to the query conditions of the specified type are first eliminated. This can ensure that the text processing model only pays attention to and processes the basic query information in subsequent steps, and will not try to understand and output these complex query conditions. Correspondingly, if the query condition recognition module identifies the second query condition, then in step S106, when performing data query, it is necessary to combine the second query condition, query information and constellation data model to perform data query. In step S1061, the query information cannot be directly used as a report query request, and the second query condition should be merged into the query information according to the report query request format to obtain complete query information, and then the merged query information is used as a report query request. The process of merging the second query condition into the query information may involve formatting the query condition, replacing or supplementing parameters, and reorganizing the query information to ensure that they comply with the format of the report query request and the input requirements of the MDX query engine.

[0180] In summary, this embodiment achieves accurate response to complex data query needs by cleverly combining the text processing model with the customized query condition identification rules, and significantly improves the overall effect of data query. This method not only overcomes the limitations of the text processing model in the prior art in understanding complex query conditions, but also reduces the pressure on the text processing model through the task separation strategy, making the identification of column names more efficient and accurate. At the same time, it enables the generation of basic query information and the identification of complex query conditions to be processed in parallel, thereby optimizing the entire query process. At the same time, since the identification of query conditions of a specified type is independent of the generation of basic query information, the system can flexibly adjust the rules according to actual needs, quickly adapt to different query scenarios and data model changes, and enhance the flexibility and scalability of the system.

[0181] Please refer to Figure 4 ,In one embodiment, the query condition of the specified type includes a time condition;

[0182] In step S1011, the query condition identification module identifies the time condition in the following manner:

[0183] Step S10111: if a time entity in the natural query sentence is identified by the pre-trained time entity recognition model, the time entity is segmented based on the pre-trained time entity segmenter to obtain at least one time entity segmentation; wherein different time entity segmentations correspond to different time dimensions;

[0184] Step S10112, identifying at least one time value covered by each of the time entity segmentations;

[0185] Step S10113, identifying the time dimension corresponding to each of the time entity segmentations;

[0186] Step S10114, obtaining at least one digital format time corresponding to each of the time entity segmentations according to the time dimension corresponding to each of the time entity segmentations, at least one time value covered, and a preset time digital format;

[0187] Step S10115, obtaining the time condition according to the digital format time corresponding to each of the time entity segmentations.

[0188] Among them, the Time Entity Recognition Model is an important tool in the field of natural language processing (NLP), which aims to automatically identify and extract time-related entity information from text data. These time entities can include specific dates (such as "July 26, 2024"), times (such as "3 pm"), time periods (such as "last Wednesday to this Friday"), and more complex time expressions (such as "three days after May Day"). Specifically in this embodiment, the JioNLP model can be used to identify time entities. In one example, the question "the contract amount from June to August in the past three years" identifies the time entity as "June to August in the past three years". The time entity recognition model is essentially a model trained by machine learning or deep learning technology, which can understand the semantic information in natural language text, identify and mark the time entities in the text. These models are usually trained based on large-scale annotated data sets to learn the characteristics and patterns of various time expressions.

[0189] A temporal entity tokenizer is a natural language processing (NLP) tool specifically designed to process time-related text. Its main function is to further subdivide the temporal entities (such as date, time, time period, etc.) identified in the text into smaller units with clear time dimensions (such as year, month, day, hour, minute, second, etc.), namely temporal entity tokenizers. These tokenizers not only retain the overall meaning of the original temporal entity, but also reveal its internal temporal structure through decomposition, which facilitates subsequent processing and analysis. Temporal entity tokenizers are usually built based on deep learning models, such as recurrent neural networks (RNNs), long short-term memory networks (LSTMs), or Transformers, which can capture the temporal information and contextual dependencies in the text, thereby accurately identifying and segmenting temporal entities. During the training process, the model will learn a large amount of annotated temporal entity data and master the tokenization rules and patterns of different temporal expressions.

[0190] In this embodiment, detailed operation steps are described for the specific type of time condition in the query condition identification process. Specifically, step S10111 first uses a pre-trained time entity recognition model to scan the natural query statement to identify all entities that may represent time in the statement, such as "tomorrow", "last Wednesday", "January 2023", etc. Then, the identified time entity is further processed by the pre-trained time entity segmenter to split it into finer-grained units. These segmentations represent different time dimensions, such as year, month, day, hour, minute, etc. For example, "January 2023" is segmented into "2023" and "January", and "June to August in the past three years" is segmented into "nearly three years" and "June to August". Step S10112 identifies and extracts at least one time value covered by each time entity segmentation. These values ​​are key information for constructing time conditions. For example, the time values ​​extracted from the time entity segmentation "in the past three years" (using this year as 2024 for illustration) include [2022, 2023, 2024], and the time values ​​extracted from the time entity segmentation "June to August" include [6, 7, 8]. At the same time, step S10113 also needs to determine the time dimension corresponding to each time entity segmentation, that is, whether it represents years, months, days or other time units, to ensure that the meaning of the time value can be correctly understood in subsequent processing. Step S10114 constructs a standardized digital format time based on the time dimension, time value and preset time digital format of each time entity segmentation. This step is a key step in converting the time description in the original text into a time format that can be directly operated by the computer. For example, according to the time entity segmentation "nearly three years" and "June to August" according to the preset time digital format, the standardized digital format time is constructed including: [2022-6; 2022-7; 2022-8; 2023-6; 2023-7; 2023-8; 2024-6; 2024-7; 2024-8]. Finally, step S10115 combines all the constructed digital format times into a complete time condition. This time condition will be used as an input parameter for subsequent query or analysis to filter or limit data within the time range.

[0191] Through the above specific embodiments, the technical solution can efficiently and accurately identify and extract time conditions from natural language query statements input by users. This process not only simplifies user operations and improves the convenience of queries, but also avoids inaccurate data retrieval caused by manual parsing errors through intelligent processing methods. In addition, by constructing standardized digital format time, a unified time benchmark is provided for subsequent data processing and analysis, further improving the efficiency and accuracy of data processing. Overall, this technical solution significantly improves the intelligence level and user experience of the information processing system.

[0192] In one embodiment, the query condition of the specified type includes a quantity restriction condition and / or a sorting condition;

[0193] In step S1011, the query condition identification module identifies the quantity restriction condition and / or the sorting condition in the following manner:

[0194] Step S10116, if it is identified that the natural query statement contains a preset quantity limit keyword, and / or a preset quantity range keyword and is adjacent to the first numerical value, a quantity limit condition is obtained according to the quantity limit keyword and / or the quantity range keyword and the first numerical value;

[0195] Step S10117: if it is identified that the natural query statement contains preset sorting keywords, a sorting condition is obtained according to the excluded keywords.

[0196] In this embodiment, regular expressions are used to detect whether there are preset quantity limit limiting keywords in the natural query statement, such as "most", "least", etc. These keywords are used to indicate the upper or lower limit of the quantity of the query results. If such quantity limit limiting keywords are identified, the quantity limiting condition is obtained according to the quantity limit limiting keywords. In one embodiment, the quantity limit limiting keywords include largest, most, best, most awesome, highest, most powerful, least, smallest, worst, lowest, last ranked, and first ranked. When the quantity limit limiting keywords are identified, the quantity limiting condition is determined to be "quantity = 1".

[0197] Similarly, regular expressions are used to detect whether there are preset quantity range limiting keywords in the natural query statement, and these keywords must be adjacent to one or more specific numerical values ​​(called the first numerical value). This structure is usually used to clearly specify the quantity range of the query results. By parsing these keywords and numerical values, a specific quantity range limiting condition is constructed. In one embodiment, the regular expression (top (\\d+)|bottom (\\d+)|top (\\d+)|bottom (\\d+)|top (\\d+)|top (\\d+))(?!(number|year|month|day|season)) can be used to detect whether there are quantity range limiting keywords and adjacent to the first numerical value (i.e., \\d+). If it is detected to exist, the quantity limiting condition is determined to be "quantity = \\d+".

[0198] In addition, the embodiment of the present application can also use regular expressions to check whether there are preset sorting keywords in the natural query statement, such as few, small, poor, low, back, inverted, ascending, etc. These keywords are used to indicate that the user wants to sort the query results according to a certain field or attribute. Therefore, once the sorting keyword is identified, the sorting field or attribute that may follow the keyword will be further parsed, as well as the direction of sorting (ascending or descending, if explicitly specified). Based on this information, a sorting condition is constructed, which will guide the query results to be arranged in the order specified by the user. In one embodiment, if the presence of "few", "small", "poor", "low", "back", "inverted", and "ascending" is detected, the sorting condition is determined to be "ascending order", otherwise it is "descending order".

[0199] This embodiment realizes accurate identification of specific types of query conditions, including quantity restriction conditions and sorting conditions, through a query condition identification module. Specifically, the query condition identification module can intelligently analyze specific keywords and their contexts in natural query statements to construct query constraints that meet the user's intentions. For quantity restriction conditions, the query condition identification module can keenly capture the preset quantity limit limiting keywords or quantity range limiting keywords and their accompanying numerical values, so as to accurately construct query conditions containing specific quantity requirements. At the same time, for sorting conditions, the query condition identification module also has efficient recognition capabilities. By identifying the preset sorting keywords, it automatically extracts the user's sorting preferences and constructs sorting conditions accordingly. This function enables the query results to be arranged in order according to the standards specified by the user, further enhancing the practicality of the query results and the user experience.

[0200] In summary, this embodiment not only enriches the types of query conditions by introducing the ability to recognize quantity restriction conditions and sorting conditions, but also significantly improves the intelligence level of the query system and user satisfaction, allowing users to obtain required information more flexibly and accurately.

[0201] In one embodiment, the step of removing the basic fields related to the query condition of the specified type from the candidate basic fields in step S1031 includes:

[0202] Step S10311, obtaining a number of character strings corresponding to the query conditions of the specified type;

[0203] Step S10312: If any basic field in the candidate basic fields is the same as any of the character strings, it is determined that the basic field is related to the query condition of the specified type, and the basic field is removed.

[0204] In this embodiment, the query conditions of the specified type generally include specific keywords, phrases or complex expressions for limiting or filtering data. Therefore, these query conditions can be decomposed into a series of basic units - character strings, thereby obtaining a number of character strings corresponding to the query conditions of each specified type.

[0205] This embodiment specifically describes the steps of how to remove the basic fields that match the query conditions of the specified type from the candidate basic fields. First, in step S10311, several character strings corresponding to the query conditions of the specified type are first obtained. Then, in step S10312, the candidate basic fields are traversed to check whether each basic field is the same as any character string obtained in step S10311. The "same" here refers to the complete match between the field name or field identifier and the character string. If any basic field in the candidate basic fields is the same as any character string, the system considers that the basic field is related to the query conditions of the specified type, so it needs to be removed from the candidate basic field list. This step ensures that only those fields that are directly related to the query conditions will be removed by exact matching, thereby avoiding unnecessary mis-removal. In this way, the field information of the complex query conditions that may interfere with the text processing model can be effectively filtered out to avoid its attempt to understand and output these complex query conditions (i.e., the query conditions of the specified type).

[0206] In one embodiment, the task execution text further includes a sixth task execution text, and the sixth task execution text is used to prompt the text processing model that the first query condition is a query condition other than the query condition of the specified type.

[0207] In this embodiment, as part of the task execution text, the sixth task execution text has a specific prompt function. It is specifically used to clearly indicate to the text processing model that the first query condition currently being processed or about to be processed is a query condition outside the specified type of query condition. This allows it to try to understand and output conditions outside the specified type of query condition, avoid outputting the specified type of query condition that it is not good at, and ensure that the accuracy of the technical concept and technical solution of this application is further strengthened and guaranteed.

[0208] In one embodiment, after the step of determining several related basic fields as candidate basic fields from the constellation data model according to the natural query statement in step S103, the following steps are further included:

[0209] Step S201, if there is no multidimensional operation field bound to any of the candidate basic fields, construct a task processing instruction according to the candidate basic fields, the natural query statement and a preset task execution text; the task execution text is used to prompt the text processing model to output query information that meets the query intent of the natural query statement, and the query information is determined according to the candidate basic fields;

[0210] Step S202, inputting the task processing instruction into a text processing model to obtain output query information;

[0211] Step S203: According to the query information and the constellation data model, query and obtain corresponding service data from a database corresponding to the constellation data model.

[0212] This embodiment illustrates the data query steps when there is no basic field bound to a multidimensional operation field in the candidate basic field. The difference between this embodiment and the above embodiment is that the candidate multidimensional operation field is not determined, and the candidate multidimensional operation field is not included when constructing the task processing instruction. At the same time, the task execution text can use other texts, and only prompt the text processing model to output query information that meets the query intention of the natural query statement, wherein the query information is determined according to the candidate basic field. Of course, in other embodiments, the task execution text can also use the text of the above embodiment. Even if there is no candidate multidimensional operation field in this embodiment, the text processing model will naturally understand and realize how to process, and the final processing effect is basically the same. In addition, the explanation of each step of this embodiment specifically refers to the above embodiment, and its technical conception and technical means are consistent, so they will not be repeated here.

[0213] Please refer to Figure 5 The embodiment of the present application also provides a data query device based on a text processing model, including:

[0214] The query statement acquisition module 201 is used to acquire the natural query statement input by the user;

[0215] The candidate basic field determination module 202 is used to obtain a constellation data model; determine a number of related basic fields from the constellation data model as candidate basic fields according to the natural query statement; wherein the constellation data model records a number of data tables, basic fields contained in each data table, and multi-dimensional operation fields bound to at least some basic fields;

[0216] A candidate multi-dimensional operation field determination module 203 is configured to determine the bound multi-dimensional operation field as a candidate multi-dimensional operation field if the candidate basic field is bound with the multi-dimensional operation field;

[0217] The task processing instruction construction module 204 is used to construct a task processing instruction according to the candidate basic fields, the candidate multi-dimensional operation fields, the natural query statement and the preset task execution text; wherein the task execution text is used to prompt the text processing model to output query information that meets the query intent of the natural query statement, and the query information is determined according to the candidate basic fields and the candidate multi-dimensional operation fields;

[0218] A query information acquisition module 205, used to input the task processing instruction into the trained text processing model to obtain output query information;

[0219] The service data query module 206 is used to query and obtain corresponding service data from a database corresponding to the constellation data model according to the query information and the constellation data model.

[0220] It should be noted that the data query device based on the text processing model provided in the above embodiment only uses the division of the above functional modules as an example when executing the data query method based on the text processing model. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the data query device based on the text processing model provided in the above embodiment and the data query method based on the text processing model belong to the same concept. The implementation process thereof is detailed in the above embodiment of the data query method based on the text processing model, which will not be repeated here.

[0221] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present application, and the present application is also intended to include such modifications and modifications.

Claims

1. A data query method based on text processing model, It is characterized in that The following steps are involved: Get the natural query statement entered by the user; Acquire a constellation data model; determine, according to the natural query statement, several relevant basic fields from the constellation data model as candidate basic fields; wherein the constellation data model records several data tables, basic fields contained in each data table, and multi-dimensional operation fields bound to at least some of the basic fields; If the candidate basic field is bound to a multi-dimensional operation field, the bound multi-dimensional operation field is determined as a candidate multi-dimensional operation field; A task processing instruction is constructed according to the candidate basic fields, the candidate multidimensional operation fields, the natural query statement and a preset task execution text; wherein the task execution text is used to prompt the text processing model to output query information that meets the query intent of the natural query statement, and the query information is determined according to the candidate basic fields and the candidate multidimensional operation fields; the task execution text includes a first task execution text; the first task execution text is used to prompt the text processing model to output the query information including a multidimensional operation field and a third basic field bound to the multidimensional operation field, the third basic field is determined from the candidate basic fields, and the multidimensional operation field is determined from the candidate multidimensional operation fields; the multidimensional operation field is used to specify the multidimensional operation of the business data of the third basic field; the task execution text also includes a third task execution text, and the third task execution text is used to prompt the text processing model to output the query information according to a preset report query request format; Inputting the task processing instruction into a trained text processing model to obtain output query information; According to the query information and the constellation data model, query and obtain corresponding service data from a database corresponding to the constellation data model; The trained text processing model is obtained in the following way: Obtain a pre-trained first text processing model; the first text processing model includes an encoder and / or a decoder, and the encoder and the decoder respectively include an attention layer and a fully connected layer connected to each other; Acquire a training data set; the training data set includes a plurality of training samples, each training sample includes a first multidimensional query statement, candidate basic fields and candidate multidimensional operation fields related to the first multidimensional query statement, the task execution text, and query tag information corresponding to the first multidimensional query statement, the query tag information includes a multidimensional operation field and a basic field bound to the multidimensional operation field; Obtain an initialized low-rank matrix, and embed the low-rank matrix in a weight matrix of a fully connected layer or an attention layer of an encoder of the first text processing model, or embed the low-rank matrix in a weight matrix of a fully connected layer or an attention layer of a decoder of the first text processing model; Input the training data set into the first text processing model, and obtain the query information corresponding to each generated training sample; calculate the loss value between the query information of each training sample and the corresponding query label information; according to the loss value, adjust the weight of the low-rank matrix until the loss value is less than a preset loss threshold or the total number of weight adjustments reaches a preset number of adjustments, thereby obtaining the trained text processing model; or, input the training data set into the first text processing model, and obtain the query information corresponding to each training sample generated within the preset time at preset intervals; calculate the loss value between the query information of each training sample generated within the preset time and the corresponding query label information, and cache the current first text processing model and the corresponding loss value; according to the loss value, adjust the weight of the low-rank matrix until the total number of weight adjustments reaches the preset number of adjustments, and determine the first text processing model with the smallest loss value as the trained text processing model from among the cached first text processing models and the corresponding loss values; The step of determining a number of related basic fields as candidate basic fields from the constellation data model according to the natural query statement includes: Perform keyword recognition on the natural query sentence to obtain a number of keywords; Vectorizing each of the keywords and each basic field in the constellation data model; calculating the vector similarity between any of the keywords and any of the basic fields; For any of the keywords, determining a first preset number of basic fields that are ranked high in similarity to the keyword as relevant basic fields of the keyword; According to the relevant basic fields of each keyword and their corresponding similarities, the relevant basic fields of each keyword are ranked by similarity; Determine a second preset number of related basic fields with the highest similarity rankings as candidate basic fields; The step of querying and obtaining corresponding service data from a database corresponding to the constellation data model according to the query information and the constellation data model comprises: The query information is used as a report query request, and the report query request and the constellation data model are input into an MDX query engine; the corresponding business data is obtained by querying from a database corresponding to the constellation data model through the MDX query engine; wherein the MDX query engine is an engine for parsing the report query request and performing data query based on the parsing results and the constellation data model.

2. The data query method based on the text processing model according to claim 1, It is characterized in that The candidate basic fields and candidate multidimensional operation fields related to the first multidimensional query statement are: a first basic field related to the first multidimensional query statement and a first multidimensional operation field bound to the first basic field, determined from the constellation data model; And / or, the candidate basic fields and candidate multidimensional operation fields related to the first multidimensional query statement are: the second basic fields and the second multidimensional operation fields related to the first multidimensional query statement, determined from the user-defined multidimensional operation fields and the custom operation information corresponding to the customized multidimensional operation fields.

3. The data query method based on the text processing model according to claim 1, It is characterized in that The step of ranking the related basic fields of each keyword by similarity according to the related basic fields of each keyword and their corresponding similarities includes: If any two of the keywords have the same related basic fields and correspond to different similarities respectively, the related basic fields with the highest similarity among the same related basic fields are retained, and the remaining related basic fields are removed.

4. The data query method based on the text processing model according to claim 1, It is characterized in that The task execution text also includes a second task execution text; the second task execution text is used to prompt that the query information output by the text processing model includes a first query condition, and the first query condition is used to limit the conditions that the business data of the third basic field query needs to meet.

5. The data query method based on the text processing model according to claim 1, It is characterized in that After the step of determining a number of related basic fields as candidate basic fields from the constellation data model according to the natural query statement, the step further includes: If there is no multi-dimensional operation field bound to any of the candidate basic fields, construct a task processing instruction according to the candidate basic fields, the natural query statement and the preset task execution text; The task execution text is used to prompt the text processing model to output query information that meets the query intent of the natural query sentence, and the query information is determined according to the candidate basic fields; Inputting the task processing instruction into a text processing model to obtain output query information; According to the query information and the constellation data model, corresponding service data is queried from a database corresponding to the constellation data model.

6. A data query device based on a text processing model, It is characterized in that include: A query statement acquisition module is used to acquire natural query statements input by users; A candidate basic field determination module is used to obtain a constellation data model; According to the natural query statement, several relevant basic fields are determined from the constellation data model as candidate basic fields; wherein the constellation data model records several data tables, basic fields contained in each data table, and multidimensional operation fields bound to at least some basic fields; the step of determining several relevant basic fields from the constellation data model as candidate basic fields according to the natural query statement includes: performing keyword recognition on the natural query statement to obtain several keywords; vectorizing each of the keywords and each basic field in the constellation data model; calculating the vector similarity between any of the keywords and any of the basic fields; for any of the keywords, determining a first preset number of basic fields with a high similarity ranking to the keyword as relevant basic fields of the keyword; performing similarity ranking on the relevant basic fields of each of the keywords according to their corresponding similarities; determining a second preset number of relevant basic fields with a high similarity ranking as candidate basic fields; A candidate multidimensional operation field determination module, configured to determine the bound multidimensional operation field as a candidate multidimensional operation field if the candidate basic field is bound with the multidimensional operation field; A task processing instruction construction module is used to construct a task processing instruction according to the candidate basic fields, the candidate multidimensional operation fields, the natural query statement and a preset task execution text; wherein the task execution text is used to prompt the text processing model to output query information that conforms to the query intent of the natural query statement, and the query information is determined according to the candidate basic fields and the candidate multidimensional operation fields; the task execution text includes a first task execution text; the first task execution text is used to prompt the query information output by the text processing model to include a multidimensional operation field and a third basic field bound to the multidimensional operation field, the third basic field is determined from the candidate basic fields, and the multidimensional operation field is determined from the candidate multidimensional operation fields; the multidimensional operation field is used to specify the multidimensional operation of the business data of the third basic field; the task execution text also includes a third task execution text, and the third task execution text is used to prompt the text processing model to output the query information according to a preset report query request format; A query information acquisition module is used to input the task processing instruction into a trained text processing model to obtain output query information; the trained text processing model is obtained in the following manner: obtaining a pre-trained first text processing model; the first text processing model includes an encoder and / or a decoder, and the encoder and the decoder respectively include an attention layer and a fully connected layer connected to each other; obtaining a training data set; the training data set includes multiple training samples, each training sample includes a first multidimensional query statement, candidate basic fields and candidate multidimensional operation fields related to the first multidimensional query statement, the task execution text, and query label information corresponding to the first multidimensional query statement, the query label information includes a multidimensional operation field and a basic field bound to the multidimensional operation field; obtaining an initialized low-rank matrix, embedding the low-rank matrix in the weight matrix of the fully connected layer or the attention layer of the encoder of the first text processing model, or embedding the low-rank matrix in the weight matrix of the fully connected layer or the attention layer of the decoder of the first text processing model matrix; input the training data set into the first text processing model, and obtain the query information corresponding to each training sample generated; calculate the loss value between the query information of each training sample and the corresponding query label information; according to the loss value, adjust the weight of the low-rank matrix until the loss value is less than the preset loss threshold or the total number of weight adjustments reaches the preset number of adjustments, and obtain the trained text processing model; or, input the training data set into the first text processing model, and obtain the query information corresponding to each training sample generated within the preset time at preset intervals; calculate the loss value between the query information of each training sample generated within the preset time and the corresponding query label information, and cache the current first text processing model and the corresponding loss value; according to the loss value, adjust the weight of the low-rank matrix until the total number of weight adjustments reaches the preset number of adjustments, and determine the first text processing model with the smallest loss value as the trained text processing model from among the cached first text processing models and the corresponding loss values; A business data query module, used to query and obtain corresponding business data from a database corresponding to the constellation data model according to the query information and the constellation data model; the step of querying and obtaining corresponding business data from a database corresponding to the constellation data model according to the query information and the constellation data model comprises: using the query information as a report query request, and inputting the report query request and the constellation data model into an MDX query engine; querying and obtaining corresponding business data from a database corresponding to the constellation data model through the MDX query engine; wherein the MDX query engine is an engine for parsing report query requests and performing data queries based on the parsing results and the constellation data model.