Text model training method and data query method based on constellation data model

Abstract

The present application relates to a text model training method and a data query method based on a constellation data model. The model training method makes full use of the constellation data model of the target data warehouse, especially the multidimensional operation fields therein, to construct a highly customized training data set. This data set not only reflects the specific business logic and query requirements of the data warehouse, but also simulates the various expression habits of users in the actual query process through the construction of a variety of query statements. Through the training of the training data set, the pre-trained text model can deeply learn and understand the complexity and diversity of the multidimensional operation fields. After the training is completed, a large data query model is obtained, which can generate more accurate query information that meets user needs, especially for multidimensional operation field queries involving complex calculations and custom formulas. Its accuracy and efficiency have been significantly improved, providing strong support for data analysis and decision-making for enterprises and organizations.

Description

Technical Field

[0001] The present application relates to the technical field of data query, and in particular to a text model training method and a data query method based on a constellation data model. Background Art

[0002] With the advent of the big data era, data warehouses are becoming increasingly important as core tools for data storage and analysis. Multidimensional data models in data warehouses (such as constellation data models) support complex business analysis and decision making, including the calculation of multidimensional operation fields (such as ratios, year-on-year growth rates, etc.), which usually involve MDX (multidimensional expressions) or user-defined calculation formulas. It is challenging for users to accurately express these query requirements.

[0003] Existing natural language processing technologies, especially data query conversion solutions using large language models, can convert users' natural language query sentences into database query statements to a certain extent, but they are unable to handle complex queries involving multidimensional operation fields. The main reason is that users have various and flexible ways to express multidimensional operation fields, while existing models lack in-depth understanding and training of multidimensional data models for specific data warehouses, making it difficult to accurately capture and parse the intent of these complex queries, resulting in inaccurate generated query statements, which in turn affects query results. Summary of the invention

[0004] Based on this, the purpose of this application is to provide a text model training method based on the constellation data model, which significantly improves the data query model's ability to understand and recognize complex multidimensional queries, and can support various complex multidimensional queries on specific data warehouses.

[0005] The text model training method based on the constellation data model provided in the embodiment of the present application includes the following steps:

[0006] Acquire a plurality of target multidimensional operation fields in a constellation data model corresponding to a target data warehouse; the constellation data model records a plurality of data tables, basic fields contained in each data table, and multidimensional operation fields bound to at least some of the basic fields;

[0007] Constructing a plurality of first natural query statements according to the plurality of target multi-dimensional operation fields; wherein the first natural query statement includes at least one target multi-dimensional operation field;

[0008] For each of the first natural query statements, generate a plurality of second natural query statements having the same query intent but different expression forms;

[0009] A training data set is constructed according to each of the second natural query statements, the preset first task execution text, and the query label information corresponding to the second natural query statement; wherein the query label information includes the target multi-dimensional operation field contained in the first natural query statement corresponding to the second natural query statement; the first task execution text is used to prompt the pre-trained text model to output query information that meets the query intent of the second natural query statement, and the query information includes a query field;

[0010] The training data set is input into a pre-trained text model for training; and the trained model is determined as a data query macro model corresponding to the target data warehouse.

[0011] The present application also provides a method for querying a large data model based on the text model training method of any one of the embodiments of the present application, comprising the following steps:

[0012] Obtaining natural query statements input by users in the query interface of the target data warehouse;

[0013] Acquire a constellation data model corresponding to the target data warehouse; determine a number of related basic fields from the constellation data model as candidate basic fields according to the natural query statement; if the candidate basic field is bound to a multidimensional operation field, determine the bound multidimensional operation field as a candidate multidimensional operation field;

[0014] According to the candidate basic fields, the candidate multi-dimensional operation fields, the natural query statement and the preset query task execution text, a task processing instruction is constructed; wherein the query task execution text is used to prompt the data query big model to output query information that meets the query intent of the natural query statement, and the query information is determined according to the candidate basic fields and the candidate multi-dimensional operation fields;

[0015] Inputting the task processing instruction into the data query model corresponding to the target data warehouse to obtain output query information;

[0016] According to the query information and the constellation data model, corresponding business data is queried from the target data warehouse.

[0017] The text model training method based on the constellation data model proposed in the embodiment of the present application deeply integrates the constellation data model characteristics of the target data warehouse through an integrated and systematic method, and significantly improves the understanding and recognition ability and comprehensive ability of the data query large model when processing complex multidimensional query tasks. The core of this method is to make full use of the constellation data model of the target data warehouse, especially the multidimensional operation field therein, to construct a highly customized training data set. This data set not only reflects the specific business logic and query requirements of the data warehouse, but also simulates the various expression habits of users in the actual query process through the construction of diversified query statements. Through the training of the training data set, the pre-trained text model can deeply learn and understand the complexity and diversity of the multidimensional operation field, so as to more accurately capture the user's query intention. The data query large model obtained after the training is completed can generate more accurate query information that meets user needs, especially for multidimensional operation field queries involving complex calculations and custom formulas, and its accuracy and efficiency have been significantly improved. Therefore, the technical solution of the embodiment of the present application not only achieves breakthroughs and innovations in the prior art at the technical level, but also brings significant benefits in practical applications. It provides users with a more convenient and efficient data query experience, and also provides strong support for data analysis and decision-making for enterprises and organizations.

[0018] For better understanding and implementation, the present application is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of a text model training method based on a constellation data model according to an embodiment of the present application;

[0020] Figure 2 A schematic diagram of the steps of determining a target multidimensional operation field of a target data warehouse in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of the steps of constructing a first natural query statement according to a target multi-dimensional operation field in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of the steps of generating a second natural query statement by using an intelligent language model in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of the steps of updating and training a pre-trained text model according to an embodiment of the present application;

[0024] Figure 6 A flowchart of a data query method based on a data query big model according to another embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present application clearer, the embodiments of the present application will be further described in detail below in conjunction with the accompanying drawings. Wherein, when the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements.

[0026] It should be clear that the implementation methods described in the embodiments described below do not represent all implementation methods consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0027] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application are also intended to include plural forms, unless the context clearly indicates other meanings. In addition, in the description of this application, unless otherwise specified, "multiple" refers to two or more. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone; the character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0028] It should be understood that, although the terms first, second, third, etc. may be used in the present application to describe various information, such information should not be limited to these terms, and these terms are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances. Depending on the context, the words "if" / "if" used in the present application can be interpreted as "at the time of" or "when" or "in response to determination".

[0029] With the advent of the big data era, data warehouses are becoming increasingly important as core tools for data storage and analysis. Multidimensional data models in data warehouses (such as constellation data models) support complex business analysis and decision making, including the calculation of multidimensional operation fields (such as ratios, year-on-year growth rates, etc.), which usually involve MDX (multidimensional expressions) or user-defined calculation formulas. It is challenging for users to accurately express these query requirements.

[0030] Existing natural language processing technologies, especially data query conversion solutions using large language models, can convert users' natural language query sentences into database query statements to a certain extent, but they are unable to handle complex queries involving multidimensional operation fields. The main reason is that users have various and flexible ways to express multidimensional operation fields, while existing models lack in-depth understanding and training of multidimensional data models for specific data warehouses, making it difficult to accurately capture and parse the intent of these complex queries, resulting in inaccurate generated query statements, which in turn affects query results.

[0031] Based on this, the embodiment of the present application aims to provide a text model training method based on the constellation data model, which significantly improves the data query model's ability to understand and recognize complex multidimensional queries, and can support various complex multidimensional queries on specific data warehouses.

[0032] Please refer to Figure 1 The text model training method based on the constellation data model provided in the embodiment of the present application comprises the following steps:

[0033] S101: Acquire several target multidimensional operation fields in a constellation data model corresponding to a target data warehouse; the constellation data model records several data tables, basic fields contained in each data table, and multidimensional operation fields bound to at least some of the basic fields;

[0034] S102: constructing a plurality of first natural query statements according to the plurality of target multi-dimensional operation fields; wherein the first natural query statement includes at least one target multi-dimensional operation field;

[0035] S103: for each of the first natural query statements, generating a plurality of second natural query statements having the same query intent but different expression forms;

[0036] S104: constructing a training data set according to each of the second natural query statements, the preset first task execution text, and the query label information corresponding to the second natural query statement; wherein the query label information includes the target multi-dimensional operation field contained in the first natural query statement corresponding to the second natural query statement; the first task execution text is used to prompt the pre-trained text model to output query information that meets the query intent of the second natural query statement, and the query information includes a query field;

[0037] S105: Input the training data set into the pre-trained text model for training; determine the trained model as the data query macro model corresponding to the target data warehouse.

[0038] The text model training method based on the constellation data model proposed in the embodiment of the present application deeply integrates the constellation data model characteristics of the target data warehouse through an integrated and systematic method, and significantly improves the understanding and recognition ability and comprehensive ability of the data query large model when processing complex multidimensional query tasks. The core of this method is to make full use of the constellation data model of the target data warehouse, especially the multidimensional operation field therein, to construct a highly customized training data set. This data set not only reflects the specific business logic and query requirements of the data warehouse, but also simulates the various expression habits of users in the actual query process through the construction of diversified query statements. Through the training of the training data set, the pre-trained text model can deeply learn and understand the complexity and diversity of the multidimensional operation field, so as to more accurately capture the user's query intention. The data query large model obtained after the training is completed can generate more accurate query information that meets user needs, especially for multidimensional operation field queries involving complex calculations and custom formulas, and its accuracy and efficiency have been significantly improved. Therefore, the technical solution of the embodiment of the present application not only achieves breakthroughs and innovations in the prior art at the technical level, but also brings significant benefits in practical applications. It provides users with a more convenient and efficient data query experience, and also provides strong support for data analysis and decision-making for enterprises and organizations.

[0039] The text model training method based on the constellation data model proposed in the embodiment of the present application is executed by a computer. In the embodiment of the present application, the data query model is specially customized for the "constellation data model" (i.e., the data model system of the data warehouse built based on a specific field or business logic) to better adapt to the complex query requirements in different fields and scenarios, and improve the accuracy and efficiency of the query.

[0040] The following first explains the pre-trained text model and task execution text, and then explains each step in detail.

[0041] In the embodiments of the present application, the pre-trained text model is a natural language processing (NLP) model that uses machine learning or deep learning technology to process, understand and generate text data. The model is pre-trained with certain text data and can learn the grammatical rules, semantic relationships and contextual dependencies of the language, thereby achieving various tasks such as understanding, extracting, classifying and / or generating text information. In general, pre-trained models, such as BERT, GPT, etc., have been preliminarily trained on massive text data, so they have certain language understanding and generation capabilities, and can quickly adapt to and improve performance on new tasks through a small amount of annotated data.

[0042] In a data query scenario of an embodiment of the present application, a pre-trained text model is a natural language processing model that can parse natural query statements input by users, understand their query intent, and generate corresponding query information based on these intents. The pre-trained text model can be based on one or more pre-trained language models, and is obtained through fine-tuning in order to adapt to the specific tasks in this application (i.e., query intent understanding and query information generation). This usually involves further training the pre-trained language model on a labeled data set containing query statements and corresponding query information, so that it can better understand the query intent and generate accurate query information. Of course, the pre-trained text model can also be an existing large language model, as long as it has the processing capability of the specific tasks of the data query scenario of the embodiment of the present application, it can be applied in the technical solution of the present application, and the embodiment of the present application is not limited to this.

[0043] It is understandable that in the embodiments of the present application, the pre-trained text model needs to be further optimized. It should be said that the existing natural language models are all trained using text data, but due to the limited amount of text data, the ambiguity and inaccuracy of the text corpus, and the large model hallucination problem, the existing models always have some limitations. Although they have certain recognition capabilities, they cannot achieve universal adaptability to various fields, various scenes, and various natural language expressions. The accuracy of their understanding is insufficient in some fields or scenes. Therefore, the embodiments of the present application are intended to be targeted, train the dedicated model of the target data warehouse, train the corresponding data query large model for the data warehouse in different fields and different scenes, and realize the specialization and precision of the data query large model. In particular, in the embodiments of the present application, although the pre-trained text model itself has undergone certain data training and has the ability to understand the query intent of natural query statements and generate query information, it is obviously insufficient in the recognition accuracy of the multidimensional operation field when understanding the query statement involving the multidimensional operation field. Therefore, the text model training method based on the constellation data model provided in the present application is mainly used to further enhance the model's understanding and recognition ability of the multidimensional operation field. Of course, the technical concept of the present application is not limited to this, and please refer to the description of each embodiment of the present application for details.

[0044] Furthermore, a task execution text is involved in the application of the pre-trained text model. The task execution text is used to require or prompt the task requirements of the pre-trained text model, including specific requirements and formatting instructions for the model output, and may also include task-related scene information or context information for the model to learn and understand the task to be processed, etc. In an embodiment of the present application, different sub-texts can be set in the task execution text according to the needs of the query, respectively used to require or prompt the corresponding task requirements of the pre-trained text model. In one embodiment, the task execution text may include a first task execution text, a second task execution text, a third task execution text... and the like, and different sub-texts contain specific requirements for model processing and output according to the task requirements, etc.

[0045] In one embodiment of the present application, the task execution text includes a first task execution text; the first task execution text is used to prompt the pre-trained text model to output query information that meets the query intent of the second natural query statement, and the query information includes a query field. Among them, the first task execution text is generally a command text used to require or prompt the pre-trained text model what task to perform, which details the specific rules that the pre-trained text model should follow when generating query information. Alternatively, the first task execution text can also be an instruction set in a specific format, and similarly, it details the specific rules that the pre-trained text model should follow when generating query information. The first task execution text can specifically determine what form it takes based on the requirements and specifications of the pre-trained text model for task processing instructions. In addition, other task execution texts, such as the second task execution text and the third task execution text, are understood by analogy based on this principle.

[0046] It should be noted that those skilled in the art understand that the task execution text prompts the pre-trained text model to do what it should do, which does not mean that the pre-trained text model must do what it should do. For example, the task execution text prompts the pre-trained text model to output query information that meets the query intent, wherein the query information includes a multidimensional operation field, but after the pre-trained text model understands the natural query statement, it believes that the query does not involve a multidimensional operation field, then it may not output the multidimensional operation field to avoid violating the core task of the task execution text prompt, that is, outputting query information that meets the query intent. Of course, if the training sample involves a query of a multidimensional operation field, but the pre-trained text model does not output the multidimensional operation field, it means that the task processing capability of the model needs to be improved, and this is also the significance of the technical solution of the embodiment of the present application.

[0047] For step S101, several target multidimensional operation fields in the constellation data model corresponding to the target data warehouse are obtained; the constellation data model records several data tables, basic fields contained in each data table, and multidimensional operation fields bound to at least some of the basic fields.

[0048] The target data warehouse is a data warehouse selected by the user. In the embodiment of the present application, the user can select one or more data warehouses in the data warehouse list, so as to train the data query model of the target data warehouse based on the constellation data model of the target data warehouse.

[0049] Among them, the constellation data model is an abstract representation used to describe the database structure. It is a data logic layer established in advance based on various data tables stored in the data warehouse. It records several data tables, the basic fields contained in each data table, and the graph relationship of each data table. In addition, it also records the binding relationship between at least some basic fields and multi-dimensional operation fields. In other words, the constellation data model does not record the specific business data of the fields of each data table, but only records the relevant information used to generate query statements. This information mainly includes the table name of the data table, the field information of the data table, and the multi-dimensional operation field information bound to the basic field. In addition, the graph relationship between each data table is established in the constellation data model, and the graph relationship at least includes a directed graph relationship.

[0050] Among them, the basic field is generally a field that records the original business data in the database. More specifically, the basic field is generally divided into a metric field and a dimension field. The metric field generally refers to the field in the fact table, and the dimension field generally refers to the field in the dimension table. Among them, the fact table and the dimension table are concepts in data modeling in the field of data warehouses. At present, the main data models involved in the field of data warehouses are star data models, snowflake data models generated on the basis of star data models (which can actually be considered as a type of star data model) and constellation data models. In the star data model, it includes a data table located in the center and other data tables connected to the center data table, wherein the center data table is the fact table, and the other data tables connected to it are dimension tables.

[0051] Among them, the multidimensional operation field, also known as the MDX operation field, is different from the basic field. It is not a native field recorded in the database, but a field obtained by definition. The data queried by it is obtained by performing complex operations (such as complex calculations) based on the data of the basic field in the database. Specifically, multidimensional operation fields for various complex operations or various data analysis indicators can be defined according to the needs of data analysis. In one embodiment, the multidimensional operation field can be "same period", "previous period", "year-on-year", "month-on-month", "global proportion", "group proportion", "global ranking" and "group ranking", etc. Obviously, the query of the multidimensional operation field involves the calculation or comparison of the data of the basic field in multiple dimensions. Unlike the query of the basic field, it only needs to obtain the data corresponding to the basic field from the database, but requires more complex analysis and processing involving multiple dimensions and / or multiple basic data. In addition, according to the needs of data analysis, users can also customize more multidimensional operation fields and corresponding calculation formulas. The query of these multidimensional operation fields often contains important information of business logic and data analysis, and is an indispensable part of deep data exploration, but it is also flexible and changeable in different industries and application scenarios, and is often difficult to be directly understood and recognized by existing large language models.

[0052] In the embodiment of the present application, at least some of the basic fields of the constellation data model are bound to multidimensional operation fields. This binding is generally established in advance by the user according to the query requirements, so that when the user enters natural language to query, the required data analysis results can be accurately and quickly queried. This binding mechanism allows any basic field to be associated with one or more multidimensional operation fields. Different multidimensional operation fields define different specific operations that can be performed on the business data represented by the basic fields to which they are bound, greatly enhancing the flexibility and dynamism of data processing.

[0053] In this step, first, the target data warehouse is determined, and the constellation data model of the target data warehouse is loaded or accessed. By accessing the metadata management system or related configuration files of the data warehouse, the multidimensional operation field information in the constellation data model is extracted. These multidimensional operation fields can include predefined calculation fields such as year-on-year, month-on-month, and growth rate, and can also include complex calculation formula fields customized by users according to business needs. The system can use these multidimensional operation fields as target fields for subsequent training data set construction.

[0054] In one embodiment, before the step of obtaining several target multi-dimensional operation fields in the constellation data model corresponding to the target data warehouse in step S101, the following steps are included:

[0055] The target data warehouse information selected by the user from the data warehouse list is obtained, and the constellation data model corresponding to the target data warehouse information is determined.

[0056] Before formally executing step S101, the system of this embodiment displays a data warehouse list to the user through the user interface. This list contains multiple available data warehouses, each of which corresponds to a different data source, data structure or business scenario. According to their needs, users can convey their selection intentions to the system by clicking, checking or other interactive methods. Among them, the user can select one or more target data warehouses from the list of available data warehouses, which have been organized and configured according to the constellation data model. Once the user selects the target data warehouse, the system obtains specific selection operation information by capturing these user operations. After obtaining the target data warehouse information selected by the user, the system will identify and determine the target data warehouse that the user wants to operate and determine the corresponding constellation data model based on this information. This usually involves accessing the metadata management system, configuration file or database model information of the data warehouse to obtain a detailed description of the data warehouse structure, data tables, fields and the relationship between them. In particular, the system needs to be able to identify which fields are multidimensional operation fields, which may be identified by specific tags, annotations or information stored in a separate metadata table. After completing this step, the system will be able to accurately obtain the multidimensional operation fields in the target data warehouse that the user is concerned about, laying a solid foundation for subsequent query statement construction and model training.

[0057] This embodiment introduces additional steps to ensure that the user can accurately specify the target data warehouse and perform subsequent operations based on the specific constellation data model of the warehouse. This not only enhances the pertinence and effectiveness of model training, but also improves the flexibility and convenience of users in the actual query process.

[0058] Please refer to Figure 2 In one embodiment, before the step of obtaining several target multi-dimensional operation fields in the constellation data model corresponding to the target data warehouse in step S101, the following steps are included:

[0059] Step S1001, obtaining historical natural query statement data of the target data warehouse;

[0060] Step S1002, determining a number of natural query statements related to the multidimensional operation field in the historical natural query statement data; analyzing, based on the number of natural query statements related to the multidimensional operation field, obtaining a multidimensional operation field whose frequency of natural language expression by the user is greater than a preset frequency;

[0061] Step S1003: determine the multi-dimensional operation field whose frequency of natural language expression is greater than a preset frequency as the target multi-dimensional operation field.

[0062] In step S1001, the system collects and obtains all or a specific range of natural query statement data executed in the data warehouse in the past period of time by accessing the log system, query history database or query record set submitted by the user of the target data warehouse. These query statements are usually written by users in natural language or SQL query language, etc., and are used to extract information from the data warehouse.

[0063] In step S1002, the system first parses the natural query statement data obtained in step S1001, and identifies the query statements involving multidimensional operation fields. This usually involves parsing elements such as field names, function calls, table connections, etc. in the query statements to determine which query statements involve multidimensional data tables or complex calculation fields in the data warehouse. Next, the system performs statistical analysis on these query statements related to multidimensional operation fields and calculates the frequency of each multidimensional operation field being mentioned in user queries. By comparing with the preset frequency threshold, those multidimensional operation fields whose frequency of being expressed by users in natural language is greater than the preset frequency are screened out. This preset frequency threshold can be flexibly set according to factors such as the actual application scenario, the scale of the data warehouse, and the user's usage habits.

[0064] Step S1003: Based on step S1002, the system determines all multidimensional operation fields that are expressed in natural language with a frequency greater than a preset frequency as target multidimensional operation fields. These fields represent the multidimensional data points that users pay the most attention to and express most frequently in natural language during the actual query process, which are of great significance for the subsequent training and optimization of the data query model.

[0065] This embodiment adds an analysis process based on historical query statement data before obtaining the target multidimensional operation field in the constellation data model corresponding to the target data warehouse. This process makes full use of the historical query records of the data warehouse, and accurately identifies the multidimensional data points that users are most concerned about and tend to use natural language expressions by statistically analyzing the frequency of using natural language expressions for multidimensional operation fields in actual queries. This data-driven method not only improves the accuracy and pertinence of the selection of target multidimensional operation fields, but also makes the training process of the data query big model closer to the actual needs of users. The data query big model obtained through training can better understand the user's diverse query intentions, especially for query requests involving complex multidimensional operations, and can generate more accurate and efficient query statements, thereby significantly improving the response speed and result accuracy of data queries. In general, the technical solution of the present application realizes the accurate identification and selection of target multidimensional operation fields by introducing the analysis process of historical query statement data, providing users with a more intelligent and efficient data query experience.

[0066] For step S102, a plurality of first natural query statements are constructed according to the plurality of target multi-dimensional operation fields; wherein the first natural query statement includes at least one target multi-dimensional operation field.

[0067] Based on the target multidimensional operation field obtained in step S101, a series of first natural query statements are constructed. These query statements are expressed in natural language, and each statement contains at least one target multidimensional operation field. For example, the following query statements can be constructed: "Query the year-on-year sales growth rate in the first quarter of this year" or "Show the month-on-month changes in product sales between last month and this month." These query statements are intended to simulate actual query requirements that users may raise.

[0068] Please refer to Figure 3 In one embodiment, the step of constructing a plurality of first natural query statements according to the plurality of target multi-dimensional operation fields in step S102 includes:

[0069] Step S1021, obtaining a plurality of preset natural query statement templates;

[0070] Step S1022: generating a plurality of natural query statements for querying the target multi-dimensional operation fields according to the plurality of natural query statement templates and the plurality of target multi-dimensional operation fields, which are first natural query statements.

[0071] In step S1021, the system retrieves and obtains a certain number of natural query statement templates from the preset query statement template library. These templates are pre-designed and stored based on the common query requirements of the data warehouse, user query habits, and the training requirements of the data query large model, and can provide a standardized query statement structure framework. The template is designed to cover common query scenarios and query types, while maintaining a certain degree of flexibility and scalability, so that it can be adaptively adjusted according to different target multi-dimensional operation fields. Specifically, the natural query statement template usually contains some common query structures and keywords, such as the query structure corresponding to the basic components of SQL query statements such as SELECT, FROM, WHERE, but expressed in natural language, and may also contain some parameter placeholders or replaceable parts for subsequent filling and replacement according to specific target multi-dimensional operation fields.

[0072] In step S1022, after obtaining the natural query statement template and the target multidimensional operation field, the system starts to execute the generation process. For each target multidimensional operation field, the system will traverse all available natural query statement templates, and insert the target multidimensional operation field into the corresponding position of the template according to the structure and rules of the template. This insertion process may be a direct text replacement, or it may be a conversion and formatting based on a certain logic. Through this step, the system can generate one or more natural query statements that conform to natural language habits, have a clear structure, and are accurately expressed for each target multidimensional operation field. These statements are the first natural query statements, which will lay the foundation for the subsequent generation of the second natural query statement with the same query intent but a different expression form.

[0073] This embodiment greatly simplifies the process of constructing the first natural query statement according to the target multidimensional operation field by introducing a natural query statement template and an automatic generation mechanism. Compared with the traditional method of manually writing natural query statements by technicians based on actual experience or relying on complex query construction logic to create natural query statements, the method of this embodiment is more efficient, flexible and easy to maintain. Through the preset query statement template, the system can quickly respond to the query requirements of different target multidimensional operation fields, generate natural query statements that meet the specifications and are easy to understand, and lay an important foundation for the subsequent construction of training data sets. At the same time, due to the replaceability and scalability of the template, the system can easily adapt to the newly added or changed multidimensional operation fields in the data warehouse without a large number of modifications and adjustments to the query construction logic. In addition, the automatically generated query statements can also reduce human errors and the workload of query optimization to a certain extent. The system can automatically select the optimal query path and strategy according to the characteristics of the target multidimensional operation field and the query requirements, thereby improving the efficiency and accuracy of the natural query statement construction. In summary, this embodiment provides users with a more convenient, efficient and intelligent natural query statement construction solution, and also provides strong support for the training and optimization of the data query large model.

[0074] In one embodiment, the natural query statement template includes a first natural query statement template including basic fields to be filled in and multi-dimensional operation fields;

[0075] The step of generating a plurality of natural query statements for querying the target multi-dimensional operation fields according to the plurality of natural query statement templates and the plurality of target multi-dimensional operation fields in step S1022 includes:

[0076] Step S10221, obtain the several target multidimensional operation fields, and obtain the basic fields bound to the target multidimensional operation fields, fill in each target multidimensional operation field and the corresponding bound basic field into the first natural query statement template respectively, and obtain several third natural query statements; determine the third natural query statement as the first natural query statement.

[0077] In this embodiment, the natural query statement template is further refined into a first natural query statement template including basic fields to be filled in and multidimensional operation fields. This template design is intended to more accurately reflect the composition of the query statement, ensuring that the generated query statement can simultaneously contain multidimensional operation fields and basic fields specified by the multidimensional operation fields. Specifically, first, the system obtains several target multidimensional operation fields. These fields are fields on which the user wants to perform multidimensional operations (such as calculation, filtering, sorting, grouping, etc.). For each target multidimensional operation field, the system further obtains the basic field bound to the field. The basic field is usually a basic data field associated with the target multidimensional operation field and used to construct the query condition. For example, if the target multidimensional operation field is the year distribution of "sales", then the basic field may be "sales" itself and "year". Next, the system fills in and replaces each target multidimensional operation field and its corresponding bound basic field according to the format and placeholder defined in the first natural query statement template. This process may involve direct insertion of field names, conversion of data types, and possible formatting. After completing the filling, the system obtains several third natural query statements that meet the requirements. These statements contain both the query conditions of the basic fields and the query requirements of the multi-dimensional operation fields. Finally, the system directly determines the generated third natural query statements as the first natural query statements. These statements will serve as the basis for the subsequent generation of the second natural query statements.

[0078] This embodiment greatly improves the efficiency and accuracy of building complex query statements by introducing a customized natural query statement template containing basic fields and multi-dimensional operation fields, and combining it with an automatic generation mechanism. This method not only simplifies the development process, but also ensures that the generated query statements can accurately and comprehensively meet the diverse query needs of users, and achieves seamless coverage from the screening of basic fields to the complex processing of multi-dimensional operation fields. At the same time, the generated query statements are closer to the user's natural language habits, providing strong support for the construction of training data sets for large models.

[0079] In step S103, for each of the first natural query statements, a plurality of second natural query statements having the same query intent but different expression forms are generated.

[0080] For each first natural query statement, multiple second natural query statements with the same query intent but different expressions are generated through language transformation, synonym replacement, sentence structure adjustment, etc. For example, for the query statement "Query the year-on-year sales growth rate in the first quarter of this year", second natural query statements such as "Year-on-year sales growth in the first quarter of this year" and "Calculate the year-on-year sales growth rate in Q1 this year" can be generated. These diverse query statements help the model learn to understand the flexibility of user query intent.

[0081] Please refer to Figure 4 In one embodiment, the step of generating a plurality of second natural query statements having the same query intent but different expression forms for each of the first natural query statements in step S103 includes:

[0082] Step S1031, constructing a second task processing instruction according to each of the first natural query statements and a preset second task execution text; the second task execution text is used to prompt the intelligent language model to generate a plurality of second natural query statements having the same query intent as the first natural query statement, and satisfying that the second natural query statement and the first natural query statement have different expression forms;

[0083] Step S1032: input the second task processing instruction into an intelligent language model to obtain a plurality of second natural query statements with the same query intent corresponding to each of the first natural query statements.

[0084] Among them, the intelligent language model is different from the pre-trained text model in this embodiment. It is a large language model with more powerful processing capabilities, ultra-large-scale parameters and ultra-large-scale data training. It is usually the most powerful generative large language model recognized in the field of natural language processing at present, such as the current ChatGPT. It has very powerful natural language understanding and text processing capabilities, and can support the implementation of the technical solution of this application - that is, this embodiment creates a rich training data set with the help of an external intelligent language model that is mature in text task execution to train and enhance the capabilities of the local pre-trained text model (data query large model).

[0085] This embodiment further refines the detailed operation of step S103, aiming to generate diversified second natural query sentences through an intelligent language model. Although these sentences have different expression forms, they maintain the same query intent as the original first natural query sentence.

[0086] In step S1031, firstly, a second task processing instruction is constructed based on each first natural query statement and the preset second task execution text. The second task execution text here is a predefined text or template, which clearly indicates the next task direction of the intelligent language model - that is, generating a query corresponding to the query meaning of the given first natural query statement. Figure 1 The key to this step is to guide the intelligent language model to understand and follow the user's fundamental query needs through precisely constructed task instructions, while encouraging it to explore and output diverse expressions.

[0087] Subsequently, in step S1032, the constructed second task processing instruction is passed as input to the intelligent language model for processing. With its powerful natural language processing and deep learning capabilities, the intelligent language model can parse the task instructions, understand the user's query intent, and generate a series of second natural query statements that are equivalent to the first natural query statements but expressed differently. These generated statements not only retain the core intent of the original query, but also show the diversity of query requirements through different vocabulary, sentence structures or expressions, providing richer training samples for subsequent model training.

[0088] In general, this embodiment realizes the automatic expansion and diversified generation of the original first natural query statement by introducing an intelligent language model. This technology not only significantly improves the flexibility and efficiency of natural query statement construction, but also enhances the robustness and adaptability of the query system through diversified query expression forms. Users do not need to manually construct multiple natural query statements to obtain a variety of expressions that fully cover the query intent, thereby improving the comprehensiveness and accuracy of the training data set.

[0089] In one embodiment, the second task execution text is further used to prompt the intelligent language model to determine a target multidimensional operation field in the first natural query statement from candidate multidimensional operation fields in the first natural query statement, and the generated second natural query statement having the same query intent as the first natural query statement does not contain the target multidimensional operation field;

[0090] The step of constructing a second task processing instruction according to each of the first natural query statements and the preset second task execution text in step S1031 includes:

[0091] Step S10311, obtaining a candidate multidimensional operation field of each of the first natural query statements according to the multidimensional operation field used when each of the first natural query statements is constructed;

[0092] Step S10312: construct a second task processing instruction according to each of the first natural query statements, the candidate multi-dimensional operation fields corresponding to each of the first natural query statements, and a preset second task execution text.

[0093] This embodiment expands the function of the second task execution text, not only requiring the intelligent language model to generate a second natural query sentence with the same query intent as the first natural query sentence but with a different expression form, but also specifically instructing the model to identify and determine the target multidimensional operation field from the candidate multidimensional operation fields of the first natural query sentence, and exclude these target multidimensional operation fields in the generated second natural query sentence. This design aims to explore the diverse expressions of query intent without relying on specific multidimensional operation fields, thereby enhancing the diversity and flexibility of training samples and improving the comprehensive understanding ability of the trained data query model for multidimensional queries.

[0094] In step S10311, the multidimensional operation fields used in constructing the first natural query statement are first traced back and analyzed, and a set of candidate multidimensional operation fields are sorted out for each first natural query statement. These candidate fields represent different dimensions or operations that may be involved in the original query, and provide key information for the subsequent task processing instruction construction.

[0095] Subsequently, in step S10312, the second task processing instruction is constructed by combining the specific content of each first natural query statement, its corresponding candidate multidimensional operation field, and the preset second task execution text. This instruction not only includes instructions for generating diversified query statements, but also clearly points out the need to identify and exclude target multidimensional operation fields from candidate multidimensional operation fields. After receiving such instructions, the intelligent language model will be able to perform tasks more accurately and generate diversified query statements that not only meet the user's query intent but also avoid dependence on specific multidimensional operation fields.

[0096] Overall, this embodiment further enriches the flexibility and strategy of the intelligent language model in generating query statements by introducing a processing mechanism for candidate multidimensional operation fields. It enables the generated second natural query statement to not only maintain the same intent as the original query, but also to show more diverse expressions by removing the restrictions of specific multidimensional operation fields. This technical design not only improves the robustness and adaptability of the query system, but also provides users with more diversified query options, which helps to more comprehensively meet users' query needs.

[0097] In one embodiment, at least part of the first natural query statement contains query condition information, and the query condition information is used to limit the conditions that the data queried by the first natural query statement needs to meet; the query tag information of the second natural query statement also includes the query condition information in the first natural query statement corresponding to the second natural query statement; the first task execution text is also used to prompt that the query information output by the pre-trained text model includes query condition information.

[0098] Among them, the query condition information is a key component in the data query process, and its form and content directly affect the accuracy and relevance of the query results. At the same time, some complex query conditions are also challenging for the recognition ability of large models. Therefore, in this embodiment, enhanced training is mainly performed on complex query conditions. Specifically, the query condition information may include range conditions, nested conditions, and the like. Among them, the range condition specifies the value domain of an attribute, requiring that the data items in the query results must fall within this range. This can be a numerical range (such as a price between 100 and 500), a date and time range (such as a creation time between January 1, 2023 and December 31, 2023), or a fuzzy matching range of a string (such as a name starting with "A"). In addition, nested conditions may also be involved in some complex query scenarios-the query condition may contain a nested structure, that is, a condition contains other conditions. This nested condition can construct a hierarchical query logic, which is suitable for processing data structures with hierarchical relationships or complex associations.

[0099] In this embodiment, the training data and task execution text are further defined and optimized in response to the training requirements of the pre-trained text model in terms of query condition recognition capabilities. Specifically, this embodiment clearly points out that at least part of the first natural query statements in the training data should contain query condition information. These condition information serve as query constraints and are used to accurately define specific requirements that the queried data must meet - specifically, the query condition information defines specific conditions or rules that need to be met when querying data, ensuring that only data that meets these conditions will be retrieved. This design ensures that the training data not only contains rich query intent information, but also covers the critical conditional limitations in the query execution process, providing a solid foundation for the model to learn how to accurately identify and apply query conditions.

[0100] At the same time, in order to ensure that the model can maintain sensitivity to query conditions when generating the second natural query statement, we further stipulate that the query label information of the second natural query statement must contain the query condition information in the corresponding first natural query statement. This approach not only sets clear expected goals for model output, but also enables the model to clearly recognize the importance of query conditions in query statements during the training process, thereby prompting the model to consciously retain and accurately express these conditions when generating new query statements. In order to achieve the above goals, this embodiment also expands the function of the first task execution text, so that it not only guides the model to generate a variety of second natural query statements, but also explicitly requires the model to include query condition information in the output query information. The addition of this instruction provides a clear optimization direction for the model, that is, to maintain the query intent while maintaining the query intent. Figure 1 While ensuring consistency, ensure the completeness and accuracy of the query conditions.

[0101] Overall, this embodiment effectively improves the training effect of the pre-trained text model in the query condition recognition ability by finely defining the training data and task execution text. During the training process, the model can not only learn how to accurately understand the user's query intention, but also learn how to identify and express query conditions, thereby achieving more accurate and efficient data query in practical applications. This improvement in comprehensive capabilities not only enhances the practicality and reliability of the model, but also provides users with a more flexible and convenient query experience.

[0102] For step S104, a training data set is constructed according to each of the second natural query statements, the preset first task execution text, and the query label information corresponding to the second natural query statement; wherein the query label information includes the target multidimensional operation field contained in the first natural query statement corresponding to the second natural query statement; the first task execution text is used to prompt the pre-trained text model to output query information that meets the query intent of the second natural query statement, and the query information includes a query field.

[0103] In this embodiment, the pre-trained text model is prompted by the first task execution text to output query information that meets the query intent of the second natural query statement, and the query information includes query fields. For example, the first task execution text can be "Please generate corresponding query fields, and the query fields include the following types, respectively, and each query field is placed in the corresponding type...". Then, corresponding query label information is assigned to each second natural query statement, and the label information includes the target multi-dimensional operation field in the first natural query statement corresponding to the query statement. Finally, the second natural query statement, the first task execution text and the query label information are combined to construct a training data set.

[0104] For step S105, the training data set is input into a pre-trained text model for training; and the trained model is determined as a data query macro model corresponding to the target data warehouse.

[0105] Select or develop a suitable pre-trained text model (such as a large language model based on a Transformer structure), and input the training data set constructed in step S104 into the model for training. During the training process, the model will learn how to convert the second natural query statement into query information (such as SQL query statements) that can accurately reflect the user's query intention. After the training is completed, the accuracy and generalization ability of the model are verified by evaluating its performance on the test data set. Finally, the trained model is determined as the data query large model corresponding to the target data warehouse for actual data query tasks.

[0106] In one embodiment, the pre-trained text model includes an encoder and / or a decoder, wherein the encoder and the decoder respectively include an attention layer and a fully connected layer connected to each other;

[0107] The step of inputting the training data set into the pre-trained text model for training in step S105 includes:

[0108] Step S1051, obtaining an initialized low-rank matrix, embedding the low-rank matrix in a weight matrix of a fully connected layer or an attention layer of an encoder of the pre-trained text model, or embedding the low-rank matrix in a weight matrix of a fully connected layer or an attention layer of a decoder of the pre-trained text model;

[0109] Step S1052, input the training data set into the pre-trained text model, obtain the query information corresponding to each generated training sample; calculate the loss value between the query information of each training sample and the corresponding query label information; according to the loss value, adjust the weight of the low-rank matrix until the loss value is less than the preset loss threshold or the total number of weight adjustments reaches the preset number of adjustments, and obtain the trained model; or, input the training data set into the pre-trained text model, and obtain the query information corresponding to each training sample generated within the preset time at preset intervals; calculate the loss value between the query information of each training sample generated within the preset time and the corresponding query label information, and cache the current pre-trained text model and the corresponding loss value; according to the loss value, adjust the weight of the low-rank matrix until the total number of weight adjustments reaches the preset number of adjustments, and determine the pre-trained text model with the smallest loss value as the trained model from among the cached pre-trained text models and the corresponding loss values.

[0110] The pre-trained text model includes an encoder and / or a decoder, that is, the pre-trained text model of this embodiment can be an encoder-only model, a decoder-only model, or an encoder-decoder model, and all three types of models can be applied in the technical solution of this application. In the encoder or decoder, an attention layer and a fully connected layer are included respectively, and the attention layer and the fully connected layer are interconnected, which can effectively process text data and realize the conversion from input to output.

[0111] In step S1051, the weight matrix of the model is optimized by embedding a low-rank matrix to reduce the number of parameters and computational complexity of the model, and at the same time may improve the generalization ability of the model. Specifically, the initialized low-rank matrix is ​​embedded in the weight matrix of the fully connected layer or attention layer of the encoder or decoder. This embedding method can be flexibly selected and adjusted according to the specific structure and performance requirements of the model. Regarding the specific structure and principle of the low-rank matrix, it can be roughly understood that it is based on the original parameter weight matrix of the model, and two low-rank column matrices A and row matrices B are determined, and these two matrices are embedded in the model as a low-rank matrix in the form of a product, so that in the data training process, without changing the original structure and original weight matrix of the model, the weight of the low-rank matrix can be adjusted to achieve the adjustment of the overall superimposed weight of the model (the weight of the low-rank matrix is ​​superimposed). After the training is completed, the weight of the low-rank matrix is ​​merged into the model to obtain a fine-tuned model. Please refer to the LoRA fine-tuning principle in this field for details. If there are any inaccuracies in the description here, please refer to the LoRA fine-tuning principle in this field, and no further description will be given here.

[0112] In step S1052, the training data set obtained in the previous step is obtained. The training data set should contain rich training samples. Each training sample is a different second natural query statement (a query statement based on natural language expression and involving multi-dimensional operation field query). In addition, it also includes data such as task execution text. These data will construct task processing instructions and input them into the pre-trained text model for training. In addition, each training sample also includes query label information corresponding to the second natural query statement. These query label information are the target query information of the second natural query statement, that is, the accurate query information expected to be generated, so as to serve as a label during model training, which is used for the model to calculate the loss value and adjust the parameters according to the loss value. Among them, since the training samples used in this embodiment are all query statements involving multi-dimensional queries, the query label information obviously also includes multi-dimensional operation fields and basic fields bound to the multi-dimensional operation fields. In one embodiment, the query label information as a label is the target query information of the second natural query statement in a preset format, so that when fine-tuning the training model, the training model can organize the query information in a preset format and then output it. Correspondingly, the preset loss function comprehensively evaluates the accuracy of the query information and the accuracy of the format and calculates the loss value, thereby more accurately adjusting the weights of the low-rank matrix so that the fine-tuned pre-trained text model meets the query information accuracy and format accuracy requirements.

[0113] In step S1052, two training strategies are proposed, one of which is to input the training data set into the pre-trained text model embedded in the low-rank matrix. The model outputs the corresponding query information based on the input second natural query sentence and the first task execution text. Then, a preset loss function is used to calculate the loss value between the query information and the query label information of each training sample. According to the loss value, the weight of the low-rank matrix is ​​adjusted by the back propagation algorithm until the loss value is less than the preset loss threshold or the total number of weight adjustments reaches the preset number of adjustments. This process continuously adjusts the weight of the low-rank matrix and iterates continuously until the model converges, and the generated query information is as close as possible to the actual query label information, and a trained text model is obtained.

[0114] In addition, another training strategy can be adopted. Compared with the previous training strategy, it sets a preset time interval during the training process. At the end of each time interval, the system collects and calculates the loss value between the query information and the query label information of all training samples in the period. At the same time, the state of the current model (including its parameters) and the corresponding loss value are cached. As the training progresses, when the total number of weight adjustments reaches the preset limit, the system selects the one with the smallest loss value from the cached multiple model versions as the final training completion model. This method allows the model to conduct more exploration during the training process while ensuring that the final selected model has the best performance. Specifically, in the step of inputting the training data set into the pre-trained text model after embedding the low-rank matrix, the training data set can be divided into multiple small batches of data sets and input into the pre-trained text model one after another; or it can be divided into multiple batches of data sets, and each time a number of training samples are randomly obtained from the training data set (self-help resampling method or sampling method without replacement) and input into the pre-trained text model. After receiving the input training sample, the pre-trained text model starts to calculate and output the query information corresponding to the training sample. This embodiment obtains query information corresponding to each training sample generated within the preset time at preset intervals, uses a preset loss function to calculate the loss value between the query information of each training sample generated within the preset time and the corresponding query label information, and caches the current pre-trained text model and the corresponding loss value; according to the loss value, adjusts the weight of the low-rank matrix to update the first pre-trained text model; continues to input the training samples into the updated pre-trained text model, and again obtains query information corresponding to each training sample generated within the preset time at preset intervals, calculates the loss value, and caches the current pre-trained text model and the corresponding loss value; until the total number of weight adjustments reaches the preset number of adjustments, the pre-trained text model with the smallest loss value is determined as the trained text model from among the cached pre-trained text models and the corresponding loss values.

[0115] The trained text model described in this embodiment is fine-tuned on the basis of the pre-trained text model. Specifically, the LoRA fine-tuning method can be adopted. By embedding a low-rank matrix in the weight matrix of the model, the number of parameters and computational complexity that need to be adjusted in the model training process are reduced. This embodiment is based on a rich natural query statement involving multi-dimensional operation field queries, and constructs a rich training data set for training, so that the model can accurately understand the multi-dimensional operation fields and other information of the natural query statement, and generate query information that meets expectations. The trained model finally obtained can be used as a large data query model for the target data warehouse. When processing query tasks for the target data warehouse, it can show higher accuracy and efficiency, and improve the performance of the overall system and user experience.

[0116] Please refer to Figure 5 In one embodiment, the text model training method based on the constellation data model further comprises the steps of:

[0117] Step S201, in response to a user's update operation on the constellation data model, obtaining update information;

[0118] Step S202: if the update information indicates that a multidimensional operation field is newly added to the constellation data model, construct a plurality of third natural query statements according to the newly added multidimensional operation field; wherein the third natural query statement includes at least one newly added multidimensional operation field;

[0119] Step S203, generating a plurality of fourth natural query statements having the same query intent according to each of the third natural query statements; wherein the fourth natural query statements are expressed in a different form from the third natural query statements;

[0120] Step S204, constructing a plurality of training samples according to each of the fourth natural query statements, the first task execution text, and the newly added multi-dimensional operation field corresponding to the fourth natural query statement;

[0121] Step S205, updating the plurality of training samples into the training data set, and retraining the pre-trained text model with the updated training data set; determining the pre-trained text model after training as the data query macro model corresponding to the target data warehouse.

[0122] In this embodiment, the text model training method based on the constellation data model is further expanded to meet the needs of dynamic updating of the constellation data model. This embodiment describes in detail how to automatically adjust and optimize the training process of the pre-trained text model when the user updates the constellation data model to ensure that the model can accurately understand and process the updated data model.

[0123] First, in step S201, the system responds to the user's update operation on the constellation data model and captures and obtains update information in real time. Such update information may include changes in the data model structure, additions and deletions of fields, adjustments to data types, etc. Through this step, the system can timely understand and respond to the latest status of the data model.

[0124] Next, in step S202, the system checks the update information, and if it is found that the constellation data model has added a multidimensional operation field, it constructs several third natural query statements based on these newly added fields. These third natural query statements are designed to fully demonstrate the potential use of the newly added multidimensional operation field in the query, ensuring that each query statement contains at least one newly added multidimensional operation field, thereby enriching the diversity of the training data.

[0125] Subsequently, in step S203, the system uses natural language processing technology to generate several fourth natural query statements with the same query intent but different expressions for each third natural query statement. The purpose of this step is to increase the generalization ability of the training data, so that the pre-trained text model can learn to understand and process a variety of different query expressions, and improve the robustness and accuracy of the model. Then, in step S204, the system constructs several new training samples based on the generated fourth natural query statement, the original first task execution text (which may contain specific requirements or constraints on the query information format), and the newly added multi-dimensional operation field corresponding to the fourth natural query statement. These training samples not only contain new query statements, but also imply the latest structure and field information after the data model is updated, providing valuable data resources for the re-training of the pre-trained text model.

[0126] Finally, in step S205, the system updates the constructed new training samples to the original training data set, and uses the updated training data set to retrain the pre-trained text model. Through this process, the model can learn the latest data model structure and query operation mode, thereby improving its accuracy and efficiency when processing query tasks containing newly added multi-dimensional operation fields. After the training is completed, the obtained pre-trained text model is determined as the data query model corresponding to the target data warehouse, ready to be put into practical application.

[0127] In general, this embodiment achieves dynamic adjustment and optimization of the pre-trained text model training process by introducing an automatic response mechanism for constellation data model updates. This not only improves the model's adaptability to the latest data model structure, but also enhances the model's generalization ability and accuracy, providing users with more flexible and efficient data query services. At the same time, by automatically generating a variety of query statements as training samples, the diversity of training data is further enriched, which helps to improve the robustness and practicality of the model.

[0128] Please refer to Figure 6 The present application also provides a method for querying a large data model based on the method of any one of the above embodiments, comprising the following steps:

[0129] Step S301, obtaining a natural query statement input by a user in a query interface of a target data warehouse;

[0130] Step S302, obtaining a constellation data model corresponding to the target data warehouse; determining a number of related basic fields from the constellation data model as candidate basic fields according to the natural query statement; if the candidate basic field is bound to a multidimensional operation field, determining the bound multidimensional operation field as a candidate multidimensional operation field;

[0131] Step S303, constructing a task processing instruction according to the candidate basic fields, the candidate multidimensional operation fields, the natural query statement, and a preset query task execution text; wherein the query task execution text is used to prompt the data query big model to output query information that meets the query intent of the natural query statement, and the query information is determined according to the candidate basic fields and the candidate multidimensional operation fields;

[0132] Step S304, inputting the task processing instruction into the data query model corresponding to the target data warehouse to obtain output query information;

[0133] Step S305: query and obtain corresponding business data from the target data warehouse according to the query information and the constellation data model.

[0134] The data query method proposed in this embodiment achieves an in-depth understanding of the user's natural query statement and outputs accurate query information by integrating the constellation data model of the target data warehouse and its trained dedicated data query model. Among them, the constellation data model, as the cornerstone of structured data, not only carries the detailed information of the data table and basic fields of the target data warehouse, but also additionally associates the multidimensional operation field and its complex calculation logic with the basic field. When the user enters the query statement in the form of natural language to query the target data warehouse, the system can quickly filter out the relevant candidate basic fields and candidate multidimensional operation fields from the constellation data model of the target data warehouse. Subsequently, combining these candidate basic fields, candidate multidimensional operation fields, natural query statements and preset query task execution texts, the system constructs an accurate task processing instruction. This instruction not only guides the data query big model on how to interpret the user's query intention, but also ensures that it can effectively identify and process the query requirements involving multidimensional operation fields, accurately output query information that meets complex query requirements, thereby accurately extracting business data from the target database, and supporting users to conduct deeper data analysis and business insights. The method of this embodiment not only supports direct query of basic database fields, but also significantly enhances the processing capability of complex operation fields (such as MDX fields), so that data query is no longer limited to the traditional simple query scope. This innovative technology not only simplifies the data query process and lowers the user operation threshold, but also greatly improves the accuracy and flexibility of the query, provides users with more comprehensive and in-depth data analysis support, and significantly optimizes the overall user experience.

[0135] The data query method of this embodiment is executed by a computer. Hereinafter, the data query model of the embodiment of the present application is first described, and then each step is described in detail.

[0136] In this embodiment, the data query model is obtained by training the constellation data model of the target data warehouse through the text model training method of the above embodiment. It can parse the natural query sentence input by the user, understand the query intention, and generate corresponding query information according to these intentions, so as to retrieve data from the database later.

[0137] For step S301, a natural query statement input by a user in a query interface of a target data warehouse is obtained.

[0138] In this embodiment, the user can input the query requirements in the form of natural language through the user query interface provided by the system (such as a web page, mobile application or command line interface). For example, the user may input "query the names and sales of the top 10 products with the highest sales in the first quarter of 2023", or for another example, the user may input "query the total sales and year-on-year growth rate of each sales region in the first quarter of this year". This step is the starting point of the data query process and provides the necessary input for subsequent steps.

[0139] For step S302, the constellation data model corresponding to the target data warehouse is obtained; according to the natural query statement, several relevant basic fields are determined from the constellation data model as candidate basic fields; if the candidate basic field is bound to a multidimensional operation field, the bound multidimensional operation field is determined as a candidate multidimensional operation field.

[0140] This step first loads or accesses the constellation data model, and then, based on the natural query statement entered by the user, filters out basic fields related to the query content, such as "product name", "sales", etc., as candidate basic fields. These candidate basic fields are column names that may be needed when constructing query statements later. After determining the candidate basic fields, this step further checks whether these basic fields are bound to multidimensional operation fields. If any of the candidate basic fields are bound to multidimensional operation fields, these multidimensional operation fields are also determined as candidate multidimensional operation fields for use in subsequent query processing.

[0141] In one embodiment, the step of determining, in step S302, a number of relevant basic fields as candidate basic fields from the constellation data model according to the natural query statement includes:

[0142] Step S3021, performing keyword recognition on the natural query sentence to obtain a number of keywords;

[0143] Step S3022, vectorizing each of the keywords and each basic field in the constellation data model; calculating the vector similarity between any of the keywords and any of the basic fields;

[0144] Step S3023, for any of the keywords, determining a first preset number of basic fields that are ranked high in similarity to the keyword as relevant basic fields of the keyword;

[0145] Step S3024, ranking the relevant basic fields of each keyword by similarity according to the relevant basic fields of each keyword and their corresponding similarities;

[0146] Step S3025: Determine a second preset number of related basic fields with top similarity rankings as candidate basic fields.

[0147] The first preset number and the second preset number can be set according to specific circumstances. In this embodiment, the first preset number is set to 8, and the second preset number can be set to 40.

[0148] This embodiment describes in detail how to filter out the basic fields related to the natural query statement input by the user from the constellation data model as the candidate basic fields. First, in step S3021, the natural query statement is identified by keywords, and a number of keywords that can represent the user's query intention are extracted using natural language processing technology (such as word segmentation, part-of-speech tagging, etc.). These keywords are the basis for subsequent analysis and are crucial for accurately understanding user needs. Then, in step S3022, the identified keywords and each basic field in the constellation data model are converted into a vector representation in a high-dimensional space using vectorization technology. Vectorization is a common method for processing text data. It can convert text information into a numerical form that is easy for a computer to process, which is convenient for subsequent similarity calculations. Then, the vector similarity between each keyword and each basic field is calculated. Similarity calculation is a key step in evaluating the proximity of two vectors in a high-dimensional space, and its result can reflect the correlation between the keyword and the basic field. Furthermore, in step S3023, for each keyword, the first preset number of basic fields with a high similarity ranking are selected as the relevant basic fields of the keyword. Through this step, the basic fields most relevant to each keyword can be screened out, reducing the amount of data to be processed later and improving the efficiency and accuracy of the query. Step S3024 performs a global similarity ranking on these basic fields according to the relevant basic fields of each keyword and their corresponding similarities. Through the global similarity ranking, the query requirements of multiple keywords can be comprehensively considered to ensure that the candidate basic fields finally selected can fully reflect the user's query intention, thereby improving the comprehensiveness and accuracy of the query results. Finally, in step S3025, the relevant basic fields of the second preset number with the highest similarity ranking are determined as candidate basic fields. Through this step, a limited number of candidate basic fields are finally determined, which are most likely to meet the user's query requirements. This not only reduces the complexity and computational complexity of subsequent query processing, but also improves the response speed and user satisfaction of the query.

[0149] In summary, this embodiment achieves efficient and accurate conversion from natural query statements to candidate basic fields through steps such as keyword recognition, vectorization processing, similarity calculation, relevant basic field screening and global ranking, providing strong support for subsequent query processing.

[0150] In one embodiment, the step of ranking the related basic fields of each keyword by similarity according to the related basic fields of each keyword and their corresponding similarities in step S3024 includes:

[0151] Step S30241: If any two of the keywords have the same related basic fields and correspond to different similarities, the related basic field with the highest similarity among the same related basic fields is retained, and the remaining related basic fields are removed.

[0152] This embodiment further refines the specific implementation method of step S3024. Specifically, when it is found during the processing that any two or more keywords have the same relevant basic fields, but these same basic fields correspond to different similarities, a specific processing strategy is adopted. First, the system needs to identify which basic fields are considered relevant by multiple keywords, that is, there are repeated basic fields. For each basic field shared by multiple keywords, the system needs to compare the similarity between these keywords and the basic field. After comparing the similarities, the system will choose to retain the basic field with the highest similarity (with the corresponding keyword), and remove other (same) basic fields (that is, they will not participate in the global ranking).

[0153] For example, the natural query sentence input by the user is "What is the contract amount of the financial industry this year?" Assume that the determined keywords include "this year", "finance", "industry" and "contract amount". Among them, the related basic fields determined by the keyword "finance" and the corresponding similarities include "industry category" (0.84), "industry subcategory" (0.82), "contract amount" (0.77), etc.; and the related basic fields determined by the keyword "contract amount" and the corresponding similarities include "contract amount" (0.98), "average contract amount" (0.88), "number of contracts" (0.75), etc. Then, it can be found that the keywords "finance" and "contract amount" have the same related basic field "contract amount", but the corresponding similarities of the two are 0.77 and 0.98 respectively. Therefore, when all related basic fields are ranked globally, the related basic field with the greatest similarity is retained, that is, the "contract amount" with a similarity of 0.98 is included in the global similarity ranking, while the "contract amount" with a similarity of 0.77 is removed and does not participate in the global similarity ranking.

[0154] Finally, in the global similarity ranking of related basic fields, for two or more identical related basic fields (corresponding to different keywords), only the related basic field with the highest similarity will be retained to participate in the ranking. Therefore, two identical basic fields (these two identical basic fields correspond to different keywords with different similarities) will not appear in the global ranking.

[0155] This embodiment limits the sorting rules for the similarity of the relevant basic fields of keywords. For the case where different keywords correspond to the same relevant basic fields, by retaining the same relevant basic fields with the highest similarity and removing other same relevant basic fields with lower similarities, it is ensured that the second preset number of candidate basic fields finally determined contain as many different basic fields as possible, effectively reducing the redundant information in the candidate basic fields and expanding the scope of the candidate basic fields selected. On the whole, the candidate basic fields cover as many basic fields as possible, which is as comprehensive and relatively accurate as possible, laying a better foundation for the subsequent processing of the large data query model and improving the accuracy, efficiency and user experience of the final query.

[0156] In one embodiment, the step of performing keyword recognition on the natural query statement to obtain a plurality of keywords in step S3021 includes:

[0157] Step S30211, segmenting the natural query sentence based on a preset segmenter to obtain a segmentation result;

[0158] Step S30212: Based on the pre-trained entity recognition model, entity recognition is performed on the natural query sentence and the word segmentation result to obtain a first entity recognition result and a second entity recognition result respectively;

[0159] Step S30213, obtaining a number of keywords according to the first entity recognition result and the second entity recognition result.

[0160] Among them, the word segmenter is a basic tool in natural language processing, and its main function is to split a continuous text string into independent vocabulary units (tokens). There are many ways to implement the word segmenter, including rule-based word segmentation, statistical-based word segmentation, and hybrid methods. In practical applications, the choice of word segmenter depends on the specific language characteristics and task requirements. In this embodiment, a suitable word segmenter can be selected according to the specific scenario.

[0161] The entity recognition model is an important component in natural language processing, which is used to identify entities with specific meanings from text. These entities can be named entities (such as names of people, places, names of institutions, etc.), or other entities with specific meanings (such as time, date, numbers, etc.). Entity recognition models are usually built based on machine learning or deep learning technology, and learn the characteristics and patterns of entities in text by training a large amount of labeled data. This embodiment accurately identifies entities in text through the entity recognition model, and can better determine candidate basic fields related to natural query statements.

[0162] This embodiment describes in detail how to perform keyword recognition on natural query statements. This process aims to extract keywords that are crucial to subsequent query processing from natural language texts to more accurately determine relevant candidate basic fields. First, in step S30211, a preset word segmenter is used to perform word segmentation on the natural query statement. Through word segmentation, complex query statements can be decomposed into more easily processed vocabulary sequences. Then, in step S30212, a pre-trained entity recognition model is used to perform dual entity recognition on the natural query statement and its word segmentation results. By performing entity recognition on the original query statement and the word segmentation result respectively, the key information in the query statement can be captured more comprehensively and the accuracy of keyword recognition can be improved. The first entity recognition result is based on the overall understanding of the original query statement, while the second entity recognition result focuses more on the vocabulary unit after word segmentation. The two complement each other and together constitute an in-depth understanding of the query statement. Finally, in step S30213, several keywords are extracted based on the comprehensive analysis of the first entity recognition result and the second entity recognition result. These keywords include not only the explicit named entities in the query, but also some words with specific meanings or importance, which together constitute the core content of the query and provide key guidance for subsequent data retrieval and processing. In one embodiment, the entities in the first entity recognition result and the second entity recognition result are combined to obtain all the keywords of the natural query.

[0163] In summary, this embodiment achieves accurate recognition of keywords in natural query sentences by introducing word segmentation and dual entity recognition. This process not only improves the accuracy and comprehensiveness of keyword recognition, thereby ensuring accurate and comprehensive determination of candidate basic fields, but also provides strong support for subsequent query information generation and data retrieval.

[0164] For step S303, a task processing instruction is constructed according to the candidate basic fields, the candidate multidimensional operation fields, the natural query statement and the preset query task execution text; wherein the query task execution text is used to prompt the data query big model to output query information that conforms to the query intent of the natural query statement, and the query information is determined according to the candidate basic fields and the candidate multidimensional operation fields.

[0165] In this step, a task processing instruction is constructed using the candidate basic fields, the candidate multidimensional operation fields, the original natural query statement, and the preset query task execution text. This task processing instruction is designed to guide the data query big model to select accurate basic fields and multidimensional operation fields from the candidate basic fields and candidate multidimensional operation fields according to the intention of the natural query statement, and generate query information that meets the query intention.

[0166] As described in the previous embodiments, the query task execution text is used to require or prompt the task requirements of the data query large model, which includes specific requirements and format instructions for the model output, and may also include task-related scene information or context information for the model to learn and understand the tasks to be processed, etc. In the embodiment of the present application, different sub-texts can be set in the query task execution text according to the needs of the query, which are respectively used to require or prompt the corresponding task requirements of the data query large model. For example, the query task execution text may include a first query task execution text, a second query task execution text, a third query task execution text... and so on, and different sub-texts contain specific requirements for model processing and output according to task requirements, etc.

[0167] In one embodiment, the query task execution text includes a first query task execution text; the first query task execution text is used to prompt that the query information output by the data query big model includes a multidimensional operation field and a first basic field bound to the multidimensional operation field, the first basic field is determined from the candidate basic fields, and the multidimensional operation field is determined from the candidate multidimensional operation fields; the multidimensional operation field is used to specify multidimensional operations on the business data of the first basic field.

[0168] The first query task execution text is generally a command text used to request or prompt the data query big model what task to perform, which details the specific rules that the data query big model should follow when generating query information. Alternatively, the first query task execution text can also be an instruction set in a specific format, which also details the specific rules that the data query big model should follow when generating query information. The first query task execution text can be specifically determined in what form according to the requirements and specifications of the data query big model for task processing instructions.

[0169] In this embodiment, the first query task execution text prompts the data query big model, and the output query information includes a multidimensional operation field and a first basic field bound to the multidimensional operation field. This will guide the data query big model to deeply understand whether the natural query statement involves a query of the multidimensional operation field, and identify the multidimensional operation field and its corresponding basic field. Among them, it specifically instructs the data query big model to determine the first basic field from the candidate basic fields, and to determine the multidimensional operation field from the candidate multidimensional operation fields, guiding the data query big model to focus on these candidate fields. Since these candidate fields are all fields related to the user's query intention determined according to the constellation data model, therefore, it actually gives the data query big model enough information to enable it to accurately lock the query intention of the natural query statement and accurately determine the basic fields and / or multidimensional operation fields involved in this query.

[0170] It should be noted that, those skilled in the art understand that the query task execution text prompts the data query big model what to do, which does not mean that the data query big model must do it. For example, the query task execution text prompts the data query big model to output query information that meets the query intent, where the query information includes multidimensional operation fields, but after the data query big model understands the natural query statement, it believes that the query does not involve multidimensional operation fields. In this case, it may not output the multidimensional operation fields to avoid violating the core task of the query task execution text prompt, that is, to output query information that meets the query intent.

[0171] In one embodiment, the first query task execution text is further used to prompt that the first basic field bound to the multidimensional operation field is determined from the candidate basic fields bound to the multidimensional operation field. As described in the embodiment of the present application, any basic field in the constellation data model can be bound to one or more multidimensional operation fields. Similarly, the same multidimensional operation field may be bound to multiple basic fields. Therefore, when determining the candidate basic fields and determining the candidate multidimensional operation fields based on the candidate basic fields, the binding relationship between the basic fields and the multidimensional operation fields can also be recorded, and these binding relationships can be used to generate task processing instructions. When the data query large model recognizes that the query intent involves a multidimensional operation field, the first basic field can be determined from the candidate basic fields bound to this multidimensional operation field.

[0172] In one embodiment, the query task execution text also includes a second query task execution text; the second query task execution text is used to prompt that the query information output by the data query big model includes query conditions, and the query conditions are determined according to the candidate basic fields; the query conditions are used to limit the conditions that the business data queried by the first basic field needs to meet.

[0173] In this embodiment, the second query task execution text prompts the data query big model to include query conditions in the output query information, and is constructed based on the candidate basic fields to further limit the scope of the query results and ensure that the final query results meet the specified conditions.

[0174] For step S304, the task processing instruction is input into the data query big model corresponding to the target data warehouse to obtain output query information.

[0175] In this step, the constructed task processing instructions are sent to the data query model of the target data warehouse. The model analyzes the intent of the natural query statement according to the requirements in the instructions, combines the candidate basic fields and the candidate multi-dimensional operation fields, and generates query information that meets the query requirements / intent. The query information is usually expressed in a structured format (such as JSON), which contains the specific field names and possible query logic used for database query.

[0176] For step S305, the corresponding business data is queried from the target data warehouse according to the query information and the constellation data model.

[0177] This step builds a complete database query statement based on the query information output by the data query model and the constellation data model. This query statement is sent to the target data warehouse for execution to retrieve business data that meets the user's query intent. This business data may include data from basic fields, calculation results of multi-dimensional operation fields, etc. This data can then be returned to the user or used for further analysis and processing.

[0178] In one embodiment, the query task execution text includes a third query task execution text, and the third query task execution text is used to prompt the data query big model to output the query information according to a preset report query request format;

[0179] The step of querying and obtaining corresponding business data from the target data warehouse according to the query information and the constellation data model in step S305 includes:

[0180] Step S3051, using the query information as a report query request, and inputting the report query request and the constellation data model into the MDX query engine; obtaining corresponding business data from the target data warehouse through the MDX query engine; wherein the MDX query engine is an engine for parsing the report query request and performing data query based on the parsing results and the constellation data model.

[0181] Among them, the MDX (Multidimensional Expressions) query engine is a component specially designed to process complex report queries. It can understand and parse query requests written in the MDX query language, and retrieve corresponding business data from the database based on these requests and the definition of the constellation data model (multidimensional data structure model, commonly used in OLAP systems).

[0182] In this embodiment, for the understanding of the MDX query engine, specific reference can be made to the working principle of the Mondrian engine (an open source online analytical processing engine developed based on Java). In the MDX query engine, it is necessary to combine the data model description information and the MDX statement for querying. The MDX statement is a query statement with a similar grammatical structure to the SQL statement. In the SQL statement or the MDX statement, the general format is: select query field from query data source; when there is a field that needs to be defined, the query statement also includes "with field operation definition" before the select field; when there is a query restriction condition that needs to be limited, the query statement also includes "where query restriction condition" after the from field. Among them, the query data source generally specifies the target data table of the target database that the user wants to query, so it generally fills in the table name of the data table. In some cases, it can also fill in another query statement (the query statement actually determines a set of data, and this set of data can also be considered to constitute a data table).

[0183] Specifically, the data model description information is generally a Schema XML (Extensible Markup Language) file, which describes the database table description information, measurement description information, dimension description information, and multidimensional operation field (MDX operation field) description information (including calculated member description information and / or named set description information) of a data model cube; the query data source of the MDX statement is determined to be the model name defined in the data model description information (generally speaking, the Schema XML defines the model name as cube, which corresponds to from cube in the MDX statement). In the MDX query engine, for a multidimensional operation field (MDX operation field), corresponding operation definition information (such as a calculation formula) is required to perform the corresponding operation. Specifically, the definition of the multidimensional operation field can be defined in the with part of the MDX statement, or in the data model description information (Schema XML file).

[0184] In this embodiment, the MDX query engine is a multidimensional query engine obtained by secondary development based on a general multidimensional analysis engine such as the Mondrian engine, that is, it not only executes queries according to MDX statements and data model description information like a general multidimensional analysis engine, but also generates MDX statements and data model description information before that. For details, please refer to the prior art description, and also refer to the description involved in the subsequent embodiments.

[0185] In this embodiment, the function of the query task execution text is further expanded, and a third query task execution text is introduced. The text has a specific function - to guide the data query big model to organize and output the query information according to the preset report query request format. Specifically, the third query task execution text can set prompt texts one by one according to the preset report query request format, so as to require the data query big model to adjust the format of the query information according to the prompt requirements one by one, so as to finally output the query information that conforms to the report query request format. This design is intended to ensure that the format of the query information matches the expected input format of the back-end data system (especially the database system that supports MDX queries), thereby improving the accuracy and efficiency of data queries.

[0186] Specifically, when performing data query, the query information output by the data query big model is directly submitted to the MDX query engine as a report query request. The MDX query engine receives the report query request, parses the target field of the user's query and the query restriction conditions involved, and then uses the constellation data model as a navigation map to perform query operations in the database and finally returns the business data that meets the conditions.

[0187] In summary, this application significantly enhances the interaction between the data query big model and the backend data system by introducing the third query task execution text and defining the implementation process of step S305 in detail. Specifically, it ensures that the format of the query information is highly consistent with the input requirements of the MDX query engine, so that based on the existing technical architecture of the MDX query engine, it is only necessary to directly input the query information as a general report query request into the MDX query engine to perform the query.

[0188] In one embodiment, the MDX query engine in step S3051 obtains the corresponding business data by querying in the following manner:

[0189] Step S30511, parsing the report query request according to a preset parsing method to obtain a target field for query; determining a basic field according to the target field;

[0190] Step S30512, determining a plurality of star data models including the basic fields from the constellation data model; wherein each of the star data models respectively records a fact table, a dimension table, and the basic fields respectively included in each data table;

[0191] Step S30513, if the target field includes a multidimensional operation field, obtain corresponding operation definition information according to the multidimensional operation field; generate data model description information according to each of the star data models and the operation definition information; generate an MDX statement according to the report query request and the operation definition information; query the target field from the target data warehouse according to the MDX statement and the data model description information; determine that the query result of the target field is the business data obtained by the query;

[0192] Step S30514, if the target field does not include a multidimensional operation field, generate data model description information according to each of the star data models; generate an MDX statement according to the report query request; query the target field from the target data warehouse according to the MDX statement and the data model description information; determine that the query result of the target field is the business data obtained by the query.

[0193] Among them, in step S30511, the report query request is parsed to obtain the target field that the user wants to query. Specifically, in different query requirements, the target field obtained by parsing is generally different. In this embodiment, the target field obtained by parsing the report query request includes basic fields and / or multi-dimensional operation fields. Generally speaking, the target field queried by the report query request can include only basic fields (meaning that only the basic data in the database is queried), or only multi-dimensional operation fields (meaning that only a certain data analysis result or a certain specified multi-dimensional data operation is queried, such as the year-on-year growth rate, without querying specific basic data), and can also include both basic fields and multi-dimensional operation fields (meaning that both the basic data and the data obtained by the specified multi-dimensional data operation are queried). Among them, if the target field of the query includes a multidimensional operation field, the multidimensional operation field will also be bound to at least one basic field - that is, the multidimensional operation field actually specifies the operation of the bound basic field. However, it should be noted that in this case, the multidimensional operation field is the target field of the query, but the basic field bound to it is not the target field of the query. For example, the query request queries the "sales volume" "same period growth rate", where "sales volume" is the target field of the query, and "sales volume" is just the basic field referenced by the target field. The final query result should be the query result of "sales volume" and does not include the data of the "sales volume" field (unless the basic field "sales volume" is also the target field of the user's query). This step determines the basic field related to the query based on the target field (same as above, not necessarily the target field of the query), so that in step S30512, the star data model related to this query can be determined based on the basic field. Specifically, if the target field of the query is a basic field, then it is the basic field related to the query; if the target field of the query is a multidimensional operation field, then the basic field bound to it is the basic field related to the query. In some special cases, there is another type of aggregate field. This type of field is associated with a basic field. The associated basic field is the basic field related to the query. For example, for the aggregate field "average sales volume", the "sales volume" field is the basic field related to the query.

[0194] In step S30512, the star data model is determined from the constellation data model. Generally speaking, in a complex constellation data model, if different data tables are centered, different star data models can be determined. A data table can be used as a fact table in a star data model and as a dimension table in other star data models.

[0195] In this step, the main goal of determining the star data model from the constellation data model is to obtain the minimum available star data model related to the query. The minimum available star data model should only contain data tables related to the query, and the data tables should only contain basic fields determined in step S30511, wherein the data tables related to the query are divided into several cases: for dimension tables, they are considered to be data tables related to the query only when they contain the basic fields determined in step S30511 or are bound to query-related restriction information; there is no limitation for fact tables. Generally speaking, a fact table is a necessary data table in a star data model, even if it does not contain basic fields. There are many ways to determine the minimum available star data model, and specific reference may be made to the prior art in the field, which will not be elaborated here.

[0196] In step S30513, the operation definition information is used to define the multidimensional operation mode corresponding to the multidimensional operation field, which can also be said to define what kind of operation is performed on the data of the basic field bound to the multidimensional operation field. Take an example of querying in an existing report query interface (including various query components) for explanation, assuming that the predefined multidimensional operation field is "sales volume" and the user selects the "sales volume" metric field during the query, and selects the parameter of its "time" dimension field as 2024, and selects the "sales volume" multidimensional operation field, submits the query, then, when the backend processes the query, it obtains the corresponding definition formula according to the "sales volume", such as "sales volume = (current data - previous data) / previous data", and substitutes the specific basic field (metric field and / or dimension field) and related parameters into the definition formula to obtain the operation definition information of the "sales volume" of the query, such as "sales volume in 2024 - sales volume in 2023) / sales volume in 2023". Of course, this example is only for illustrative purposes to facilitate understanding of the multidimensional operation fields and their operation definition information. The specific implementation details can be set according to the situation. How each of the star data models generates data model description information and how the report query request generates an MDX statement are prior arts in the art. Please refer to the prior arts for details and will not be elaborated here.

[0197] In step S30514, the target data warehouse includes data of each data table corresponding to the constellation data model. That is, the target data warehouse is actually the basis for modeling the constellation data model. In one embodiment, the target data warehouse is preferably an OLAP (On-Line Analytical Processing) database.

[0198] In this embodiment, the basic field is determined according to the target field queried by the user, and several star data models containing the basic field are determined. If the target field includes a multidimensional operation field, the corresponding operation definition information is obtained according to the multidimensional operation field through the MDX query engine; data model description information is generated according to each of the star data models and the operation definition information; MDX statements are generated according to the report query request and the operation definition information; and query results of the target field are obtained from the database according to the MDX statements and the data model description information. If the target field does not include a multidimensional operation field, data model description information is directly generated according to each of the star data models, and MDX statements are directly generated according to the report query request; the subsequent steps are similar according to the MDX statements and the data model description information. Through this embodiment, the MDX query engine can accurately parse the report query request, generate data model description information and MDX statements, and thus, in combination with the data model description information and the MDX statements, it can efficiently query the query results of the target field from the target data warehouse, and determine that the query results of the target field are the business data obtained by the query.

[0199] In one embodiment, after the step of determining several related basic fields as candidate basic fields from the constellation data model according to the natural query statement in step S302, the method further includes the following steps:

[0200] Step S401: if there is no multi-dimensional operation field bound to any of the candidate basic fields, a task processing instruction is constructed according to the candidate basic fields, the natural query statement, and a preset query task execution text; the query task execution text is used to prompt the data query big model to output query information that meets the query intent of the natural query statement, and the query information is determined according to the candidate basic fields;

[0201] Step S402, inputting the task processing instruction into the data query model corresponding to the target data warehouse to obtain output query information;

[0202] Step S403: query and obtain corresponding business data from the target data warehouse according to the query information and the constellation data model.

[0203] This embodiment illustrates the data query steps when there is no basic field binding multidimensional operation field in the candidate basic field. The difference between it and the above embodiment is that the candidate multidimensional operation field is not determined, and the candidate multidimensional operation field is not included when constructing the task processing instruction. At the same time, the query task execution text can use other texts, and only prompt the data query big model to output query information that meets the query intention of the natural query statement, wherein the query information is determined according to the candidate basic field. Of course, in other embodiments, the query task execution text can also use the text of the above embodiment. Even if there is no candidate multidimensional operation field in this embodiment, the data query big model will naturally understand and realize how to process it, and the final processing effect is basically the same. In addition, the explanation of each step of this embodiment specifically refers to the above embodiment, and its technical conception and technical means are consistent, so they will not be repeated here.

[0204] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present application, and the present application is also intended to include such modifications and modifications.

Claims

1. A text model training method based on a constellation data model, characterized in that: The following steps are involved: Obtaining several target multidimensional operation fields in the constellation data model corresponding to the target data warehouse; The constellation data model records a number of data tables, basic fields contained in each data table, and multi-dimensional operation fields bound to at least some of the basic fields; Constructing a plurality of first natural query statements according to the plurality of target multi-dimensional operation fields; wherein the first natural query statement includes at least one target multi-dimensional operation field; For each of the first natural query statements, generate a plurality of second natural query statements having the same query intent but different expression forms; A training data set is constructed according to each of the second natural query statements, the preset first task execution text, and the query label information corresponding to the second natural query statement; wherein the query label information includes the target multi-dimensional operation field contained in the first natural query statement corresponding to the second natural query statement; the first task execution text is used to prompt the pre-trained text model to output query information that meets the query intent of the second natural query statement, and the query information includes a query field; Inputting the training data set into the pre-trained text model for training; determining the trained model as the data query macro model corresponding to the target data warehouse; Before the step of obtaining several target multi-dimensional operation fields in the constellation data model corresponding to the target data warehouse, the steps include: Obtain historical natural query statement data of the target data warehouse; Determine a number of natural query statements related to the multidimensional operation field in the historical natural query statement data; and analyze, based on the number of natural query statements related to the multidimensional operation field, obtain a multidimensional operation field in which a frequency of natural language expression by users is greater than a preset frequency; Determine a multidimensional operation field whose frequency of natural language expression is greater than a preset frequency as a target multidimensional operation field; The step of generating, for each of the first natural query statements, a plurality of second natural query statements having the same query intent but different expression forms comprises: Constructing a second task processing instruction according to each of the first natural query statements and a preset second task execution text; the second task execution text is used to prompt the intelligent language model to generate a plurality of second natural query statements having the same query intent as the first natural query statement, and satisfying that the second natural query statement and the first natural query statement have different expression forms; Inputting the second task processing instruction into an intelligent language model to obtain a plurality of second natural query statements with the same query intent corresponding to each of the first natural query statements; The second task execution text is also used to prompt the intelligent language model to determine a target multidimensional operation field in the first natural query statement from candidate multidimensional operation fields in the first natural query statement, and the generated second natural query statement having the same query intent as the first natural query statement does not contain the target multidimensional operation field; The step of constructing a second task processing instruction according to each of the first natural query statements and the preset second task execution text includes: According to the multidimensional operation fields used when constructing each of the first natural query statements, a candidate multidimensional operation field of each of the first natural query statements is obtained; A second task processing instruction is constructed according to each of the first natural query statements, the candidate multi-dimensional operation fields corresponding to each of the first natural query statements, and a preset second task execution text.

2. The text model training method based on the constellation data model according to claim 1 is characterized in that: The step of constructing a plurality of first natural query statements according to the plurality of target multi-dimensional operation fields includes: Obtain several preset natural query statement templates; According to the plurality of natural query statement templates and the plurality of target multi-dimensional operation fields, a plurality of natural query statements for querying the target multi-dimensional operation fields are generated, which are first natural query statements.

3. The text model training method based on the constellation data model according to claim 2 is characterized in that: The natural query statement template includes a first natural query statement template including basic fields to be filled in and multi-dimensional operation fields; The step of generating a plurality of natural query statements for querying the target multidimensional operation fields according to the plurality of natural query statement templates and the plurality of target multidimensional operation fields comprises: Obtain the target multidimensional operation fields, and obtain the basic fields bound to the target multidimensional operation fields, fill each target multidimensional operation field and the corresponding bound basic field into the first natural query statement template respectively, and obtain a plurality of third natural query statements; determine the third natural query statement as the first natural query statement.

4. The text model training method based on the constellation data model according to any one of claims 1 to 3, characterized in that: At least part of the first natural query statement includes query condition information, where the query condition information is used to limit conditions that data queried by the first natural query statement needs to satisfy; The query tag information of the second natural query statement also includes query condition information in the first natural query statement corresponding to the second natural query statement; the first task execution text is also used to prompt that the query information output by the pre-trained text model includes query condition information.

5. The text model training method based on the constellation data model according to claim 1 is characterized in that: The method further comprises the steps of: In response to a user's update operation on the constellation data model, acquiring update information; If the update information indicates that a multidimensional operation field is newly added to the constellation data model, construct a plurality of third natural query statements according to the newly added multidimensional operation field; wherein the third natural query statement includes at least one newly added multidimensional operation field; Generating a plurality of fourth natural query statements having the same query intent according to each of the third natural query statements; wherein the fourth natural query statements are expressed in a different form from the third natural query statements; constructing a plurality of training samples according to each of the fourth natural query statements, the first task execution text, and the newly added multi-dimensional operation field corresponding to the fourth natural query statement; The plurality of training samples are updated into the training data set, and the pre-trained text model is re-trained through the updated training data set; the pre-trained text model that has been trained is determined as the data query macro model corresponding to the target data warehouse.

6. The text model training method based on the constellation data model according to claim 1 is characterized in that: Before the step of obtaining several target multi-dimensional operation fields in the constellation data model corresponding to the target data warehouse, the step includes: The target data warehouse information selected by the user is obtained, and the constellation data model corresponding to the target data warehouse information is determined.

7. A data query method based on the data query large model obtained by any one of the methods of claims 1 to 6, characterized in that: The following steps are involved: Obtaining natural query statements input by users in the query interface of the target data warehouse; Acquire a constellation data model corresponding to the target data warehouse; Determining, according to the natural query statement, several relevant basic fields from the constellation data model as candidate basic fields; If the candidate basic field is bound to a multi-dimensional operation field, the bound multi-dimensional operation field is determined as a candidate multi-dimensional operation field; According to the candidate basic fields, the candidate multi-dimensional operation fields, the natural query statement and the preset query task execution text, a task processing instruction is constructed; wherein the query task execution text is used to prompt the data query big model to output query information that meets the query intent of the natural query statement, and the query information is determined according to the candidate basic fields and the candidate multi-dimensional operation fields; Inputting the task processing instruction into the data query big model corresponding to the target data warehouse to obtain output query information; According to the query information and the constellation data model, corresponding business data is queried from the target data warehouse.

Images