Data processing method and device, electronic equipment and storage medium

Abstract

The present disclosure discloses a data processing method and device, electronic equipment and storage medium, relates to the field of artificial intelligence, in particular to the field of intelligent question answering, deep learning and large model, and can be applied to the scene of content generation based on artificial intelligence. The specific implementation scheme is as follows: obtaining problem content information of a user question and a target virtual table in a target question scenario to which the user question belongs; wherein the target virtual table is used to indicate first meta information of a first physical table in the target question scenario; generating an intermediate state query language for querying data from the target virtual table under the constraint of constraint information by using a large model according to the problem content information and the target virtual table; obtaining target information by analyzing the intermediate state query language; generating an executable query language for querying data from the first physical table according to the target information; obtaining a query result by executing the executable query language; and performing a display operation according to the query result.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to the fields of intelligent data collection, deep learning, and large models, and can be applied to artificial intelligence-generated content (AIGC) scenarios. In particular, it relates to a data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In recent years, with the continuous evolution and application of large model technology, more and more organizations and institutions have joined the ranks of building efficient business systems based on large model capabilities. Among various large model-based system applications, how to organically combine data analysis and large model capabilities to form an interactive question-answering system that supports natural language is currently a hot research and application direction in the industry. Summary of the Invention

[0003] This disclosure provides a data processing method, apparatus, electronic device, and storage medium.

[0004] According to a first aspect of this disclosure, a data processing method is provided, comprising: acquiring problem content information of a user's problem and a target virtual table under a target problem scenario to which the user's problem belongs; wherein the target virtual table is used to indicate first metadata of a first physical table under the target problem scenario; generating an intermediate query language for querying data from the target virtual table using a large model under constraint information based on the problem content information and the target virtual table; parsing the intermediate query language to obtain target information; generating an executable query language for querying data from the first physical table based on the target information; obtaining a query result by executing the executable query language; and performing a display operation based on the query result.

[0005] According to a second aspect of this disclosure, a data processing apparatus is provided, comprising: a first acquisition module, configured to acquire problem content information of a user's question and a target virtual table under a target problem scenario to which the user's question belongs; wherein the target virtual table is used to indicate first metadata of a first physical table under the target problem scenario; a first generation module, configured to generate an intermediate query language for querying data from the target virtual table using a large model under constraint information, based on the problem content information and the target virtual table; a parsing module, configured to parse the intermediate query language to obtain target information; a second generation module, configured to generate an executable query language for querying data from the first physical table based on the target information; a first processing module, configured to obtain a query result by executing the executable query language; and a second processing module, configured to perform a display operation based on the query result.

[0006] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the data processing method as described in the first aspect.

[0007] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided that stores computer instructions for causing the computer to perform the data processing method as described in the first aspect.

[0008] According to a fifth aspect of this disclosure, a computer program product is provided, including computer instructions that, when executed by a processor, implement the data processing method as described in the first aspect.

[0009] The data processing method, apparatus, electronic device, and storage medium disclosed herein have the following beneficial effects:

[0010] This method involves obtaining the problem content information of a user's question and a target virtual table within the target problem scenario. The target virtual table indicates the first element information of the first physical table within the target problem scenario. Based on the problem content information and the target virtual table, a large model is used to generate an intermediate query language for retrieving data from the target virtual table, under constraints. The intermediate query language is parsed to obtain target information. An executable query language is generated based on the target information to retrieve data from the first physical table. The query results are obtained by executing the executable query language. Finally, a display operation is performed based on the query results. This method, by generating an intermediate query language for retrieving data from the target virtual table using a large model under constraints, based on the user's question content and the virtual table within the target problem scenario, significantly reduces the difficulty for the large model to understand the data, thereby improving the accuracy of the large model's output results.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0013] Figure 1 This is a schematic flowchart of the data processing method provided according to the first embodiment of this disclosure;

[0014] Figure 2 This is a schematic flowchart of the data processing method provided according to the second embodiment of this disclosure;

[0015] Figure 3 This is a flowchart illustrating the process of generating an executable query language according to the third embodiment of this disclosure;

[0016] Figure 4 This is a schematic flowchart of the data processing method provided according to the fourth embodiment of this disclosure;

[0017] Figure 5 This is a flowchart illustrating the data processing method provided according to the fifth embodiment of this disclosure;

[0018] Figure 6 This is a schematic flowchart of the data processing method provided according to the sixth embodiment of this disclosure;

[0019] Figure 7 This is a schematic flowchart of the data processing method provided according to the seventh embodiment of this disclosure;

[0020] Figure 8This is a schematic diagram of the structure of a conversational data analysis system provided according to the eighth embodiment of this disclosure;

[0021] Figure 9 This is a flowchart of the scenario interface management of a conversational data analysis system provided in the ninth embodiment of this disclosure;

[0022] Figure 10 This is a schematic diagram showing the results of a conversational data analysis system provided according to the tenth embodiment of this disclosure;

[0023] Figure 11 This is a schematic diagram of the structure of a data processing apparatus according to the eleventh embodiment of this disclosure;

[0024] Figure 12 A schematic block diagram of an example electronic device 1200 that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0025] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0026] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision, and disclosure of users' personal information are all carried out with the consent of the users, and all comply with the provisions of relevant laws and regulations, and do not violate public order and good morals.

[0027] In recent years, with the continuous evolution and application of large model technology, more and more organizations and institutions have joined the ranks of building efficient business systems based on large model capabilities. Among various large model-based system applications, how to organically combine data analysis and large model capabilities to form an interactive question-answering system that supports natural language is currently a hot research and application direction in the industry.

[0028] Focusing on the field of intelligent data analysis, traditional BI (Business Intelligence) systems currently offer users a fixed analytical model. This model only supports users in analyzing the data content that has already been developed and is available. If users want to explore more data possibilities, they need to develop additional fixed models. However, interactive intelligent analysis systems based on large models enable users to explore and gain insights into business data through natural language, thereby bringing more possibilities for data value discovery to the field of data analysis in a more flexible, natural, and efficient way.

[0029] In related technologies, when using large models to process interactive question answering, the large models need to understand massive amounts of data, which is difficult to understand and results in low accuracy of the output results.

[0030] To address the aforementioned problems, this disclosure provides a data processing method, apparatus, electronic device, and storage medium.

[0031] The data processing method, apparatus, electronic device, and storage medium of this disclosure are described below with reference to the accompanying drawings.

[0032] It should be noted that the data processing method in this embodiment is executed by a data processing device, which can be implemented by software and / or hardware and can be configured in an electronic device.

[0033] Figure 1 This is a flowchart illustrating the data processing method provided according to the first embodiment of this disclosure.

[0034] like Figure 1 As shown, the data processing method includes:

[0035] Step 101: Obtain the problem content information of the user's problem and the target virtual table under the target problem scenario to which the user's problem belongs.

[0036] User questions can be questions asked by users in natural language.

[0037] As an example, user questions could be such as "What was the order quantity of product xx for xx company in Q3 2023?" or "How many employees does xx company have?"

[0038] The question content of a user question refers to the question content information obtained after processing the questions asked by users in natural language.

[0039] As an example, we can identify jargon in user questions, such as the "Q3" in the user question "What is the order quantity of xx product in Q3 2023 for xx company?" Then, we can replace the jargon, for example, by replacing Q3 with the third quarter, to obtain more natural and standardized question content information.

[0040] Each user issue has its own associated problem scenario. Problem scenarios can include business scenarios, human resource scenarios, etc.

[0041] As an example, the user question "What was the order quantity of product xx in Q3 2023 for xx company?" belongs to a business scenario, and the user question "How many employees does xx company have?" belongs to a human resources scenario, etc.

[0042] In this embodiment, the target problem scenario is the problem scenario to which the user's problem belongs.

[0043] The target virtual table is used to indicate the first metadata of the first physical table in the target problem scenario.

[0044] The first physical table is the data table that actually stores the corresponding data. The number of first physical tables can be one or more, and this embodiment does not limit this.

[0045] Among them, the first metadata is the field metadata of the first physical table.

[0046] In this embodiment, a problem scenario corresponds to a virtual table, and the target virtual table is the virtual table corresponding to the target problem scenario (the virtual table under the target problem scenario). The target virtual table is only used to indicate the first metadata of the first physical table under the target problem scenario, and does not actually store data.

[0047] As an example, the target virtual table can consist of the first metadata of at least one first physical table. That is, the field set in the target virtual table is assembled from the field set of at least one first physical table in the target problem scenario.

[0048] Step 102: Based on the problem content information and the target virtual table, use the large model to generate an intermediate query language for querying data from the target virtual table under the constraints of the constraint information.

[0049] Among them, constraint information is used to constrain the output results of large models.

[0050] As an example, constraint information could be instructing the large model to return a piece of text, or instructing the large model to return a piece of code, or instructing the large model to return a query language, etc. In this embodiment, constraint information is used to instruct the large model to return a query language.

[0051] Among them, the large model has undergone SFT (Supervised Fine-Tuning) and can output results relatively accurately.

[0052] In this embodiment, the problem content information, the target virtual table, and the constraint information can be used as inputs to the large model to obtain an intermediate query language generated by the large model for querying data from the target virtual table.

[0053] In this disclosure, by weaving data from the first metadata of at least one first physical table under different problem scenarios, a virtual table is obtained for each problem scenario. An intermediate query language for querying data from this virtual table is generated using a large model. This significantly reduces the difficulty for the large model to understand the data, thereby improving the accuracy of the output results of the large model (the accuracy of the large model in generating results when facing a single table is higher than the accuracy when facing multiple tables).

[0054] It should be noted that the intermediate query language is used to query data from the target virtual table. However, the target virtual table is only used to indicate the first meta-information of the first physical table in the target problem scenario and does not actually store data. Therefore, the intermediate query language generated by the large model is not executable.

[0055] Step 103: Parse the intermediate query language to obtain the target information.

[0056] Since intermediate query language is not executable, it needs to be parsed to generate executable query language in order to obtain the query results corresponding to the user's question. In this embodiment, the intermediate query language can be parsed to obtain target information used to generate executable query language.

[0057] As an example, target information is used to indicate the physical table on which the intermediate query language depends.

[0058] Step 104: Generate an executable query language based on the target information to query data from the first physical table.

[0059] It is understandable that the target virtual table is used to indicate the first meta-information of the first physical table in the target problem scenario, and the intermediate query language is used to query data from the target virtual table. Therefore, the target information obtained by parsing the intermediate query language is information related to the first physical table, so that an executable query language for querying data from the first physical table can be generated based on the target information.

[0060] Step 105: Obtain the query results by executing the executable query language.

[0061] It should be noted that the executable query language is used to query data from the first physical table, which is the data table that actually stores the corresponding data. Therefore, the query results requested by the user can be obtained by executing the executable query language.

[0062] Step 106: Perform the display operation based on the query results.

[0063] In this embodiment, the query results can be displayed after they are obtained.

[0064] As an example, query results can be displayed in the form of text and / or images.

[0065] In this embodiment, the problem content information of the user's question and the target virtual table under the target problem scenario to which the user's question belongs are obtained. The target virtual table is used to indicate the first metadata of the first physical table under the target problem scenario. Based on the problem content information and the target virtual table, an intermediate query language for querying data from the target virtual table is generated using a large model under constraint information. The intermediate query language is parsed to obtain target information. An executable query language for querying data from the first physical table is generated based on the target information. The query result is obtained by executing the executable query language. A display operation is performed based on the query result. This disclosure, by generating an intermediate query language for querying data from the target virtual table using a large model under constraint information based on the user's question content and the virtual table under the target problem scenario, can significantly reduce the difficulty of the large model understanding the data, thereby improving the accuracy of the large model's output results.

[0066] Figure 2 This is a flowchart illustrating the data processing method provided according to the second embodiment of this disclosure.

[0067] like Figure 2 As shown, the data processing method includes:

[0068] Step 201: Obtain the problem content information of the user's problem and the target virtual table under the target problem scenario to which the user's problem belongs.

[0069] Step 202: Based on the problem content information and the target virtual table, use the large model to generate an intermediate query language for querying data from the target virtual table under the constraints of the constraint information.

[0070] Step 203: Parse the intermediate query language to obtain the target information.

[0071] The target information includes result fields and condition columns.

[0072] Step 204: Determine the second physical table from the first physical table based on the target information.

[0073] In this embodiment, the physical tables that the intermediate query language generated by the large model depends on can be identified based on the result fields and condition columns.

[0074] The second physical table is the physical table to which the result field in the first physical table belongs, and the physical table to which the condition column belongs.

[0075] One possible approach is to determine the second physical table to be queried from the first physical tables based on which first physical tables the result fields originate from, and which first physical tables the condition columns originate from.

[0076] Step 205: Generate an executable query language based on the second physical table and the target information.

[0077] In this embodiment, an executable query language for querying data from the second physical table can be generated based on the determined second physical table to be queried, result fields, and condition columns.

[0078] To ensure that users can only access the data they are authorized to access and to improve data security, in one possible implementation of this disclosure, a candidate query language is generated based on a second physical table and target information; the candidate query language is updated based on the user identifier corresponding to the user question and the permission table to obtain an executable query language with access control.

[0079] Among them, user identifiers are used to identify user identity information, such as username, user ID (Identity), etc.

[0080] The permissions table is used to record the access permissions of at least one user to the data in the physical table.

[0081] Among them, the executable query language is an executable query language with access control. This means that when the executable query language is executed, it will only access the data that the user is authorized to access, and will not access the data that is not authorized to access, thereby improving data security.

[0082] One possible implementation is to obtain an executable query language with access control by adding permission filtering conditions, such as the range of data and field access permissions, to the candidate query language based on the user identifier corresponding to the user question and the permission table.

[0083] To clearly illustrate the process from a user's question to an executable query language, an example is provided below.

[0084] Figure 3 This is a flowchart illustrating the process of generating an executable query language according to the third embodiment of this disclosure.

[0085] like Figure 3As shown, firstly, based on the user's question, the input of the large model is determined (question content information, target virtual table, constraint information). Then, the large model generates an intermediate query language corresponding to the user's question based on the single target virtual table from its input (the large model generates the intermediate query language based on the target virtual table). This language is a single-table query language oriented towards the target virtual table. Next, the intermediate query language is parsed to obtain the corresponding set of output fields (result fields) and the set of columns in the query conditions (condition columns). These two sets are used to identify which physical tables the intermediate query language given by the large model specifically depends on. For example, the result field set identifies a dependency on physical table 1, and the condition column set identifies a dependency on physical table 3. Figure 3 The two physical tables are connected by a solid line, and then a multi-table join query language (an executable query language based on multiple physical tables) is dynamically constructed using the unique keys of the two tables.

[0086] Optionally, the permission table association strategy can be dynamically added based on the identity of the accessing user to generate a physically executable final-state query language with permissions (an executable query language with permission control).

[0087] Step 206: Obtain the query results by executing the executable query language.

[0088] Step 207: Perform the display operation based on the query results.

[0089] It should be noted that the explanations of steps 201, 202, 203, 206 and 207 can be found in the relevant descriptions in any embodiment of this disclosure, and will not be repeated here.

[0090] In this embodiment, the problem content information of the user's question and the target virtual table under the target problem scenario to which the user's question belongs are obtained. The target virtual table indicates the first metadata of the first physical table under the target problem scenario. Based on the problem content information and the target virtual table, a large model is used to generate an intermediate query language for querying data from the target virtual table under constraint information. The intermediate query language is parsed to obtain target information. Based on the target information, a second physical table is determined from the first physical table. Based on the second physical table and the target information, an executable query language is generated. The query result is obtained by executing the executable query language. A display operation is performed based on the query result. Therefore, by parsing the obtained result fields and condition columns, the specific physical tables that the intermediate query language given by the large model depends on are identified. Then, based on the determined physical tables, result fields, and condition columns, an executable query language is generated, ensuring that the generated executable query language can accurately match the user's question and optimize query performance.

[0091] Figure 4This is a flowchart illustrating the data processing method provided according to the fourth embodiment of this disclosure.

[0092] like Figure 4 As shown, the data processing method includes:

[0093] Step 401: Obtain the problem content information of the user's problem and the target virtual table under the target problem scenario to which the user's problem belongs.

[0094] Step 402: Based on the problem content information and the target virtual table, use the large model to generate an intermediate query language for querying data from the target virtual table under the constraints of the constraint information.

[0095] Step 403: Parse the intermediate query language to obtain the target information.

[0096] The target information includes result fields and condition columns.

[0097] Step 404: Determine the second physical table from the first physical table based on the target information.

[0098] Step 405: Display target information.

[0099] To ensure the accuracy of the generated executable query language, in this embodiment, the user is asked to confirm the target information before generating the executable query language.

[0100] One possible approach is to display the target information in natural language.

[0101] Step 406: In response to receiving a confirmation operation for the target information, generate an executable query language based on the second physical table and the target information.

[0102] In this embodiment, after receiving a confirmation operation for the target information, an executable query language can be generated based on the second physical table and the target information.

[0103] To reduce user interaction and improve user experience, as an example, a specified display duration can be set. When the target information is displayed for a duration exceeding the specified duration, it is assumed that the user has confirmed the information. The specified display duration can be any short duration, such as 3 seconds or 5 seconds.

[0104] It should be noted that, when displaying target information to the user for confirmation, in order to facilitate the user's timely correction of erroneous target information, in one possible implementation of this disclosure, a first index value corresponding to the target information is displayed; in response to receiving a modification operation on the target information, the target information is updated based on a second index value specified from the first index value. Thus, by allowing the user to modify the target information based on the displayed index value when confirming an error, the user can further narrow down the problem through correction, improving the accuracy of the query results obtained by subsequently executing the executable query language.

[0105] As one possible implementation, the first index value is obtained based on the index column configuration of the target problem scenario. The index column configuration is used for at least one of the following: configuring the index of the dimension field based on the enumerated value of the dimension field in the target virtual table; configuring the index of the indicator field based on the indicator type of the indicator field in the target virtual table. Thus, by limiting the index value corresponding to the target information to be obtained based on the field in the virtual table corresponding to the target problem scenario, the effectiveness of the index value corresponding to the target information provided to the user for selection is improved.

[0106] As an example, a dimension field refers to a field whose enumeration value is at least one character, such as the "Company" field. The enumeration value of this field is a string, and let's assume the enumeration values ​​for the "Company" field are "Company 1", "Company 2", and "Company 3". If the user's question is "What was the order quantity of product xx in the third quarter of year xx for Company 1?", then normally, one of the target information obtained by parsing the intermediate query language generated by the large model would be "Company: Company 1". If the intermediate query language obtained by parsing the large model is "Company: Company 2", then the user can click on the area containing "Company 2", causing the interface to display a dropdown list corresponding to "Company 2". The contents of this dropdown list are the first index values ​​corresponding to "Company 2" ("Company 1", "Company 3"). Thus, the user can select "Company 1" to replace "Company 2".

[0107] As an example, a metric field refers to a field whose enumerated values ​​are specific numbers, such as operating revenue. The enumerated values ​​of this field are different numbers and have no specific practical meaning. In this case, it's not necessary to use the enumerated values ​​of this field as an index. Instead, indexes should be created for different types of metric fields, such as order quantity, product quantity, and order date. If the user's question is "What was the order quantity of product xx in the third quarter of 201xx?", then normally, one of the target information obtained from parsing the intermediate query language generated by the large model would be "metric: order quantity". Assuming the intermediate query language generated by parsing the large model yields "metric: product quantity", the user can click on the area containing "product quantity" to display a dropdown list corresponding to "product quantity". The contents of this dropdown list are the first index value corresponding to "product quantity" ("order quantity", "order date", etc.). Thus, the user can select "order quantity" to replace "product quantity".

[0108] Understandably, when a user modifies the target information, it indicates that the output of the large model is not as expected. One possible approach is to generate training samples based on the second index value, the intermediate query language, and the prompt information; these training samples are then used to train the large model. Thus, by using unexpected problem cases encountered in real-world applications as training samples to train the large model, the model's accuracy in analyzing the corresponding problem domain is improved, the accuracy of the intermediate query language output by the large model is enhanced, and the user experience is further improved.

[0109] Step 407: Obtain the query results by executing the executable query language.

[0110] Step 408: Perform the display operation based on the query results.

[0111] It should be noted that the explanations of steps 401, 402, 403, 404, 407 and 408 can be found in the relevant descriptions in any embodiment of this disclosure, and will not be repeated here.

[0112] In this embodiment, the process involves obtaining the problem content information of a user's question and a target virtual table within the target problem scenario to which the user's question belongs. The target virtual table indicates the first metadata of a first physical table within the target problem scenario. Based on the problem content information and the target virtual table, an intermediate query language is generated using a large model under constraints to retrieve data from the target virtual table. The intermediate query language is parsed to obtain target information. Based on the target information, a second physical table is determined from the first physical table. The target information is then displayed. In response to receiving a confirmation operation for the target information, an executable query language is generated based on the second physical table and the target information. The executable query language is executed to obtain query results. Finally, a display operation is performed based on the query results. Therefore, by parsing the intermediate query language and presenting the parsed target information in natural language to allow the user to confirm their intent, further convergence of the user's question can be achieved.

[0113] Figure 5 This is a flowchart illustrating the data processing method provided according to the fifth embodiment of this disclosure.

[0114] like Figure 5 As shown, the data processing method includes:

[0115] Step 501: Format the user's question to obtain the question content information.

[0116] Formatting refers to replacing jargon in user questions to obtain more natural and standardized question content information.

[0117] For explanations of "user issues" and "user content information," please refer to the relevant descriptions in any embodiment of this disclosure, which will not be repeated here.

[0118] Step 502: Based on the user's question, use a pre-prediction model to determine the target problem scenario and the target virtual table.

[0119] The target virtual table is used to indicate the first metadata of the first physical table under the target problem scenario. Explanations of "user problem," "problem scenario," "target virtual table," "target problem scenario," "first physical table," and "first metadata" can be found in the relevant descriptions in any embodiment of this disclosure, and will not be repeated here.

[0120] In this embodiment, a pre-prediction model can be used to determine the target problem scenario to which the user's problem belongs, as well as the target virtual table under the target problem scenario, based on the user's problem.

[0121] It should be noted that in one possible implementation of this disclosure, the user problem is a problem input under a problem scenario selected by the user, which leads to the question: whether the problem scenario selected by the user is consistent with the target problem scenario determined by the pre-prediction model.

[0122] The problem scenario selected by the user may or may not match the target problem scenario determined by the pre-prediction model. One possible approach is to obtain constraint information and execute subsequent processes if the target problem scenario matches the selected problem scenario.

[0123] When the target problem scenario differs from the selected problem scenario, the process terminates and stops executing. The situations where the target problem scenario differs from the selected problem scenario include the following two scenarios:

[0124] 1. If the target problem scenario determined by the pre-prediction model is incorrect, the pre-prediction model needs to be retrained.

[0125] Second, if the user selects the wrong question scenario, such as selecting the human resources scenario but asking a question related to the finance scenario, the user can be prompted to ask the question in the correct question scenario.

[0126] Therefore, by comparing the target problem scenario determined by the pre-prediction model with the problem scenario selected by the user, the accuracy of the target problem scenario determined by the pre-prediction model can be verified. Then, based on the accurate target problem scenario and the virtual table corresponding to the accurate target problem scenario, the subsequent process is executed, thereby improving the accuracy of the output results of the subsequent process.

[0127] Step 503: Based on the problem content information and the target virtual table, use the large model to generate an intermediate query language for querying data from the target virtual table under the constraints of the constraint information.

[0128] Step 504: Parse the intermediate query language to obtain the target information.

[0129] Step 505: Generate an executable query language based on the target information to query data from the first physical table.

[0130] Step 506: Obtain the query results by executing the executable query language.

[0131] Step 507: Perform the display operation based on the query results.

[0132] It should be noted that the explanations of steps 503, 504, 505, 506 and 507 can be found in the relevant descriptions in any embodiment of this disclosure, and will not be repeated here.

[0133] In this embodiment, user questions are formatted to obtain question content information. Based on the user question, a pre-prediction model is used to determine the target problem scenario and the target virtual table. The target virtual table indicates the first metadata of the first physical table under the target problem scenario. Based on the question content information and the target virtual table, a large model is used to generate an intermediate query language for querying data from the target virtual table under constraint information. The intermediate query language is parsed to obtain target information. An executable query language for querying data from the first physical table is generated based on the target information. The query result is obtained by executing the executable query language. A display operation is performed based on the query result. Therefore, through formatting, jargon in user questions can be replaced, making the question content information more natural and standardized. Furthermore, by using a pre-prediction model to determine the target problem scenario to which the user question belongs and the corresponding virtual table, problem processing efficiency can be improved.

[0134] Figure 6 This is a schematic flowchart of the data processing method provided according to the sixth embodiment of this disclosure.

[0135] like Figure 6 As shown, the data processing method includes:

[0136] Step 601: Obtain the problem content information of the user's problem and the target virtual table under the target problem scenario to which the user's problem belongs.

[0137] Step 602: Based on the problem content information and the target virtual table, use the large model to generate an intermediate query language for querying data from the target virtual table under the constraints of the constraint information.

[0138] Step 603: Parse the intermediate query language to obtain the target information.

[0139] Step 604: Generate an executable query language for querying data from the first physical table based on the target information.

[0140] Step 605: Obtain the query results by executing the executable query language.

[0141] Step 606: Determine the target display strategy based on the data characteristics of the query results.

[0142] Among them, the target display strategy is used to indicate the data visualization method of query results.

[0143] Data visualization methods include, but are not limited to: bar charts, pie charts, scatter plots, line charts, histograms, and tables.

[0144] As an example, if the query result is a two-dimensional dataset (each data point includes two values ​​(x and y)) and only one dimension needs to be compared, then the target display strategy can be determined to be to display the query result using a bar chart.

[0145] Step 607: Display at least one of the following: query results and summary content of query results. The query results are displayed according to the target display strategy, and the summary content is obtained by summarizing the query results using a large model.

[0146] In this embodiment, the query results can be displayed according to the target display strategy, and / or, a summary of the query results can be displayed.

[0147] It should be noted that the explanations of steps 601, 602, 603, 604 and 605 can be found in the relevant descriptions in any embodiment of this disclosure, and will not be repeated here.

[0148] In this embodiment, the process involves obtaining the problem content information of a user's question and a target virtual table within the target problem scenario to which the user's question belongs. The target virtual table indicates the first metadata of the first physical table within the target problem scenario. Based on the problem content information and the target virtual table, a large model is used to generate an intermediate query language for retrieving data from the target virtual table under constraint information. The intermediate query language is parsed to obtain target information. An executable query language is generated based on the target information to retrieve data from the first physical table. The query results are obtained by executing the executable query language. A target display strategy is determined based on the data characteristics of the query results. At least one of the following is displayed according to the target display strategy: the query results and a summary of the query results. Thus, by selecting an appropriate display strategy based on the data characteristics of the query results and displaying a summary of the query results, the combination of text and graphics enhances the user experience.

[0149] Figure 7 This is a flowchart illustrating the data processing method provided according to the seventh embodiment of this disclosure.

[0150] like Figure 7 As shown, the data processing method includes:

[0151] Step 701: Obtain the data source configuration for the target problem scenario.

[0152] The data source configuration is used to configure the data source connection information for the target problem scenario and for at least one of the following table entries: the first physical table and the permission table; wherein the permission table is used to generate an executable query language with permission control.

[0153] Step 702: Obtain the second element information of the table entry based on the data source connection information.

[0154] The second element information includes the table name, table usage instructions, field names, field meanings, and a limited set of dimension enumeration values.

[0155] In this embodiment, since the table entry includes at least one of the following: a first physical table and a permission table, the second metadata of the table entry obtained based on the data source connection information also includes at least one of the following: the second metadata of the first physical table and the second metadata of the permission table.

[0156] Step 703: In response to the table entry including the first physical table and receiving a configuration operation for the second meta information, obtain the virtual table configuration for the target problem scenario.

[0157] Step 704: Generate the target virtual table according to the virtual table configuration.

[0158] Step 705: In response to the table entries including the first physical table and the permission table, and receiving the configuration operation for the second meta information, obtain the data permission configuration for the target problem scenario.

[0159] The data permission configuration is used to indicate the relationship between the permission table and the first physical table. In other words, the permission table and the physical table can be associated through the data permission configuration.

[0160] Step 706: Obtain the problem content information of the user's problem and the target virtual table under the target problem scenario to which the user's problem belongs.

[0161] Step 707: Based on the problem content information and the target virtual table, use the large model to generate an intermediate query language for querying data from the target virtual table under the constraints of the constraint information.

[0162] Step 708: Parse the intermediate query language to obtain the target information.

[0163] Step 709: Generate an executable query language based on the target information to query data from the first physical table.

[0164] Step 710: Obtain the query results by executing the executable query language.

[0165] Step 711: Perform the display operation based on the query results.

[0166] It should be noted that the explanations of steps 706, 707, 708, 709, 710 and 711 can be found in the relevant descriptions in any embodiment of this disclosure, and will not be repeated here.

[0167] In this embodiment of the disclosure, a data source configuration for a target problem scenario is obtained; wherein, the data source configuration is used to configure data source connection information for at least one of the following table entries under the target problem scenario: a first physical table and a permission table; wherein, the permission table is used to generate an executable query language with permission control; second metadata of the table entries is obtained according to the data source connection information; in response to the table entry including the first physical table and receiving a configuration operation for the second metadata, a virtual table configuration for the target problem scenario is obtained; a target virtual table is generated according to the virtual table configuration; in response to the table entry including the first physical table and the permission table and receiving a configuration operation for the second metadata, data permission configuration for the target problem scenario is obtained; wherein The data permission configuration is used to indicate the relationship between the permission table and the first physical table; the problem content information of the user's question and the target virtual table under the target problem scenario to which the user's question belongs are obtained; the target virtual table is used to indicate the first meta-information of the first physical table under the target problem scenario; based on the problem content information and the target virtual table, an intermediate query language for querying data from the target virtual table is generated using a large model under the constraint information; the intermediate query language is parsed to obtain the target information; an executable query language for querying data from the first physical table is generated based on the target information; the query results are obtained by executing the executable query language; and the display operation is performed based on the query results. Therefore, by configuring the data source based on the target problem scenario, the real-time performance of the data can be improved by obtaining the data from the physical tables and / or permission tables on which the target problem scenario depends. By operating on the metadata of the physical tables on which the target problem scenario depends, the virtual table configuration of the target problem scenario can be obtained, and a virtual table corresponding to the target problem scenario can be generated, ensuring that one problem scenario corresponds to one virtual table. The virtual table is obtained by configuring the metadata of the physical tables on which the problem scenario depends. Through data permission configuration, the permission table can be associated with the physical table, so as to generate an executable query language with permission control.

[0168] To clearly illustrate the above embodiments, examples will now be provided.

[0169] Figure 8 This is a schematic diagram of the structure of a conversational data analysis system provided according to the eighth embodiment of this disclosure.

[0170] In related technologies, conversational data analysis systems have the following problems:

[0171] First, conversational data analytics systems often require targeted data development to meet diverse data needs. Furthermore, once developed, the corresponding tasks typically involve long-term scheduling and continuous iteration based on the required data cycle, resulting in high development and maintenance costs.

[0172] Second, conversational data analysis systems are generally highly customized, and without iterative expansion, they are difficult to adapt to the complex and ever-changing analytical needs of businesses, lacking universality.

[0173] The conversational data analysis system provided in this embodiment organically combines models, data, and engineering. Users can ask data-related questions to the system using natural language. The model analyzes the data based on the user's questions, and the engineering team confirms the user's intent based on the model's analysis results and presents the analysis results to the user. This entire process avoids redundant data development and improves the universality of data analysis.

[0174] like Figure 8 As shown, this conversational data analysis system performs the following five steps:

[0175] In the first step, query recognition and formatting are performed.

[0176] First, the user asks the system a question in natural language (i.e., a user question). Figure 8 The query in the query, such as "What is the order quantity of xx product in Q3 2023 for xx company?", will first undergo intent recognition. Intent recognition refers to using a pre-prediction model to determine the target question scenario to which the user question belongs and the target virtual table under the target question scenario. Query formatting refers to recognizing the jargon in the user question and replacing the recognized jargon, such as replacing Q3 with the third quarter, to obtain the content of the user question.

[0177] In the second step, dynamic prompt generation is implemented.

[0178] Secondly, after determining the target problem scenario, the target virtual table within the target problem scenario, and the user's question content, a corresponding prompt can be dynamically generated. This prompt includes: question content information, the target virtual table within the target problem scenario, and constraint information. The constraint information is used to constrain the output results of the large model. Since the large model only faces a single table (the target virtual table), the difficulty of understanding the data for the large model can be significantly reduced, thereby improving the accuracy of the large model's output results (the accuracy of the large model generating results when facing a single table is higher than the accuracy when facing multiple tables).

[0179] Figure 8 In this context, data weaving semantics refers to a single virtual table that is only oriented towards the large model. Different problem scenarios correspond to different virtual tables, which can be generated on the basis of the business data warehouse through the meta service.

[0180] In the third stage, the intermediate state query language is reverse-solved and the intent is confirmed.

[0181] The prompt is then input into the large model after SFT (Simplified Chinese Query Theory), resulting in an intermediate query language generated by the large model for retrieving data from the target virtual table. This achieves the generation of an intermediate query language from the target virtual table based on the problem content and the target virtual table, under constraints. Furthermore, this process can perform reasoning from natural language to intermediate query language using the given prompt, converting the user's natural language question into an intermediate query language based on data-woven semantics (target virtual table). The system then parses this intermediate query language to obtain the target information and presents it to the user in natural language to confirm their intent, further narrowing down the user's question. Simultaneously, based on the target information, the physical tables used are determined, and based on the physical tables used and the target information, an executable query language with access control and supporting multi-table joins is generated.

[0182] During the intent confirmation process, the project will index the target information in advance using ElasticSearch (ES) to obtain the first index value corresponding to the target information. This index value is used to recommend similar values ​​during confirmation. For example, if the model identifies "order quantity" as "product quantity" in the user's question, "order quantity" can be recommended through similar values, so that the user can correct their intent and achieve the goal of question convergence.

[0183] The fourth step involves presenting and summarizing the results.

[0184] The executable query language obtains the final query result set through the business data warehouse. The engineering team recommends charts based on the characteristics of the result set and requests the large model to summarize conclusions by combining user questions and results.

[0185] In the fifth stage, the SFT data flywheel is performed.

[0186] It is understandable that when a user modifies the target information, it means that the output of the large model does not meet expectations. Therefore, in the SFT stage, unexpected problems that occur in actual applications can be used as seed problems to generalize and expand the training set, thereby strengthening the large model's accurate analysis capabilities in this field and improving the user experience.

[0187] In the third stage, the intermediate query language is further enhanced into a physically executable query language. The core process of this can be found in [link to relevant documentation]. Figure 3 , Figure 3The system implements dynamic materialization of data weaving semantics (target virtual table), which means dynamically enhancing virtual queries into physically executable queries. First, the main model generates an intermediate query language corresponding to the user's question based on the single virtual table given in the prompt (the main model generates the intermediate query language based on the target virtual table). This language is a single-table query language oriented towards the virtual table. Next, the system parses the intermediate query language to obtain the corresponding set of output fields (result fields) and the set of columns in the query conditions (condition columns). These two sets are used to identify which physical tables the intermediate query language given by the main model specifically depends on. For example, the result field set identifies a dependency on physical table 1, and the condition column set identifies a dependency on physical table 3. Figure 3 The system connects two physical tables (corresponding to two physical tables connected by a solid line), and then dynamically constructs a multi-table join query language (an executable query language based on multiple physical tables) using the unique keys of the two tables. Finally, the system dynamically adds the join strategy of the permission table according to the identity of the accessing user to generate a final-state query language that carries permissions and can be physically executed (an executable query language with permission control).

[0188] As one possible implementation, this conversational data analytics system also supports user-initiated access scenarios; the overall process can be found in [reference needed]. Figure 9 The flowchart shown is for the scenario integration management process. Figure 9 The system can maintain corresponding metadata through scenario access for subsequent processes such as dynamic prompts and Elasticsearch indexing strategies. Specifically: Data source configuration: Configures the data source connection information for the business, used by the system to retrieve specific data based on the connection information. Virtual wide table model configuration: Configures the metadata of the data weaving semantics (virtual table) corresponding to the scenario, including table usage instructions, field names, field meanings, and field descriptions, used for dynamically generating prompts. Index column attribute configuration: For the fields of the aforementioned wide table, they are generally divided into two categories: dimensions and metrics. For dimension fields, an indexing strategy can be set to support Elasticsearch in identifying which column enumeration values ​​need to be indexed, thus more accurately recommending similar values ​​during the user intent confirmation stage. For metric fields, similar dimension metric fields can be used as their respective indexes to recommend similar values ​​to the user during the user intent confirmation stage.

[0189] As an example, the results of this conversational data analytics system are displayed as follows: Figure 10 As shown. Figure 10A user asks the system a question using natural language: "What were the order quantities for YY products from xx company in each quarter of 2023?" The system model responds, including clarifications of the user's intent in natural language, query results, and a summary. The system model's response process is as follows: First, the system model clarifies the user's intent in natural language (referring to...) Figure 10 (The content following "Model identifies your query:") If the clarified intent's key dimensions or metrics do not meet the user's expectations, the user can initiate intent modification via "Adjust Dimensions". Figure 10 The text below "Model recognizes your query:" contains bolded sections with a background shadow (e.g., "1" Query data metrics: followed by "Date" or "Order Quantity"; "2" Query conditions: followed by "XX", "YY", "2023", etc.). Users can select recommended similar values ​​from a dropdown list to make corrections. If the intent information clarified by the system model to the user is correct (assuming the system model's clarification of intent is correct if the user does not perform any modification operation within the set time), the query process will be automatically triggered, and the system model will display the query results to the user. Figure 10 The query results are displayed as a histogram, and the system model will then provide a textual summary of the results.

[0190] In summary, the conversational data analysis system provided in this embodiment has the following beneficial effects:

[0191] I. Reduce data development and maintenance costs: By linking large models and data engineering to build a conversational data analysis system, repetitive data development patterns and cumbersome data architectures are avoided.

[0192] II. Enhancing the Flexibility and Versatility of Data Analysis: The conversational data analysis system provided in this embodiment cleverly implements an interactive question-and-answer analysis process based on a large model, which can flexibly respond to complex and ever-changing business analysis needs and has stronger data analysis universality.

[0193] Figure 11 This is a schematic diagram of the structure of a data processing apparatus provided according to the eleventh embodiment of this disclosure.

[0194] like Figure 11 As shown, the data processing apparatus includes:

[0195] The first acquisition module 1101 is used to acquire the problem content information of the user's problem and the target virtual table under the target problem scenario to which the user's problem belongs; wherein, the target virtual table is used to indicate the first meta information of the first physical table under the target problem scenario;

[0196] The first generation module 1102 is used to generate an intermediate query language for querying data from the target virtual table based on the problem content information and the target virtual table, using a large model under the constraint information.

[0197] Parsing module 1103 is used to parse the intermediate query language to obtain the target information;

[0198] The second generation module 1104 is used to generate an executable query language for querying data from the first physical table based on the target information.

[0199] The first processing module 1105 is used to obtain query results by executing an executable query language;

[0200] The second processing module 1106 is used to perform display operations based on the query results.

[0201] As one possible implementation of this disclosure, the target information includes a result field and a condition column; the second generation module 1104 includes: a determining unit, configured to determine a second physical table from a first physical table based on the target information; and a generation unit, configured to generate an executable query language based on the second physical table and the target information.

[0202] As one possible implementation of this disclosure, the generation unit is further configured to: display target information; and in response to receiving a confirmation operation for the target information, generate an executable query language based on the second physical table and the target information.

[0203] As one possible implementation of this disclosure, the apparatus further includes: a display module for displaying a first index value corresponding to the target information; and an update module for updating the target information based on a second index value specified from the first index value in response to receiving a modification operation on the target information.

[0204] As one possible implementation of this disclosure, the first index value is obtained based on the index column configuration of the target problem scenario; the index column configuration is used for at least one of the following: configuring the index of the dimension field based on the enumeration value of the dimension field in the target virtual table; configuring the index of the indicator field based on the indicator type of the indicator field in the target virtual table.

[0205] As one possible implementation of this disclosure, the apparatus further includes: a third generation module, configured to generate training samples based on a second index value, an intermediate query language, and prompt information; wherein the training samples are used to train a large model.

[0206] As one possible implementation of this disclosure, the generation unit is further configured to: generate candidate query languages ​​based on the second physical table and the target information; and update the candidate query languages ​​based on the user identifier corresponding to the user question and the permission table to obtain an executable query language with permission control.

[0207] As one possible implementation of this disclosure, the first acquisition module is further configured to: format the user question to obtain question content information; and determine the target question scenario and target virtual table based on the user question using a pre-prediction model.

[0208] As one possible implementation of this disclosure, the user question is a question entered in a problem scenario selected by the user; the above-mentioned device further includes: a second acquisition module, used to acquire constraint information when the target problem scenario is consistent with the selected problem scenario.

[0209] As one possible implementation of this disclosure, the second processing module is further configured to: determine a target display strategy based on the data characteristics of the query results; and display at least one of the following: query results and summary content of the query results, wherein the query results are displayed according to the target display strategy, and the summary content is obtained by summarizing the query results using a large model.

[0210] As a possible implementation of this disclosure, the apparatus further includes: a third acquisition module, configured to acquire data source configuration for the target problem scenario; wherein the data source configuration is configured to configure data source connection information for at least one of the following table entries under the target problem scenario: a first physical table and a permission table; wherein the permission table is used to generate an executable query language with permission control; and a fourth acquisition module, configured to acquire second metadata of the table entries based on the data source connection information.

[0211] As a possible implementation of this disclosure, the apparatus further includes: a fifth acquisition module, configured to acquire a virtual table configuration for a target problem scenario in response to a table entry including a first physical table and receiving a configuration operation for second metadata; and a fourth generation module, configured to generate a target virtual table based on the virtual table configuration.

[0212] As a possible implementation of this disclosure, the apparatus further includes: a sixth acquisition module, configured to acquire data permission configuration for a target problem scenario in response to a table entry including a first physical table and a permission table, and receiving a configuration operation for second metadata; wherein the data permission configuration is used to indicate the association relationship between the permission table and the first physical table.

[0213] It should be noted that the foregoing explanation of the data processing method also applies to the data processing device of this embodiment, and will not be repeated here.

[0214] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0215] Figure 12 A schematic block diagram of an example electronic device 1200 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0216] like Figure 12 As shown, device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes based on a computer program stored in ROM (Read-Only Memory) 1202 or loaded from storage unit 1208 into RAM (Random Access Memory) 1203. RAM 1203 may also store various programs and data required for the operation of device 1200. The computing unit 1201, ROM 1202, and RAM 1203 are interconnected via bus 1204. I / O (Input / Output) interface 1205 is also connected to bus 1204.

[0217] Multiple components in device 1200 are connected to I / O interface 1205, including: input unit 1206, such as keyboard, mouse, etc.; output unit 1207, such as various types of monitors, speakers, etc.; storage unit 1208, such as disk, optical disk, etc.; and communication unit 1209, such as network card, modem, wireless transceiver, etc. Communication unit 1209 allows device 1200 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0218] The computing unit 1201 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by the computing unit 1201, one or more steps of the data processing methods described above may be performed. Alternatively, in other embodiments, the computing unit 1201 may be configured to perform a data processing method by any other suitable means (e.g., by means of firmware).

[0219] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0220] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0221] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0222] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0223] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, and blockchain networks.

[0224] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0225] It's important to note that artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0226] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0227] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A data processing method, comprising: Obtain the problem content information of the user's question and the target virtual table under the target problem scenario to which the user's question belongs; wherein, the target virtual table is used to indicate the first meta-information of the first physical table under the target problem scenario; Based on the problem content information and the target virtual table, an intermediate query language for querying data from the target virtual table is generated using a large model under the constraints of the constraint information. Parse the intermediate query language to obtain the target information; Generate an executable query language for querying data from the first physical table based on the target information; The query results are obtained by executing the executable query language. The display operation will be performed based on the query results.

2. The method according to claim 1, wherein, The target information includes result fields and condition columns; The step of generating an executable query language for querying data from the first physical table based on the target information includes: Based on the target information, a second physical table is determined from the first physical table; The executable query language is generated based on the second physical table and the target information.

3. The method according to claim 2, wherein, The step of generating the executable query language based on the second physical table and the target information includes: Display the target information; In response to receiving a confirmation operation for the target information, the executable query language is generated based on the second physical table and the target information.

4. The method according to claim 3, wherein, The method further includes: Display the first index value corresponding to the target information; In response to receiving a modification operation for the target information, the target information is updated based on a second index value specified from the first index value.

5. The method according to claim 4, wherein, The first index value is obtained based on the index column configuration of the target problem scenario; The index column configuration is used for at least one of the following: configuring the index of the dimension field based on the enumerated value of the dimension field in the target virtual table; configuring the index of the indicator field based on the indicator type of the indicator field in the target virtual table.

6. The method according to claim 4, wherein, The method further includes: Training samples are generated based on the second index value, the intermediate query language, and the prompt information; wherein the training samples are used to train the large model.

7. The method according to claim 2, wherein, The step of generating the executable query language based on the second physical table and the target information includes: Based on the second physical table and the target information, a candidate query language is generated; Based on the user identifier corresponding to the user question and the permission table, update the candidate query language to obtain the executable query language with permission control.

8. The method according to claim 1, wherein, The process of obtaining the problem content information of the user's question and the target virtual table under the target problem scenario to which the user's question belongs includes: The user's question is formatted to obtain the question content information; Based on the user's question, a pre-prediction model is used to determine the target problem scenario and the target virtual table.

9. The method according to claim 8, wherein, The user question is a question entered within the problem scenario selected by the user. The method further includes: If the target problem scenario matches the selected problem scenario, the constraint information is obtained.

10. The method according to claim 1, wherein, The step of performing the display operation based on the query results includes: Based on the data characteristics of the query results, determine the target display strategy; Display at least one of the following: the query results, and a summary of the query results, wherein the query results are displayed according to the target display strategy, and the summary is obtained by summarizing the query results using the large model.

11. The method according to claim 1, wherein, The method further includes: Obtain the data source configuration for the target problem scenario; wherein, the data source configuration is used to configure data source connection information for the target problem scenario and for at least one of the following table entries: the first physical table and the permission table; wherein, the permission table is used to generate the executable query language with permission control; The second element information of the table entry is obtained based on the data source connection information.

12. The method according to claim 11, wherein, The method further includes: In response to the table entry including the first physical table and receiving a configuration operation for the second metadata, the virtual table configuration of the target problem scenario is obtained; The target virtual table is generated based on the virtual table configuration.

13. The method according to claim 11, wherein, The method further includes: In response to the table entry including the first physical table and the permission table, and receiving a configuration operation for the second metadata, the data permission configuration for the target problem scenario is obtained; wherein, the data permission configuration is used to indicate the association relationship between the permission table and the first physical table.

14. A data processing apparatus, comprising: The first acquisition module is used to acquire the problem content information of the user's problem and the target virtual table under the target problem scenario to which the user's problem belongs; wherein, the target virtual table is used to indicate the first meta-information of the first physical table under the target problem scenario; The first generation module is used to generate an intermediate query language for querying data from the target virtual table based on the problem content information and the target virtual table, using a large model under the constraint information. The parsing module is used to parse the intermediate query language to obtain the target information; The second generation module is used to generate an executable query language for querying data from the first physical table based on the target information; The first processing module is used to obtain query results by executing the executable query language; The second processing module is used to perform display operations based on the query results.

15. The apparatus according to claim 14, wherein, The target information includes result fields and condition columns; The second generation module includes: A determining unit is configured to determine a second physical table from the first physical table based on the target information; The generation unit is used to generate the executable query language based on the second physical table and the target information.

16. The apparatus according to claim 15, wherein, The generation unit is further configured to: Display the target information; In response to receiving a confirmation operation for the target information, the executable query language is generated based on the second physical table and the target information.

17. The apparatus according to claim 16, wherein, The device further includes: The display module is used to display the first index value corresponding to the target information; An update module is configured to, in response to receiving a modification operation on the target information, update the target information based on a second index value specified from the first index value.

18. The apparatus according to claim 17, wherein, The first index value is obtained based on the index column configuration of the target problem scenario; The index column configuration is used for at least one of the following: configuring the index of the dimension field based on the enumerated value of the dimension field in the target virtual table; configuring the index of the indicator field based on the indicator type of the indicator field in the target virtual table.

19. The apparatus according to claim 17, wherein, The device further includes: The third generation module is used to generate training samples based on the second index value, the intermediate query language, and the prompt information; wherein the training samples are used to train the large model.

20. The apparatus according to claim 15, wherein, The generation unit is further configured to: Based on the second physical table and the target information, a candidate query language is generated; Based on the user identifier corresponding to the user question and the permission table, update the candidate query language to obtain the executable query language with permission control.

21. The apparatus according to claim 14, wherein, The first acquisition module is further configured to: The user's question is formatted to obtain the question content information; Based on the user's question, a pre-prediction model is used to determine the target problem scenario and the target virtual table.

22. The apparatus according to claim 21, wherein, The user question is a question entered within the problem scenario selected by the user. The device further includes: The second acquisition module is used to acquire the constraint information when the target problem scenario is consistent with the selected problem scenario.

23. The apparatus according to claim 14, wherein, The second processing module is further configured to: Based on the data characteristics of the query results, determine the target display strategy; Display at least one of the following: the query results, and a summary of the query results, wherein the query results are displayed according to the target display strategy, and the summary is obtained by summarizing the query results using the large model.

24. The apparatus according to claim 14, wherein, The device further includes: The third acquisition module is used to acquire the data source configuration of the target problem scenario; wherein, the data source configuration is used to configure the data source connection information for the target problem scenario and for at least one of the following table entries: the first physical table and the permission table; wherein, the permission table is used to generate the executable query language with permission control; The fourth acquisition module is used to acquire the second element information of the table entry based on the data source connection information.

25. The apparatus according to claim 24, wherein, The device further includes: The fifth acquisition module is used to acquire the virtual table configuration of the target problem scenario in response to the table entry including the first physical table and receiving a configuration operation for the second meta information; The fourth generation module is used to generate the target virtual table according to the virtual table configuration.

26. The apparatus according to claim 24, wherein, The device further includes: The sixth acquisition module is used to acquire the data permission configuration of the target problem scenario in response to the table entry including the first physical table and the permission table, and receiving a configuration operation for the second metadata; wherein the data permission configuration is used to indicate the association relationship between the permission table and the first physical table.

27. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-13.

28. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-13.

29. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-13.

Images