Method and device for determining training data of large language model and medium

Abstract

The invention discloses a method and device for determining training data of a large language model and a medium, and relates to the technical field of large language models.The method comprises the steps that data features of each data table in a plurality of data tables under a plurality of preset feature types are determined; determining a target data table from the data tables based on the data features of the data tables; based on the data features of the target data table, determining candidate answers obtained by performing data analysis on the target data table; generating a target question corresponding to the candidate answer based on the candidate answer and a large language model; based on the target question, the candidate answer and the target data table, training data is generated, the training data is used for conducting model training on an analysis large model, and the analysis large model is used for determining the answer of the input question based on the input question of the user and the data table provided by the user. Therefore, corresponding effective answers inevitably exist after the questions in the training data are analyzed, and the accuracy and effectiveness of training based on the training data are ensured.

Description

Technical Field

[0001] This disclosure relates to the field of large language model technology, and more specifically, to a method, apparatus, and medium for determining training data for large language models. Background Technology

[0002] Complex data analysis is a data analysis capability within the field of artificial intelligence. Its complexity lies not only in requiring systems to perform simple data extraction but also in demanding capabilities similar to human data analysts, such as anomaly detection and analytical decision-making. Therefore, training large models with these complex data analysis capabilities requires constructing training data that meets these requirements. Related technologies typically employ techniques such as prompt engineering (PE) and model fine-tuning to allow large language models to simulate datasets. However, datasets simulated through this process may contain data that does not reflect real-world scenarios (e.g., synthetic anomalies differ significantly from real-world anomalies), leading to excessively large training errors when training models based on questions and answers built on this dataset. Summary of the Invention

[0003] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] In a first aspect, this disclosure provides a method for determining training data for a large language model, the method comprising: Determine the data characteristics of each of the multiple data tables under multiple preset feature types; Based on the data characteristics of the data table, determine the target data table from the data table; Based on the data characteristics of the target data table, candidate answers obtained by data analysis of the target data table are determined; Based on the candidate answers and the large language model, the target question corresponding to the candidate answers is generated; Training data is generated based on the target question, the candidate answers, and the target data table. The training data is used to train the large-scale analysis model, which is used to determine the answer to the input question based on the user's input question and the data table provided by the user.

[0005] Secondly, this disclosure provides an apparatus for determining training data for a large language model, the apparatus comprising: The first determining module is used to determine the data characteristics of each of the multiple data tables under multiple preset feature types; The second determining module is used to determine the target data table from the data table based on the data characteristics of the data table; The third determining module is used to determine candidate answers obtained by data analysis of the target data table based on the data characteristics of the target data table; The first generation module is used to generate the target question corresponding to the candidate answer based on the candidate answer and the large language model; The second generation module is used to generate training data based on the target question, the candidate answers, and the target data table. The training data is used to train the large analysis model, and the large analysis model is used to determine the answer to the input question based on the user's input question and the data table provided by the user.

[0006] Thirdly, this disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect.

[0007] Fourthly, this disclosure provides an electronic device, comprising: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method described in the first aspect.

[0008] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0009] The above technical solution allows for the selection of target data tables for training data generation from data tables generated in actual business operations. This improves the matching degree between the generated training data dataset and the actual business scenario, thereby ensuring the accuracy and efficiency of training large-scale analytical models. Furthermore, in this disclosed solution, when determining training data, candidate answers are first analyzed from the target data table, and then target questions are determined based on these candidate answers to obtain the training data. This ensures that after analyzing the questions in the training data during model training, corresponding valid answers will inevitably exist. This avoids the situation in related technologies where question-answer pairs directly generated from datasets often fail to yield valid answers, guaranteeing the accuracy and effectiveness of training based on the training data. This further improves the accuracy of model training based on this training data and enhances the accuracy of complex data analysis.

[0010] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0011] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings: Figure 1 This is a flowchart of a method for determining training data for a large language model according to one embodiment of the present disclosure; Figure 2 This is a schematic diagram illustrating feature extraction from a data table according to one embodiment of the present disclosure; Figure 3 This is a block diagram of an apparatus for determining training data for a large language model according to one embodiment of the present disclosure; Figure 4 A schematic diagram of the structure of an electronic device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0012] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0013] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0014] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0015] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0016] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0017] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0018] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0019] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0020] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0021] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0022] Meanwhile, it is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0023] In the field of large-scale model technology, to improve the analytical capabilities of large-scale models in complex data analysis scenarios, it is necessary to pre-construct questions and answers capable of performing complex data analysis for training the large-scale model. In related technologies, questions and answers can be set through manual annotation; however, this method requires a large amount of manual work and is time-consuming. Alternatively, prompt word engineering can be used to allow the large-scale model to automatically generate datasets and construct questions and answers based on these datasets. However, this method may result in invalid questions and answers due to discrepancies between the constructed dataset and the actual data, and there may also be cases where the generated questions do not have valid answers after analysis of the dataset, leading to poor training performance of the large-scale model. Therefore, this disclosure provides the following embodiments to generate training data for large-scale models to perform complex data analysis.

[0024] Figure 1 The diagram shows a flowchart of a method for determining training data for a large language model according to one embodiment of this disclosure. Figure 1 As shown, the method may include: In step 11, the data characteristics of each data table in the multiple data tables are determined under multiple preset feature types.

[0025] In this context, multiple feature types that need to be extracted can be pre-set based on actual application scenarios. For example, the preset feature types may include, but are not limited to, multiple types such as business type, row and column type, time series type, dimension type, and indicator type. Among them, the data features of the business type can be used to represent the characteristics of the business to which the data in the data table belongs; the data features of the row and column type can be used to represent the row information and column information in the data table; the data features of the time series type can be used to represent the time series field of the corresponding time in the data table and the features of roll-up or drill-down based on the time series field; the data features of the dimension type can be used to represent the field features in the data table used for data analysis; and the data features of the indicator type can represent the field features in the data table used for numerical calculation.

[0026] In this embodiment, the multiple data tables can be actual data tables generated in actual business scenarios, which can be obtained after being confirmed and used by the corresponding users. Therefore, the training data in this disclosure can be determined based on the actual data tables, thereby ensuring the authenticity and usability of the subsequently generated training data.

[0027] In step 12, the target data table is determined from the data table based on the data characteristics of the data table.

[0028] Generally speaking, the more data features a data table has, the higher the complexity of the analysis based on that data table. In this case, the greater the possibility of generating questions and answers for complex data analysis based on that data table, the more likely it is that the data table can be used to generate training data. Therefore, in this step, the complexity of data analysis based on the data table can be ranked according to the data features of the data table, and the target data table for generating training data can be selected from the ranked data.

[0029] In step 13, based on the data characteristics of the target data table, candidate answers obtained by data analysis of the target data table are determined.

[0030] This step involves performing data analysis on the target data table based on its data characteristics to identify feature points discovered through complex data analysis. For example, these feature points may include at least one of the following: outlier data in the target data table, correlated indicators, and multi-dimensional features associated with the outlier data. Accordingly, the feature points obtained from the analysis can be used as candidate answers, which can then be used as answers in question-answer pairs within the training data.

[0031] In step 14, the target question corresponding to the candidate answer is generated based on the candidate answer and the large language model.

[0032] This step involves constructing prompt text based on candidate answers and prompt words used for question generation. The prompt words can include rules for data analysis in real-world business scenarios. The prompt text is then input into a large language model, which generates a target question corresponding to the candidate answer, thus improving the matching degree between the candidate answer and the target question.

[0033] In step 15, training data is generated based on the target question, candidate answers, and target data table. The training data is used to train the large analysis model, which is used to determine the answer to the input question based on the user's input question and the data table provided by the user. The large analysis model can be a large language model for performing complex data analysis.

[0034] Specifically, question-answer pairs can be generated based on candidate answers and the target question, such as by directly combining the target question and candidate answers to obtain question-answer pairs. Training data can then be determined based on the question-answer pairs and the target data table. The large-scale analysis model can then be trained using the training data determined in this disclosure.

[0035] Therefore, through the above technical solution, the target data table for generating training data can be selected from the data tables generated in actual business operations. This improves the matching degree between the dataset used for generating training data and the actual business scenario, thereby ensuring the accuracy and efficiency of model training for large analytical models. Furthermore, in the solution disclosed herein, when determining training data, candidate answers are first analyzed from the target data table, and then target questions are determined based on these candidate answers to obtain training data. This ensures that after analyzing the questions in the training data during model training, there will inevitably be corresponding valid answers. This avoids the situation in related technologies where question-answer pairs directly generated from datasets are unlikely to yield valid answers, ensuring the accuracy and effectiveness of training based on the training data, further improving the accuracy of model training based on this training data, and enhancing the accuracy of complex data analysis.

[0036] In some possible embodiments, before determining the data features of each of the multiple data tables under preset feature types, the data tables can be preprocessed, such as determining whether the units of the indicators in the data tables are consistent, or whether the data formats are consistent. For example, missing values ​​in the data tables can be identified. Data tables with missing values ​​can be ignored, or missing values ​​can be supplemented through interpolation. Then, data features are extracted based on the supplemented data tables.

[0037] In some possible embodiments, determining the data characteristics of each of the multiple data tables under a preset plurality of feature types may include: For each of the data tables, input information is determined based on the data table and the prompt words of the large language model, wherein the prompt words contain rules for feature extraction under the preset feature type.

[0038] The input information is input into the large language model to obtain the data features of the data table.

[0039] Specifically, different feature types and rules for feature extraction can be predefined in the prompt words. This allows the large language model to understand and extract features based on these rules, thereby obtaining the data features of the data table under multiple feature types. This enables automatic analysis and extraction of data table features. Furthermore, by setting constraints on the feature types in the prompt words, multiple data features of the data table can be extracted, improving the comprehensiveness of the data table analysis. This provides data support for evaluating the value of the data table during analysis and increases the likelihood of performing complex data analysis on the target data table subsequently identified.

[0040] In some possible embodiments, determining the target data table from the data table based on the data characteristics of the data table may include: For each of the data tables, a feature score is determined based on the data characteristics of the data table.

[0041] As an example, for each data feature's feature type, a mapping relationship between the data features and feature scores under that feature type can be pre-set. Accordingly, the score of each data feature can be determined based on the data features and the mapping relationship.

[0042] For example, regarding business type, if the identified data feature is a major category, the feature score is N1; if the data feature is a minor category, the feature score is N2. Regarding row and column type, if the identified number of rows and columns falls within the first range, the feature score is N3; if it falls within the second range, the feature score is N4; and if it falls within the third range, the feature score is N5. Regarding time series type, if the number of time series fields in the identified data feature falls within the fourth range, the feature score is N6; and if it falls within the fifth range, the feature score is N7. For dimension types, if the number of independent fields in the determined data features falls within the sixth range, the feature score is N8; if it falls within the seventh range, the feature score is N9; and if the number of fields that can be rolled up and drilled down falls within the eighth range, the feature score is N10. Independent dimensions represent dimensions without a conversion relationship between them. For example, dates and years have a conversion relationship, so they are not independent dimensions. Fields that can be rolled up and drilled down are those with hierarchical relationships, such as fields corresponding to provinces, cities, and districts, or fields used to classify product categories. For indicator types, if the determined data features include indicators with formulaic relationships, the feature score is N11; if they include indicators with behavioral chain relationships, the feature score is N12. Behavioral chain relationships represent a progressive relationship between the values ​​of multiple fields. For example, in product promotion and purchase scenarios, the following fields corresponding to a product—display, view, follow, add to cart, place order, etc.—have a progressive relationship, forming a behavioral chain relationship. The specific settings for the first to eighth ranges and the feature scores N1 to N12 mentioned in the above example can be set based on actual application scenarios, and this disclosure does not impose any limitations on them. As an example, the above rules can be directly set in a large language model, so that the large language model outputs the feature scores of the data table while outputting the data features, such as... Figure 2 As shown.

[0043] As an example, knowledge base information can be added to the prompts of the large language model. For instance, the knowledge base information could include behavioral chain relationships within the industry, enabling the large language model to accurately and effectively identify the indicator features in the data table based on the knowledge base information. Alternatively, industry classification system information can be added to the prompts of the large language model. This industry classification system information can be uploaded and provided by users, allowing the large language model to effectively extract industry features based on it, thereby improving the accuracy of the data features in the data table.

[0044] The weights corresponding to different feature types can be preset and adjusted based on the actual application scenario. After determining the score of each data feature in the data table, the scores of each data feature in the data table can be weighted and summed to obtain the feature score of the data table.

[0045] Data tables whose feature scores exceed the score threshold are identified as candidate data tables.

[0046] The score threshold can be a pre-set value. When the feature score exceeds the score threshold, it can be considered that the data table is more likely to be used for complex data analysis. Question-answer pairs can then be set based on the data table to determine the training data for analyzing the training process of large models, thereby improving the ability of large models to perform complex data analysis.

[0047] If the candidate data table contains a related field, then the related data table corresponding to the related field is obtained, and the data table obtained by associating the candidate data table and the related data table is determined as the target data table; If the candidate data table does not contain a related field, then the candidate data table shall be used as the target data table.

[0048] In practical applications, different data tables may be used for different purposes. For example, a user table records user information, an order table records order information, and a logistics table records logistics information. These different data tables are linked through association fields; for instance, the user table and the order table are linked by user ID, and the order table and the logistics table are linked by order ID. The associated data tables corresponding to the association fields include data tables directly associated with the association fields and data tables indirectly associated with the association fields. For example, data tables containing the association fields can be added to a data table set. For each data table in the data table set, the association fields for that data table are determined, and data tables containing the association fields are added to the data table set until no new data tables are added. The data tables in the data table set are then considered as the associated data tables corresponding to the association fields. For example, taking the user table as an example, the order table is a directly associated data table, and the logistics table is an indirectly associated data table.

[0049] To increase the complexity of data analysis, multiple data tables can be analyzed as a whole. Furthermore, candidate data tables and related data tables can be associated to obtain a wide table, which can then be used as the target data table.

[0050] Therefore, through the above technical solution, data tables can be filtered based on their data characteristics, thereby determining whether a dataset supports complex data analysis. This identifies datasets suitable for complex data analysis. Furthermore, by associating multiple data tables, a target data table is obtained, further increasing the complexity of the data in the target data table. This ensures that the generated training data contains problems suitable for complex data analysis, thereby improving the training efficiency of large-scale analytical models for complex data analysis.

[0051] As an example, before determining the data characteristics of each of the multiple data tables under several preset feature types, the following steps can be performed: For each data table, if the data table contains a related field, obtain the related data table corresponding to the related field, and determine the data table obtained by associating the data table and the related data table as a new data table. Then, the step of determining the data characteristics of each of the multiple data tables under several preset feature types is performed, that is, first merging the data tables and then extracting the data characteristics from the data tables. The method of merging the data tables has been detailed above and will not be repeated here.

[0052] Accordingly, when generating training data based on the target question, the candidate answers, and the target data table, the question-answer pairs generated from the target question and the candidate answers, as well as the original data table used to construct the target data table, can be used as training data.

[0053] In some possible embodiments, to further enhance the diversity of complex data analyses that can be performed on the target data table, predictive generation can be performed on the data in the data table to provide dimensional diversity. For example, the feature type includes indicator features, where the relationship between different fields in the indicator features is a formulaic relationship. For instance, the target data table contains the indicator feature "sales amount" and the formula relationship "gross profit margin = (gross profit / sales amount) * 100%". Therefore, the fields in the target data table can be expanded based on existing data in the target data table.

[0054] Accordingly, before the step of determining candidate answers obtained by data analysis of the target data table based on the data characteristics of the target data table, the method may further include: Based on the fields in the target data table and the relationship of the formula, determine the prediction fields that need to be used for data generation.

[0055] In this case, fields in the formula relationship that do not belong to the target data table can be identified as prediction fields that need to be generated. Continuing with the example above, the prediction fields can be sales gross profit and gross profit margin.

[0056] Based on the data in the target data table, generate the field values ​​for the prediction fields; The predicted field values ​​are added to the target data table to obtain the updated target data table.

[0057] In this step, data from the target data table is input into the large language model to obtain the values ​​of some predicted fields. Then, by combining these predicted field values ​​with the data and formula relationships in the target data table, the values ​​of the remaining predicted fields are determined. For example, the large language model can be used to predict the value of gross profit. After obtaining the predicted gross profit value, the value of the gross profit is determined by combining the sales amount data and formula relationships in the target data table. The determined predicted field values ​​are then added to the target data table, further increasing the number of fields in the target data table. This increases the number of fields that need to be analyzed when performing data analysis based on the target data table, thereby increasing the complexity of data analysis based on the question-and-answer pairs generated from the target data table. This enhances the ability of the trained analytical model to process complex data.

[0058] In some possible embodiments, the data features include dimensional features and indicator features. Accordingly, determining candidate answers obtained through data analysis of the target data table based on its data features may include: For each indicator in the indicator features, identify the abnormal data of the indicator in each dimension of the dimension features.

[0059] For example, each indicator can be analyzed to identify those that are abnormal. This identification can be based on common anomaly detection methods in the field, such as statistical methods that consider data exceeding the mean ± 3σ range as abnormal based on the normal distribution assumption.

[0060] For example, for a target data table, the indicator features include 5 indicators A1-A5, and the dimension features include 10 dimensions B1-B10. Then it can be determined whether indicator A1 has anomalies in dimensions B1-B10, whether indicator A2 has anomalies in dimensions B1-B10, and so on, to achieve anomaly detection for each indicator in each dimension.

[0061] If the dimension corresponding to the abnormal data belongs to a multi-level dimension, then the candidate answer is determined based on each dimension in the multi-level dimension and the abnormal data. The multiple dimensions in the multi-level dimension are associated with each other based on roll-up and drill-down. The candidate answer includes the dimension in the multi-level dimension where there is abnormal data and the existing abnormal data.

[0062] For example, after the above steps, it is determined that indicator A1 has abnormal data under dimension B1. Dimension B1 is a multi-level dimension, such as dimension B1 being a district, dimension B2 being a city, and dimension B3 being a province. Then B1, B2, and B3 form a multi-level dimension.

[0063] As an example, if the dimension corresponding to the abnormal data is a multi-level dimension, and there is corresponding abnormal data in at least one up-dimension corresponding to the dimension, then the dimension in the multi-level dimension that contains abnormal data and the existing abnormal data are taken as candidate answers.

[0064] For example, if the identified outlier data corresponds to dimension B1, then we need to determine whether there is corresponding outlier data in its superordinate dimension B2. If there is, we can continue to determine whether there is corresponding outlier data in its superordinate dimension B3. If there is also corresponding outlier data in dimension B3, then the outlier data of indicator A1 in dimensions B1, B2 and B3 are taken as candidate answers.

[0065] As another example, for multi-level dimensions, abnormal data analysis can begin from the lowest-level dimension. If the identified abnormal data has corresponding abnormal data in N consecutively scrolled-up dimensions, candidate answers can be generated based on the dimensions containing the abnormal data and the existing abnormal data. Here, N can be determined based on the actual application scenario. For example, if the identified abnormal data corresponds to dimension B1, then it is determined whether there is corresponding abnormal data in its scrolled-up dimension B2. If there is no corresponding abnormal data in dimension B2, then the abnormal data is considered to have no analytical value and can be ignored; other abnormal data can then be detected.

[0066] Therefore, through the above technical solution, comprehensive detection of abnormal data in the target data table can be achieved by traversing and detecting indicators and dimensions, thereby improving the accuracy and effectiveness of the identified candidate answers. Furthermore, by determining the existence of abnormal data through hierarchical roll-up within multi-level dimensions, multi-level analysis of abnormal data is achieved, increasing the analytical level required to obtain answers and identifying candidate answers that require drill-down analysis. This improves the adaptability of the obtained training data to complex data analysis application scenarios.

[0067] As an example, in this scenario, when generating the target question corresponding to the candidate answer based on the candidate answer and the large language model, the target question can be generated based on the abnormal data of the uppermost convolution dimension of the candidate answer and the large language model. This allows the large analysis model to perform drill-down analysis based on the target question during training, enabling the large analysis model to perform complex analysis and increasing the complexity of the training data.

[0068] As an example, prior to the step of generating training data based on the target question, the candidate answers, and the target data table, the method may further include: The path corresponding to the candidate answer and the target question are input into the large language model to obtain the target answer. The path is a path formed based on the multi-level dimensions corresponding to the candidate answer, determined by drilling down from the top level. The path corresponding to the candidate answer is a path formed by drilling down from the top layer to determine the multi-level dimensions corresponding to the candidate answer. Taking the above example, the path corresponding to the candidate answer is the analysis path B3-B3-B1, which is the path of analysis from the provincial dimension, city dimension, and district dimension. It is the process of analysis from coarse-grained to fine-grained based on the target question and data table.

[0069] In this embodiment, to improve the fluency of the natural language descriptions of the answers in the training data, candidate answers can be regenerated based on a large language model. As an example, prompts for re-describing the answer can be pre-set. This step combines the path corresponding to the candidate answer and the target question with the prompts to obtain the input text, enabling the large language model to generate a target answer that more closely matches the natural language description of the target question. The target answer and the candidate answer are the same answer with different descriptions.

[0070] Furthermore, generating training data based on the target question, the candidate answers, and the target data table can include: generating training data based on the target question, the target answers, and the target data table. The implementation method has been described above.

[0071] Therefore, through the above technical solution, for candidate answers that can be drilled down, the target answer corresponding to the target question can be obtained based on the path corresponding to the candidate answer, thereby realizing the re-description of the answer in the question and answer, improving the matching degree between the target question and the target answer, and improving the accuracy and effectiveness of the training data obtained subsequently.

[0072] In some possible embodiments, the data features include dimensional features and indicator features. Accordingly, determining candidate answers obtained through data analysis of the target data table based on its data features may include: For each indicator in the aforementioned indicator features, abnormal data for that indicator are identified in each dimension of the aforementioned dimensional features. The implementation of this step has been detailed above and will not be repeated here.

[0073] If the abnormal data in multiple dimensions is the same abnormal data, then the candidate answer is determined based on the dimension corresponding to the abnormal data and the abnormal data.

[0074] In practical applications, there may be multiple dimensions that cause data anomalies. This disclosure provides the following embodiments for multi-dimensional analysis scenarios in complex data analysis. For a given indicator, the abnormal dimensions corresponding to the abnormal data of that indicator can be obtained. Furthermore, it can be determined whether the abnormal data under the abnormal dimensions are the same abnormal data. For example, taking indicator A1 as an example, indicator A1 has abnormal data D1 under dimension B4, abnormal data D2 under dimension B5, and abnormal data D1 under dimension B6. In this example, it can be determined that abnormal data D1 is abnormal under both dimension B4 and dimension B6, and the abnormal data and the corresponding dimensions can be used as candidate answers.

[0075] As another example, the dimensions corresponding to the abnormal data, the abnormal data, and the target question can be input into a large language model to obtain new candidate answers, further improving the descriptive completeness and accuracy of the candidate answers.

[0076] Therefore, the above technical solution can analyze the target data table from multiple dimensions that may lead to data anomalies, thereby achieving multi-dimensional analysis of the target data table. This allows for the identification of candidate answers from multi-dimensional analysis scenarios, and further, the identification of question-answer pairs in multi-dimensional analysis scenarios, thereby improving the diversity and richness of the identified training data.

[0077] In some possible embodiments, the data features include dimensional features and indicator features. Accordingly, determining candidate answers obtained through data analysis of the target data table based on its data features may include: Based on the indicator characteristics of the target data table, a candidate indicator group is determined in the target data table, wherein the candidate indicator group contains any two indicators from the indicator characteristics.

[0078] As an example, any two indicators in the indicator features can be combined to obtain a candidate indicator group. For example, combining indicators A1-A5 can yield 10 candidate indicator groups.

[0079] Determine the conditional dimension corresponding to each candidate indicator group, wherein the conditional dimension includes at least one dimension among the dimensional features.

[0080] For example, the candidate indicator group includes indicators A1 and A2. The maximum number of conditional dimensions for data analysis can be preset; for example, a maximum of 2 can be set, allowing the selection of one or two dimensions from the dimensional features as conditional dimensions. Continuing the example above, if there are 10 dimensions in the dimensional features, and one conditional dimension is used, there can be 10 types of conditional dimensions; if two conditional dimensions are used, there can be 45 combinations of conditional dimensions. In some possible embodiments, multiple levels within a multi-level dimension are related, allowing the selection of conditional dimensions from dimensions that are not part of the multi-level dimension.

[0081] If two indicators in the candidate indicator group are correlated under the target value of the condition dimension, then the candidate answer is determined based on the candidate indicator group and the target value of the condition dimension.

[0082] In this embodiment, for the candidate indicator group and the condition dimension, different values ​​for the condition dimension can be determined, and for each value of the condition dimension, it can be determined whether the two indicators in the candidate indicator group are correlated. The correlation between different indicators can be determined using correlation testing methods in the art, such as calculating the correlation coefficient between two indicators, or verifying correlation through a chi-square test. Thus, the correlation of different indicators under certain conditions can be determined through a comprehensive test. The target value of the condition dimension can represent the precondition that the two indicators are correlated. In this embodiment, the candidate indicator group and the target value of the condition dimension can be used as candidate answers.

[0083] Therefore, through the above technical solution, the correlation test of different indicators under different values ​​of dimensions can be used to determine whether different indicators are correlated under certain conditions. This improves the analytical level of obtaining answers based on the target data table, making the obtained answers more suitable for complex data analysis application scenarios. It also helps to build effective and accurate training data for complex data analysis scenarios of large models without the need for manual annotation and settings, thus saving manual workload.

[0084] Based on the technical solution provided in this disclosure, candidate answers can be determined by combining one or more of the above-mentioned methods for determining candidate answers, thereby determining training data. This allows for the determination of question-answer pairs that can be generated based on the target data table from multiple perspectives, while improving the diversity of data analysis.

[0085] As an example, when determining candidate answers, the corresponding knowledge points can be identified based on the type of the candidate answer and preset knowledge point matching rules. These knowledge points are then used as the knowledge points for the training data generated from the candidate answer. The training data under each knowledge point is then categorized and stored according to these knowledge points, improving the management and configuration of the training data. For instance, during the training of a large analytical model, training can be performed based on training data under one or more user-specified knowledge points. This improves the accuracy of the large analytical model's data analysis under that type of knowledge point, allows for fine-tuning of the large analytical model, expands the application scenarios of the large analytical model trained on the training data constructed according to this disclosure, and enhances the user experience.

[0086] Based on the same inventive concept, this disclosure also provides a device for determining training data for large language models, such as... Figure 3 As shown, the determining device 10 includes: The first determining module 100 is used to determine the data characteristics of each of the multiple data tables under a preset set of multiple feature types; The second determining module 200 is used to determine a target data table from the data table based on the data characteristics of the data table; The third determining module 300 is used to determine candidate answers obtained by data analysis of the target data table based on the data characteristics of the target data table; The first generation module 400 is used to generate the target question corresponding to the candidate answer based on the candidate answer and the large language model; The second generation module 500 is used to generate training data based on the target question, the candidate answers, and the target data table. The training data is used to train the large analysis model, and the large analysis model is used to determine the answer to the input question based on the user's input question and the data table provided by the user.

[0087] Optionally, the second determining module 200 includes: The first determining submodule is used to determine the feature score of each data table based on the data characteristics of the data table. The second determining submodule is used to determine the data table whose feature scores exceed the score threshold as a candidate data table; The first processing submodule is used to obtain the associated data table corresponding to the associated field if the candidate data table contains an associated field, and determine the data table obtained by associating the candidate data table and the associated data table as the target data table. The third determining submodule is used to take the candidate data table as the target data table if the candidate data table does not contain an associated field.

[0088] Optionally, the first determining module 100 includes: The fourth determining submodule is used to determine input information for each data table based on the data table and the prompt words of the large language model, wherein the prompt words contain rules for feature extraction under the preset feature type; The second processing submodule is used to input the input information into the large language model to obtain the data features of the data table.

[0089] Optionally, the feature type includes indicator features, and the relationship between different fields in the indicator features is a formulaic relationship; Before the step of determining candidate answers obtained by data analysis of the target data table based on the data characteristics of the target data table, the determining device 10 further includes: The fourth determining module is used to determine the prediction fields that need to be generated based on the fields in the target data table and the formula relationship before the third determining module 300 determines the candidate answers obtained by data analysis of the target data table based on the data characteristics of the target data table. The third generation submodule is used to generate the field value of the prediction field based on the data in the target data table; The first processing module is used to add the field value of the predicted field to the target data table to obtain the updated target data table.

[0090] Optionally, the data features include dimensional features and indicator features; The third determining module 300 includes: The fifth determination submodule is used to determine the abnormal data of each indicator in each dimension of the dimensional features for each indicator in the indicator features. The sixth determining submodule is used to determine the candidate answer based on each dimension in the multi-level dimension and the abnormal data if the dimension corresponding to the abnormal data belongs to a multi-level dimension. The multiple dimensions in the multi-level dimension are associated based on roll-up and drill-down. The candidate answer includes the dimension in the multi-level dimension where abnormal data exists and the existing abnormal data.

[0091] Optionally, the determining device 10 further includes: The second processing module is used to input the path corresponding to the candidate answer and the target question into the large language model before the second generation module 500 generates training data based on the target question, the candidate answer and the target data table, to obtain the target answer. The path is a path formed based on the multi-level dimensions corresponding to the candidate answer, determined by drilling down from the top layer. The second generation module 500 includes: The training data is generated based on the target question, the target answer, and the target data table.

[0092] Optionally, the data features include dimensional features and indicator features; The second generation module 500 includes: The fifth determination submodule is used to determine the abnormal data of each indicator in each dimension of the dimensional features for each indicator in the indicator features. The seventh determination submodule is used to determine the candidate answer based on the dimension corresponding to the abnormal data and the abnormal data if the abnormal data under multiple dimensions are the same abnormal data.

[0093] Optionally, the data features include dimensional features and indicator features; The second generation module 500 includes: The eighth determining submodule is used to determine a candidate indicator group in the target data table based on the indicator characteristics of the target data table, wherein the candidate indicator group contains any two indicators from the indicator characteristics; The ninth determining submodule is used to determine the condition dimension corresponding to each candidate index group, wherein the condition dimension includes at least one dimension among the dimension features; The tenth determination submodule is used to determine the candidate answer based on the candidate indicator group and the target value of the condition dimension if two indicators in the candidate indicator group are correlated under the target value of the condition dimension.

[0094] This disclosure also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the above-described method.

[0095] This disclosure also provides an electronic device, including: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the above method.

[0096] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0097] The following is for reference. Figure 4 The diagram illustrates a structural schematic of an electronic device (e.g., a terminal device or a server) 600 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0098] like Figure 4 As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0099] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0100] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the methods of embodiments of this disclosure.

[0101] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0102] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0103] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0104] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: determine data features of each of the multiple data tables under preset multiple feature types; determine a target data table from the data tables based on the data features of the data tables; determine candidate answers obtained by data analysis of the target data table based on the data features of the target data table; generate a target question corresponding to the candidate answer based on the candidate answer and a large language model; and generate training data based on the target question, the candidate answer, and the target data table, wherein the training data is used to train the large analysis model, and the large analysis model is used to determine the answer to the input question based on the user's input question and the data table provided by the user.

[0105] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0106] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0107] The modules described in the embodiments of this disclosure can be implemented in software or in hardware. The names of the modules do not necessarily limit the module itself; for example, the first determining module can also be described as "a module for determining the data characteristics of each of the multiple data tables under preset multiple feature types".

[0108] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0109] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0110] According to one or more embodiments of this disclosure, Example 1 provides a method for determining training data for a large language model, the method comprising: Determine the data characteristics of each of the multiple data tables under multiple preset feature types; Based on the data characteristics of the data table, determine the target data table from the data table; Based on the data characteristics of the target data table, candidate answers obtained by data analysis of the target data table are determined; Based on the candidate answers and the large language model, the target question corresponding to the candidate answers is generated; Training data is generated based on the target question, the candidate answers, and the target data table. The training data is used to train the large-scale analysis model, which is used to determine the answer to the input question based on the user's input question and the data table provided by the user.

[0111] According to one or more embodiments of this disclosure, Example 2 provides the method of Example 1, wherein determining a target data table from the data table based on the data characteristics of the data table includes: For each of the data tables, a feature score is determined based on the data characteristics of the data table. Data tables whose feature scores exceed the score threshold are identified as candidate data tables; If the candidate data table contains a related field, then the related data table corresponding to the related field is obtained, and the data table obtained by associating the candidate data table and the related data table is determined as the target data table; If the candidate data table does not contain a related field, then the candidate data table shall be used as the target data table.

[0112] According to one or more embodiments of this disclosure, Example 3 provides the method of Example 1, wherein determining the data characteristics of each of the multiple data tables under a preset plurality of feature types includes: For each of the data tables, input information is determined based on the data table and the prompt words of the large language model, wherein the prompt words contain rules for feature extraction under the preset feature type; The input information is input into the large language model to obtain the data features of the data table.

[0113] According to one or more embodiments of this disclosure, Example 4 provides the method of Example 1, wherein the feature type includes indicator features, and the relationship between different fields in the indicator features is a formula relationship; Before the step of determining candidate answers obtained by data analysis of the target data table based on the data characteristics of the target data table, the method further includes: Based on the fields in the target data table and the relationship of the formula, determine the prediction fields that need to be used for data generation; Based on the data in the target data table, generate the field values ​​for the prediction fields; The predicted field values ​​are added to the target data table to obtain the updated target data table.

[0114] According to one or more embodiments of this disclosure, Example 5 provides the method of Example 1, wherein the data features include dimensional features and indicator features; The process of determining candidate answers based on the data characteristics of the target data table, obtained through data analysis of the target data table, includes: For each indicator in the indicator features, identify the abnormal data of the indicator in each dimension of the dimensional features; If the dimension corresponding to the abnormal data belongs to a multi-level dimension, then the candidate answer is determined based on each dimension in the multi-level dimension and the abnormal data. The multiple dimensions in the multi-level dimension are associated with each other based on roll-up and drill-down. The candidate answer includes the dimension in the multi-level dimension where there is abnormal data and the existing abnormal data.

[0115] According to one or more embodiments of this disclosure, Example 6 provides the method of Example 5, which, prior to the step of generating training data based on the target question, the candidate answers, and the target data table, further includes: The path corresponding to the candidate answer and the target question are input into the large language model to obtain the target answer. The path is a path formed based on the multi-level dimensions corresponding to the candidate answer, determined by drilling down from the top level. The step of generating training data based on the target question, the candidate answers, and the target data table includes: The training data is generated based on the target question, the target answer, and the target data table.

[0116] According to one or more embodiments of this disclosure, Example 7 provides the method of Example 1, wherein the data features include dimensional features and indicator features; The process of determining candidate answers based on the data characteristics of the target data table, obtained through data analysis of the target data table, includes: For each indicator in the indicator features, identify the abnormal data of the indicator in each dimension of the dimensional features; If the abnormal data in multiple dimensions is the same abnormal data, then the candidate answer is determined based on the dimension corresponding to the abnormal data and the abnormal data.

[0117] According to one or more embodiments of this disclosure, Example 8 provides the method of Example 1, wherein the data features include dimensional features and indicator features; The process of determining candidate answers based on the data characteristics of the target data table, obtained through data analysis of the target data table, includes: Based on the indicator characteristics of the target data table, a candidate indicator group is determined in the target data table, wherein the candidate indicator group contains any two indicators from the indicator characteristics; Determine the conditional dimension corresponding to each candidate indicator group, wherein the conditional dimension includes at least one dimension among the dimensional features; If two indicators in the candidate indicator group are correlated under the target value of the condition dimension, then the candidate answer is determined based on the candidate indicator group and the target value of the condition dimension.

[0118] According to one or more embodiments of this disclosure, Example 9 provides an apparatus for determining training data for a large language model, the apparatus comprising: The first determining module is used to determine the data characteristics of each of the multiple data tables under multiple preset feature types; The second determining module is used to determine the target data table from the data table based on the data characteristics of the data table; The third determining module is used to determine candidate answers obtained by data analysis of the target data table based on the data characteristics of the target data table; The first generation module is used to generate the target question corresponding to the candidate answer based on the candidate answer and the large language model; The second generation module is used to generate training data based on the target question, the candidate answers, and the target data table. The training data is used to train the large analysis model, and the large analysis model is used to determine the answer to the input question based on the user's input question and the data table provided by the user.

[0119] According to one or more embodiments of the present disclosure, Example 10 provides a computer-readable medium having a computer program stored thereon that, when executed by a processing device, implements the steps of the method described in any one of Examples 1-8.

[0120] According to one or more embodiments of this disclosure, Example 11 provides an electronic device, including: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of any one of the methods in Examples 1-8.

[0121] According to one or more embodiments of the present disclosure, Example 12 provides a computer program product including a computer program that, when executed by a processor, implements the steps of the method described in any one of Examples 1-8.

[0122] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0123] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0124] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative forms of implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which the various modules perform their operations has been described in detail in the embodiments relating to the method, and will not be elaborated upon here.

Claims

1. A method for determining training data of a large language model, characterized in that, The method comprises: determining data features of each of a plurality of data tables under a plurality of preset feature types; determining a target data table from the data tables based on the data features of the data tables; determining a candidate answer obtained by performing data analysis on the target data table based on the data features of the target data table; generating a target question corresponding to the candidate answer based on the candidate answer and a large language model; generating training data based on the target question, the candidate answer, and the target data table, wherein the training data is used for model training of an analysis large model, and the analysis large model is used to determine an answer to an input question of a user based on the input question and a data table provided by the user.

2. The method of claim 1, wherein, The method comprises: for each of the data tables, determining a feature score of the data table based on the data features of the data table; determining a candidate data table whose feature score exceeds a score threshold; if the candidate data table contains an association field, obtaining an association data table corresponding to the association field, and determining a data table obtained by associating the candidate data table and the association data table as the target data table; if the candidate data table does not contain an association field, determining the candidate data table as the target data table.

3. The method of claim 1, wherein, The method comprises: for each of the data tables, determining input information based on the data table and a prompt word of a large language model, wherein the prompt word contains rules for feature extraction under the preset feature types; inputting the input information into the large language model to obtain the data features of the data table.

4. The method of claim 1, wherein, The feature types include index features, and the association relationship between different fields in the index features is a formula relationship; Before the step of determining a candidate answer obtained by performing data analysis on the target data table based on the data features of the target data table, the method further comprises: determining a predicted field that needs to be generated based on the fields in the target data table and the formula relationship; generating a field value of the predicted field based on the data in the target data table; adding the field value of the predicted field to the target data table to obtain an updated target data table.

5. The method of claim 1, wherein, The data features include dimension features and index features. The method comprises: for each of the index features, determining abnormal data of the index under each of the dimension features; if the dimension corresponding to the abnormal data belongs to a multi-level dimension, determining the candidate answer based on each of the dimensions in the multi-level dimension and the abnormal data, wherein the dimensions in the multi-level dimension are associated based on uprolling and down drilling, and the candidate answer contains the dimensions in the multi-level dimension that have abnormal data and the abnormal data.

6. The method of claim 5, wherein, Before the step of generating training data based on the target question, the candidate answer and the target data table, the method further comprises: inputting the path corresponding to the candidate answer and the target question into the large language model to obtain a target answer, the path being a path formed by dimensions determined based on the multi-level dimensions corresponding to the candidate answer and gradually drilling down from the top level; the step of generating training data based on the target question, the candidate answer and the target data table comprises: generating the training data based on the target question, the target answer and the target data table.

7. The method of claim 1, wherein, The data features include dimension features and index features. The step of determining a candidate answer obtained by data analysis on the target data table based on the data features of the target data table comprises: for each index in the index features, respectively determine the abnormal data of the index in each dimension of the dimension features; if the abnormal data in multiple dimensions is the same abnormal data, determine the candidate answer based on the dimension corresponding to the abnormal data and the abnormal data.

8. The method of claim 1, wherein, The data features include dimension features and index features. The step of determining a candidate answer obtained by data analysis on the target data table based on the data features of the target data table comprises: determining a candidate index group in the target data table based on the index features of the target data table, wherein the candidate index group contains any two indexes in the index features; determining a conditional dimension corresponding to each candidate index group, wherein the conditional dimension contains at least one dimension in the dimension features; if two indexes in the candidate index group have correlation under the target value of the conditional dimension, determine the candidate answer based on the candidate index group and the target value of the conditional dimension.

9. A determination apparatus for training data of a large language model, characterized in that, The determining device comprises: a first determining module configured to determine data features of each data table in a plurality of data tables under a plurality of preset feature types; a second determining module configured to determine a target data table from the data tables based on the data features of the data tables; a third determining module configured to determine a candidate answer obtained by data analysis on the target data table based on the data features of the target data table; a first generating module configured to generate a target question corresponding to the candidate answer based on the candidate answer and a large language model; a second generating module configured to generate training data based on the target question, the candidate answer and the target data table, wherein the training data is used for model training of an analysis large model, and the analysis large model is used for determining an answer to an input question of a user based on the input question of the user and a data table provided by the user.

10. A computer readable medium having stored thereon a computer program, characterized in that, The computer program is executed by a processing device to implement the steps of the method of any one of claims 1-8.

11. An electronic device, comprising: comprises: a storage device having a computer program stored thereon; a processing device configured to execute the computer program in the storage device to implement the steps of the method of any one of claims 1-8.

12. A computer program product comprising a computer program, characterized in that, The computer program is executed by a processor to implement the steps of the method of any one of claims 1-8.

Images