Rapid data visualization generation method and device based on large language model

Abstract

The embodiment of the invention provides a rapid data visualization generation method and device based on a large language model, and the method comprises the steps: receiving and analyzing a data analysis demand sent by a user, inputting a target business index and target business dimension information in the data analysis demand into a demand classification model, obtaining a type index and a measurement index output by the demand classification model; determining the hierarchical structure and layout of the report according to the logical relationship between the type indexes, and adding computational logic for the report according to the computational formula, aggregation mode and display format of the measurement indexes to generate a corresponding business report; according to the method, data visualization can be efficiently and accurately realized.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a method and device for rapid data visualization generation based on a large language model. Background Art

[0002] As an important part of data analysis and business intelligence, data visualization plays an increasingly critical role in today's data-driven business environment. It can transform complex data sets into intuitive and easy-to-understand graphical representations, helping decision makers quickly understand patterns, trends, and anomalies in the data. However, despite the significant progress made in data visualization technology in recent years, existing methods still have many limitations and challenges when faced with rapidly changing business needs and increasingly complex data structures.

[0003] First, traditional data visualization methods usually rely on professional data analysts or visualization experts. These professionals need to have a deep understanding of data structure and business requirements, and master complex visualization tools and techniques. Although this method can produce high-quality visualization results, it has obvious defects: it is time-consuming and labor-intensive, costly, and difficult to respond quickly to changing business needs. In today's fast-paced business environment, the efficiency of this traditional method can no longer meet the real-time decision-making needs of enterprises.

[0004] Secondly, there are many automated data visualization tools on the market that are designed to simplify the visualization process. These tools usually provide a series of predefined chart types and templates, and users only need to import data and select the required chart type. Although these tools increase the speed of generating visualizations, they often lack a deep understanding of business context. As a result, they may generate visualization results that are technically correct but difficult to understand or irrelevant in business terms. Such visualizations that are out of business context not only fail to provide effective support for decision-making, but may also lead to misunderstandings and wrong decisions.

[0005] In summary, although data visualization technology has made significant progress in the past few years, existing methods still have many limitations when facing the complex and changing data analysis needs of modern enterprises. Summary of the invention

[0006] In response to the problems in the prior art, the present application provides a method and device for rapid data visualization generation based on a large language model, which can realize data visualization efficiently and accurately.

[0007] In order to solve at least one of the above problems, the present application provides the following technical solutions:

[0008] In a first aspect, the present application provides a method for fast data visualization generation based on a large language model, comprising:

[0009] Obtaining original data features of user historical data analysis needs, performing feature analysis on the original data features, determining the indicator relevance, feature importance, and principal component contribution of the original data features, inputting the indicator relevance, feature importance, and principal component contribution into the input layer of a multi-layer perceptron model for model training, and obtaining a demand classification model;

[0010] Receive and analyze the data analysis requirements sent by the user, input the target business indicators and target business dimension information in the data analysis requirements into the demand classification model, and obtain the type indicators and measurement indicators output by the demand classification model;

[0011] According to the logical relationship between the types of indicators, the hierarchy and layout of the report are determined, and according to the calculation formula, aggregation method and display format of the measurement indicators, calculation logic is added to the report to generate the corresponding business report.

[0012] Furthermore, the feature analysis of the original data features to determine the indicator correlation, feature importance and principal component contribution of the original data features includes:

[0013] Determining the indicator correlation of each of the original data features according to the linear correlation degree between the original data features and the target business indicators;

[0014] According to the Gini index gain generated when the original data features are split at the preset decision tree nodes, the feature importance of each of the original data features is determined, and according to the eigenvalues ​​and eigenvectors of the covariance matrix corresponding to the original data features, the principal component contribution of each of the original data features is determined.

[0015] Furthermore, the indicator correlation, the feature importance and the principal component contribution are input into the input layer of a multi-layer perceptron model for model training to obtain a demand classification model, including:

[0016] Constructing a model input layer whose number of nodes matches the correlation of the indicator, the importance of the feature, and the number of features of the principal component contribution, constructing a model output layer whose number of nodes corresponds to the number of possible type indicators and measurement indicators, and constructing an initial classification model based on the model input layer, the model output layer, and the preset model hidden layer;

[0017] The indicator correlation, the feature importance and the principal component contribution are input into the model input layer of the multi-layer perceptron model and the model is trained using a small batch gradient descent method. The model parameters are updated using a back propagation algorithm to obtain a demand classification model.

[0018] Furthermore, the target business indicators and target business dimension information in the data analysis requirements are input into the requirements classification model to obtain the type indicators and measurement indicators output by the requirements classification model, including:

[0019] Perform semantic recognition on the data analysis requirements through a preset natural semantic recognition algorithm, determine the corresponding target business indicators and target business dimensions, and convert them into feature vectors;

[0020] The feature vector is input into the demand classification model after model training, so that the demand classification model performs forward propagation calculation, applies activation function in each hidden layer, and finally outputs type indicators and measurement indicators in the output layer.

[0021] Furthermore, determining the hierarchy and layout of the report according to the logical relationship between the type indicators includes:

[0022] Identify the hierarchical relationship, subordinate relationship and associated relationship between type indicators through a predefined business knowledge graph, and construct a relationship network based on the hierarchical relationship, the subordinate relationship and the associated relationship;

[0023] The highest level type indicator in the relationship network is used as the main dimension to determine the report hierarchy and structure.

[0024] Furthermore, adding calculation logic to the report according to the calculation formula, aggregation method and display format of the metric includes:

[0025] Determine the required original data items and intermediate calculation steps according to the calculation formula of the metric, determine the corresponding aggregation function and aggregation level according to the aggregation method of the metric, and determine the corresponding tree structure or directed graph structure according to the display format of the metric;

[0026] The calculation logic is added to the report according to the required original data items and intermediate calculation steps, the corresponding aggregation functions and aggregation levels, and the corresponding tree structure or directed graph structure.

[0027] Furthermore, the generating of the corresponding business report also includes:

[0028] Analyze the calculation formula and aggregation method of each metric, determine the corresponding data processing flow and extract the required raw data from the data source, and implement the calculation process of each metric according to the predefined calculation order, including intermediate calculation steps and final aggregation;

[0029] Process the calculation logic under different dimensional combinations and adopt parallel calculation or pre-calculation strategies, fill the calculation results into the report structure, and apply the preset data format and display style to generate the corresponding business report.

[0030] In a second aspect, the present application provides a fast data visualization generation device based on a large language model, comprising:

[0031] A model building module is used to obtain the original data features of the user's historical data analysis needs, perform feature analysis on the original data features, determine the indicator relevance, feature importance and principal component contribution of the original data features, input the indicator relevance, feature importance and principal component contribution into the input layer of the multi-layer perceptron model for model training, and obtain a demand classification model;

[0032] An indicator determination module is used to receive and analyze the data analysis requirements sent by the user, input the target business indicators and target business dimension information in the data analysis requirements into the demand classification model, and obtain the type indicators and measurement indicators output by the demand classification model;

[0033] The report generation module is used to determine the hierarchy and layout of the report according to the logical relationship between the types of indicators, add calculation logic to the report according to the calculation formula, aggregation method and display format of the measurement indicators, and generate corresponding business reports.

[0034] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for rapid data visualization generation based on a large language model when executing the program.

[0035] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for fast data visualization generation based on a large language model.

[0036] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the method for fast data visualization generation based on a large language model.

[0037] It can be seen from the above technical solution that the present application provides a method and device for rapid data visualization generation based on a large language model. By receiving and parsing the data analysis requirements sent by users, the target business indicators and target business dimension information in the data analysis requirements are input into the demand classification model, and the type indicators and measurement indicators output by the demand classification model are obtained; according to the logical relationship between the type indicators, the hierarchy and layout of the report are determined, and according to the calculation formula, aggregation method and display format of the measurement indicators, calculation logic is added to the report to generate corresponding business reports, thereby realizing data visualization efficiently and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 This is one of the flow charts of the method for rapid data visualization generation based on a large language model in an embodiment of the present application;

[0040] Figure 2 This is the second flow chart of the method for rapid data visualization generation based on a large language model in an embodiment of the present application;

[0041] Figure 3 This is a flowchart of a method for rapid data visualization generation based on a large language model in an embodiment of the present application;

[0042] Figure 4 This is a fourth flow chart of a method for rapid data visualization generation based on a large language model in an embodiment of the present application;

[0043] Figure 5 This is a fifth flow chart of a method for rapid data visualization generation based on a large language model in an embodiment of the present application;

[0044] Figure 6 This is a sixth flow chart of a method for rapid data visualization generation based on a large language model in an embodiment of the present application;

[0045] Figure 7 This is the seventh flow chart of the method for rapid data visualization generation based on a large language model in the embodiment of the present application;

[0046] Figure 8 A structural diagram of a fast data visualization generation device based on a large language model in an embodiment of the present application;

[0047] Fig. 9 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application.

[0048] Reference numerals:

[0049] Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0051] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0052] In view of the problems existing in the prior art, the present application provides a method and device for rapid data visualization generation based on a large language model, which receives and parses the data analysis requirements sent by users, inputs the target business indicators and target business dimension information in the data analysis requirements into the demand classification model, and obtains the type indicators and measurement indicators output by the demand classification model; determines the hierarchy and layout of the report based on the logical relationship between the type indicators, adds calculation logic to the report based on the calculation formula, aggregation method and display format of the measurement indicators, and generates corresponding business reports, thereby realizing data visualization efficiently and accurately.

[0053] In order to realize data visualization efficiently and accurately, the present application provides an embodiment of a method for generating fast data visualization based on a large language model, see Figure 1 The fast data visualization generation method based on the large language model specifically includes the following contents:

[0054] Step S101: obtaining original data features of the user's historical data analysis needs, performing feature analysis on the original data features, determining the indicator relevance, feature importance and principal component contribution of the original data features, inputting the indicator relevance, feature importance and principal component contribution into the input layer of a multi-layer perceptron model for model training, and obtaining a demand classification model;

[0055] Optionally, in this embodiment, step S101 is a key link in the method for rapid data visualization generation based on a large language model, mainly involving data feature analysis and model training. The technical process of this step is first to obtain the original data features of the user's historical data analysis needs. These features may include the frequency of user queries, the complexity of queries, the business areas involved, etc. The methods for obtaining these features may include log analysis, user behavior tracking, etc.

[0056] Next, these raw data features are subjected to in-depth feature analysis. This analysis process includes three main aspects: indicator correlation, feature importance, and principal component contribution. Indicator correlation is usually calculated using the Pearson correlation coefficient or the Spearman rank correlation coefficient, which reflects the strength of the linear relationship between each feature and the target business indicator. The calculation of feature importance may be determined using a decision tree model by calculating the Gini index gain generated by each feature when the node is split. The calculation of principal component contribution involves the principal component analysis (PCA) method, which determines the contribution of each feature to the principal component by calculating the eigenvalues ​​and eigenvectors of the covariance matrix.

[0057] These three analysis methods each have their own technical principles and advantages. Correlation analysis can quickly identify features that are strongly correlated with the target indicator, but may ignore nonlinear relationships. Feature importance analysis can capture the nonlinear relationship between features and targets, but may be affected by overfitting. Principal component analysis can effectively reduce the dimension of features and extract the most representative feature combination, but may reduce the interpretability of features.

[0058] The technical problem solved in this step is mainly how to extract the most valuable information from a large number of raw data features, and how to effectively organize this information to provide high-quality input for subsequent model training. By combining these three analysis methods, the system can comprehensively evaluate the importance of each feature, taking into account both linear relationships and nonlinear relationships, while also identifying potential feature combinations.

[0059] After the analysis is completed, the indicator correlation, feature importance, and principal component contribution are used as inputs and sent to the input layer of the multi-layer perceptron model for model training. The multi-layer perceptron is a feedforward neural network that can learn complex nonlinear relationships between features. The training process usually uses back propagation algorithms and stochastic gradient descent methods to optimize model parameters. Through training, the system eventually obtains a model that can automatically classify user needs based on data features.

[0060] The technical effect achieved by this step is significant. First, it achieves effective compression and information extraction of raw data features, greatly reducing the complexity of subsequent processing. Second, through the training of the multi-layer perceptron model, the system acquires the ability to automatically understand and classify user needs, which lays the foundation for subsequent automated report generation. Third, this method is highly adaptable and scalable. When faced with new business scenarios or new data features, it only needs to update the training data and retrain the model.

[0061] Step S102: receiving and parsing the data analysis requirements sent by the user, inputting the target business indicators and target business dimension information in the data analysis requirements into the demand classification model, and obtaining the type indicators and measurement indicators output by the demand classification model;

[0062] Optionally, in this embodiment, step S102 is a key link in the fast data visualization generation method based on a large language model, which mainly involves receiving, parsing and classifying user needs. The technical process of this step is first to receive data analysis needs sent by users. These needs are usually expressed in the form of natural language, such as "I want to see the sales comparison of various regions last month" or "Please analyze our customer retention rate trend in the past three years."

[0063] After receiving user requirements, the system needs to parse these natural language expressions. This parsing process involves natural language processing (NLP) technology, and its main purpose is to extract key business indicators and dimension information from the user's description. For example, from the sentence "Comparison of sales in various regions last month", the system needs to identify that "sales" is the target business indicator, and "region" and "last month" are the target business dimensions.

[0064] The technical principles of the parsing process may include lexical analysis, syntactic analysis, and semantic analysis. Lexical analysis mainly divides the input text into words or phrases. Syntactic analysis analyzes the grammatical relationship between these words and constructs the syntax tree of the sentence. Semantic analysis understands the actual meaning of these words and grammatical structures in the current context. In this process, the system may use pre-trained language models (such as BERT or GPT) to improve the accuracy of parsing.

[0065] After the analysis is completed, the system converts the extracted target business indicators and target business dimension information into feature vectors. This conversion process may involve feature engineering techniques, such as one-hot encoding, word embedding, etc. The converted feature vectors will be used as input to the demand classification model trained in step S101.

[0066] The demand classification model is calculated through forward propagation, applying activation functions (such as ReLU or Sigmoid) in each hidden layer, and finally giving the prediction results in the output layer. This result includes type indicators and measurement indicators. Type indicators may include the type of report (such as trend analysis, comparative analysis, distribution analysis, etc.), while measurement indicators may include specific indicators that need to be calculated (such as sales, profit margin, number of customers, etc.).

[0067] The technical problem solved in this step is mainly how to convert unstructured natural language requirements into structured data analysis tasks. This is a typical natural language understanding problem, involving multiple difficulties such as semantic parsing and intent recognition. By combining NLP technology and machine learning models, the system can automatically complete this conversion process, greatly reducing the need for manual intervention.

[0068] In addition, this step also solves the problem of how to concretize the user's vague needs. Users may use vague or unprofessional terms when expressing their needs. The system needs to be able to understand these expressions and map them to specific business indicators and dimensions. This requires the system to have certain domain knowledge and context understanding capabilities.

[0069] The technical effect achieved by this step is significant. First, it realizes the automatic parsing and classification of user needs, greatly improving the efficiency of data analysis. In traditional methods, data analysts need to communicate with users for multiple rounds to clarify the needs, while this automated process can complete the same work in a very short time. Secondly, this method improves the accuracy and consistency of demand understanding. Due to the use of standardized models, errors caused by human understanding deviations are reduced. Thirdly, this method improves the usability of the system, allowing users who do not have professional data analysis skills to easily propose complex data analysis requirements.

[0070] Step S103: Determine the hierarchy and layout of the report based on the logical relationship between the type indicators, add calculation logic to the report based on the calculation formula, aggregation method and display format of the measurement indicators, and generate a corresponding business report.

[0071] Optionally, in this embodiment, step S103 is a key link in the fast data visualization generation method based on the large language model, which mainly involves the structural design of the report and the implementation of the calculation logic. The technical process of this step is first to determine the hierarchy and layout of the report according to the type indicator obtained in step S102. The logical relationship between type indicators may include hierarchical relationships, parallel relationships, or cross relationships. The system needs to analyze these relationships and convert them into a suitable report structure.

[0072] When determining the report structure, the system may use decision trees or graph theory algorithms to represent and analyze the relationship between indicators. For example, if the type indicators include "region", "product category" and "time", the system may decide to use a three-layer tree structure, with region as the highest layer, product category as the middle layer, and time as the lowest layer. Alternatively, the system may choose to use a crosstab format, with region and product category as row dimensions and time as column dimension. This decision process may take into account multiple factors such as the characteristics of the data, user habits, and visualization effects.

[0073] After determining the basic structure of the report, the system will add calculation logic to the report based on the calculation formula, aggregation method and display format of the metric. This process involves the automatic generation of database query language (such as SQL), the selection of data aggregation algorithm and data formatting processing.

[0074] For calculation formulas, the system needs to convert the metrics in user requirements (such as "sales" and "profit margin") into specific calculation logic. This may involve simple summation operations or complex conditional calculations or cross-table associations. The system may use syntax analysis technology to convert the calculation logic described in natural language into structured expressions, and then into specific SQL queries or program codes.

[0075] The choice of aggregation method involves knowledge of statistics and data analysis. The system needs to select the appropriate aggregation method based on the type and distribution characteristics of the data. For example, for continuous data, statistics such as mean, median or standard deviation may be used; for discrete data, statistics such as count or mode may be used. The system may use heuristic algorithms or machine learning models to automatically select the most appropriate aggregation method.

[0076] The display format needs to take into account the characteristics of the data and the reading habits of the user. For example, percentage data may need to retain two decimal places and add a percentage sign, and amount data may need to use a thousand separator, etc. The system may maintain a predefined format library to automatically select the appropriate display format based on the data type and context.

[0077] The technical problems solved in this step mainly include how to transform abstract report requirements into specific data structures and calculation logic, and how to ensure the consistency and readability of reports while ensuring flexibility. The traditional report development process usually requires data analysts to manually design report structures and write query statements, which is not only time-consuming and labor-intensive, but also prone to errors. By automating this process, the system greatly improves the efficiency and accuracy of report generation.

[0078] In addition, this step also solves the problem of how to handle complex multidimensional data analysis needs. In actual business, users may need to analyze multiple dimensions and indicators at the same time, which requires the system to be able to flexibly combine different dimensions and indicators to generate complex multidimensional reports. By using algorithms such as graph theory or matrix operations, the system can efficiently handle such complex multidimensional analysis needs.

[0079] The technical effect achieved by this step is significant. First, it realizes the full automation of the report generation process, from the user's request to the generation of the final report, the entire process does not require human intervention. This not only greatly improves efficiency, but also reduces the possibility of human error. Secondly, this method improves the quality and consistency of reports. Due to the use of standardized algorithms and templates, the generated reports are highly consistent in structure and format, which is easy for users to understand and compare. Thirdly, this method greatly improves the flexibility and scalability of the system. When faced with new business needs, you only need to update the corresponding rules or models to quickly adapt to new reporting needs.

[0080] From the above description, it can be seen that the fast data visualization generation method based on the large language model provided in the embodiment of the present application can receive and parse the data analysis requirements sent by the user, input the target business indicators and target business dimension information in the data analysis requirements into the demand classification model, and obtain the type indicators and measurement indicators output by the demand classification model; according to the logical relationship between the type indicators, the hierarchy and layout of the report are determined, and according to the calculation formula, aggregation method and display format of the measurement indicators, calculation logic is added to the report to generate corresponding business reports, thereby realizing data visualization efficiently and accurately.

[0081] In one embodiment of the fast data visualization generation method based on a large language model of the present application, see Figure 2 , and can also include the following:

[0082] Step S201: determining the indicator correlation of each of the original data features according to the linear correlation degree between the original data features and the target business indicators;

[0083] Step S202: Determine the feature importance of each of the original data features according to the Gini index gain generated when the original data features are split at the preset decision tree nodes, and determine the principal component contribution of each of the original data features according to the eigenvalues ​​and eigenvectors of the covariance matrix corresponding to the original data features.

[0084] Optionally, in this embodiment, steps S201 and S202 are important links in the method for rapid data visualization generation based on a large language model, and mainly involve the analysis and evaluation of data features. The technical process of these two steps is intended to comprehensively evaluate the original data features from different angles, so as to provide a reliable basis for subsequent model training and data analysis.

[0085] In step S201, the system first calculates the linear correlation between the original data features and the target business indicators to determine the indicator correlation of each original data feature. This process usually uses the Pearson correlation coefficient or the Spearman rank correlation coefficient. The Pearson correlation coefficient is applicable to continuous variables, while the Spearman rank correlation coefficient is applicable to ordinal variables. During the calculation process, the system will perform a one-to-one correlation analysis between each original data feature and the target business indicator to obtain a series of correlation coefficients. The closer the absolute value of these correlation coefficients is to 1, the stronger the linear correlation between the feature and the target indicator.

[0086] The technical principle of correlation analysis is based on the concept of covariance in statistics. For the Pearson correlation coefficient, it is actually the covariance of two variables divided by the product of their standard deviations. This calculation method can eliminate the influence of dimension, so that the correlation between variables of different scales can be directly compared. For the Spearman rank correlation coefficient, the original data is first converted into rank data, and then the correlation between the ranks is calculated.

[0087] Step S202 evaluates the importance of the original data features from two other perspectives. First, the system uses a decision tree model to calculate the importance of features. Specifically, the system builds a preset decision tree model and then observes the Gini index gain generated by each feature when the node is split. The Gini index is an indicator to measure the purity of a data set. The smaller its value, the purer the data set. When a feature is used for node splitting, if it can significantly reduce the Gini index of the child node (i.e., increase data purity), then this feature is considered important. The system records the sum of the Gini index gains generated by each feature in all node splits, and this sum represents the importance of the feature.

[0088] The advantage of the decision tree model is that it can capture the nonlinear relationship between features and targets, which is not possible with simple correlation analysis. In addition, the decision tree model is insensitive to the scale of features and can handle both continuous and discrete variables, which makes it widely applicable in feature importance assessment.

[0089] Next, the system calculates the principal component contribution. This process involves principal component analysis (PCA) technology. The system first calculates the covariance matrix of the original data features, and then solves the eigenvalues ​​and eigenvectors of this matrix. The eigenvalue represents the variance of the corresponding principal component, while the eigenvector represents the direction of the principal component. The system calculates the sum of the squares of the projections of each original feature on each principal component, and weights them according to the importance of each principal component (that is, the size of the corresponding eigenvalue), and finally obtains the principal component contribution of each original feature.

[0090] The technical principle of principal component analysis is based on linear algebra and statistics. Its core idea is to project high-dimensional data into a low-dimensional space while retaining as much original information as possible. By analyzing the eigenvalues ​​and eigenvectors of the covariance matrix, PCA can find the main direction of change in the data (i.e., the principal component) and quantify the contribution of each original feature to these main changes.

[0091] The technical problems solved in these two steps mainly include how to comprehensively and objectively evaluate the importance of the original data features, and how to reduce the dimension of the data while retaining key information. Traditional data analysis methods often only focus on the evaluation of features in a single dimension, such as only looking at correlation or only looking at variance, which may miss some important information. By combining multiple evaluation methods, the system can comprehensively evaluate the importance of each feature from different angles to avoid missing key information.

[0092] In addition, these two steps also solve the problem of how to deal with high-dimensional data. In actual business, there may be hundreds or thousands of raw data features. Directly using all features for analysis is not only computationally intensive, but also prone to introducing noise. Through feature importance evaluation and principal component analysis, the system can effectively reduce the data dimension and extract the most critical information.

[0093] The technical effects achieved by these two steps are significant. First, they provide a comprehensive feature evaluation framework that can evaluate the importance of features from three perspectives: linear correlation, nonlinear importance, and principal component contribution. This multi-dimensional evaluation method can more accurately identify truly important features, providing a reliable foundation for subsequent model training and data analysis. Secondly, this method improves the efficiency and accuracy of data analysis. By identifying important features and reducing data dimensions, the system can greatly reduce the amount of calculation and speed up analysis while retaining key information. Thirdly, this method improves the interpretability of the model. By clarifying the importance and contribution of each feature, the analysis results become more transparent and interpretable, making it easier for business personnel to understand and apply.

[0094] In one embodiment of the fast data visualization generation method based on a large language model of the present application, see Figure 3 , and can also include the following:

[0095] Step S301: construct a model input layer whose number of nodes matches the indicator correlation, the feature importance and the number of features of the principal component contribution, construct a model output layer whose number of nodes corresponds to the number of possible type indicators and measurement indicators, and obtain an initial classification model based on the model input layer, the model output layer and the preset model hidden layer;

[0096] Step S302: input the indicator correlation, the feature importance and the principal component contribution into the model input layer of the multi-layer perceptron model and use the small batch gradient descent method to train the model, update the model parameters through the back propagation algorithm, and obtain the demand classification model.

[0097] Optionally, in this embodiment, steps S301 and S302 are the core links in the fast data visualization generation method based on the large language model, which mainly involve the construction and training process of the classification model. The technical process of these two steps is to build a deep learning model that can accurately classify user needs and provide a reliable foundation for subsequent report generation.

[0098] In step S301, the system first builds the input layer of the model. The number of nodes in the input layer matches the number of features of indicator correlation, feature importance, and principal component contribution calculated in the previous step. This means that if we have n raw data features, the number of nodes in the input layer is 3n, because each feature has three evaluation indicators. This design ensures that the model can fully utilize all feature evaluation information obtained in the previous steps.

[0099] Next, the system builds the output layer of the model. The number of nodes in the output layer corresponds to the number of possible type indicators and measurement indicators. For example, if the system predefines m possible type indicators and k possible measurement indicators, then the number of nodes in the output layer is m+k. Each output node represents a possible indicator, and its activation value represents the probability of the indicator being selected.

[0100] Between the input layer and the output layer, the system will add a certain number of hidden layers according to the preset model structure. The number of hidden layers and the number of nodes in each layer usually need to be determined based on the complexity of the specific problem and the dataset. Generally speaking, complex problems may require more hidden layers and nodes to capture nonlinear relationships in the data.

[0101] The construction process of this initial classification model adopts the multi-layer perceptron (MLP) structure commonly used in deep learning. Each layer of MLP consists of multiple neurons, and the neurons between adjacent layers are fully connected. Each neuron receives the output of all neurons in the previous layer, and after weighted summation and nonlinear activation function processing, it passes the result to the next layer. This structure can effectively learn the complex nonlinear relationship between input features and output indicators.

[0102] Step S302 is the training process of the model. First, the system inputs the index correlation, feature importance and principal component contribution as input data into the input layer of the model. Then, the system uses the mini-batch gradient descent method to train the model. The mini-batch gradient descent method is a commonly used optimization algorithm that only uses a small portion of the training data to calculate the gradient in each iteration, which not only ensures the efficiency of training, but also avoids falling into the local optimal solution.

[0103] During the training process, the system uses the back-propagation algorithm to update the model parameters. The core idea of ​​the back-propagation algorithm is to calculate the partial derivatives of the loss function with respect to each model parameter, and then update the parameters based on these partial derivatives to minimize the loss function. Specifically, the algorithm first performs forward propagation to calculate the predicted output of the model; then compares the predicted output with the true label to calculate the loss; finally, starting from the output layer, reversely calculate the contribution of each parameter to the loss layer by layer, and adjust the parameters accordingly.

[0104] The technical problems solved in these two steps mainly include how to effectively transform the feature evaluation results into a model that can be used for demand classification, and how to efficiently train this model. Traditional demand classification methods often rely on manually formulated rules or simple statistical models, which often do not work well when faced with complex business scenarios. By building and training deep learning models, the system can automatically learn complex feature combinations and nonlinear relationships, thereby more accurately classifying user needs.

[0105] In addition, these two steps also solve the problem of how to deal with high-dimensional features and large-scale training data. In actual business, there may be hundreds or thousands of original data features. Directly using all the original features for classification is not only computationally intensive, but also prone to introducing noise. By using the feature evaluation results obtained in the previous step as input, the system greatly reduces the input dimension, improves training efficiency and model generalization ability.

[0106] The technical effects achieved by these two steps are significant. First, they achieve end-to-end automated processing from raw data features to demand classification. The system is able to automatically build and train classification models suitable for specific business scenarios without manual intervention. Second, this approach improves the accuracy and efficiency of demand classification. By leveraging the powerful expressive power of deep learning models, the system is able to capture complex feature interactions and nonlinear relationships, thereby more accurately understanding and classifying user needs. Third, this approach improves the scalability and adaptability of the system. When faced with new business scenarios or data features, the system can quickly adapt by retraining the model without redesigning the classification rules.

[0107] Let's take a specific example: suppose a large e-commerce platform wants to automatically generate personalized product recommendation reports based on users' browsing and purchasing behaviors. The system collects dozens of features such as users' click-through rate, dwell time, purchase frequency, and ratings, and calculates the relevance, importance, and principal component contribution of these features through the previous steps. In step S301, the system constructs a multi-layer perceptron model with 90 nodes in the input layer (assuming 30 original features, each with 3 evaluation indicators) and 15 nodes in the output layer (assuming 10 possible type indicators and 5 possible measurement indicators). The model also contains two hidden layers, each with 50 nodes. In step S302, the system uses historical data to train the model, randomly selecting 64 samples each time for small batch gradient descent. After multiple rounds of training, the model can accurately predict the most suitable report type and key indicators based on the user's behavioral characteristics. For example, for users who browse frequently but rarely buy, the model may recommend generating a report containing indicators such as "browsed product category distribution" and "product price sensitivity" to help the operation team formulate targeted promotion strategies. This automatically generated classification model not only greatly improves the efficiency and accuracy of personalized recommendations, but also can automatically adjust as user behavior changes and always maintain good performance.

[0108] In one embodiment of the fast data visualization generation method based on a large language model of the present application, see Figure 4 , and can also include the following:

[0109] Step S401: semantically identifying the data analysis requirements through a preset natural semantic recognition algorithm, determining corresponding target business indicators and target business dimensions, and converting them into feature vectors;

[0110] Step S402: The feature vector is input into the demand classification model after model training, so that the demand classification model performs forward propagation calculation, applies an activation function in each hidden layer, and finally outputs type indicators and measurement indicators in the output layer.

[0111] Optionally, in this embodiment, steps S401 and S402 are key links in the fast data visualization generation method based on a large language model, and mainly involve the semantic understanding and classification process of user needs. The technical process of these two steps is to convert the user's natural language needs into a form that the system can understand and process, and to accurately classify the needs through the trained model, so as to provide accurate guidance for subsequent report generation.

[0112] In step S401, the system first uses a preset natural semantic recognition algorithm to perform semantic recognition on the data analysis requirements proposed by the user. This process usually involves multiple natural language processing (NLP) technologies, including but not limited to word segmentation, part-of-speech tagging, named entity recognition, dependency syntax analysis, etc. The combination of these technologies enables the system to accurately understand the key information in the user's needs, such as target business indicators and target business dimensions.

[0113] Specifically, word segmentation technology divides the continuous text input by the user into meaningful word units; part-of-speech tagging assigns a part of speech to each word, such as noun, verb, adjective, etc.; named entity recognition can identify specific categories of entities in the text, such as names of people, places, and organizations; and dependency parsing can analyze the syntactic relationship between words in a sentence. Through the synergy of these technologies, the system can extract key business concepts and analysis requirements from the user's natural language description.

[0114] After identifying the target business indicators and target business dimensions, the system needs to convert this information into feature vectors. This process usually uses word embedding technology, such as Word2Vec, GloVe, or BERT. Word embedding technology can map words to a high-dimensional vector space, so that words with similar semantics are closer in this space. In this way, the system can convert business indicators and dimensions described in natural language into numerical feature vectors, preparing for subsequent model input.

[0115] Step S402 is to input the feature vector obtained in step S401 into the previously trained demand classification model. The demand classification model is a trained deep neural network, usually using a multi-layer perceptron (MLP) or other more complex network structures.

[0116] When the feature vector is input into the model, the system will perform forward propagation calculations. Specifically, the input vector is first multiplied by the weight matrix from the input layer to the first hidden layer, and then the bias term is added. The result is passed through an activation function, such as ReLU (Rectified Linear Unit) or Sigmoid function, to introduce nonlinear transformations. This process is repeated in each hidden layer until the output layer is reached.

[0117] At the output layer, the system usually uses the Softmax activation function to convert the raw output of the neural network into a probability distribution. This probability distribution reflects the model's prediction confidence for each possible type indicator and metric. The system will select the type indicator and metric with the highest confidence as the final classification result.

[0118] The technical problems solved in these two steps mainly include how to accurately understand the user's natural language needs and how to convert this understanding into a numerical representation that can be used for accurate classification. Traditional demand analysis methods often rely on manual interpretation and classification, which is not only inefficient but also easily affected by personal subjective judgment. By introducing natural language processing and deep learning technology, the system can automatically understand and classify user needs, greatly improving efficiency and consistency.

[0119] In addition, these two steps also solve the problem of how to deal with diverse and complex user needs. In actual business, users may describe their analysis needs in a variety of different ways of expression. By using advanced natural language processing technology and flexible deep learning models, the system can adapt to various expressions and accurately capture the essence of user needs.

[0120] The technical effects achieved by these two steps are significant. First, they achieve end-to-end automated processing from natural language requirements to accurate classification results. The system can automatically understand user requirements and give corresponding type indicators and metrics without manual intervention. Second, this approach greatly improves the accuracy and efficiency of demand classification. By leveraging advanced NLP technology and deep learning models, the system is able to capture the nuances in user requirements and provide more accurate classification results. Third, this approach improves the scalability and adaptability of the system. When faced with new business scenarios or expressions, the system can quickly adapt by updating the language model and retraining the classification model.

[0121] To give a specific example: suppose a financial analysis platform receives the following user demand: "I want to understand the trend of return on investment in various industries over the past year, with a special focus on the technology and medical industries." In step S401, the system first uses natural language processing technology to analyze this sentence. Through word segmentation and part-of-speech tagging, the system identifies keywords such as "return on investment", "industry", "trend", "technology", and "medical". Through named entity recognition, the system determines that "technology" and "medical" are specific industry categories. Through dependency syntax analysis, the system understands that "return on investment" is the main analysis target, "industry" is the dimension of analysis, and "trend" indicates that time series analysis is required.

[0122] The system converts these understandings into feature vectors, which may be in the form of [0.8, 0.6, 0.9, 0.7, 0.5, ...], where each value represents the strength of a certain semantic feature. In step S402, the system inputs this feature vector into the previously trained demand classification model. The model calculates through a multi-layer neural network and finally gives a prediction result at the output layer. For example, the model may predict the type indicator as "time series analysis" with a confidence level of 90%, and predict the metric indicator as "return on investment" with a confidence level of 85%. Based on this classification result, the system can further generate a visualization report containing the return on investment trends of various industries (especially the technology and medical industries) in the past year. This automated demand understanding and classification process not only greatly improves the analysis efficiency, but also can accurately capture the details of user needs, providing precise guidance for subsequent data analysis and visualization.

[0123] In one embodiment of the fast data visualization generation method based on a large language model of the present application, see Figure 5 , and can also include the following:

[0124] Step S501: Identify the hierarchical relationship, subordinate relationship and associated relationship between type indicators through a predefined business knowledge graph, and construct a relationship network according to the hierarchical relationship, the subordinate relationship and the associated relationship;

[0125] Step S502: using the highest level type indicator in the relationship network as the main dimension to determine the report hierarchy and structure.

[0126] Optionally, in this embodiment, steps S501 and S502 are the core links in the fast data visualization generation method based on the large language model, which mainly involve the identification of business indicator relationships and the determination of report structure. The technical process of these two steps is to use the predefined business knowledge graph to build a relationship network between indicators, and determine the hierarchy and overall structure of the final report based on this network, providing a clear framework for subsequent data visualization.

[0127] In step S501, the system first uses the predefined business knowledge graph to identify various relationships between type indicators. Business knowledge graph is a structured knowledge representation method that contains entities, concepts and various relationships between them in a specific field. In this scenario, the knowledge graph mainly contains various business indicators and the relationship information between them.

[0128] Specifically, the system recognizes three main types of relationships: hierarchical relationships, subordinate relationships, and associative relationships. Hierarchical relationships indicate the superior-subordinate relationship between indicators, for example, "Total Sales" may be the parent indicator of "Regional Sales". Subordinate relationships indicate that one indicator belongs to the scope of another indicator, for example, "Customer Satisfaction" may be subordinate to the broader concept of "Customer Service Quality". Associative relationships indicate the mutual influence or correlation between indicators, for example, there may be an associative relationship between "Advertising Spend" and "Number of New Customers".

[0129] The system traverses the nodes and edges in the knowledge graph to identify all indicators and their relationships related to the current analysis needs. This process may involve graph traversal algorithms such as depth-first search (DFS) or breadth-first search (BFS) to ensure that all relevant relationships are fully captured.

[0130] After identifying these relationships, the system constructs a relationship network. This network can be represented by a graph structure, where nodes represent individual indicators and edges represent the relationships between indicators. The type and properties of the edges can be used to distinguish different types of relationships. For example, directed edges can be used to represent hierarchical relationships and subordinate relationships, and undirected edges can be used to represent association relationships. The weight of the edge can be used to represent the strength or importance of the relationship.

[0131] Step S502 determines the hierarchy and overall structure of the report based on the constructed relationship network. The system first needs to identify the highest level type indicators in the relationship network. This can usually be achieved by analyzing the in-degree and out-degree of the nodes. The highest level indicators usually have more out-degree (edges pointing to other nodes) and few or no in-degree (edges from other nodes).

[0132] Once the highest-level type indicator is identified, the system will use it as the primary dimension of the report. The primary dimension will become the top level of the report structure and determine the main organization of the report. For example, if "sales performance" is identified as the highest-level indicator, the entire report may be organized around sales performance, and other related indicators will be used as sub-dimensions or detailed indicators.

[0133] Based on the primary dimension, the system will further determine the hierarchical structure of the report. This process usually involves building a tree structure, where the primary dimension is at the root node and other indicators are arranged at different levels according to their relationship with the primary dimension. The hierarchical relationship and subordinate relationship directly determine the position of the indicator in the tree, while the association relationship may affect the horizontal arrangement or grouping of the indicator.

[0134] The technical problems solved in these two steps mainly include how to effectively use domain knowledge to organize and structure analysis results, and how to automatically design a report structure that meets user needs and business logic. Traditional report design methods often rely on the experience and intuition of analysts, which is not only time-consuming and labor-intensive, but also prone to overlooking some potentially important relationships. By introducing knowledge graphs and automated relationship analysis, the system can comprehensively consider the complex relationships between various business indicators, thereby generating a more reasonable and insightful report structure.

[0135] In addition, these two steps also solve the problem of how to deal with multi-dimensional and multi-level complex business scenarios. In actual business, the relationship between various indicators is often intricate, and a single hierarchical structure may not be able to fully reflect this complexity. By building a relationship network and determining the report structure based on it, the system can better capture and present this complexity.

[0136] The technical effects achieved by these two steps are significant. First, they realize the automated conversion from abstract business knowledge to specific report structure. The system can automatically identify and utilize various relationships between business indicators to generate a report structure that conforms to business logic, greatly reducing the workload of manual design. Second, this method improves the rationality and comprehensiveness of the report structure. By systematically analyzing the indicator relationship, the report structure can more comprehensively reflect all aspects of the business and avoid possible omissions in manual design. Third, this method improves the interpretability and usability of the report. Since the report structure is built based on clear business relationships, users can more easily understand the organizational logic of the report, thereby making more effective use of the report for decision-making.

[0137] To give a specific example: suppose a retail enterprise's data analysis platform is generating a comprehensive performance report for senior managers. In step S501, the system analyzes relevant indicators through a predefined retail knowledge graph. The system identifies that "overall performance" is the highest-level indicator, which has a hierarchical relationship with "sales performance", "operational efficiency" and "customer satisfaction". "Sales performance" is in a subordinate relationship with "online sales" and "offline sales". At the same time, the system also identifies that there is a correlation between "promotional activities" and "sales performance".

[0138] Based on these relationships, the system constructs a relationship network. In this network, "overall performance" is at the top, directly connected to "sales performance", "operational efficiency" and "customer satisfaction". The "sales performance" node is further connected to "online sales" and "offline sales". There is an edge between "promotion activities" and "sales performance" to indicate the association.

[0139] In step S502, the system uses "overall performance" as the main dimension and determines the hierarchical structure of the report. The final report structure may be as follows: the top level is the "overall performance" overview, which is divided into three main parts: "sales performance", "operational efficiency" and "customer satisfaction". Under the "sales performance" part, it is further divided into two sub-parts: "online sales" and "offline sales". At the same time, the system will add an analysis module on the impact of promotional activities in the "sales performance" part to reflect the correlation between the two.

[0140] This automatically generated report structure not only comprehensively covers all business aspects, but also clearly shows the logical relationship between various indicators, allowing senior managers to quickly obtain key information and gain a deep understanding of the mutual impact between various business links. This greatly improves the practicality and decision-making support capabilities of the report, and also provides a clear guiding framework for subsequent data visualization and in-depth analysis.

[0141] In one embodiment of the fast data visualization generation method based on a large language model of the present application, see Figure 6 , and can also include the following:

[0142] Step S601: determining the required original data items and intermediate calculation steps according to the calculation formula of the metric, determining the corresponding aggregation function and aggregation level according to the aggregation method of the metric, and determining the corresponding tree structure or directed graph structure according to the display format of the metric;

[0143] Step S602: adding calculation logic to the report according to the required original data items and intermediate calculation steps, the corresponding aggregation functions and aggregation levels, and the corresponding tree structure or directed graph structure.

[0144] Optionally, steps S601 and S602 are key links in the fast data visualization generation method based on the large language model, mainly involving the determination and implementation process of the report calculation logic. The technical process of these two steps is to transform abstract metrics into specific data processing and calculation processes, and provide accurate data support and structured presentation methods for the final report generation.

[0145] In step S601, the system first needs to determine the required raw data items and intermediate calculation steps based on the calculation formula of the metric. This process involves parsing and decomposing the calculation formula. The system may use an abstract syntax tree (AST) or other similar data structures to represent and analyze the calculation formula. By traversing this structure, the system can identify all the required raw data items and the various intermediate steps in the calculation process.

[0146] For example, if the metric is "profit margin" and its calculation formula is "(income-cost) / income", the system will identify "income" and "cost" as the original data items, and determine that the intermediate step of "income-cost" needs to be calculated first. This decomposition not only ensures the accuracy of the calculation, but also provides clear guidance for subsequent data acquisition and processing.

[0147] Next, the system needs to determine the corresponding aggregation function and aggregation level based on the aggregation method of the metric. Aggregation methods may include sum, average, maximum, minimum, etc. The system needs to select appropriate SQL aggregation functions or other data processing functions to implement these aggregation operations. At the same time, the system also needs to determine the aggregation level, which usually corresponds to the hierarchical structure of the report.

[0148] For example, if you need to calculate "monthly sales", the system may choose SUM as the aggregation function and set the aggregation level to Month. If the report also needs to show quarterly and annual data, the system also needs to determine how to aggregate these higher levels.

[0149] Finally, the system needs to determine the corresponding tree structure or directed graph structure based on the display format of the metric. This step focuses on the organization and presentation of data. The tree structure is usually used to represent hierarchical relationships, such as organizing sales data by region, city, and store. The directed graph structure is more suitable for representing complex relationship networks, such as showing the sales correlation between different products.

[0150] The system may use specific data structures, such as nested dictionaries or custom tree node classes, to represent these structures. The choice of structure usually depends on the nature of the data and the needs of the report. For example, if the report needs to show cross-analysis of multiple dimensions, a multidimensional array or similar data structure may be chosen.

[0151] Step S602 is to integrate the various elements determined in S601 into the calculation logic of the report. This process involves building a complete data processing pipeline, from the acquisition of raw data to the calculation and organization of the final results.

[0152] First, the system builds data acquisition logic based on the identified raw data items. This may involve writing SQL queries, calling APIs, or reading specific data files. The system needs to ensure that all necessary raw data can be acquired and perform necessary preprocessing, such as data cleaning and format conversion.

[0153] Next, the system builds the calculation logic according to the determined intermediate calculation steps. This usually involves writing a series of calculation functions or expressions. The system needs to ensure that these calculations are performed in the correct order and correctly handle possible special cases, such as division by zero errors, null value processing, etc.

[0154] The system then implements the data aggregation logic based on the determined aggregation function and aggregation level. This may involve the use of grouping operations (such as GROUP BY in SQL) and corresponding aggregation functions. The system needs to ensure that the aggregation operation can be correctly applied to different levels and handle the relationship between levels.

[0155] Finally, the system organizes the calculation results according to a determined tree structure or directed graph structure. This involves filling the calculated data into a pre-defined data structure. The system needs to ensure that the data is correctly placed in the right place in the structure and handle the relationship between different nodes or edges.

[0156] The technical problems solved in these two steps mainly include how to transform abstract business indicators into specific data processing processes and how to automatically build complex report calculation logic. Traditional report development methods usually require data analysts or developers to manually write a large amount of data processing code, which is not only time-consuming and labor-intensive, but also prone to errors. By introducing an automated calculation logic generation process, the system can quickly and accurately build complex report calculation logic, greatly improving the efficiency and accuracy of report development.

[0157] In addition, these two steps also solve the problem of how to deal with diverse and complex business indicators. In actual business, different indicators may have different calculation methods, aggregation methods, and display requirements. By decomposing these factors and processing them systematically, the system can flexibly respond to various different indicator requirements and provide a general and powerful report generation framework.

[0158] The technical effects achieved by these two steps are significant. First, they realize the automatic conversion from abstract indicators to specific calculation logic. The system can automatically parse the indicator definition and generate the corresponding data processing and calculation process, which greatly reduces the workload of manual coding. Secondly, this method improves the accuracy and consistency of report calculations. Since the calculation logic is automatically generated, it can ensure that the calculation method of each indicator is always consistent, reducing the possibility of human error. Thirdly, this method improves the flexibility and scalability of report development. When faced with new indicators or report requirements, the system can quickly generate the corresponding calculation logic without significantly modifying the existing code.

[0159] In one embodiment of the fast data visualization generation method based on a large language model of the present application, see Figure 7 , and can also include the following:

[0160] Step S701: Analyze the calculation formula and aggregation method of each metric indicator, determine the corresponding data processing flow and extract the required raw data from the data source, and implement the calculation process of each metric indicator according to the predefined calculation order, including intermediate calculation steps and final aggregation;

[0161] Step S702: Process the calculation logic under different dimensional combinations and adopt parallel calculation or pre-calculation strategies, fill the calculation results into the report structure, and apply the preset data format and display style to generate the corresponding business report.

[0162] Optionally, in this embodiment, steps S701 and S702 are key execution stages in the fast data visualization generation method based on a large language model, mainly involving actual data processing, calculation and report generation processes. The technical process of these two steps is to convert the calculation logic and report structure determined in the previous steps into specific data processing operations and visualization outputs, thereby generating the final business report.

[0163] In step S701, the system first needs to analyze the calculation formula and aggregation method of each metric. This process involves further refinement and operationalization of the calculation logic determined in the previous step. The system needs to convert the calculation formula of each indicator into specific data processing steps, which may include basic arithmetic operations, complex statistical calculations or specific business logic.

[0164] For example, for a "customer lifetime value" (CLV) indicator, its calculation formula may involve the customer's average order value, purchase frequency, and expected customer relationship duration. The system needs to break down this formula into a series of calculation steps, including calculating the average order value, determining the purchase frequency, estimating the customer relationship duration, and finally combining these intermediate results to get the final CLV value.

[0165] Next, the system needs to extract the required raw data from the data source according to the determined data processing flow. This process may involve complex database queries, API calls, or file reading operations. The system needs to ensure that all necessary data can be obtained efficiently while also considering the integrity and consistency of the data.

[0166] For example, to calculate CLV, the system may need to extract the customer's historical order information from the order database, obtain the customer's registration time and last activity time from the customer relationship management system, and may also need to obtain product profit margin information from the product database. The system needs to design an efficient query strategy, which may include the use of indexes, partitioned queries, or parallel processing to ensure good performance when processing large amounts of data.

[0167] Then, the system needs to implement the calculation process of each metric in a predefined calculation order. This includes executing various intermediate calculation steps and finally performing aggregation operations. In this process, the system needs to handle various possible abnormal situations, such as handling missing data, handling division by zero errors, handling outliers, etc.

[0168] For example, when calculating CLV, the system may first calculate the average order value of each customer, then calculate the purchase frequency, then estimate the duration of the customer relationship, and finally multiply these values ​​to get CLV. In this process, the system needs to handle special cases such as new customers with no historical orders and customers who have not been active for a long time.

[0169] Step S702 integrates the results calculated in S701 into the final report. This process first involves processing the calculation logic under different dimension combinations. In actual business reports, it is usually necessary to analyze data from multiple dimensions, such as by time, region, product category, etc. The system needs to be able to flexibly handle these different dimension combinations and calculate the corresponding indicator value for each combination.

[0170] In order to improve computing efficiency, the system may adopt parallel computing or pre-computation strategies. Parallel computing can use multi-core processors or distributed computing resources to process the calculation of multiple dimensions or indicators at the same time. The pre-computation strategy is to calculate and store some commonly used intermediate results or summary data in advance to speed up subsequent query and calculation processes.

[0171] For example, when calculating CLV in different regions and time periods, the system can process data from different regions in parallel, or pre-calculate daily summary data to speed up the calculation of monthly and annual CLV.

[0172] After the calculation is completed, the system needs to fill the results into the previously defined report structure. This process needs to ensure that the data is correctly placed in the corresponding position of the report, while also handling possible data missing or abnormal situations.

[0173] Finally, the system needs to apply the preset data format and display style to generate the final business report. This may include setting the precision and presentation format of numbers, adding percentage symbols, setting conditional formatting (such as changing the cell color based on the value size), etc. The system also needs to handle the overall layout of the report, add charts and legends, generate summary information, etc.

[0174] The technical problems solved in these two steps mainly include how to efficiently process large amounts of data, how to flexibly respond to complex computing requirements, how to handle multi-dimensional data analysis, and how to generate reports that are both beautiful and practical. Traditional report generation methods usually require a lot of manual coding and debugging, and are difficult to cope with rapidly changing business needs. By introducing automated data processing and report generation processes, the system can greatly improve the efficiency and flexibility of report generation.

[0175] The technical effects achieved by these two steps are significant. First, they achieve fully automated processing from raw data to final reports. The system is able to automatically perform complex data extraction, calculation, and aggregation operations, greatly reducing the need for manual intervention. Second, this approach improves the efficiency and accuracy of report generation. By adopting parallel computing and pre-computation strategies, the system is able to quickly process large amounts of data, while also reducing the possibility of errors due to reduced manual operations. Third, this approach improves the flexibility and customizability of reports. When faced with new analytical requirements, the system can quickly adjust the calculation logic and report structure without significantly modifying the underlying code.

[0176] To give a specific example: suppose a retail chain needs to generate a comprehensive sales analysis report, which includes key indicators such as "sales", "gross profit margin" and "average customer spending", and needs to be analyzed by dimensions such as time (day, month, year), region and product category.

[0177] In step S701, the system first analyzes the calculation formulas of these indicators. "Sales" needs to be directly summarized from the order data, "Gross Profit" needs to calculate (Sales - Cost) / Sales, and "Average Order Price" needs to calculate Sales / Order Quantity. The system determines that it needs to extract the original data such as order amount, product cost, and order quantity from the order database.

[0178] The system has designed an efficient data extraction strategy, using partition query technology to extract data in parallel by time range. For each indicator, the system implements the corresponding calculation function. For example, the function for calculating gross profit margin not only performs basic calculations, but also handles possible zero division errors and performs special processing for abnormal situations where costs are higher than sales.

[0179] In step S702, the system needs to process calculations of different dimensional combinations. The system uses a multidimensional array structure to organize data, where each dimension (time, region, product category) corresponds to an array axis. The system uses parallel computing technology to simultaneously process aggregation operations of different dimensions, greatly improving computing efficiency.

[0180] To further optimize performance, the system also implements a pre-calculation strategy. Every night, the system pre-calculates the indicators for the day and stores them in an intermediate result table. In this way, when generating monthly or annual reports, the system can directly use these pre-calculated results without reprocessing the original data.

[0181] Finally, the system fills the calculation results into the predefined report structure. The report includes an overview section that shows the overall sales situation, followed by detailed analysis by region and by product category. The system applies preset data formats, such as formatting sales as currency and formatting gross profit margin and growth rate as percentage. The system also adds conditional formatting, so that cells above the average are displayed in green and those below the average are displayed in red, which intuitively reflects business performance.

[0182] In addition, the system automatically generates several key charts, including sales trend charts, pie charts of sales share by region, and bar charts of sales of top 10 products. These charts greatly enhance the readability and analytical value of the report.

[0183] Through this automated report generation process, enterprises can quickly obtain comprehensive and in-depth sales analysis, providing strong support for the formulation of business strategies. Moreover, when it is necessary to adjust the analysis dimension or add new indicators, the system can also respond quickly, reflecting extremely high flexibility and efficiency. This method not only greatly reduces the time and labor cost of report generation, but also improves the quality and depth of data analysis, providing a strong tool support for the data-driven decision-making of enterprises.

[0184] In order to realize data visualization efficiently and accurately, the present application provides an embodiment of a fast data visualization generation device based on a large language model for realizing all or part of the content of the fast data visualization generation method based on a large language model, see Figure 8 The fast data visualization generation device based on the large language model specifically includes the following contents:

[0185] The model building module 10 is used to obtain the original data features of the user's historical data analysis needs, perform feature analysis on the original data features, determine the indicator relevance, feature importance and principal component contribution of the original data features, input the indicator relevance, feature importance and principal component contribution into the input layer of the multi-layer perceptron model for model training, and obtain a demand classification model;

[0186] The indicator determination module 20 is used to receive and analyze the data analysis requirements sent by the user, input the target business indicators and target business dimension information in the data analysis requirements into the demand classification model, and obtain the type indicators and measurement indicators output by the demand classification model;

[0187] The report generation module 30 is used to determine the hierarchy and layout of the report according to the logical relationship between the types of indicators, add calculation logic to the report according to the calculation formula, aggregation method and display format of the measurement indicators, and generate corresponding business reports.

[0188] From the above description, it can be seen that the fast data visualization generation device based on the large language model provided in the embodiment of the present application can receive and parse the data analysis requirements sent by the user, input the target business indicators and target business dimension information in the data analysis requirements into the demand classification model, and obtain the type indicators and measurement indicators output by the demand classification model; according to the logical relationship between the type indicators, the hierarchy and layout of the report are determined, and according to the calculation formula, aggregation method and display format of the measurement indicators, calculation logic is added to the report to generate corresponding business reports, thereby realizing data visualization efficiently and accurately.

[0189] From the hardware level, in order to realize data visualization efficiently and accurately, the present application provides an embodiment of an electronic device for realizing all or part of the content of the method for rapid data visualization generation based on a large language model, and the electronic device specifically includes the following content:

[0190] Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between the fast data visualization generation device based on the large language model and the core business system, user terminal and related database and other related devices; the logic controller can be a desktop computer, a tablet computer and a mobile terminal, etc., but the present embodiment is not limited thereto. In the present embodiment, the logic controller can be implemented with reference to the embodiment of the fast data visualization generation method based on the large language model and the embodiment of the fast data visualization generation device based on the large language model in the embodiment, and the contents thereof are incorporated herein, and the repeated parts are not repeated.

[0191] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0192] In practical applications, part of the method for rapid data visualization generation based on a large language model can be executed on the electronic device side as described above, or all operations can be completed in the client device. The selection can be made based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor.

[0193] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0194] Fig. 9 FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig. 9 As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig. 9 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0195] In one embodiment, the function of the fast data visualization generation method based on the large language model can be integrated into the central processing unit 9100. The central processing unit 9100 can be configured to perform the following control:

[0196] Step S101: obtaining original data features of the user's historical data analysis needs, performing feature analysis on the original data features, determining the indicator relevance, feature importance and principal component contribution of the original data features, inputting the indicator relevance, feature importance and principal component contribution into the input layer of a multi-layer perceptron model for model training, and obtaining a demand classification model;

[0197] Step S102: receiving and parsing the data analysis requirements sent by the user, inputting the target business indicators and target business dimension information in the data analysis requirements into the demand classification model, and obtaining the type indicators and measurement indicators output by the demand classification model;

[0198] Step S103: Determine the hierarchy and layout of the report based on the logical relationship between the type indicators, add calculation logic to the report based on the calculation formula, aggregation method and display format of the measurement indicators, and generate a corresponding business report.

[0199] From the above description, it can be seen that the electronic device provided in the embodiment of the present application receives and parses the data analysis requirements sent by the user, inputs the target business indicators and target business dimension information in the data analysis requirements into the demand classification model, and obtains the type indicators and measurement indicators output by the demand classification model; according to the logical relationship between the type indicators, the hierarchy and layout of the report are determined, and according to the calculation formula, aggregation method and display format of the measurement indicators, calculation logic is added to the report to generate corresponding business reports, thereby realizing data visualization efficiently and accurately.

[0200] In another embodiment, the fast data visualization generation device based on the large language model can be configured separately from the central processing unit 9100. For example, the fast data visualization generation device based on the large language model can be configured as a chip connected to the central processing unit 9100, and the function of the fast data visualization generation method based on the large language model can be implemented through the control of the central processing unit.

[0201] like Fig. 9 As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig. 9 In addition, the electronic device 9600 may also include Fig. 9 For components not shown, reference may be made to the prior art.

[0202] like Fig. 9 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0203] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0204] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0205] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0206] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0207] The communication module 9110 is a transmitter / receiver that sends and receives signals via the antenna 9111. The communication module 9110 (transmitter / receiver) is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0208] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module, etc. The communication module 9110 (transmitter / receiver) is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.

[0209] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps in the method for fast data visualization generation based on a large language model in the above-mentioned embodiment, where the execution subject is a server or a client. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, all the steps in the method for fast data visualization generation based on a large language model in the above-mentioned embodiment are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0210] Step S101: obtaining original data features of the user's historical data analysis needs, performing feature analysis on the original data features, determining the indicator relevance, feature importance and principal component contribution of the original data features, inputting the indicator relevance, feature importance and principal component contribution into the input layer of a multi-layer perceptron model for model training, and obtaining a demand classification model;

[0211] Step S102: receiving and parsing the data analysis requirements sent by the user, inputting the target business indicators and target business dimension information in the data analysis requirements into the demand classification model, and obtaining the type indicators and measurement indicators output by the demand classification model;

[0212] Step S103: Determine the hierarchy and layout of the report based on the logical relationship between the type indicators, add calculation logic to the report based on the calculation formula, aggregation method and display format of the measurement indicators, and generate a corresponding business report.

[0213] From the above description, it can be seen that the computer-readable storage medium provided in the embodiment of the present application receives and parses the data analysis requirements sent by the user, inputs the target business indicators and target business dimension information in the data analysis requirements into the demand classification model, and obtains the type indicators and measurement indicators output by the demand classification model; according to the logical relationship between the type indicators, the hierarchy and layout of the report are determined, and according to the calculation formula, aggregation method and display format of the measurement indicators, calculation logic is added to the report to generate corresponding business reports, thereby realizing data visualization efficiently and accurately.

[0214] The embodiments of the present application also provide a computer program product capable of implementing all the steps of the method for fast data visualization generation based on a large language model in the above embodiments, where the execution subject is a server or a client. When the computer program / instruction is executed by a processor, the steps of the method for fast data visualization generation based on a large language model are implemented. For example, the computer program / instruction implements the following steps:

[0215] Step S101: obtaining original data features of the user's historical data analysis needs, performing feature analysis on the original data features, determining the indicator relevance, feature importance and principal component contribution of the original data features, inputting the indicator relevance, feature importance and principal component contribution into the input layer of a multi-layer perceptron model for model training, and obtaining a demand classification model;

[0216] Step S102: receiving and parsing the data analysis requirements sent by the user, inputting the target business indicators and target business dimension information in the data analysis requirements into the demand classification model, and obtaining the type indicators and measurement indicators output by the demand classification model;

[0217] Step S103: Determine the hierarchy and layout of the report based on the logical relationship between the type indicators, add calculation logic to the report based on the calculation formula, aggregation method and display format of the measurement indicators, and generate a corresponding business report.

[0218] From the above description, it can be seen that the computer program product provided in the embodiment of the present application receives and parses the data analysis requirements sent by the user, inputs the target business indicators and target business dimension information in the data analysis requirements into the demand classification model, and obtains the type indicators and measurement indicators output by the demand classification model; according to the logical relationship between the type indicators, the hierarchy and layout of the report are determined, and according to the calculation formula, aggregation method and display format of the measurement indicators, calculation logic is added to the report to generate corresponding business reports, thereby realizing data visualization efficiently and accurately.

[0219] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0220] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0221] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0222] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0223] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A fast data visualization generation method based on a large language model, characterized in that: The method comprises: Obtaining original data features of user historical data analysis needs, performing feature analysis on the original data features, determining the indicator relevance, feature importance, and principal component contribution of the original data features, inputting the indicator relevance, feature importance, and principal component contribution into the input layer of a multi-layer perceptron model for model training, and obtaining a demand classification model; Receive and analyze the data analysis requirements sent by the user, input the target business indicators and target business dimension information in the data analysis requirements into the demand classification model, and obtain the type indicators and measurement indicators output by the demand classification model; According to the logical relationship between the types of indicators, the hierarchy and layout of the report are determined, and according to the calculation formula, aggregation method and display format of the measurement indicators, calculation logic is added to the report to generate the corresponding business report.

2. The method for rapid data visualization generation based on a large language model according to claim 1, characterized in that: The performing feature analysis on the original data features to determine the indicator correlation, feature importance and principal component contribution of the original data features includes: Determining the indicator correlation of each of the original data features according to the linear correlation degree between the original data features and the target business indicators; According to the Gini index gain generated when the original data features are split at the preset decision tree nodes, the feature importance of each of the original data features is determined, and according to the eigenvalues ​​and eigenvectors of the covariance matrix corresponding to the original data features, the principal component contribution of each of the original data features is determined.

3. The method for rapid data visualization generation based on a large language model according to claim 1, characterized in that: The step of inputting the indicator correlation, the feature importance and the principal component contribution into the input layer of the multi-layer perceptron model for model training to obtain a demand classification model includes: Constructing a model input layer whose number of nodes matches the correlation of the indicator, the importance of the feature, and the number of features of the principal component contribution, constructing a model output layer whose number of nodes corresponds to the number of possible type indicators and measurement indicators, and constructing an initial classification model based on the model input layer, the model output layer, and the preset model hidden layer; The indicator correlation, the feature importance and the principal component contribution are input into the model input layer of the multi-layer perceptron model and the model is trained using a small batch gradient descent method. The model parameters are updated using a back propagation algorithm to obtain a demand classification model.

4. The method for rapid data visualization generation based on a large language model according to claim 1, characterized in that: The step of inputting the target business indicators and target business dimension information in the data analysis requirements into the demand classification model to obtain the type indicators and measurement indicators output by the demand classification model includes: Perform semantic recognition on the data analysis requirements through a preset natural semantic recognition algorithm, determine the corresponding target business indicators and target business dimensions, and convert them into feature vectors; The feature vector is input into the demand classification model after model training, so that the demand classification model performs forward propagation calculation, applies activation function in each hidden layer, and finally outputs type indicators and measurement indicators in the output layer.

5. The method for rapid data visualization generation based on a large language model according to claim 1, characterized in that: Determining the hierarchy and layout of the report according to the logical relationship between the type indicators includes: Identify the hierarchical relationship, subordinate relationship and associated relationship between type indicators through a predefined business knowledge graph, and construct a relationship network based on the hierarchical relationship, the subordinate relationship and the associated relationship; The highest level type indicator in the relationship network is used as the main dimension to determine the report hierarchy and structure.

6. The method for rapid data visualization generation based on a large language model according to claim 1, characterized in that: Adding calculation logic to the report according to the calculation formula, aggregation method and display format of the metric includes: Determine the required original data items and intermediate calculation steps according to the calculation formula of the metric, determine the corresponding aggregation function and aggregation level according to the aggregation method of the metric, and determine the corresponding tree structure or directed graph structure according to the display format of the metric; The calculation logic is added to the report according to the required original data items and intermediate calculation steps, the corresponding aggregation functions and aggregation levels, and the corresponding tree structure or directed graph structure.

7. The method for rapid data visualization generation based on a large language model according to claim 1, characterized in that: The generating of the corresponding business report also includes: Analyze the calculation formula and aggregation method of each metric, determine the corresponding data processing flow and extract the required raw data from the data source, and implement the calculation process of each metric according to the predefined calculation order, including intermediate calculation steps and final aggregation; Process the calculation logic under different dimensional combinations and adopt parallel calculation or pre-calculation strategies, fill the calculation results into the report structure, and apply the preset data format and display style to generate the corresponding business report.

8. A fast data visualization generation device based on a large language model, characterized in that: The device comprises: A model building module is used to obtain the original data features of the user's historical data analysis needs, perform feature analysis on the original data features, determine the indicator relevance, feature importance and principal component contribution of the original data features, input the indicator relevance, feature importance and principal component contribution into the input layer of the multi-layer perceptron model for model training, and obtain a demand classification model; An indicator determination module is used to receive and analyze the data analysis requirements sent by the user, input the target business indicators and target business dimension information in the data analysis requirements into the demand classification model, and obtain the type indicators and measurement indicators output by the demand classification model; The report generation module is used to determine the hierarchy and layout of the report according to the logical relationship between the types of indicators, add calculation logic to the report according to the calculation formula, aggregation method and display format of the measurement indicators, and generate corresponding business reports.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method for fast data visualization generation based on a large language model as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for fast data visualization generation based on a large language model described in any one of claims 1 to 7 are implemented.

Images