Multi-dimensional data attribution analysis method and device based on large language model

Abstract

The present disclosure relates to the technical field of large language model, in particular to a multi-dimensional data attribution analysis method and device based on large language model, which can reduce the combination explosion problem in the super multi-dimensional scene, more accurately identify the key influence factors, and improve the accuracy of multi-dimensional data attribution analysis. The method comprises: obtaining target data for attribution analysis based on natural language description; determining an initial analysis dimension for attribution analysis by a first large language model, and determining an influence probability of the initial analysis dimension by a first prediction model, and screening a target analysis dimension with an influence probability higher than a preset probability threshold; performing business prediction based on the target data and the target analysis dimension by a second prediction model to obtain a predicted index value; and performing attribution analysis on the target analysis dimension based on the actual index value corresponding to the target analysis dimension and the predicted index value by the first large language model to obtain an attribution analysis result.

Description

Technical Field

[0001] This disclosure relates to the field of large language model technology, specifically to a method and apparatus for multidimensional data attribution analysis based on large language models. Background Technology

[0002] Data attribution analysis aims to determine the causes of data changes or outcomes. By deeply analyzing the data, it identifies the key factors influencing specific metrics or events. For example, when analyzing the reasons for sales growth, e-commerce platforms might start with data from multiple aspects such as user purchasing behavior, product promotions, and the market environment to determine whether the increase in sales was due to factors such as the increase in new users or the increase in repeat purchases by existing users.

[0003] However, when performing multidimensional data attribution analysis, the relevant technologies mainly rely on preset analysis templates and fixed calculation logic, and cannot intelligently identify potential attribution paths and key influencing factors. Especially when facing complex business scenarios, it is difficult to automatically discover the implicit relationships between data. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a method and apparatus for multidimensional data attribution analysis based on a large language model, in order to solve the deficiencies in related technologies.

[0005] According to a first aspect of the present disclosure, a multidimensional data attribution analysis method based on a large language model is provided, comprising: In response to the natural language description input by the user, the user intent is obtained by performing intent recognition on the natural language description through the first major language model, and if the user intent represents the attribution analysis intent, the target data for attribution analysis is obtained based on the natural language description. Based on the data features of the target data and the user intent, the first large language model determines the initial analysis dimension for attribution analysis, and the first prediction model determines the influence probability of the initial analysis dimension based at least on the target data and historical data. Target analysis dimensions with influence probabilities higher than a preset probability threshold are then selected from the initial analysis dimensions. The first large language model determines key business indicators based on the natural language description, and extracts the actual indicator values ​​corresponding to the key business indicators from the target data. The second prediction model performs business prediction based on the target data and the target analysis dimension to obtain the predicted indicator values. The second prediction model is used to predict the indicator values ​​corresponding to the key business indicators after adjusting the target data based on the target analysis dimension. Attribution analysis is performed using the first language model based on the actual and predicted indicator values ​​corresponding to the target analysis dimension to obtain the attribution analysis results.

[0006] According to a second aspect of the present disclosure, a multidimensional data attribution analysis apparatus based on a large language model is provided, comprising: The data acquisition module is used to respond to the natural language description input by the user, perform intent recognition on the natural language description through the first language model to obtain the user intent, and, if the user intent represents the attribution analysis intent, acquire the target data for attribution analysis based on the natural language description. The dimension determination module is used to determine the initial analysis dimension for attribution analysis based on the data features of the target data and the user intent through the first large language model, and to determine the influence probability of the initial analysis dimension based on at least the target data and historical data through the first prediction model, and to filter the target analysis dimension whose influence probability is higher than a preset probability threshold from the initial analysis dimension. The business forecasting module is used to determine key business indicators based on the natural language description using the first large language model, extract the actual indicator values ​​corresponding to the key business indicators from the target data, and perform business forecasting based on the target data and the target analysis dimension using the second forecasting model to obtain the predicted indicator values. The second forecasting model is used to predict the indicator values ​​corresponding to the key business indicators after adjusting the target data based on the target analysis dimension. The attribution analysis module is used to perform attribution analysis based on the actual and predicted indicator values ​​corresponding to the target analysis dimension using the first large language model, and obtain the attribution analysis results.

[0007] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: The multidimensional data attribution analysis method based on a large language model provided in this disclosure can automatically identify potential attribution paths affecting business indicators through the reasoning capabilities of the large language model, thereby improving the efficiency and accuracy of multidimensional data attribution analysis. Furthermore, using the probability output of the first prediction model as a heuristic rule for attribution path search can reduce the combinatorial explosion problem in highly multidimensional scenarios, further improving the efficiency of multidimensional data attribution analysis. By adjusting the target data based on the target analysis dimension using a second prediction model, the corresponding indicator values ​​of key business indicators can be analyzed not only based on observed actual data but also on adjusted simulated data, thereby more accurately identifying key influencing factors and further improving the accuracy of multidimensional data attribution analysis. Attached Figure Description

[0008] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0009] Figure 1 This is a flowchart illustrating a multidimensional data attribution analysis method based on a large language model, as shown in an exemplary embodiment of this disclosure; Figure 2 This is a schematic diagram of the structure of a multidimensional data attribution analysis device based on a large language model, as illustrated in an exemplary embodiment of this disclosure. Detailed Implementation

[0010] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure.

[0011] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” used in this disclosure are also intended to include the plural forms unless the context clearly indicates otherwise. It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this disclosure, and similarly, second information may also be referred to as first information.

[0012] Firstly, at least one embodiment of this disclosure provides a multidimensional data attribution analysis method based on a large language model. Please refer to the appendix. Figure 1 The diagram illustrates the process of the method, including steps S101 to S104.

[0013] In step S101, in response to the natural language description input by the user, the intention of the natural language description is identified by the first large language model to obtain the user intention, and the target data for attribution analysis is obtained based on the natural language description when the user intention represents the attribution analysis intention.

[0014] In step S102, the first language model determines the initial analysis dimension for attribution analysis based on the data features and user intent of the target data, and the first prediction model determines the influence probability of the initial analysis dimension based on at least the target data and historical data. Target analysis dimensions with influence probabilities higher than a preset probability threshold are selected from the initial analysis dimensions.

[0015] In step S103, key business indicators are determined based on natural language description using a first language model, and the actual indicator values ​​corresponding to the key business indicators are extracted from the target data. Then, a second prediction model is used to perform business prediction based on the target data and target analysis dimensions to obtain predicted indicator values. Specifically, the second prediction model is used to predict the indicator values ​​corresponding to the key business indicators after adjusting the target data based on the target analysis dimensions.

[0016] In step S104, attribution analysis is performed using the first major language model based on the actual and predicted indicator values ​​corresponding to the target analysis dimension to obtain the attribution analysis results.

[0017] Taking e-commerce as an example, the natural language description entered by the user is: Analyze the reasons for the 15% year-on-year decline in sales during the event. Potential influencing factors include: increased promotional efforts by competitors, reduced advertising spending, stockouts of key products, and decreased coupon conversion rates. Traditional solutions require traversing all dimension combinations. For example, considering the aforementioned four potential influencing factors, plus three sales time granularities and five sales region granularities, a total of 60 attribution paths need to be traversed, resulting in low efficiency in attribution analysis.

[0018] In this embodiment of the disclosure, after determining the initial analysis dimension, the first prediction model determines the probability of its impact based at least on the target data and historical data. Continuing with the above example, the first prediction model predicts the impact probabilities of competitor promotional efforts, advertising exposure, main product inventory, and coupon conversion rate to be 68%, 92%, 45%, and 31%, respectively. If the preset probability threshold is 60%, then the target analysis dimensions can be selected as advertising exposure and competitor promotions. Therefore, high-probability attribution paths can be automatically retained, reducing the analysis of low-probability attribution paths, saving computational resources, more accurately identifying key influencing factors, and improving the accuracy of multidimensional data attribution analysis.

[0019] Furthermore, attribution analysis is performed using a second prediction model based on adjusted simulated data. Continuing with the previous example, assuming ad impressions remain at the same level as last year, the second prediction model predicts sales revenue. The difference between predicted and actual sales revenue is only 3%, indicating that ad impressions are not the primary cause of the sales decline. Assuming competitors did not offer a buy-one-get-one-free promotion, the second prediction model predicts sales revenue that is 22% higher than actual sales revenue, indicating that competitor promotions are the main cause of the sales decline. Finally, the first language model outputs the attribution analysis results as follows: "Core Conclusion: Competitors' sudden increase in buy-one-get-one-free promotions led to customer churn; Supporting Findings: Although advertising decreased, its delivery efficiency improved; Recommended Action: Immediately launch a limited-time buy-one-get-one-free promotion, targeting competitor customers."

[0020] Therefore, by leveraging the reasoning capabilities of large language models, potential attribution paths affecting business metrics are automatically identified, thereby improving the efficiency and accuracy of multidimensional data attribution analysis. Furthermore, the probability output of the first prediction model is used as a heuristic rule for attribution path search, reducing the combinatorial explosion problem in highly multidimensional scenarios and further enhancing the efficiency of multidimensional data attribution analysis. The second prediction model can perform attribution analysis not only based on observed actual data but also on adjusted simulated data, thus more accurately identifying key influencing factors and further improving the accuracy of multidimensional data attribution analysis. For example, in sales attribution analysis, a deep attribution analysis involving 15+ dimensions and tens of millions of data points can be completed within 5 seconds, with a 91% success rate in accurately identifying key influencing factors.

[0021] To make it easier to understand, the above steps will be further illustrated with examples below.

[0022] For example, in step S101, an Artificial Intelligence (AI) interactive page can be displayed to the user. Then, in response to the natural language description input by the user on the AI ​​interactive page, the user's intention is determined by the first major language model associated with the AI ​​interactive page. This intention recognition process can refer to relevant technologies and will not be elaborated here. Given that the user's intention represents an attribution analysis intent, target data for attribution analysis is obtained based on the natural language description. Taking an e-commerce scenario as an example, target data for attribution analysis can be obtained, such as e-commerce sales data, competitor promotional data, customer service work order data, and order system log data.

[0023] In some embodiments, obtaining target data for attribution analysis based on natural language description includes: extracting second key information from the natural language description using a first large language model; obtaining a target SQL statement based on the second key information using a second large language model, and extracting constraints in natural language form from the target SQL statement; converting the second key information into a first text vector, and converting the constraints into a second text vector, and calculating the vector similarity between the first text vector and the second text vector; and obtaining target data for attribution analysis based on the target SQL statement if the vector similarity is higher than a first preset similarity threshold.

[0024] For example, the first preset similarity threshold can be set according to actual business needs, such as 0.8. This embodiment of the present disclosure does not limit this.

[0025] It should be understood that traditional data attribution analysis requires professional SQL (Structured Query Language) developers to write complex multidimensional query statements, including multi-table joins, grouping and summarizing, and time series analysis. When business requirements change, the SQL code needs to be rewritten and debugged, resulting in long development cycles, high maintenance costs, and a high risk of syntax errors and logical flaws. In this embodiment, users can describe their analysis requirements using natural language, such as "analyze the main reasons for the decline in sales in the East China region over the past three months," and then automatically convert this into complex SQL statements containing multi-table joins, time filtering, and grouping and aggregation, thereby improving the efficiency of multidimensional data attribution analysis.

[0026] Furthermore, it should be understood that because large language models may have biases in their understanding of certain constraints, the SQL statements generated by these models may not fully and accurately reflect the user's true needs, potentially affecting the accuracy of subsequent attribution analysis. In this embodiment, by comparing the consistency between user intent and SQL semantics, and utilizing semantic vectorization and similarity comparison, the final generated SQL statement accurately expresses the user's intent, thereby improving the accuracy of subsequent attribution analysis.

[0027] For example, the user inputs a natural language description: "Analyze the main reasons for the decline in sales in region A over the past three months." The first language model extracts the second key information from this description as: sales volume in region A over the past three months. The SQL statement generated based on this second key information is: `SELECT * FROM sales WHERE region = 'A' AND date>= DATE_SUB(CURRENT_DATE, INTERVAL 3 MONTH) AND metric = 'sales volume'`. The constraint extracted from this SQL statement is: sales data for region A over the past three months. Then, the two natural language statements containing the second key information and the constraint are vectorized and their similarity is compared. If the similarity is higher than or equal to a first preset similarity threshold, it indicates that the SQL statement matches the user's intent. Figure 1 This allows for the retrieval of target data for attribution analysis based on SQL statements. If the similarity is below a first preset similarity threshold, it indicates a significant discrepancy between the SQL statement and the user's intent, prompting manual verification of the SQL statement. Thus, through semantic constraint verification, the accuracy of SQL generation can reach over 93%, and development efficiency can be improved by 70%, significantly lowering the professional SQL skill threshold and enhancing the efficiency and accuracy of multidimensional data attribution analysis.

[0028] In some embodiments, obtaining the target SQL statement based on the second key information through the second language model includes: generating an initial SQL statement based on the second key information through the second language model, performing data feature analysis on the initial SQL statement to obtain feature analysis results, and rewriting the initial SQL statement based on the feature analysis results to obtain the target SQL statement. The feature analysis results include the SQL fields to be optimized in the initial SQL statement and optimization suggestions for the SQL fields.

[0029] Taking sales attribution as an example, the user's input in natural language is: "Analyze the reasons for the abnormal sales in region B over the past week." After generating the initial SQL statement using the second language model, data feature analysis reveals that each execution requires a full table scan, resulting in long query times. Furthermore, even though pre-calculated quantiles exist, the initial SQL statement still performs quantile calculations. In this case, the initial SQL statement can be rewritten using the second language model based on the feature analysis structure. For example, subquery materialization and partition pruning can be used to obtain the target SQL statement.

[0030] Therefore, the initial SQL statement can be automatically reorganized by operators, improving the efficiency of complex queries and thus enhancing the efficiency of multidimensional data attribution analysis.

[0031] In some embodiments, obtaining target data for attribution analysis based on natural language description includes: obtaining initial data for attribution analysis from multiple data sources based on natural language description; parsing unstructured data in the initial data into a unified data format using a third language model to obtain first data, and mapping structured data in the initial data into a unified data format using a field mapping engine to obtain second data; and combining the first data and the second data into target data for attribution analysis.

[0032] It should be understood that traditional data attribution analysis systems lack a unified data understanding and processing framework when handling heterogeneous data from multiple sources, such as third-party platforms, locally uploaded files, and ETL (Extract Transform Load) data streams. Furthermore, they require manual configuration of complex data mapping relationships, leading to low data integration efficiency and an inability to quickly respond to business analysis needs. In this embodiment, the semantic understanding capabilities of a large language model can be utilized to automatically identify, parse, and standardize multi-source heterogeneous data. It can automatically understand the database schema structure of different data sources, uniformly mapping data from different e-commerce platforms, office platforms, and locally uploaded files to a standard data model. This can improve data integration efficiency by more than 85%, thereby enhancing the efficiency of multidimensional data attribution analysis.

[0033] For example, structured and unstructured data in the initial data can be determined using a fourth language model used to parse the database schema. For unstructured data, such as the text data "payment amount is 299" obtained from a business document on an office platform, it is parsed into a unified data format using a third language model, resulting in the first data being {"payment": 299}. For structured data, such as the promotional price field {"discounted_price": 189} obtained from a business data table on an e-commerce platform, it is mapped into a unified data format using a field mapping engine, resulting in the second data being {"discountPrice": 189}. Accordingly, the target data for attribution analysis is obtained by combining the first and second data. Thus, data from different data sources can be uniformly mapped to a standard data model, achieving intelligent unified processing of multi-source data and improving the efficiency of multi-dimensional data attribution analysis.

[0034] In some embodiments, in step S102, a first entity can be matched in the business knowledge graph based on user intent using a first large language model, and a pre-defined analysis dimension associated with the first entity can be used as the first analysis dimension. Based on the data characteristics of the target data, a second analysis dimension can be determined. The business knowledge graph is used to store the analysis dimension entities for attribution analysis of business problems. The first analysis dimension and the second analysis dimension are combined into an initial analysis dimension for attribution analysis.

[0035] For example, based on user intent using the first language model, when a user raises the question of "declining sales," the business knowledge graph automatically loads entities representing the three core dimensions: product, channel, and time. The product dimension entity can be associated with product inventory and price fluctuations (i.e., pre-defined analysis dimensions); the channel dimension entity can be associated with platform traffic and promotional activities (i.e., pre-defined analysis dimensions); and the time dimension entity can be associated with year-on-year data and seasonal trends (i.e., pre-defined analysis dimensions). Furthermore, if the first language model identifies a surge in complaints due to "damaged packaging" from customer service tickets, the second analysis dimension can be determined to be the product packaging dimension (i.e., a non-pre-defined analysis dimension). Accordingly, the initial analysis dimensions used for attribution analysis can include the pre-defined analysis dimensions associated with the product, channel, and time dimension entities, as well as the non-pre-defined packaging dimension.

[0036] Therefore, the reasoning capabilities of large language models can be used to automatically identify potential dimensions affecting business metrics, thereby improving the accuracy of multidimensional data attribution analysis.

[0037] In some embodiments, in step S102, first key information can be extracted from the natural language description using a first major language model, and the business scenario corresponding to the target data can be determined based on the first key information; target data quality rules can be generated based on the business scenario using the first major language model, and the target data can be quality checked based on the data quality rules to obtain quality check results; if the quality check results indicate that the target data has no data quality problems, the initial analysis dimensions for attribution analysis can be determined based on the data features and user intent of the target data using the first major language model.

[0038] It should be understood that traditional attribution analysis's data quality check rules are complex to configure and cannot automatically generate suitable quality monitoring strategies based on business scenarios. In this embodiment, by analyzing business scenarios using a large language model, suitable data quality check rules are automatically generated. This can improve the data quality problem detection rate by 60% and reduce the false positive rate by 40%, significantly enhancing the credibility of data assets and thus improving the accuracy of multidimensional data attribution analysis.

[0039] For example, a user inputs a natural language description: "Analyze the reasons for the 15% year-on-year decline in sales during the event." The first language model can extract the first key information from this description as "event period," and based on this key information, determine the business scenario as a major promotional event. Then, the first language model can set key field missing detection rules, consistency detection rules, accuracy detection rules, and promotion-specific detection rules based on this business scenario. The threshold in the accuracy detection rule can be appropriately relaxed; for example, ±3σ in a normal scenario, it can be adjusted to ±5σ in the current major promotional event scenario, where σ represents the adjustment factor. Additionally, the promotion-specific detection rules are used to strengthen price consistency checks, avoiding inconsistencies in promotional prices during the major promotional event.

[0040] Therefore, by analyzing business scenarios through large language models, data quality inspection rules suitable for business scenarios can be automatically generated, thereby improving the data quality problem detection rate and thus improving the accuracy of multidimensional data attribution analysis.

[0041] After determining the initial analysis dimensions, the influence probability of these dimensions can be determined using a first prediction model, based at least on the target data and historical data. Target analysis dimensions with influence probabilities higher than a preset probability threshold are then selected from the initial analysis dimensions. The first prediction model can be an LSTM (Long Short-Term Memory) network model, trained using real historical data. Input data can include the target data and historical data from the past 30 days. Based on the past 30 days of historical data, time-series characteristic data such as historical sales volume, traffic fluctuation patterns, and price change curves can be determined. Based on the target data, real-time status data such as current inventory status, real-time conversion rate, and promotional activity intensity, as well as business scenario data such as major promotion indicators, holiday coefficients, and competitor activity status, can be determined. Alternatively, the input data for the first prediction model can also include external environmental data such as weather data, economic data, and heat data.

[0042] After obtaining the impact probability of the initial analysis dimension based on the first prediction model, target analysis dimensions with impact probabilities higher than a preset probability threshold can be selected, and then step S103 is executed. The preset probability threshold can be set according to actual business needs, for example, it can be set to 60%, but this embodiment does not limit this setting.

[0043] For example, in step S103, the key business indicator is the target indicator for attribution analysis. For instance, in natural language, it is described as: "Analyze the reasons for the 15% year-on-year decline in sales during the activity period." In this case, the key business indicator is sales, and the actual indicator value corresponding to the key business indicator is the specific value of sales.

[0044] In some embodiments, in step S103, actual feature values ​​that can affect key business indicators can be extracted from the target data; the target feature values ​​corresponding to the target analysis dimension in the actual feature values ​​are replaced with preset simulated feature values ​​to obtain new feature data; and business prediction is performed based on the new feature data by the second prediction model to obtain the predicted indicator values ​​corresponding to the key business indicators.

[0045] For example, the second prediction model can be an LSTM model used to predict business metrics (such as sales revenue) based on historical data. It should be understood that the second prediction model can be trained using sample historical data and the corresponding sample business metrics. During inference, for given target data, the actual feature values ​​that can influence key business metrics are first extracted from the target data, such as ad impressions X, product price Y, and promotion intensity Z. Then, keeping other feature values ​​unchanged, the target feature values ​​corresponding to the target analysis dimension are replaced with preset simulated feature values. For example, if the target analysis dimension is ad impressions, then other feature values ​​such as product price Y and promotion intensity Z remain unchanged, and ad impressions are replaced with X', forming new feature data. Then, the second prediction model performs business prediction based on the new feature data to obtain the predicted metric values ​​corresponding to key business metrics under the simulated business scenario. Therefore, attribution analysis can be performed not only based on observed actual data but also on adjusted simulated data, thus more accurately identifying key influencing factors and further improving the accuracy of multidimensional data attribution analysis.

[0046] In some embodiments, in step S104, the target difference between the actual indicator value and the predicted indicator value corresponding to the target analysis dimension can be determined by the first large language model, and attribution weights are assigned to the target analysis dimension based on the target difference. Attribution analysis results are generated based on the attribution weights corresponding to each target analysis dimension, wherein the target difference corresponding to each target analysis dimension is proportional to the attribution weight.

[0047] For example, continuing with the above example, assuming the ad impressions remain at the same level as last year (i.e., setting the ad impressions to a preset simulated characteristic value, such as the average ad impressions in the same period last year), and then using the second prediction model to predict sales, if the difference between the predicted and actual sales is only 3%, then the ad impressions can be assigned an attribution weight of 23%. Assuming competitors did not offer a buy-one-get-one-free promotion (i.e., setting the competitor's promotional efforts to a preset simulated characteristic value), and then using the second prediction model to predict sales, if the predicted sales are 22% higher than the actual sales, then the competitor's promotion can be assigned an attribution weight of 82%.

[0048] Therefore, the first major language model can perform quantitative attribution analysis based on the attribution weights of each target analysis dimension, thereby improving the accuracy of multidimensional data attribution analysis.

[0049] In some embodiments, a first language model can be used to recommend visualization chart types to users based on data features of user intent and attribution analysis results; in response to a user's triggering operation on a target chart type among the visualization chart types, attribution analysis results can be displayed based on the target chart type.

[0050] It should be understood that traditional attribution analysis tools primarily rely on users manually selecting chart types and configuring parameters, lacking intelligent recommendation capabilities based on data characteristics and analytical intent. Users need professional data visualization knowledge to create suitable analytical charts, resulting in high learning costs and unreliable effectiveness. In this embodiment, the most suitable visualization chart type, including trend charts, bar charts, heatmaps, and funnel charts, can be automatically recommended based on data characteristics and analytical intent. As a result, the chart recommendation accuracy reaches 88%, and the success rate of users creating effective visualization charts increases by 65%, significantly improving the user experience.

[0051] It should be understood that enterprise-level data attribution analysis platforms need to support complex row and column permission controls. Traditional solutions struggle to achieve dynamic permission configuration based on multiple dimensions such as user attributes, departmental levels, and data sensitivity. Furthermore, they lack intelligent permission rule understanding capabilities and cannot automatically generate permission configurations based on natural language descriptions. Therefore, in some embodiments, semantic understanding based on large language models can be used to achieve natural language descriptions and automatic configuration of permission rules. For example, it can support intelligent parsing of complex permission rules such as "East China regional sales managers can only view sales data for the past six months in their region," automatically generating corresponding row and column permission configurations. This improves permission configuration efficiency by 80% and significantly enhances security. In this embodiment, permission configurations can be automatically generated based on natural language descriptions.

[0052] It should be understood that existing ETL scheduling systems lack intelligent task orchestration and exception handling capabilities when dealing with complex data dependencies. However, the embodiments of this disclosure can achieve intelligent ETL task orchestration and scheduling optimization. Therefore, in some embodiments, intelligent ETL task orchestration and scheduling optimization can also be achieved based on the understanding of task dependencies using a large language model. The system can automatically analyze data dependencies between tasks, optimize the execution order, and handle exception retry logic. Task scheduling efficiency is improved by 45%, and data processing latency is reduced by 30%.

[0053] In some embodiments, the multidimensional data attribution analysis method provided in this disclosure can also be based on a hybrid architecture of the self-developed FabricEngine computing engine and the GP engine, supporting dynamic adjustment of pipeline parallelism. During peak sales periods, it can handle 50,000+ query requests per second, with complex attribution analysis response time controlled within 3 seconds, meeting enterprise-level real-time analysis needs. Furthermore, it can achieve end-to-end security assurance from data access, storage, computation to display, supporting functions such as OSS data backup, operation log auditing, and sensitive data anonymization. Data security compliance reaches enterprise-level standards, providing reliable assurance for large-scale commercial applications.

[0054] According to a second aspect of the embodiments of this disclosure, a multidimensional data attribution analysis device based on a large language model is provided. Please refer to the appendix. Figure 2 The multidimensional data attribution analysis device 200 based on a large language model includes: The data acquisition module 201 is used to respond to the natural language description input by the user, perform intent recognition on the natural language description through the first language model to obtain the user intent, and, if the user intent represents the attribution analysis intent, acquire target data for attribution analysis based on the natural language description. The dimension determination module 202 is used to determine the initial analysis dimension for attribution analysis based on the data features of the target data and the user intent through the first large language model, and to determine the influence probability of the initial analysis dimension based on at least the target data and historical data through the first prediction model, and to filter the target analysis dimension whose influence probability is higher than a preset probability threshold from the initial analysis dimension. The business forecasting module 203 is used to determine key business indicators based on the natural language description using the first large language model, extract the actual indicator values ​​corresponding to the key business indicators from the target data, and perform business forecasting based on the target data and the target analysis dimension using the second forecasting model to obtain the predicted indicator values. The second forecasting model is used to predict the indicator values ​​corresponding to the key business indicators after adjusting the target data based on the target analysis dimension. The attribution analysis module 204 is used to perform attribution analysis based on the actual and predicted indicator values ​​corresponding to the target analysis dimension using the first large language model, and obtain the attribution analysis results.

[0055] In some embodiments, the dimension determination module 202 is specifically used for: The first entity is matched in the business knowledge graph based on the user intent using the first large language model, and the preset analysis dimension corresponding to the first entity is used as the first analysis dimension. Based on the data characteristics of the target data, the second analysis dimension is determined. The business knowledge graph is used to store the analysis dimension entities for attribution analysis of business problems. The first and second analysis dimensions are combined to form the initial analysis dimensions for attribution analysis.

[0056] In some embodiments, the dimension determination module 202 is specifically used for: The first key information is extracted from the natural language description using the first large language model, and the business scenario corresponding to the target data is determined based on the first key information. Based on the business scenario, the first large language model generates target data quality rules, and the target data is subjected to quality detection based on the data quality rules to obtain quality detection results. If the quality inspection results indicate that the target data has no data quality issues, the first large language model determines the initial analysis dimensions for attribution analysis based on the data features of the target data and the user intent.

[0057] In some embodiments, the business forecasting module 203 is specifically used for: Extract actual feature values ​​that can affect the key business indicators from the target data; Replace the target feature value corresponding to the target analysis dimension in the actual feature value with a preset simulated feature value to obtain new feature data; The second prediction model performs business predictions based on the new feature data to obtain the predicted indicator values ​​corresponding to the key business indicators.

[0058] In some embodiments, the attribution analysis module 204 is specifically used for: The first large language model determines the target difference between the actual and predicted indicator values ​​corresponding to the target analysis dimension. Based on the target difference, attribution weights are assigned to the target analysis dimension. Attribution analysis results are generated based on the attribution weights corresponding to each target analysis dimension. The target difference corresponding to each target analysis dimension is proportional to the attribution weight.

[0059] In some embodiments, the data acquisition module 201 is specifically used for: The second key information is extracted from the natural language description using the first large language model; The target SQL statement is obtained by using the second major language model based on the second key information, and the constraints in natural language form are extracted from the target SQL statement. The second key information is converted into a first text vector, and the constraint condition is converted into a second text vector. The vector similarity between the first text vector and the second text vector is calculated. If the vector similarity is greater than a first preset similarity threshold, target data for attribution analysis is obtained based on the target SQL statement.

[0060] In some embodiments, the data acquisition module 201 is specifically used for: An initial SQL statement is generated based on the second key information using the second language model, and data feature analysis is performed on the initial SQL statement to obtain feature analysis results. The initial SQL statement is then rewritten based on the feature analysis results to obtain the target SQL statement. The feature analysis results include the SQL fields to be optimized in the initial SQL statement and optimization suggestions for the SQL fields.

[0061] In some embodiments, the data acquisition module 201 is specifically used for: Initial data for attribution analysis is obtained from multiple data sources based on the natural language description described above; For the unstructured data in the initial data, the unstructured data is parsed into a unified data format through the third language model to obtain the first data. For the structured data in the initial data, the structured data is mapped into a unified data format through the field mapping engine to obtain the second data. The first data and the second data are combined into target data for attribution analysis.

[0062] In some embodiments, the multidimensional data attribution analysis device based on a large language model further includes a chart module, which is used to recommend visual chart types to the user based on the user intent and the attribution analysis results through the first large language model; and to display the attribution analysis results based on the target chart type in response to the user's trigger operation on the target chart type.

[0063] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments of the method in the first aspect, and will not be elaborated upon here.

[0064] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0065] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0066] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A multidimensional data attribution analysis method based on a large language model, characterized in that, include: In response to the natural language description input by the user, the user intent is obtained by performing intent recognition on the natural language description through the first major language model, and if the user intent represents the attribution analysis intent, the target data for attribution analysis is obtained based on the natural language description. Based on the data features of the target data and the user intent, the first large language model determines the initial analysis dimension for attribution analysis. Specifically, the first large language model matches a first entity in the business knowledge graph based on the user intent, and the preset analysis dimension corresponding to the first entity is used as the first analysis dimension. A second analysis dimension is determined based on the data features of the target data. The business knowledge graph stores the analysis dimension entities for attribution analysis of business issues. The first and second analysis dimensions are combined to form the initial analysis dimension for attribution analysis. The first prediction model determines the impact probability of the initial analysis dimension based at least on the target data and historical data, and then filters the target analysis dimension whose impact probability is higher than a preset probability threshold from the initial analysis dimension. The first large language model determines key business indicators based on the natural language description, and extracts the actual indicator values ​​corresponding to the key business indicators from the target data. The second prediction model performs business prediction based on the target data and the target analysis dimension to obtain the predicted indicator values. The second prediction model is used to predict the indicator values ​​corresponding to the key business indicators after adjusting the target data based on the target analysis dimension. Attribution analysis is performed using the first language model based on the actual and predicted indicator values ​​corresponding to the target analysis dimension to obtain the attribution analysis results. The step of performing business prediction based on the target data and the target analysis dimension using a second prediction model to obtain the predicted indicator values ​​corresponding to the key business indicators includes: Extract actual feature values ​​that can affect the key business indicators from the target data; Replace the target feature value corresponding to the target analysis dimension in the actual feature value with a preset simulated feature value to obtain new feature data; The second prediction model performs business predictions based on the new feature data to obtain the predicted indicator values ​​corresponding to the key business indicators.

2. The multidimensional data attribution analysis method based on a large language model according to claim 1, characterized in that, The step of determining the initial analysis dimensions for attribution analysis based on the data features of the target data and the user intent using the first large language model includes: The first key information is extracted from the natural language description using the first large language model, and the business scenario corresponding to the target data is determined based on the first key information. Based on the business scenario, the first large language model generates target data quality rules, and the target data is subjected to quality detection based on the data quality rules to obtain quality detection results. If the quality inspection results indicate that the target data has no data quality issues, the first large language model determines the initial analysis dimensions for attribution analysis based on the data features of the target data and the user intent.

3. The multidimensional data attribution analysis method based on a large language model according to claim 1, characterized in that, The attribution analysis, performed using the first large language model based on the actual and predicted indicator values ​​corresponding to the target analysis dimension, yields the following results: The first large language model determines the target difference between the actual and predicted indicator values ​​corresponding to the target analysis dimension. Based on the target difference, attribution weights are assigned to the target analysis dimension. Attribution analysis results are generated based on the attribution weights corresponding to each target analysis dimension. The target difference corresponding to each target analysis dimension is proportional to the attribution weight.

4. The multidimensional data attribution analysis method based on a large language model according to any one of claims 1-3, characterized in that, The acquisition of target data for attribution analysis based on the natural language description includes: The second key information is extracted from the natural language description using the first large language model; The target SQL statement is obtained by using the second major language model based on the second key information, and the constraints in natural language form are extracted from the target SQL statement. The second key information is converted into a first text vector, and the constraint condition is converted into a second text vector. The vector similarity between the first text vector and the second text vector is calculated. If the vector similarity is greater than a first preset similarity threshold, target data for attribution analysis is obtained based on the target SQL statement.

5. The multidimensional data attribution analysis method based on a large language model according to claim 4, characterized in that, The process of obtaining the target SQL statement based on the second key information using the second major language model includes: An initial SQL statement is generated based on the second key information using the second language model, and data feature analysis is performed on the initial SQL statement to obtain feature analysis results. The initial SQL statement is then rewritten based on the feature analysis results to obtain the target SQL statement. The feature analysis results include the SQL fields to be optimized in the initial SQL statement and optimization suggestions for the SQL fields.

6. The multidimensional data attribution analysis method based on a large language model according to any one of claims 1-3, characterized in that, The acquisition of target data for attribution analysis based on the natural language description includes: Initial data for attribution analysis is obtained from multiple data sources based on the natural language description described above; For the unstructured data in the initial data, the unstructured data is parsed into a unified data format through the third language model to obtain the first data. For the structured data in the initial data, the structured data is mapped into a unified data format through the field mapping engine to obtain the second data. The first data and the second data are combined into target data for attribution analysis.

7. The multidimensional data attribution analysis method based on a large language model according to any one of claims 1-3, characterized in that, The method further includes: Based on the user intent and the attribution analysis results, the first language model recommends visualization chart types to the user. In response to the user's triggering action on the target chart type in the visualization chart type, the attribution analysis results are displayed based on the target chart type.

8. A multidimensional data attribution analysis device based on a large language model, characterized in that, include: The data acquisition module is used to respond to the natural language description input by the user, perform intent recognition on the natural language description through the first language model to obtain the user intent, and, if the user intent represents the attribution analysis intent, acquire the target data for attribution analysis based on the natural language description. The dimension determination module is used to determine initial analysis dimensions for attribution analysis based on the data features of the target data and the user intent using a first large language model; determine the influence probability of the initial analysis dimensions based on at least the target data and historical data using a first prediction model; filter target analysis dimensions whose influence probability is higher than a preset probability threshold from the initial analysis dimensions; match a first entity in a business knowledge graph based on the user intent using the first large language model; use a preset analysis dimension corresponding to the first entity as the first analysis dimension; determine a second analysis dimension based on the data features of the target data; wherein the business knowledge graph is used to store analysis dimension entities for attribution analysis of business issues; and combine the first analysis dimension and the second analysis dimension to form the initial analysis dimensions for attribution analysis. The business forecasting module is used to determine key business indicators based on the natural language description using the first large language model, extract the actual indicator values ​​corresponding to the key business indicators from the target data, and perform business forecasting based on the target data and the target analysis dimension using the second forecasting model to obtain the predicted indicator values. The second forecasting model is used to predict the indicator values ​​corresponding to the key business indicators after adjusting the target data based on the target analysis dimension. The attribution analysis module is used to perform attribution analysis based on the actual and predicted indicator values ​​corresponding to the target analysis dimension using the first large language model, and obtain the attribution analysis results. The business forecasting module is further used for: Extract actual feature values ​​that can affect the key business indicators from the target data; Replace the target feature value corresponding to the target analysis dimension in the actual feature value with a preset simulated feature value to obtain new feature data; The second prediction model performs business predictions based on the new feature data to obtain the predicted indicator values ​​corresponding to the key business indicators.

Images