A data analysis method and system for enterprise reporting

Abstract

The present application belongs to the technical field of data processing, and particularly relates to a data analysis method and system for enterprise report, comprising: obtaining a mode vector library and a co-occurrence weight matrix constructed based on historical records or table fields; converting a natural language query into a query vector, matching a candidate table set with a similar score sequence, and extracting query keywords; calculating a concentration index and an overlap rate index, in response to both indexes being out of limit, taking an intersection of an overlap field set composed of cross-table same-name fields and the query keywords to determine a target confusion field set; calculating a disambiguation score based on the co-occurrence weight matrix, correcting to obtain a rearrangement score sequence and screening target tables; and finally generating a query statement to obtain business data and rendering a visual report. The present application solves the query failure caused by cross-table field confusion, and improves the accuracy of enterprise report generation.

Description

Technical Field

[0001] This invention relates to the field of data processing technology. More specifically, this invention relates to a data analysis method and system for enterprise reports. Background Technology

[0002] As enterprises continue their digital transformation, data analysis and report generation are playing an increasingly crucial role in daily operations and strategic decision-making. Enterprises typically deploy large and complex business databases, covering multiple business modules such as procurement, sales, inventory, and finance. Business personnel lacking technical backgrounds often struggle to directly write professional SQL statements to retrieve data. Therefore, the industry has gradually developed natural language query technology, which automatically converts users' natural language queries into underlying database query commands and generates visual reports, thereby lowering the data access barrier and improving the overall operational efficiency and data value extraction capabilities of enterprises.

[0003] In real-world enterprise production environments, database tables for different business modules are often designed and maintained independently. Due to consistent business naming conventions, a large number of fields with the same name or ambiguous fields commonly exist in the database tables of different modules. For example, multiple business tables may contain common fields such as amount, date, and number, but these fields have completely different semantic meanings in different business contexts. This semantic overlap of cross-table fields presents unique challenges to the practical application of natural language to query technology and is a key bottleneck restricting the accuracy of automated report generation.

[0004] Chinese patent application CN116821168B discloses an improved NL2SQL method based on a generative large language model. The proposed technical solution includes preprocessing database table information to obtain a set of annotated text feature vectors and calculating the weights of each column; segmenting and expanding the word segmentation of natural language questions to generate text feature vectors; matching the target table by calculating the weighted cosine similarity of the table and columns; inputting the matched table structure information and the natural language question into a generative large language model; and using a thought chain approach to construct prompts and generate SQL query statements. This solution improves the applicability to environments with frequently updated database table structures.

[0005] This existing technology uses weighted cosine similarity matching based on table and column annotations to recall target tables, with column weights calculated solely based on the frequency of field occurrences across the entire database. In large-scale enterprise database scenarios, common fields with the same name appear frequently across business modules and are assigned low column weights, making it difficult for weighted similarity calculations to effectively distinguish the semantic attribution of these fields in different business scenarios. When a user query involves such common fields, vector retrieval simultaneously recalls multiple candidate data tables containing that field, and the matching scores of these candidate tables are often quite similar. Subsequently, all the recalled candidate table structure information is fully injected into a large language model to generate query statements. The large amount of redundant table structure information severely dilutes the attention weight of the large language model to field meaning, making it unable to accurately lock onto the target fields corresponding to the actual business intent. The final generated query statements frequently suffer from field mapping errors and field attribution errors, resulting in a high query execution failure rate and even returning completely incorrect business data, failing to meet the accuracy and reliability requirements of automated report generation in complex enterprise business scenarios. Summary of the Invention

[0006] To address the technical problem in large-scale enterprise database scenarios where interference from fields with the same name or multiple meanings across business modules leads to the retrieval of irrelevant tables during routine searches, resulting in diluted model attention, incorrect field mapping and attribution, and a high failure rate in query execution, this invention provides a data analysis method and system for enterprise reports.

[0007] In a first aspect, the present invention provides a data analysis method for enterprise reports, comprising: S1, acquiring a pattern vector library and a co-occurrence weight matrix of an enterprise database; the co-occurrence weight matrix is ​​constructed based on the co-occurrence features between fields in historically successfully executed query statements, or initialized based on the co-occurrence relationships of fields within the same data table when the number of historical query statements is less than a preset number; S2, converting the user's natural language query into a query vector; calculating the similarity between the query vector and each data table in the pattern vector library to obtain a candidate table set and a corresponding similarity score sequence; extracting entity features from the natural language query to obtain query keywords; S3, calculating a concentration index based on the similarity score sequence, and based on the candidate... In the table set, the overlap rate index is calculated for each table field. If both the concentration index and overlap rate index are greater than the concentration threshold, cross-table fields with the same name are extracted from the candidate table set to obtain the overlapping field set. The intersection of the overlapping field set and the query keywords is determined as the target obfuscation field set. In step S4, based on the co-occurrence weight matrix and the target obfuscation field set, the disambiguation score of each candidate table in the candidate table set is calculated. The similarity score sequence is corrected based on the disambiguation score to obtain a rearranged score sequence. The target table is selected from the candidate table set based on the rearranged score sequence. In step S5, a query statement is generated based on the target table's table structure information and natural language query. The query statement is executed to obtain business data, and a visualization report is rendered based on the business data.

[0008] By adopting the above technical solution, this invention obtains a candidate table set by acquiring a pattern vector library and a co-occurrence weight matrix, calculating the similarity between the query vector and each data table, and accurately extracting the target confusing field set when both the concentration index and the overlap rate index are greater than the corresponding thresholds. Then, the disambiguation score is calculated using the co-occurrence weight matrix, and the similarity score sequence is corrected accordingly to filter the target table. Finally, a query statement is generated based on the table structure information of the target table, and a visual report is rendered. This solution achieves automatic detection of ambiguity risk through algorithm-inherent dual indicators and deeply integrates historical co-occurrence features accumulated in the offline stage with vector retrieval results in the online stage, completing the numerical disambiguation and reordering score sequence calculation for ambiguous field scenarios. This allows the system to maintain a stable high query execution success rate even when facing high business complexity scenarios, thus solving the problem of high query failure rates caused by diluted model attention weights due to homonymous or ambiguous fields when processing large-scale enterprise databases, leading to field mapping errors and cross-table attribution errors.

[0009] Preferably, calculating the concentration index based on the similarity score sequence includes: calculating the average score and standard deviation of each similarity in the similarity score sequence; obtaining a stability constant and a similarity benchmark threshold, and using the sum of the standard deviation and the stability constant as the target denominator; obtaining the difference between the average score and the similarity benchmark threshold as the target numerator; and determining the ratio of the target numerator to the target denominator as the concentration index.

[0010] By adopting the above technical solution, the calculation process uses the statistical dispersion formula to characterize the degree of clustering of each similarity score in the candidate table set, and the introduction of this stability constant effectively prevents mathematical calculation anomalies caused by a zero denominator. A robust numerical evaluation index derived entirely from the algorithm's endogenous intermediate data is constructed, thereby solving the technical defect that a single vector matching score cannot intuitively reflect the semantic ambiguity of the retrieval results, and providing an accurate and objective judgment basis for the automatic triggering of the subsequent disambiguation mechanism.

[0011] Preferably, the overlap rate index is calculated based on the fields of each table in the candidate table set, including: obtaining the first field set and the second field set of any two candidate tables in the candidate table set; calculating the number of intersections and the number of unions of the first field set and the second field set, and determining the ratio of the number of intersections to the number of unions as the local overlap rate of the corresponding two candidate tables; calculating the average value of the local overlap rates corresponding to all candidate table combinations in the candidate table set, and determining the average value as the overlap rate index.

[0012] Preferably, the disambiguation score satisfies the following relationship:

[0013] in, Indicates the first in the candidate table set Disambiguation score of candidate list; This indicates the index number of the candidate table in the candidate table set; Represents the first in the candidate table set Zhang Candidate List; Indicates the target obfuscated field set; Indicates the total number of field elements contained in the target obfuscation field set; Represents the fields in the target obfuscation field set; Indicates the first The set of table fields for the candidate table; This represents the fields in the table's field set; Indicates the weight of historical co-occurrence; Indicates the target obfuscation field set. The initial response score.

[0014] By employing the above technical solution, this invention uses an aggregation mechanism that first extracts the maximum value and then calculates the average when calculating the disambiguation score. First, the maximum value of the historical co-occurrence weights of the table field set with the target obfuscated field set is determined as the initial response score. This step accurately identifies the core fields relevant to the business and directly filters out data noise caused by a large number of irrelevant fields in the candidate table. Then, the average of each initial response score is calculated, and the disambiguation score of a single candidate table is obtained through aggregation. This mechanism measures the comprehensive fit between the candidate table and the target obfuscated field set without noise at the physical level, overcoming the semantic bias caused by relying solely on vector retrieval, and ensuring that the target table, which conforms to the enterprise's actual business logic, can stand out in the re-ranked score sequence.

[0015] Preferably, the rearrangement scores of each candidate table in the rearrangement score sequence satisfy the following relationship:

[0016] in, Indicates the first in the candidate table set The reordering score of the candidate list; This indicates the index number of the candidate table in the candidate table set; Represents the first in the candidate table set Zhang Candidate List; Indicates the first similarity score sequence The similarity between the candidate lists; This represents the gain coefficient corresponding to the business intent; 1 represents the disambiguation score; 1 represents the reference constant used to ensure the numerical reference.

[0017] By adopting the above technical solution, this invention uses the business intent of natural language queries as a moderating variable to achieve adaptive matching between the disambiguation gain strength and the obfuscation characteristics of fields in each business module, effectively avoiding excessive intervention while ensuring the disambiguation effect. It solves the technical defect of using a single, unchanging constant coefficient globally, which leads to erroneous over-correction of some low-ambiguity business modules, thereby improving the overall accuracy of query statement generation.

[0018] Preferably, obtaining the gain coefficient corresponding to the business intent includes: obtaining sample query statements that belong to the business intent and have been successfully executed in the historical records; generating multiple candidate coefficients with a fixed step size within a preset numerical range; searching among the multiple candidate coefficients with the goal of maximizing the execution success rate of the sample query statements, and determining the candidate coefficient that maximizes the execution success rate as the gain coefficient corresponding to the business intent.

[0019] Preferably, the historical co-occurrence weights satisfy the following relationship:

[0020] in, Indicates the weight of historical co-occurrence; This represents the fields in the table's field set; Represents the fields in the target obfuscation field set; This indicates the number of times a field in the table's field set and a field in the target obfuscation field set co-occur together in historical query statements; This represents the total number of times a field in the target obfuscation field set appears in all successfully executed historical query statements. This represents the smoothing constant.

[0021] By adopting the above technical solution, the present invention can accurately characterize and evaluate the degree of closeness between the two fields to be evaluated in the same business scenario in the enterprise's real business logic. Ultimately, it solves the technical pain point of disambiguation of ambiguous fields due to the lack of objective numerical basis from the data source, and provides solid data support for subsequent high-precision target table screening based on rearranged score sequences.

[0022] Preferably, when constructing the co-occurrence weight matrix, the method further includes: in response to the total number of historical query statements being less than a preset number, extracting multiple internal fields from the same data table in the enterprise database; treating any two internal fields in the same data table as homologous co-occurrence pairs, setting the initial value of the corresponding co-occurrence count to 1, and generating the initial co-occurrence weight matrix by replacing the historical query statements based on the initial value.

[0023] Preferably, extracting entity features from a natural language query to obtain query keywords includes: performing feature extraction on the natural language query to obtain a set of Chinese characters; mapping each Chinese character in the set of Chinese characters to a corresponding English field name in a pre-constructed bidirectional index table, and determining the set of English field names as the query keywords.

[0024] Secondly, the present invention provides a data analysis system for enterprise reports, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned data analysis method for enterprise reports is implemented.

[0025] By adopting the above technical solution, a data analysis method for enterprise reports is generated into a computer program and stored in a memory for loading and execution by a processor. This allows for the creation of a terminal device based on the memory and processor, facilitating its use.

[0026] The technical solution of the present invention has the following beneficial technical effects: This invention achieves automatic identification and index-based disambiguation of ambiguity risks in polysemous fields by constructing a pattern vector library and a co-occurrence weight matrix, and introducing a dual risk detection mechanism based on concentration and overlap indicators. By deeply fusing historical co-occurrence features accumulated in the offline phase with vector retrieval results in the online phase, candidate tables are disambiguated and re-ranked, solving the problem of diluted attention weights in large language models caused by fields with the same name in large-scale enterprise databases. This avoids field mapping errors and cross-table attribution errors, improves the accuracy of automated report generation, and ensures that the system maintains a stable high query execution success rate even when facing scenarios with high business complexity.

[0027] Furthermore, this invention achieves continuous evolution of system analysis performance by introducing a business intent gain coefficient and a historical query feedback closed-loop mechanism. The adaptive optimization process of the gain coefficient ensures that the disambiguation gain strength dynamically and accurately matches the field confusion characteristics of different business modules, avoiding misjudgments caused by over-correction. Simultaneously, the system constructs a self-enhancing data evolution mechanism by periodically refreshing successfully executed historical query statements to the co-occurrence weight matrix, causing disambiguation accuracy to continuously converge and strengthen with the continued use of business tools. This mechanism not only solves the cold start pain point caused by insufficient historical data accumulation in the early stages of system deployment but also lowers the barrier for business personnel lacking technical backgrounds to access data, improving the overall operational efficiency of the enterprise. Attached Figure Description

[0028] Figure 1 This is a flowchart of a data analysis method for enterprise reports in this invention; Figure 2 This is a comparison chart of the query execution success rates of the embodiments of the present invention and traditional solutions under different business complexities; Figure 3 This is a trend graph of the field attribution error rate as a function of system runtime in an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0030] The application scenario of this embodiment is based on an enterprise database consisting of multiple business data tables. This enterprise database covers multiple business modules such as procurement, sales, inventory, and finance. Furthermore, business personnel can input natural language queries, such as calculating the total sales amount for each region in the previous quarter and sorting by amount in descending order. The analysis method automatically generates corresponding query statements and renders visual reports. Since multiple business modules in the enterprise database contain fields with the same name, conventional direct retrieval methods are prone to errors in field attribution. This embodiment of the invention discloses a data analysis method for enterprise reports, referring to… Figure 1 This includes steps S1-S5: S1: Obtain the pattern vector library and co-occurrence weight matrix.

[0031] In one embodiment, in order to provide a logical basis for accurate semantic retrieval and disambiguation correction of polysemous fields in the process of automatically generating internal enterprise data reports, it is necessary to pre-build a data structure that supports the underlying logic before the online response. This structure includes a pattern vector library for semantic matching and a co-occurrence weight matrix that contains business logic associations.

[0032] To extract semantic features from database table structures, this invention first extracts the pattern description text for each table in an enterprise database offline. Next, a text embedding model is used to perform a high-dimensional mapping transformation on the pattern description text, thereby mapping discrete text character information into a 1024-dimensional continuous dense vector. Subsequently, the continuous dense vector and the corresponding original set of field names are stored in a vector database, thus constructing a pattern vector library.

[0033] To address the computational needs of business relationship logic between fields, this invention further introduces a co-occurrence weight matrix. During the construction of the co-occurrence weight matrix, considering that incorrect query logic may lead to incorrect table structure relationships, to avoid data pollution of the co-occurrence weight matrix by these invalid relationships at the source, only successfully executed historical query statements are selected for calculation when retrieving historical records. This step extracts these successfully executed historical query statements as data basis, aiming to characterize and measure the closeness of the two fields to be evaluated belonging to the same business scenario in the enterprise's real business logic. In the specific calculation process, the co-occurrence frequency of the two fields to be evaluated in these successfully executed historical query statements is counted, as well as the total frequency of the target field in all successfully executed historical query statements. Subsequently, a smoothing constant is obtained, and the sum of the aforementioned total frequency and the smoothing constant is used as the denominator base. Finally, by calculating the ratio of the aforementioned co-occurrence frequency to the denominator base, the historical co-occurrence weight between the two fields to be evaluated is determined.

[0034] Specifically, the historical co-occurrence weights satisfy the following relation:

[0035] in, Indicates the weight of historical co-occurrence; and For any two table fields participating in the evaluation during offline computation, in the online disambiguation scenario of subsequent steps in this invention, Specifically, these correspond to the fields in the table's field set. Specifically, these correspond to the fields in the target obfuscation field set; This indicates the number of times a field in the table's field set and a field in the target obfuscation field set co-occur together in historical query statements; This represents the total number of times a field in the target obfuscation field set appears in all successfully executed historical query statements. This represents the smoothing constant, with a preferred reference value of 1, used to prevent mathematical anomalies where the denominator is zero when the total number of terms is zero.

[0036] Understandably, a higher historical co-occurrence weight indicates that the two fields being evaluated appear more frequently in the same historical query, meaning that they are more likely to belong to the same business scenario in the enterprise's actual business logic. For example, the sales region field almost always co-occurs with the amount field in the order details table, but usually does not co-occur with the amount field in the inventory table. This objective rule directly reflects the true business semantic attribution. The constructed co-occurrence weight matrix is ​​stored in a cache database in the form of key-value pairs and is periodically refreshed as historical data accumulates.

[0037] In another embodiment, considering the potential cold start pain point of insufficient successful records during initial deployment, if the total number of query statements in the historical records is less than a preset number, the present invention switches to an initialization strategy. In this case, multiple internal fields from the same data table in the enterprise database are extracted; then, any two internal fields from the same data table are considered as co-occurring pairs, and the initial value of the corresponding co-occurrence count is set to 1. Finally, the missing historical query records are replaced based on the initial value, thereby generating an initial co-occurrence weight matrix.

[0038] The preferred reference value for the preset number is fifty records. When the business is launched for the first time and the number of successful execution records in the historical records is less than the preset number, a cold start strategy will be forcibly implemented. The engineering and physical significance of this cold start strategy is that internal fields located in the same data table naturally belong to the same business context, thus possessing the most basic co-occurrence prior attribute. By assigning an initial value of 1 to co-occurring pairs, it can be ensured that this method still has basic field association judgment capabilities during the cold start phase when historical data is lacking. As successful historical query statements are continuously accumulated during subsequent use, when the accumulated number of statements exceeds the preset number, it will automatically and seamlessly switch to a calculation chain based on real statistical values. At that time, the prior weights set in the initial stage will be gradually diluted and covered by the real business occurrence patterns that evolve over time.

[0039] Thus, this step completes the construction of the pattern vector library and co-occurrence weight matrix through the offline preprocessing stage. This process transforms the discrete database table structure into high-dimensional features that can be quickly calculated by machines. At the same time, it solidifies the historical successful query experience of enterprise personnel into an objective and numerical co-occurrence weight matrix, providing solid data support and historical basis for the subsequent online stage of rapid semantic retrieval and accurate disambiguation of cross-table fields with the same name.

[0040] S2: Retrieve the candidate table set and extract the query keywords.

[0041] In one embodiment, after the offline data structure is pre-built, when business personnel initiate a data query request online, in order to quickly narrow down the search scope and anchor the core business object in the enterprise's large and complex database, semantic matching and feature extraction operations must be performed in real time.

[0042] Specifically, this invention first acquires the user's natural language query, then uses a text embedding model to encode the natural language query into a continuous dense vector with the same dimensions as the pattern vector library, serving as the query vector. Subsequently, in the pattern vector library, the standard cosine similarity formula is used to calculate the cosine similarity between the query vector and the corresponding pattern vectors of all data tables. All calculated cosine similarities are sorted in descending order of score, and a top-ranked set of data tables is selected to form a candidate table set. The cosine similarity scores corresponding to these data tables are then used to construct a similarity score sequence. A preferred reference value for the number of data tables selected is five, which can be dynamically adjusted by technicians within the range of three to eight tables depending on the specific size of the enterprise database. To ensure fast online response, all extracted data is processed in memory and not written to disk.

[0043] While performing the aforementioned vector retrieval task, considering that ambiguous natural language expressions may not be directly aligned with the underlying database physical model, this invention generates query keywords through a feature extraction process to accurately anchor specific business data columns. First, using well-known natural language processing techniques such as named entity recognition, a set of Chinese characters explicitly related to business fields is extracted from the natural language query. For example, when the natural language query is to calculate sales revenue for each region, the extracted set of Chinese characters contains the core terms "region" and "sales revenue."

[0044] Because the table structures of underlying enterprise databases typically use English field names, directly using Chinese character sets cannot participate in subsequent set operations. To bridge this structural gap in data format, this invention utilizes a pre-built offline bidirectional index table to accurately translate and map each Chinese character in the extracted Chinese character set to its corresponding English field name. Finally, the set composed of these English field names is determined as the query keywords.

[0045] Thus, this step, through online vectorization and standard similarity calculation, rapidly compresses the retrieval scope of the target table from hundreds of tables across the entire database to a few semantically most relevant candidate tables, reducing the context length for subsequent language model processing. Simultaneously, by employing entity feature extraction and normalized mapping in tandem, it outputs similarity score sequences and query keywords, providing indispensable key input data for subsequent accurate evaluation of recall quality and implementation of polysemous field disambiguation.

[0046] S3: Calculate the dual indicators and determine the target obfuscated field set.

[0047] In one embodiment, after obtaining the candidate table set and the corresponding similar score sequence, in order to accurately assess whether there is a risk of confusion with ambiguous fields in the current recall results, and thus decide whether to trigger the subsequent disambiguation and re-ranking process, it is necessary to automatically detect the current degree of ambiguity from the algorithm's endogenous data.

[0048] To analyze the dispersion and clustering of similarity scores in each data table and filter out spurious clustering interference caused by full recall of low-relevance tables, we first calculate the average score and standard deviation of each similarity in the similarity score sequence; then, we obtain the stability constant and the similarity benchmark threshold, and use the sum of the standard deviation and the stability constant as the target denominator; finally, we obtain the difference between the average score and the similarity benchmark threshold as the target numerator, and the ratio of the aforementioned target numerator to the target denominator is determined as the concentration index.

[0049] Specifically, the concentration index satisfies the following relationship:

[0050] in, Indicators of concentration; This represents the average score; This represents the similarity baseline threshold, used to intercept irrelevant searches with low overall similarity for recall purposes. Indicates standard deviation; This represents the stability constant, and its preferred reference value is... , is a dimensionless parameter used to prevent mathematical calculation anomalies where the target denominator is zero when the standard deviation is zero.

[0051] Furthermore, considering that analyzing only the dispersion of similar score sequences is insufficient to fully reflect the field overlap of each data table at the physical structure level, this step also introduces an overlap rate index to examine field overlap from the physical table structure level. The calculation process of the overlap rate index is as follows: First, obtain the first field set and the second field set of any two candidate tables in the candidate table set. To avoid diluting the true business overlap due to the excessive number of common audit fields such as system primary keys or timestamps, common non-business fields are pre-removed when obtaining the first field set and the second field set. Next, calculate the number of intersections and the number of unions of the first field set and the second field set after removing interfering fields, and determine the ratio of the number of intersections to the number of unions as the local overlap rate of the corresponding two candidate tables. Finally, the overlap rate index is obtained by averaging the local overlap rates corresponding to all combinations of candidate tables in the candidate table set.

[0052] Specifically, the overlap rate index satisfies the following relationship:

[0053] in, This refers to the overlap rate metric, which measures the degree of structural overlap between fields across tables from a global perspective. and Both are index numbers of the candidate tables in the candidate table set, and ; This indicates the first [field] after removing general non-business fields. The set of fields in the candidate table; This indicates the first [field] after removing general non-business fields. The set of fields in the candidate table; This indicates the total number of candidate tables in the candidate table set; This represents the number of intersections between the first and second field sets after removing general non-business fields; This represents the number of elements in the union of the first and second field sets after removing general non-business fields.

[0054] After obtaining the above two indicators, considering that a single judgment condition may lead to misjudgment—for example, if the concentration index is high but the overlap rate index is low, it means that the scores are close but the fields do not overlap, which is a normal multi-table join query—this method adopts a dual threshold verification mechanism. Only when both the concentration index and the overlap rate index are greater than the concentration threshold and the overlap rate index are greater than the overlap rate threshold, is it determined that there is a substantial risk of ambiguous field confusion in the current environment.

[0055] For example, the concentration threshold can be the median of the concentration index sequence obtained from historical queries, and a fixed initial value of 3 is used as the benchmark when there are fewer than 100 historical records. The preferred reference value for the overlap rate threshold can be set to 0.1 to ensure extreme sensitivity to the risk of confusion for a small number of core business fields with the same name. Once the above high confusion risk judgment is triggered, this step immediately traverses and summarizes the cross-table fields with the same name between each candidate table, and forms an overlapping field set after merging and deduplication. Since not all fields with the same name will interfere with the current query, in order to accurately identify the target confusion field set that is clearly involved in the current user query from the structural conflicts, the overlapping field set is further subjected to a set intersection operation with the query keywords extracted from the previous step. In the result of the intersection operation, the extracted elements are the target confusion field set that is clearly involved in the user query and exists in multiple candidate tables. These elements within the intersection are strictly determined as the target confusion field set and used as the calculation input for the subsequent disambiguation step.

[0056] Thus, this step achieves automatic detection of disambiguation timing without any external manual annotation through two objective indicators derived entirely from the algorithm's endogenous intermediate data. This ensures accurate identification of the target obfuscated field set and avoids unnecessary intervention and wasted computing power for unambiguous queries.

[0057] S4: Calculate the disambiguation and rearrangement score and filter the target table.

[0058] In one embodiment, after determining that the current recall results have the risk of ambiguous field confusion and extracting the target confusion field set, considering that relying solely on vector retrieval may result in semantic bias, in order to make the data table that truly conforms to the enterprise's business logic stand out, this step needs to introduce historical business experience to perform disambiguation calculation and re-ranking on the candidate table.

[0059] To scientifically score the current candidate table by leveraging the implicit business semantic attribution patterns in historical queries, the table field set of any candidate table is first obtained. Then, in the co-occurrence weight matrix, the historical co-occurrence weights of each field in the table field set and each field in the target obfuscation field set are queried. Subsequently, the maximum value of each historical co-occurrence weight is determined as the initial response score of the corresponding field in the target obfuscation field set corresponding to the candidate table. Finally, the average value of the initial response scores of each field in the target obfuscation field set corresponding to the candidate table is calculated, and the aforementioned average value is determined as the disambiguation score of the candidate table.

[0060] Specifically, the disambiguation score satisfies the following relation:

[0061] in, Indicates the first in the candidate table set Disambiguation score of candidate list; This indicates the index number of the candidate table in the candidate table set; Represents the first in the candidate table set Zhang Candidate List; Indicates the target obfuscated field set; Indicates the total number of field elements contained in the target obfuscation field set; Represents the fields in the target obfuscation field set; Indicates the first Zhang Candidate List The set of table fields; This represents the fields in the table's field set; Indicates the weight of historical co-occurrence; Indicates the target obfuscation field set. The initial response score.

[0062] Through the above derivation process, the internal field with the highest historical co-occurrence weight with each field in the target obfuscated field set is identified among all fields in the candidate table. The highest weight is then taken as the optimal response of the candidate table to the field, i.e., the initial response score. By averaging the initial response scores of all field elements, the high-dimensional field-level matching scores are aggregated and collapsed into a comprehensive disambiguation score for a single candidate table. Its engineering and physical significance lies in evaluating the comprehensive fit between the candidate table and the target obfuscated field set in historical real-world business scenarios. The higher the disambiguation score, the more frequently the candidate table is used in conjunction with the fields queried by the user in historical records, and the clearer its business semantic attribution. For example, if the target obfuscated field set includes amount, and the order details table shows frequent historical co-occurrence of amount and region, while the purchase table shows amount mostly co-occurring with supplier, then the disambiguation score of the order details table will be significantly higher than that of the purchase table, thus correctly identifying it as the target table in subsequent steps.

[0063] In another embodiment, to smoothly integrate the calculated disambiguation score into the preceding vector retrieval framework, this step modifies the aforementioned similarity score sequence based on the disambiguation score to obtain a rearranged score sequence. First, the business intent of the natural language query is identified and the gain coefficient corresponding to the business intent is obtained; then, the product of the disambiguation score and the gain coefficient corresponding to any candidate table is added to a baseline constant of 1 to obtain the correction multiplier for the candidate table; subsequently, the similarity corresponding to the candidate table is multiplied by the correction multiplier to calculate the rearranged score of the candidate table; finally, the rearranged scores of each candidate table are summarized to obtain the rearranged score sequence.

[0064] It should be noted that, in order to achieve adaptive dynamic optimization of the gain coefficient, the following steps are taken: first, sample query statements that belong to the business intent and have been successfully executed in the historical records are obtained; then, multiple candidate coefficients are generated within a preset numerical range with a fixed step size; finally, with the optimization objective of maximizing the execution success rate of the sample query statements, the search is conducted among the multiple candidate coefficients, and the candidate coefficient with the highest execution success rate is determined as the gain coefficient corresponding to the business intent.

[0065] Specifically, the rearranged scores satisfy the following relation:

[0066] in, Indicates the first in the candidate table set The reordering score of the candidate list; This indicates the index number of the candidate table in the candidate table set; Represents the first in the candidate table set Zhang Candidate List; Indicating the first similarity score in the aforementioned sequence The similarity between the candidate lists; This represents the gain coefficient corresponding to the business intent; 1 represents the disambiguation score; 1 represents the baseline constant used to ensure that the rearrangement score is not lower than the similarity score before correction. This represents the correction multiplier for the candidate table.

[0067] Understandably, compared to using a single, unchanging constant coefficient globally, using the business intent of the natural language query as a moderating variable allows the disambiguation gain strength to adaptively match the field obfuscation characteristics of each business module. For example, this step can use a grid search algorithm to automatically optimize within a preset value range of 0.5 to 3 with a fixed step size of 0.1. Under this mechanism, business modules with severe field overlap and a high density of ambiguous fields, such as financial intents, will receive stronger historical disambiguation correction through automatic optimization, while modules with already high field discriminativeness will receive weaker correction, thus avoiding excessive intervention and improving the overall query generation accuracy without manual parameter tuning. After calculating all rearrangement scores, the candidate tables are sorted in descending order according to the rearrangement score sequence, and the top-ranked data table is selected as the target table. A maximum number of target tables can also be set; if the maximum number is 3, the top 3 data tables are selected as the target tables.

[0068] In this way, this step organically integrates offline historical field co-occurrence knowledge, online vector retrieval similarity results, and overlapping field location results. Without introducing additional external labeled data, it completes the accurate disambiguation and reordering of candidate tables in the case of ambiguous fields, solves the field attribution error caused by fields with the same name, and lays a solid data foundation for the accurate generation of subsequent query statements.

[0069] S5: Generate query statements to retrieve data and render visual reports.

[0070] In one embodiment, after disambiguating the ambiguous fields and accurately filtering out the target table, in order to ultimately respond to the query needs of business personnel, the ambiguous natural language must be converted into database executable instructions and the final statistical analysis results must be presented in an intuitive form.

[0071] Specifically, the system first obtains the table structure information of the target tables selected in the previous steps. It should be noted that the target table can be a single data table or multiple data tables selected based on the rearranged score sequence; when there are multiple target tables, the system reads the relationship fields between these tables from the metadata of the enterprise database.

[0072] Next, the table structure information, such as the field names, field types, and Chinese field comments of the target table, the related fields of each target table, and the user's natural language query are assembled into a structured prompt text, which is then input into a large language model to generate an SQL query statement containing the correct JOIN clause. Subsequently, the query statement is submitted to the enterprise database for execution to obtain business data.

[0073] Furthermore, to ensure that the historical statistical patterns relied upon in the preliminary steps can continuously evolve and form a self-reinforcing data flywheel as the business develops, the following feedback process is included after the analysis method renders visual reports based on business data. In response to the condition that a query statement is executed successfully, the natural language query, successfully executed historical query statements, and execution success status are written to a history record table. Subsequently, by periodically extracting newly added successfully executed historical query statements from the history record table, the co-occurrence relationships between fields are recalculated, and the co-occurrence weight matrix is ​​refreshed.

[0074] It's important to note that the historical record table serves as the underlying storage medium for accumulating real-world query experience within the enterprise. Writing successful execution statuses along with the natural language query and query statement ensures, from the source, that only correctly validated retrieval logic, can participate in subsequent statistical calculations, thus preventing erroneous field relationships from contaminating the underlying features. The optimal time period for periodic extraction is 24 hours, meaning a refresh mechanism is triggered daily during off-peak business hours. As enterprise personnel continuously perform data queries, the number of successful historical query samples increases, enriching the business relationship features contained in the historical record table, thereby continuously improving the statistical confidence of the co-occurrence weight matrix from previous steps.

[0075] In this way, while completing the underlying data query and upper-level report rendering and directly meeting the business needs of enterprise personnel, this step seamlessly archives the successful interaction results into the history, providing a continuous source of real business data for the offline co-occurrence weight matrix to be continuously refreshed. This mechanism ensures that the overall data analysis solution forms a complete and highly automated data flow loop, allowing the disambiguation accuracy of ambiguous fields to continuously converge and strengthen with the continuous use of business tools, thereby improving the practicality and engineering value of automated report generation.

[0076] Figure 2This chart compares the query execution success rates of the embodiments of the present invention and traditional solutions under different levels of business complexity. In an enterprise database environment, as the complexity of business logic increases, the interference from fields with the same name and ambiguous fields within the database intensifies. Observing the bar distribution in the chart, it can be seen that the bars filled with intersecting diagonal lines represent traditional vector retrieval solutions, while the bars filled with solid colors represent the disambiguation solution of the present invention. In scenarios with low business complexity, the difference in query execution success rates between the two solutions is small. However, when the business complexity increases to a high level, the query execution success rate of traditional vector retrieval solutions drops significantly because they cannot effectively identify semantic confusion caused by overlapping physical structures. In contrast, the disambiguation solution of the present invention, by introducing risk detection based on concentration and overlap rate indicators and combining disambiguation reordering with a co-occurrence weight matrix, can still maintain a query execution success rate of over 90% even with high business complexity. This comparison result strongly verifies that the technical means provided by the present invention has higher accuracy and stability when handling complex business logic.

[0077] Figure 3 This paper presents a trend graph showing the field assignment error rate over system runtime in an embodiment of the present invention. The graph records the field assignment error rate trend from the initial stage of system operation to the twelfth week. Observing the fluctuating curve trajectory, it can be seen that in the first to third weeks of system operation, due to limited historical data accumulation and the co-occurrence weight matrix not yet undergoing sufficient iteration, the field assignment error rate is at a relatively high level. As the system runtime progresses, the present invention, through a feedback closed-loop mechanism, continuously archives successful interaction results and periodically refreshes the co-occurrence weight matrix. This continuous evolution process steadily improves disambiguation accuracy, and the field assignment error rate shows a significant downward trend, finally stabilizing at a low level after the eighth week of operation. This result demonstrates that the technical solution of the present invention possesses adaptive learning and continuous reinforcement capabilities, and can continuously converge the error rate with the use of business tools, achieving extremely high engineering practical value.

[0078] This invention also discloses a data analysis system for enterprise reports, including a processor and a memory. The memory stores computer program instructions, which, when executed by the processor, implement a data analysis method for enterprise reports according to this invention.

[0079] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

[0080] It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the scope of protection of this invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A data analysis method for enterprise reports, characterized in that, include: S1, obtain the pattern vector library and co-occurrence weight matrix of the enterprise database; The co-occurrence weight matrix is ​​constructed based on the co-occurrence features between fields in historically successfully executed query statements, or initialized based on the co-occurrence relationship of fields within the same data table when the number of historical query statements is less than a preset number; S2, the user's natural language query is converted into a query vector; the similarity between the query vector and each data table is calculated in the pattern vector library to obtain a candidate table set and the corresponding similarity score sequence; Extract entity features from natural language queries to obtain query keywords; S3, calculate the concentration index based on the similarity score sequence, and calculate the overlap rate index based on the fields of each table in the candidate table set; In response to the concentration index being greater than the concentration threshold and the overlap rate index being greater than the overlap rate threshold, cross-table same-name fields are extracted from the candidate table set to obtain the overlapping field set; the intersection of the overlapping field set and the query keywords is determined as the target confusion field set; S4, based on the co-occurrence weight matrix and the target confusion field set, the disambiguation score of each candidate table in the candidate table set is calculated; Based on the disambiguation score, the similarity score sequence is corrected to obtain the rearranged score sequence; the target table is selected from the candidate table set according to the rearranged score sequence; S5, a query statement is generated based on the table structure information of the target table and natural language query; the query statement is executed to obtain business data, and a visualization report is rendered based on the business data.

2. The data analysis method for enterprise reports according to claim 1, characterized in that, Concentration indices are calculated based on similarity score sequences, including: Calculate the average score and standard deviation of each similarity in the similarity score sequence; Obtain the stability constant and the similarity benchmark threshold, and use the sum of the standard deviation and the stability constant as the target denominator; The difference between the average score and the similarity benchmark threshold is used as the target molecule; The ratio of the target numerator to the target denominator is determined as the concentration index.

3. The data analysis method for enterprise reports according to claim 1, characterized in that, The overlap rate metric is calculated based on the fields of each table in the candidate table set, including: Obtain the first field set and the second field set of any two candidate tables in the candidate table set; Calculate the number of intersections and the number of unions between the first field set and the second field set, and determine the ratio of the number of intersections to the number of unions as the local overlap rate of the corresponding two candidate tables; Calculate the average local overlap rate corresponding to all candidate table combinations in the candidate table set, and determine the average value as the overlap rate index.

4. The data analysis method for enterprise reports according to claim 1, characterized in that, The disambiguation score satisfies the following relationship: in, Indicates the first in the candidate table set Disambiguation score of candidate list; This indicates the index number of the candidate table in the candidate table set; Represents the first in the candidate table set Zhang Candidate List; Indicates the target obfuscated field set; Indicates the total number of field elements contained in the target obfuscation field set; Represents the fields in the target obfuscation field set; Indicates the first The set of table fields for the candidate table; This represents the fields in the table's field set; Indicates the weight of historical co-occurrence; Indicates the target obfuscation field set. The initial response score.

5. A data analysis method for enterprise reports according to claim 1, characterized in that, The rearrangement scores of each candidate table in the rearrangement score sequence satisfy the following relation: in, Indicates the first in the candidate table set The reordering score of the candidate list; This indicates the index number of the candidate table in the candidate table set; Represents the first in the candidate table set Zhang Candidate List; Indicates the first similarity score sequence The similarity between the candidate lists; This represents the gain coefficient corresponding to the business intent; 1 represents the disambiguation score; 1 represents the reference constant used to ensure the numerical reference.

6. A data analysis method for enterprise reports according to claim 5, characterized in that, The gain coefficients corresponding to the obtained business intent include: Retrieve sample query statements from the historical records that belong to the stated business intent and were successfully executed; Multiple candidate coefficients are generated within a preset numerical range with a fixed step size; With the goal of maximizing the execution success rate of the sample query statement, a search is conducted among the multiple candidate coefficients, and the candidate coefficient that maximizes the execution success rate is determined as the gain coefficient corresponding to the business intent.

7. A data analysis method for enterprise reports according to claim 4, characterized in that, The historical co-occurrence weights satisfy the following relationship: in, Indicates the weight of historical co-occurrence; This represents the fields in the table's field set; Represents the fields in the target obfuscation field set; This indicates the number of times a field in the table's field set and a field in the target obfuscation field set co-occur together in historical query statements; This represents the total number of times a field in the target obfuscation field set appears in all successfully executed historical query statements. This represents the smoothing constant.

8. A data analysis method for enterprise reports according to claim 1, characterized in that, When constructing the co-occurrence weight matrix, the following is also included: If the total number of historical query statements is less than a preset number, multiple internal fields from the same data table in the enterprise database are extracted. Any two internal fields within the same data table are considered as co-occurring pairs, and the initial value of the corresponding co-occurrence count is set to 1. The initial co-occurrence weight matrix is ​​generated by replacing the historical query statement with the initial value.

9. A data analysis method for enterprise reports according to claim 1, characterized in that, Extracting entity features from natural language queries yields query keywords, including: Feature extraction is performed on the natural language query to obtain a set of Chinese characters; In a pre-built bidirectional index table, each Chinese character in the Chinese character set is mapped to a corresponding English field name, and the set of English field names is determined as the query keyword.

10. A data analysis system for enterprise reports, characterized in that, include: A processor and a memory, the memory storing computer program instructions that, when executed by the processor, implement a data analysis method for enterprise reports according to any one of claims 1-9.

Images