Method, device, equipment and medium for automatically generating report of batch database table

Abstract

This application belongs to the field of data processing technology and relates to a method, apparatus, device, and medium for automatically generating reports from batch database tables. The method includes: acquiring a batch of tables for which reports are to be generated, and generating a knowledge graph; acquiring business text data and obtaining augmented vectors; obtaining a hybrid index based on the knowledge graph and augmented vectors; acquiring user query statements and outputting fusion vectors; quantizing and reconstructing the fusion vectors, and concatenating them with semantic tag vectors to obtain a subspace; performing a retrieval based on the subspace and hybrid index, and employing a two-level strategy of coarse recall and fine ranking for internal recall to obtain a priority context, and inserting tags at the concatenation points; aligning the priority context with intent and pattern based on the inserted tags, and performing validation to obtain query results; using the query results as input to a large language model to obtain a report. This application enables the automatic generation of reports from batch tables.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to methods, apparatus, devices and media for automatically generating reports from batch database tables. Background Technology

[0002] With the advancement of science and the development of technology, the way data is processed has also changed.

[0003] In existing technologies, SQL queries are typically written manually to extract data from the database, and then the data is manually summarized, analyzed, and reported.

[0004] The above report generation method has the following problems:

[0005] 1. Data retrieval relies on traditional database queries, which cannot understand user semantics, resulting in low retrieval efficiency and inaccurate retrieval results.

[0006] 2. Report generation relies on manual labor, resulting in low automation and a long generation cycle.

[0007] 3. When user intent changes, it cannot respond in real time and generate new reports. Summary of the Invention

[0008] Therefore, it is necessary to provide methods, apparatus, devices, and media for automatically generating reports from batch database tables to address the aforementioned technical problems. These methods and apparatuses can understand the user's semantics for batch tables, quickly retrieve data related to the user's needs from the database, and automatically generate high-quality statistical analysis reports using a large language model.

[0009] Methods for automatically generating reports from batch database tables include:

[0010] The system retrieves the batch database tables containing the reports to be generated, exports and parses the corresponding dynamic link libraries to obtain the fields, meanings, relationships, and semantic tags of each field in the dynamic link libraries, in order to generate a knowledge graph; it also retrieves and encodes business text data to obtain data vectors, which are then concatenated with semantic tag vectors to obtain augmented vectors; the knowledge graph is stored in a graph database, and the augmented vectors are stored in a vector database, resulting in a hybrid index of the graph and vectors.

[0011] The system retrieves the user's query statement, obtains a JSON string, and feeds it into the Transformer dual towers sharing a frozen backbone, outputting a fused vector. The fused vector is quantized and reconstructed, and concatenated with the semantic label vector to form an entry vector, thus obtaining a subspace. The system performs retrieval based on the subspace and the hybrid index, and adopts a two-level strategy of coarse recall and fine ranking for internal recall, obtaining the priority context and insertion tag.

[0012] Based on the insertion mark, the priority context is aligned with intent and pattern, validated, and an executable statement is generated; based on the executable statement, the query results are obtained.

[0013] The query results are used as input to a large language model for focusing and matching of core dimensions to generate a report.

[0014] In one embodiment, a batch of tables containing reports to be generated are obtained, the corresponding dynamic link libraries are exported and parsed to obtain the fields, meanings, relationships, and semantic tags of each field of the dynamic link libraries, in order to generate a knowledge graph, including:

[0015] Obtain the batch database tables for the reports to be generated, and export the corresponding dynamic link libraries;

[0016] A parser is used to extract the primary and foreign keys, table field values, table field names, and data types of the database tables in the dynamic link library, so as to obtain the fields, meanings, and relationships of the dynamic link library, and add semantic tags to each field;

[0017] A knowledge graph is generated by using fields as nodes, relationships as edges, and data types, meanings, and semantic tags as node attributes.

[0018] In one embodiment, the knowledge graph is stored in a graph database, the augmented vectors are stored in a vector database, and a hybrid index of the graph and vectors is obtained, including:

[0019] The knowledge graph is stored in a graph database, the augmented vectors are stored in a vector database, and the graph database and vector database are associated based on the node's identity number to obtain a hybrid index of graph and vector.

[0020] In one embodiment, the user's query statement is obtained as a JSON string, and then fed into the two Transformer towers sharing a frozen backbone. The output fused vector includes:

[0021] Obtain the user's query statement, extract keywords, and combine them with a knowledge graph to obtain a JSON string;

[0022] The user's query and JSON string are fed into the two Transformer towers sharing the frozen backbone, and the fused vector is output.

[0023] In one embodiment, the fused vector is quantized and reconstructed, and then concatenated with the semantic label vector to form an entry vector, thereby obtaining a subspace, including:

[0024] The fused vector is quantized and reconstructed to obtain a residual vector, which is then concatenated with the semantic label vector to form an entry vector.

[0025] The subspace is obtained based on the entry vector.

[0026] In one embodiment, retrieval is performed based on subspaces and hybrid indexes, and an internal recall is conducted using a two-level strategy of coarse recall and fine ranking to obtain priority context and insertion tags, including:

[0027] The retrieval is performed based on subspaces and hybrid indexes, and a two-level strategy of coarse recall and fine ranking is used for internal recall to output candidate vectors.

[0028] Based on the candidate vectors, the priority context is obtained and a marker is inserted.

[0029] In one embodiment, the priority context is aligned with intent and pattern based on the inserted tags, and then validated to generate an executable statement; based on the executable statement, query results are obtained, including:

[0030] Based on the inserted tags, regular expressions are used to align the intent and pattern of the priority context;

[0031] After alignment, the DSL-Skeleton generator is called, and the identity number of the node corresponding to the candidate vector is used as the leaf node. The minimum connected subgraph algorithm is run on the knowledge graph, and the simplest multi-table connection path is generated along the edge.

[0032] After generating the simplest multi-table join path, it is validated to generate an executable statement;

[0033] The executable statement is sent to the database to obtain the raw result set;

[0034] If the original result set is found to be empty, the filtering range is broadened to obtain the query results; if the original result set is not found to be empty, a statistical JSON statement is generated to obtain the query results.

[0035] A device for automatically generating reports from batch database tables, comprising:

[0036] The first module is used to obtain the batch database tables of the reports to be generated, export the corresponding dynamic link libraries and parse them to obtain the fields, meanings, relationships and semantic tags of each field of the dynamic link libraries in order to generate a knowledge graph; obtain business text data and encode it to obtain data vectors, and concatenate it with semantic tag vectors to obtain augmented vectors; store the knowledge graph in the graph database, store the augmented vectors in the vector database, and obtain a hybrid index of graph and vectors;

[0037] The second module is used to obtain the user's query statement, obtain a JSON string, and send it to the Transformer dual towers with a shared frozen backbone to output a fused vector; the fused vector is quantized and reconstructed, and concatenated with the semantic label vector to form an entry vector to obtain a subspace; retrieval is performed based on the subspace and the hybrid index, and a two-level strategy of coarse recall and fine ranking is used for internal recall to obtain the priority context and insertion mark;

[0038] The third module is used to align the intent and pattern of the priority context according to the insertion mark, perform validation, and generate an executable statement; based on the executable statement, the query results are obtained.

[0039] The fourth module is used to focus on the query results as input to the large language model, match the core dimensions, and generate a report.

[0040] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described method.

[0041] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0042] The aforementioned method, apparatus, computer equipment, and storage medium for automatically generating reports from batch database tables, based on retrieval-enhanced generation, achieve automated generation of statistical analysis reports. This breaks free from the limitations of traditional fixed templates and dynamically generates personalized reports according to users' real-time needs. This means users do not need to wait for preset template updates or manually adjust them; the system can respond instantly and generate the required reports, greatly improving the flexibility and efficiency of data processing. It meets users' high demands for real-time data analysis and high-quality, highly readable report generation, while ensuring the accuracy and timeliness of data acquisition. Specifically, the combination of RAG-enhanced retrieval (retrieval-enhanced generation) and vector databases efficiently and quickly retrieves the most relevant contextual information from large-scale data, significantly improving the efficiency and accuracy of report generation. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating a method for automatically generating reports from batch database tables in one embodiment.

[0044] Figure 2 This is a structural block diagram of an apparatus for automatically generating reports from batch database tables in one embodiment;

[0045] Figure 3 This is an internal structural diagram of a computer device in one embodiment.

[0046] Explanation of reference numerals in the attached figures:

[0047] 201 Module 1, 202 Module 2, 203 Module 3, 204 Module 4. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0049] Furthermore, the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of those features. In the description of this application, "multiple sets" means at least two sets, such as two sets, three sets, etc., unless otherwise explicitly specified.

[0050] In this application, unless otherwise expressly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection, an electrical connection, a physical connection, or a wireless communication connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two elements or the interaction between two elements, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0051] Furthermore, the technical solutions of the various embodiments of this application can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by this application.

[0052] This application provides a method for automatically generating reports from batch database tables, such as... Figure 1 The flowchart shown, in one embodiment, includes:

[0053] Step 101: Obtain the batch database table of the report to be generated, export and parse the corresponding dynamic link library to obtain the fields, meanings, relationships and semantic tags of each field of the dynamic link library to generate a knowledge graph; obtain business text data and encode it to obtain a data vector, and concatenate it with the semantic tag vector to obtain an augmented vector; store the knowledge graph in the graph database, store the augmented vector in the vector database, and obtain a hybrid index of graph and vector.

[0054] Specifically:

[0055] The process involves: acquiring a batch of database tables (including multiple database tables named `file_tags` and the time range for each table) to generate reports, and exporting the corresponding dynamic link libraries (DDL files); using a parser to extract the primary and foreign keys (including primary and foreign keys), table field values ​​(belonging to row data), table field names (belonging to column data), and data types from the database tables in the DDL; using the primary and foreign keys as the fields of the DDL; obtaining the meaning of the DDL based on the table field values ​​and table field names (including file size and access type); performing self-associations on different primary and foreign keys to obtain the relationships between different database tables in the DDL (including direction and constraints); adding semantic tags to each field to generate a 128-dimensional semantic tag vector; using fields as nodes, relationships as edges, and data types, meanings, and semantic tags as node attributes to label each node with an identification number (i.e., node ID) and assign a type label to each node to generate a knowledge graph;

[0056] Based on the node's identity identifier (i.e., the node's ID), data related to the knowledge graph is obtained as business text data (specifically, it can be free text such as varchar / text type fields and notes in the business database). Within this business text data, data related to the business type is extracted on a sentence-by-sentence basis. This data undergoes desensitization processing (text cleaning to remove version numbers, suffixes, and special symbols; the specific desensitization process is based on existing technology) and data encoding (word segmentation and stop word removal followed by encoding) to obtain a 384-dimensional data vector corresponding to each sentence (meaning data extraction is performed on a sentence-by-sentence basis, and each sentence generates a 384-dimensional data vector). The 384-dimensional data vector of each sentence is then concatenated with a 128-dimensional semantic tag vector to obtain a 512-dimensional augmented vector for each sentence (each sentence corresponds to a 512-dimensional augmented vector, and all sentences collectively correspond to a single knowledge graph).

[0057] The knowledge graph is stored in a graph database (e.g., TuGraph), and the augmented vectors are stored in a vector database (e.g., Milvus, which stores both vectors and scalars). The graph database and the vector database are then linked based on the node's identity identifier (i.e., the node's ID) to obtain a hybrid index of graph and vector data.

[0058] in:

[0059] Knowledge graphs can be continuously updated based on changes to database tables (specifically: obtain newly added database tables, get the corresponding fields, meanings, and relationships, and update the knowledge graph accordingly).

[0060] For example:

[0061] Obtain the batch database tables for the reports to be generated (taking report A from an office in 2025 as an example, with a time range of 2025-01-01 to 2025-09-11, involving 4 database tables: files, users, departments, and file_access); export the DDL files corresponding to the above 4 database tables using mysqldump-no-data, thus obtaining filedb_2025A1.sql; use a parser (existing technology, which can be built using a Python script) to extract the primary and foreign keys of the database tables in filedb_2025A1.sql: files.creator_id→users.user_id, file_access.file_id→files.file_id, file_access.user_id→users.user_id. The `.user_id`, `users.dept_id`, and `departments.parent_dept_id` are used to extract table field values, field names, and data types, resulting in the fields, meanings, and relationships in the DDL file. Using the open-source `schema_labels_zh` dataset, semantic labels are added to each field (e.g., `files.type` → "document format / type", `departments.dept_name` → "organizational unit / department name"), generating semantic label vectors. Fields are used as nodes, relationships as edges, and data types, meanings, and semantic label vectors as node attributes. Each node is labeled with a node ID and assigned a type label, generating a knowledge graph. The knowledge graph is continuously updated (e.g., if a `file_tags` table is added on September 12th, 3 new nodes and 2 FK edges are written to TuGraph, resulting in an updated knowledge graph, achieving "continuous evolution" of the knowledge graph with zero changes to historical nodes).

[0062] Based on the node ID, retrieve the knowledge graph-related data "2025 Method B.pdf" and "2025 Notice C.docx" as business text data. From this business text data, use an ETL tool (existing technology) to extract data, and after anonymization, use an open-source Adapter (Sentence-Transformer + MLP, specifically existing technology) for data encoding to obtain a 384-dimensional data vector. This vector is then concatenated with a 128-dimensional semantic tag vector to obtain a 512-dimensional augmented vector: "Document Title Method B" and "Organizational Unit Department D".

[0063] The knowledge graph is stored in a graph database (in TuGraph), and the augmented vectors are stored in a vector database (in Milvus using row-level node IDs constructed from "table name.primary key column name.primary key value"). The graph database and the vector database are then linked based on the node IDs (both databases use the same row-level node IDs as foreign keys—Milvus treats them as primary keys, and TuGraph treats them as node names), resulting in a hybrid index of graph and vector data.

[0064] In this step, the batch database tables for generating reports are sorted and parsed to generate a hybrid index of graphs and vectors.

[0065] Step 102: Obtain the user's query statement, get the JSON string, and send it to the Transformer dual towers with shared frozen backbone to output the fused vector; quantize and reconstruct the fused vector, and concatenate it with the semantic label vector to form the entry vector to obtain the subspace; perform retrieval based on the subspace and the hybrid index, and use a two-level strategy of coarse recall and fine ranking for internal recall to obtain the priority context and insertion tag.

[0066] Specifically:

[0067] Obtain the user's query statement, extract keywords, and combine them with a knowledge graph to obtain a JSON string;

[0068] The user's query and JSON string are fed into the two Transformer towers sharing the frozen backbone, and the fused vector is output.

[0069] The fused vector is quantized and reconstructed to obtain a residual vector, which is then concatenated with the semantic label vector to form an entry vector; based on the entry vector, the subspace is obtained.

[0070] The retrieval is performed based on subspaces and hybrid indexes, and a two-level strategy of coarse recall and fine ranking is used for internal recall to output candidate vectors.

[0071] Based on the candidate vectors, the priority context is obtained and a marker is inserted.

[0072] More specifically:

[0073] The user's query is obtained, and the Jieba word segmentation library (existing technology) is used for word segmentation to obtain a candidate word set. The candidate word set is then filtered by character-level intersection with semantic tags and knowledge graph (i.e., the intersection of the candidate word set and semantic tags is taken, the intersection of the candidate word set and knowledge graph is taken, and the intersection of the two intersections is taken again; how to take the intersection is an existing technology). The keywords are then selected in descending order according to TF-IDF weights (the selection ratio and selection method are both existing technologies).

[0074] Using keywords as the entry point (i.e., input), Cypher CONTAINS (existing technology) is used to locate the starting node in the knowledge graph, and combined with the subgraphs of the knowledge graph (how to obtain subgraphs from the knowledge graph belongs to existing technology), a JSON string is obtained (how to obtain the string belongs to existing technology).

[0075] The user's query statement and JSON string are fed into the Transformer dual towers that share the frozen backbone, and a 768-dimensional fusion vector of business (content of the representation table) and perception (relationship of the representation table) is output.

[0076] For the fused vector, hierarchical residual quantization is performed first, followed by reconstruction to obtain a 512-dimensional residual vector; the residual vector is then concatenated with a 128-dimensional semantic tag vector to form a 640-dimensional entry vector.

[0077] Based on the 640-dimensional entry vector, IVF-PQ clustering is performed to obtain the subspace;

[0078] The retrieval is performed based on subspaces and hybrid indexes, and a two-level strategy of coarse recall and fine ranking is used for internal recall to output candidate vectors.

[0079] The candidate vectors are scored for keyword overlap with the user's query, and then concatenated according to the scores to obtain the priority context, with a marker inserted at the concatenation point.

[0080] For example:

[0081] The user's query q is obtained and segmented using the Jieba word segmentation library to obtain a candidate word set. The candidate word set is then subjected to character-level intersection filtering with semantic tags (the "semantic tag" attribute of all nodes in TuGraph) and the knowledge graph (the original text attributes of all ROW nodes in TuGraph), retaining only the business words that have appeared. The Top-K words are then selected in descending order of TF-IDF weight (K≤3) to obtain the keywords.

[0082] Using keywords as the entry point, Cypher CONTAINS is used to locate the starting node in the knowledge graph, and two steps are taken outward along the existing primary and foreign key edges. Combined with the subgraph g of the knowledge graph, the JSON string is obtained in real time through linearization.

[0083] The user's query and JSON string are fed into a shared frozen backbone of the Transformer dual towers, outputting a 768-dimensional fusion vector of business and perception. (The user's query is input into the q tower, and the JSON string is input into the g tower. The q tower and g tower share a 12-layer 768-dimensional bert-base-chinese structure. The 1st to 11th layers are used for weight freezing, and the 12th layer is used to output a 768-dimensional CLS vector. The q tower is specifically for capturing the semantics of the query, and the g tower is specifically for capturing the connectivity path of the table structure. The two are complementary. The q tower outputs q_vec, and the g tower outputs g_vec. Based on the output of the dual towers and the two obtained gating matrices Wq and Wg, element-wise weighting is performed to obtain the weighted value σ = Wq·q_vec + Wg·g_vec. Then, the 768-dimensional fusion vector is obtained based on the weighted value.)

[0084] For the fused vector, perform hierarchical residual quantization of 768→512→256→128 to obtain the quantized vector (first, randomly initialize three fully connected layers (768→512, 512→256, 256→128) and their corresponding codebooks (learnable vector centers), then train iteratively using 10k fused vectors with K-means preset centers: calculate the residuals according to the forward→nearest neighbor index→residual method, obtain the loss, and iterate according to the loss and the previous residual until the residuals meet the preset conditions, then stop the iteration and obtain the quantized vector).

[0085] The quantized vector is reconstructed (how to reconstruct it using existing techniques) to obtain a 512-dimensional residual vector;

[0086] The 512-dimensional residual vector is concatenated with the 128-dimensional semantic tag vector to form a 640-dimensional entry vector.

[0087] Based on the 640-dimensional entry vector, IVF-PQ is used to perform k-means clustering to divide the vector space into 64 business-related spaces, thus obtaining subspaces;

[0088] The retrieval is performed based on subspaces and a hybrid index. First, subspaces are filtered by semantic tags, then by type tags (a two-layer filtering approach improves retrieval accuracy and efficiency), resulting in subspaces matching both tags. Irrelevant subspaces are then excluded. Within each subspace matching both tags, a two-level strategy of coarse recall and fine ranking is employed for internal recall (further improving retrieval accuracy and efficiency). Candidate vectors are output. (Internal recall specifically includes: within each subspace, coarse recall followed by fine ranking; coarse recall uses quantitative indexing (existing technologies such as PQ and IVF) to output a candidate vector list (e.g., a Top-200 candidate vector list), avoiding a full database scan, reducing storage by 60%, and maintaining 99% reconstruction accuracy; fine ranking introduces hybrid similarity to calculate the similarity between each vector in the candidate vector list and all other vectors (hybrid similarity S = ...). α ·cosine(q_fields,d_fields)+ β `graph_distance(q_fields, d_fields)`, where `cosine(q, d)` represents the cosine similarity between `q_fields` and `d_fields`, and `graph_distance` represents the shortest path length between nodes calculated by Schema-Graph using the TuGraph shortest path algorithm (i.e., the shortest path length between `q_fields` and `d_fields` in Schema-Graph; this shortest path length reflects the semantic / business association strength of the fields, and the shorter the distance, the more similar they are). `q_fields` represents the query-side fields, and `d_fields` represents the document-side fields (i.e., candidate data-side fields). α , β Different dynamic similarity coefficients are represented and updated in real time through online Bayesian optimization based on user click-through rate (the specific update method is based on existing technology). They are arranged in descending order of mixed similarity and the final candidate vector list (e.g., selecting the Top-10 from the Top-200) is retained as the output candidate vectors. This two-level intra-recall strategy is significantly better than traditional single cosine retrieval.

[0089] TF-IDF is calculated for the original text blocks corresponding to the candidate vectors (i.e., the original text blocks used for the query statement), and keyword overlap is scored with the user's query statement (how to score is based on existing technology). The priority context is obtained by concatenating the text blocks according to the descending score, and a marker is inserted at the concatenation point. <field>(For example: A user asks "Which departments have downloaded the recruitment management regulations?" → The 768-dimensional fusion vector is quantized and reconstructed to obtain a 512-dimensional residual. The front end is then appended with the "document title" tag in a one-hot encoding to form a 640-dimensional entry. IVF-PQ uses this for clustering, first filtering the "document title" subspace, then recalling the Top-200 within it, and outputting "Human Resources Department".)

[0090] In this step, enhanced retrieval is performed based on the user's query, priority context is generated, and tags are inserted.

[0091] Employing a dual-tower encoding approach combining semantic-structural and business-awareness methods, the generated fused vectors simultaneously carry both "what to ask" and "where to look." This ensures that subsequent vector retrieval includes not only semantic information but also database table structure relationships, avoiding the missed retrieval of cross-table vouchers or the false recall of synonymous but different business fields that may occur with purely semantic or keyword-based solutions. This improves semantic matching effectiveness and recall rate, thereby enhancing the accuracy of generated reports. The retrieval process utilizes subspaces and hybrid indexes, leveraging both data graph relationships and vector similarity to further improve retrieval accuracy and efficiency.

[0092] It should be noted that the specific method for reconstructing the residual vector is existing technology and will not be elaborated here.

[0093] Step 103: Align the intent and pattern of the priority context according to the insertion mark, perform validation, and generate an executable statement; obtain the query results based on the executable statement.

[0094] Specifically:

[0095] Based on the insertion mark, regular expressions are used to align the intent and pattern of the priority context;

[0096] After alignment, the DSL-Skeleton generator is called, and the identity number of the node corresponding to the candidate vector is used as the leaf node. The minimum connected subgraph algorithm is run on the knowledge graph, and the simplest multi-table connection path is generated along the edge.

[0097] After generating the simplest multi-table join path, it is validated to generate an executable statement;

[0098] The executable statement is sent to the database to obtain the raw result set;

[0099] If the original result set is found to be empty, the filtering range is broadened to obtain the query results; if the original result set is not found to be empty, a statistical JSON statement is generated to obtain the query results.

[0100] For example:

[0101] Use regular expressions based on the insertion mark. <field> (. ?)< / field> The priority context is matched. If the priority context matches the regular expression, it is considered a match. If it does not match, the original result set is considered to be empty. If it matches, the priority context is compared with the node ID in the knowledge graph (that is, based on the inserted tag, the ID in the priority context is found, and the content in the knowledge graph is found according to the ID and the content is compared) to ensure the alignment of intent and pattern (that is, to ensure that the content to be found is a consistent pattern).

[0102] After alignment, the DSL-Skeleton generator (existing technology) is called, and the node ID corresponding to the candidate vector is used as the leaf node ("F_need leaf") to run the minimum connected subgraph algorithm (existing technology) on the knowledge graph, and generate the simplest multi-table join path along the edge (the specific method of generating the simplest multi-table join path is existing technology: the algorithm can immediately construct the FROM / JOIN clause and replace it with SQL_skeleton with named placeholders. The SQL_skeleton with named placeholders (including FROM / JOIN clause and SELECT field list) is checked with whitelist and bound to variables).

[0103] After generating the simplest multi-table join path, a skeleton is formed based on the simplest multi-table join path (how to form the skeleton is existing technology). The skeleton is validated using a whitelist syntax tree. The validation uses ANTLR to parse the syntax tree (existing technology) and compare it with the whitelist (existing technology) field list and blacklist (existing technology) function list to ensure that only authorized table fields are used, blacklist expressions are disabled, and sensitive operations are granted secondary authorization. After the validation passes, an executable SQL statement is generated based on the skeleton (with placeholders replaced by bind variables).

[0104] The executable statement is sent to the business database via a database connection (e.g., JDBC) to obtain the original result set JSON.

[0105] If the original result set JSON is found to be empty, the filtering range is widened (e.g., time range / numerical interval) and the user is prompted to obtain the query results;

[0106] If the original result set JSON is not empty, pandas-profiling (existing technology) is called on the numerical columns of the business database to generate statistical JSON statements. The original result set JSON, statistical JSON statements, and executable SQL statements are then sent to the LangChain PromptTemplate large model (existing technology), which generates the query results (e.g., from January 9, 2025 to September 12, 2025, the Human Resources Department downloaded the recruitment management regulations 18 times, accounting for 42% of the entire company).

[0107] In this step, the query results are generated.

[0108] Step 104: Use the query results as input to the large language model, focus on it, match the core dimensions, and obtain a report.

[0109] Specifically:

[0110] The query results are used as input to a large language model for focusing and matching core dimensions. Descriptions and charts are generated for the core dimensions, and introductions, conclusions, and table sources are produced to generate a report.

[0111] For example:

[0112] The system uses the query results as input to a large language model, focusing on the user's query, the retrieved priority context, and the query results (i.e., the user's query is only performed within the retrieved priority context and query results). It then matches core dimensions (using vector semantic matching to match report-related dimensions (vocabulary sets), while irrelevant dimensions are directly collapsed; the specific matching method is based on existing technologies, such as Sentence-BERT). Descriptions and charts are generated for the core dimensions (charts include year-on-year, month-on-month, and regional comparisons, automatically selecting bar charts or line charts based on user-mentioned business keywords such as "street" and "department"). A lead (explaining "why focus on these dimensions"), conclusions, and table sources are also generated (how to select charts, and how to generate the lead and conclusions are based on existing technologies, such as LangChain PromptTemplate), resulting in the report.

[0113] In this step, a report is automatically generated based on the query results using a large language model. A follow-up question template is embedded at the end of the report, allowing users to continue asking follow-up questions on related topics from their original query. This application can incrementally rewrite the same SQL query without re-searching, thus achieving personalized report generation.

[0114] The aforementioned method for automatically generating reports from batch database tables, based on retrieval-enhanced generation, achieves automated generation of statistical analysis reports. It breaks free from the limitations of traditional methods that rely on fixed templates, dynamically generating personalized reports based on real-time user needs. This means users do not need to wait for preset template updates or manually adjust them; the system can respond instantly and generate the required reports, greatly improving the flexibility and efficiency of data processing. It meets users' high demands for real-time data analysis and high-quality, highly readable report generation, while ensuring the accuracy and timeliness of data acquisition. Specifically, the combination of RAG-enhanced retrieval (retrieval-enhanced generation) and vector databases efficiently and quickly retrieves the most relevant contextual information from large-scale data, significantly improving the efficiency and accuracy of report generation.

[0115] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0116] This application also provides an apparatus for automatically generating reports from batch database tables, such as... Figure 2 As shown, in one embodiment, it includes: a first module 201, a second module 202, a third module 203, and a fourth module 204, wherein:

[0117] The first module 201 is used to obtain the batch database tables of the reports to be generated, export the corresponding dynamic link libraries and parse them to obtain the fields, meanings, relationships and semantic tags of each field of the dynamic link libraries in order to generate a knowledge graph; obtain business text data and encode it to obtain data vectors, and concatenate them with semantic tag vectors to obtain augmented vectors; store the knowledge graph in the graph database, store the augmented vectors in the vector database, and obtain a hybrid index of graph and vectors;

[0118] The second module 202 is used to obtain the user's query statement, obtain a JSON string, and send it to the Transformer dual towers with a shared frozen backbone to output a fused vector; the fused vector is quantized and reconstructed, and concatenated with the semantic label vector to form an entry vector to obtain a subspace; retrieval is performed based on the subspace and the hybrid index, and a two-level strategy of coarse recall and fine ranking is used for internal recall to obtain the priority context and insertion tag;

[0119] The third module 203 is used to align the intent and pattern of the priority context according to the insertion mark, perform validation, and generate an executable statement; based on the executable statement, the query results are obtained;

[0120] The fourth module, 204, is used to focus on the query results as input to the large language model, match the core dimensions, and generate a report.

[0121] Specific limitations regarding the apparatus for automatically generating reports from batch database tables can be found in the limitations of the method for automatically generating reports from batch database tables described above, and will not be repeated here. Each module in the above apparatus can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0122] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a method for automatically generating reports from batch database tables. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0123] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0124] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the method described above.

[0125] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0126] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0127] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

[0128] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0129] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended application documents.< / field>

Claims

1. A method for automatically generating reports from batch database tables, characterized in that, include: Obtain the batch database tables for the reports to be generated, and export the corresponding dynamic link libraries; A parser is used to extract the primary and foreign keys, table field values, table field names, and data types of database tables in dynamic link libraries. Primary and foreign keys are used as fields in the dynamic link library (DLL). The meaning of the DLL is derived from the table field values ​​and names. Self-associations are performed on different primary and foreign keys to obtain the relationships between different database tables within the DLL. Semantic tags are added to each field, generating a 128-dimensional semantic tag vector. Each node is labeled with an identity number and a type label, using fields as nodes, relationships as edges, and data type, meaning, and semantic tags as node attributes, thus generating a knowledge graph. Data related to the knowledge graph is retrieved based on the node's identity number as business text data. Within the business text data, data related to the business type is extracted on a sentence-by-sentence basis, and after anonymization and data encoding, a 384-dimensional data vector corresponding to each sentence is obtained. The 384-dimensional data vector of each sentence is concatenated with the 128-dimensional semantic tag vector to obtain a 512-dimensional augmented vector for each sentence. The knowledge graph is stored in a graph database, and the augmented vectors are stored in a vector database. A hybrid index of graph and vector data is obtained by associating the graph database and vector database based on the node's identity number. The system retrieves the user's query statement, obtains a JSON string, and feeds it into the Transformer dual towers sharing the frozen backbone, outputting a fused vector. The fused vector is quantized and reconstructed, and concatenated with the semantic label vector to form an entry vector, thus obtaining a subspace. The system performs retrieval based on the subspace and the hybrid index, and adopts a two-level strategy of coarse recall and fine ranking for internal recall, obtaining the priority context and insertion tag. Based on the insertion mark, the priority context is matched using regular expressions. If the priority context matches the regular expression, it is considered a hit; otherwise, the original result set is considered an empty set. If a hit occurs, the priority context is compared with the node ID in the knowledge spectrum to ensure the alignment of intent and pattern. After alignment, the DSL-Skeleton generator is invoked, using the node IDs corresponding to the candidate vectors as leaf nodes, to run the minimum connected subgraph algorithm on the knowledge graph and generate the simplest multi-table join path along the edges. After generating the simplest multi-table join path, a skeleton is formed based on the simplest multi-table join path. The skeleton is validated by a whitelist syntax tree. The validation uses ANTLR to parse the syntax tree and compare it with the whitelist field list, and filters it with the blacklist function list to ensure that only authorized table fields are used, blacklist expressions are disabled, and sensitive operations are granted secondary authorization. After the validation passes, an executable SQL statement is generated based on the skeleton. The executable statement is sent to the business database via a database connection to obtain the original result set JSON; if the original result set JSON is empty, the filtering range is widened and the user is prompted to obtain the query results. If the original result set JSON is not empty, then pandas-profiling is called on the numerical columns of the business database to generate statistical JSON statements. The original result set JSON, statistical JSON statements and executable SQL statements are then sent to the LangChain PromptTemplate large model, which generates the query results. The query results are used as input to a large language model for focusing and matching of core dimensions to generate a report.

2. The method for automatically generating reports from batch database tables according to claim 1, characterized in that, Obtain the user's query statement, obtain a JSON string, and feed it into the two Transformer towers sharing a frozen backbone. Output the fused vector, including: Obtain the user's query statement, extract keywords, and combine them with a knowledge graph to obtain a JSON string; The user's query and JSON string are fed into the two Transformer towers sharing the frozen backbone, and the fused vector is output.

3. The method for automatically generating reports from batch database tables according to claim 2, characterized in that, The fused vector is quantized and reconstructed, and then concatenated with the semantic label vector to form an entry vector, thus obtaining a subspace, including: The fused vector is quantized and reconstructed to obtain a residual vector, which is then concatenated with the semantic label vector to form an entry vector. The subspace is obtained based on the entry vector.

4. The method for automatically generating reports from batch database tables according to claim 3, characterized in that, Retrieval is performed based on subspaces and hybrid indexes, and an internal recall strategy of coarse recall and fine ranking is used to obtain priority context and insertion tags, including: The retrieval is performed based on subspaces and hybrid indexes, and a two-level strategy of coarse recall and fine ranking is used for internal recall to output candidate vectors. Based on the candidate vectors, the priority context is obtained and a marker is inserted.

5. A device for automatically generating reports from batch database tables, characterized in that, The method for automatically generating reports from batch database tables according to any one of claims 1 to 4 includes: The first module is used to obtain the batch database tables for generating reports and export the corresponding dynamic link libraries (DLLs). A parser is used to extract the primary and foreign keys, table field values, table field names, and data types from the database tables in the DLLs. The primary and foreign keys are used as fields in the DLLs. The meaning of the DLLs is derived from the table field values ​​and names. Self-associations are performed on different primary and foreign keys to obtain the relationships between different database tables in the DLLs. Semantic tags are added to each field, generating a 128-dimensional semantic tag vector. Each node is labeled with an identification number, using fields as nodes, relationships as edges, and data types, meanings, and semantic tags as node attributes. Assign type labels to generate a knowledge graph; based on the node's identity number, obtain data related to the knowledge graph as business text data; in the business text data, extract data related to the business type on a sentence-by-sentence basis, and after desensitization and data encoding, obtain a 384-dimensional data vector corresponding to each sentence; concatenate the 384-dimensional data vector of each sentence with a 128-dimensional semantic label vector to obtain a 512-dimensional augmented vector for each sentence; store the knowledge graph in a graph database, store the augmented vector in a vector database, and associate the graph database and the vector database based on the node's identity number to obtain a hybrid index of graph and vector; The second module is used to obtain the user's query statement, obtain a JSON string, and send it to the Transformer dual towers with a shared frozen backbone to output a fused vector; the fused vector is quantized and reconstructed, and concatenated with the semantic label vector to form an entry vector to obtain a subspace; retrieval is performed based on the subspace and the hybrid index, and a two-level strategy of coarse recall and fine ranking is used for internal recall to obtain the priority context and insertion mark; The third module is used to match the priority context using regular expressions based on the insertion markers. If the priority context matches the regular expression, it is considered a match; otherwise, the original result set is considered empty. Upon a match, the priority context is compared with the node IDs in the knowledge graph to ensure alignment of intent and pattern. After alignment, the DSL-Skeleton generator is called, using the node IDs corresponding to the candidate vectors as leaf nodes, to run the minimum connected subgraph algorithm on the knowledge graph and generate the simplest multi-table join paths along the edges. After generating the simplest multi-table join paths, a skeleton is formed based on these paths. The skeleton is then validated using a whitelisted syntax tree, with validation performed using ANTLR. The syntax tree is parsed and compared with the whitelist of fields and the blacklist of functions to ensure that only authorized table fields are used, blacklisted expressions are disabled, and sensitive operations are granted secondary authorization. After the verification is passed, an executable SQL statement is generated based on the skeleton. The executable statement is sent to the business database via the database connection to obtain the original result set JSON. If the original result set JSON is empty, the filtering range is widened and the user is prompted to obtain the query results. If the original result set JSON is not empty, pandas-profiling is called on the numerical columns of the business database to generate statistical JSON statements. The original result set JSON, statistical JSON statements, and executable SQL statements are sent together into the LangChainPromptTemplate large model, which generates the query results. The fourth module is used to focus on the query results as input to the large language model, match the core dimensions, and generate a report.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images