Automatic report generation system and method based on hybrid analytical model and template engine

Abstract

The invention provides an automatic report generation system and method based on a hybrid analytic model and a template engine, and the system comprises a hybrid document analysis module which converts an input semi-structured document into a structured YAML data file, builds and retains a context hierarchical relation through maintaining a stack data structure, and generates an automatic report when a title with a level of N is analyzed; popping up all title nodes with levels greater than or equal to N in the stack, and then pressing the title nodes with the current level of N into the stack; the multi-source data fusion and processing module is used for expanding static data extracted from the YAML data file based on the YAML data file; and the report generation module based on the template engine is used for realizing separation of contents and formats and dynamically generating table rows or paragraphs. According to the method and the device, the structured information with the context relation can be accurately and completely extracted from the semi-structured source document or the mixed document with variable formats, and the analysis precision, the data processing flexibility and the expansibility of the mixed document are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to a report generation system, in particular to an automatic report generation system based on a hybrid parsing model and a template engine, and further relates to an automatic report generation method applied to the automatic report generation system based on the hybrid parsing model and the template engine. BACKGROUND

[0002] In the current information age, report writing is one of the core work of knowledge-intensive industries (such as scientific research, business analysis, finance, aerospace engineering, etc.). The traditional report generation process highly depends on manual work, which not only consumes time and effort, but also makes it difficult to guarantee the accuracy and consistency of information extraction when dealing with complex formats, graphic-text mixed documents.

[0003] To solve this problem, some existing technologies have also proposed some automatic report generation schemes. For example, patent US10614527B2 discloses a "system and method for automatically generating reports based on electronic documents", the core of which is to create a structured data template to parse unstructured data. However, the technical solution of US10614527B2 patent document mainly relies on pre-defined templates that are strongly bound to specific report requirements. When facing complex and variable logical structures within the document, such as cross-page tables, more than three levels of nested lists, and conditional content, its parsing ability and context fidelity are obviously limited, and it is difficult to achieve complete reproduction of the inherent hierarchical relationship in the document. In addition, although Adobe's automated report tool can handle structured data filling, its technical starting point is already structured data, that is, it needs to be based on structured data to complete, and it lacks deep parsing ability for front-end semi-structured source documents or mixed documents, and cannot solve the core challenge of "from unstructured to structured".

[0004] In summary, the existing technologies have the following significant deficiencies: first, the parsing accuracy and context fidelity are insufficient, it is difficult to accurately and completely extract structured information with context relationships from semi-structured source documents or mixed documents with variable formats, and there is a technical bottleneck that is not obvious. Second, the flexibility and expandability are poor, the data extraction logic and report generation template are highly coupled, and it is difficult to adapt to different types of report requirements. Third, it lacks dynamic data fusion capability, most schemes focus on static information processing of a single document, and lack the function of real-time fusion and analysis with external databases. SUMMARY

[0005] The technical problem solved by the present application is to provide an automatic report generation system based on a hybrid parsing model and a template engine, aiming to accurately and completely extract structured information with context relationships from semi-structured source documents or hybrid documents with variable formats, improve the parsing accuracy, data processing flexibility and scalability of hybrid documents, and have the fusion and processing capacity of multi-source data through the optimization design of the overall technical solution.

[0006] To this end, the present application provides an automatic report generation system based on a hybrid parsing model and a template engine, comprising: A hybrid document parsing module is used to convert the input semi-structured document into a structured YAML data file, and a stack data structure is maintained to build and retain the context hierarchical relationship during the conversion process. When a title with a level of N is parsed, all title nodes with a level greater than or equal to N in the stack are popped out, and then the current title node with a level of N is pushed into the stack, where N is a natural number. A multi-source data fusion and processing module is based on the YAML data file and extends the static data extracted from the YAML data file according to a preset logic. And a report generation module based on a template engine is used to realize content and format separation, insert InlineImage object placeholders and control statements, and dynamically generate table rows or paragraphs.

[0007] The present application is further improved, and the implementation process of the hybrid document parsing module comprises the following steps: Step A1, first load the source document using the python-docx library, parse the input semi-structured document into a document object model, and then sequentially traverse the top-level elements in the document object model, including paragraph elements and table elements. Step A2, the corresponding relationship between the style name in the document and the standard title level is defined in advance as a style mapping file, and the style attribute of each paragraph element is found through the style mapping file. If the style attribute of the paragraph element does not belong to the style mapping file, it is judged as ordinary content. Step A3, a stack data structure is maintained to dynamically track the current title level, and the initial state of the stack is empty, representing the root node of the document. When a title with a level of N is parsed, all title nodes with a level greater than or equal to N in the stack are popped out, and then the current title node with a level of N is pushed into the stack, so as to ensure that the top of the stack is the highest level title node to which the current content belongs. In step A4, the paragraphs and tables identified as non-headings are defined as content nodes, and the content nodes are added to a list corresponding to a preset key by programming, and the list is taken as an attribute of a top-of-stack heading node, and the list is associated with the heading node; In step A5, content extraction and pattern matching are performed by reading a configuration file, and when a user adjusts or adds a new extraction rule, the adjustment and addition of the extraction rule are realized by adding a new object. In step A6, for table elements, the text content of the table is extracted row by row and cell by cell, and is converted into a two-dimensional list or a dictionary list structure.

[0008] Further improvement of the application is that in step A4, for multiple consecutive content nodes, they are sequentially added to the list corresponding to the same key.

[0009] Further improvement of the application is that step A5 includes the following sub-steps: Step A501, read one or more configuration files, each configuration file is a JSON array, and each object in the JSON array represents an extraction rule, and the extraction rule includes at least a rule name and a regular expression. Step A502, when a user adjusts or adds a new extraction rule, open the configuration file, add a new object to the JSON array of the configuration file, save the configuration file after adding the new object, and read the latest saved configuration file when running next time.

[0010] Further improvement of the application is that step A6 includes the following sub-steps: Step A601, check the first row of the table, if the content of the first row meets the pre-defined table header characteristics, take the first row as a key, and jump to step A602, otherwise, jump to step A603. Step A602, traverse the remaining rows of the table, pair the data of each row with the key of the table header, generate a dictionary, and after the traversal is completed, all dictionaries are combined into a list. Step A603, convert the entire table into a two-dimensional list composed of a list, and store it under the node of the corresponding YAML data file.

[0011] Further improvement of the application is that the implementation process of the multi-source data fusion and processing module includes the following steps: Step B1, in the configuration file, a SQL statement template containing placeholders is pre-defined; in the running, based on the YAML data file generated by the hybrid document parsing module, first, according to the preset logic, the values corresponding to the preset logic are bound to the placeholders of the SQL statement template, and then the query is executed. Step B2, merge the query result as a new key-value pair with the YAML data file.

[0012] Further improvement of the present application is that in step B2, when the query result needs to be associated to a specific sub-node of the YAML data file, recursively traverse the structure of the YAML data file to find the corresponding node of the preset logic first, and then add the query result as a new attribute of the node; in the process of merging by adding, solve data conflicts based on timestamp or priority marking rules.

[0013] Further improvement of the present application is that the implementation process of the report generation module based on the template engine includes the following steps: Step C1, in the position where dynamic content needs to be generated, input Jinja2 syntax in pure text form, for InlineImage object placeholder, create an InlineImage instance in the Python script and assign it to the key with the same name as the placeholder in the upper and lower context dictionaries; for control statements, read instructions for controlling rendering logic in the Jinja2 template engine, including loop instructions and conditional judgment instructions; Step C2, when the Jinja2 template engine processes the document with loop instructions under the drive of the docxtpl library, first parse the XML structure in the loop body, at this time, the XML structure represents one or more table rows / paragraphs; then, according to the length of the corresponding list in the context object, copy and render the XML structure N times, each time filling the placeholder with an element in the list, thereby dynamically generating multiple table rows or multiple paragraphs in the final document.

[0014] Further improvement of the present application is that it also includes an error handling module, which uses a try...except block to wrap the creation of InlineImage object placeholder and file reading operation in the process of providing image information; if a path error occurs, or an image file damage exception is thrown by the Pillow library, the exception will be captured, a detailed error log will be recorded, the insertion of the image will be skipped, and a prompt text will be prompted to indicate that the image loading has failed, so that the report generation process will not be interrupted due to a single image problem; boundary check is also set to avoid size out-of-range.

[0015] The present application also provides an automatic report generation method based on a hybrid parsing model and a template engine, which is applied to the automatic report generation system based on a hybrid parsing model and a template engine as described above, and includes the following steps: Step S1, for converting the input semi-structured document into a structured YAML data file, in the conversion process, the context hierarchy is constructed and retained by maintaining a stack data structure, in the maintenance process, when a title of level N is parsed, all title nodes with level greater than or equal to N in the stack are popped out, and then the title node of the current level N is pushed into the stack; Step S2, based on the YAML data file, the static data extracted from the YAML data file is extended; And step S3, for realizing content and format separation, inserting InlineImage object placeholder and control statement, and dynamically generating table row or paragraph.

[0016] Compared with the prior art, the beneficial effects of the present application are that: firstly, the input semi-structured document is converted into a structured YAML data file, in the conversion process, the context hierarchy is constructed and retained by maintaining a stack data structure, and in the maintenance process, when a title of level N is parsed, all title nodes with level greater than or equal to N in the stack are popped out, and then the title node of the current level N is pushed into the stack, so as to ensure that the top of the stack is the latest and highest level title node to which the current content belongs; then, based on the YAML data file, the static data extracted from the YAML data file is extended to realize multi-source data fusion and processing; finally, based on the report generation module of the template engine, the content and format are separated, the InlineImage object placeholder and the control statement are inserted, and the table row or paragraph is dynamically generated. Therefore, through the optimization design of the overall technical scheme, the present application can accurately and completely extract structured information with context relationship from semi-structured source documents or mixed documents with variable formats, effectively improve the parsing accuracy, data processing flexibility and expandability of mixed documents, and has the fusion and processing capability of multi-source data. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is the overall architecture schematic diagram of an embodiment of the present application; Figure 2 is the workflow schematic diagram of the mixed document parsing module of an embodiment of the present application; Figure 3 is the mixed document parsing flowchart of an embodiment of the present application; Figure 4 is the rendering flowchart in the report generation process of an embodiment of the present application. DETAILED DESCRIPTION

[0018] In the description of this invention, the term "several" means one or more; the term "multiple" means two or more; the terms "greater than," "less than," and "exceeding" are all understood to exclude the stated number; and the terms "above," "below," and "within" are all understood to include the stated number. The terms "first," "second," etc., are understood to be used only to distinguish identical or similar technical feature names, and should not be construed as implying / indicating the relative importance of the technical features, the number of technical features, or the sequential relationship between the technical features.

[0019] The preferred embodiments of the present invention will now be described in further detail with reference to the accompanying drawings.

[0020] like Figures 1 to 4 As shown, this embodiment provides an automatic report generation system based on a hybrid parsing model and a template engine, including: The hybrid document parsing module is used to convert the input semi-structured document into a structured YAML data file. During the conversion process, the context hierarchy is built and preserved by maintaining a stack data structure. During the maintenance process, when a title of level N is parsed, all title nodes of level N or higher are popped from the stack, and then the title node of level N is pushed onto the stack. N is a natural number. The multi-source data fusion and processing module, based on YAML data files, expands the static data extracted from the YAML data files according to preset logic; Additionally, a report generation module based on a template engine is used to separate content from format, insert InlineImage object placeholders and control statements, and dynamically generate table rows or paragraphs.

[0021] The hybrid document parsing module described in this embodiment is configured by default to identify structural elements by traversing the underlying XML object model of the source document (i.e., the input semi-structured document). It then combines this with a hot-swappable regular expression template dynamically loaded from an external configuration file to perform regular expression-based pattern matching. By maintaining a stack data structure, it parses the semi-structured document into a structured data file that preserves hierarchical relationships, thereby generating a YAML data file that retains its hierarchical structure. A YAML data file is a text file used to represent structured data, following the syntax specifications of YAML (YAML is not Markup Language).

[0022] The multi-source data fusion and processing module described in this embodiment is configured by default to resolve conflicts when data from different data sources conflict, using a user-configurable rule based on timestamps or priority markers.

[0023] like Figure 1 As shown, in addition to the report generation module based on the template engine, a dynamic chart generation module is also preferably included. This dynamic chart generation module generates charts based on an external JSON configuration file, and the report generation module embeds the charts into the report.

[0024] In this embodiment, firstly, the input semi-structured document is converted into a structured YAML data file. During the conversion process, a stack data structure is maintained to construct and preserve the contextual hierarchy. During this maintenance, when a title of level N is parsed, all title nodes at levels greater than or equal to N are popped from the stack, and then the title node at level N is pushed onto the stack to ensure that the top of the stack is the newest and highest-level title node to which the current content belongs. Next, based on the YAML data file, static data extracted from the YAML data file is expanded to achieve multi-source data fusion and processing. Finally, based on the report generation module of the template engine, content and format are separated, and table rows or paragraphs are dynamically generated by inserting InlineImage object placeholders and control statements. Therefore, this invention, through the optimized design of the overall technical solution, can accurately and completely extract structured information with contextual relationships from semi-structured source documents or hybrid documents with varying formats, effectively improving the parsing accuracy, data processing flexibility, and scalability of hybrid documents, and possessing the ability to fuse and process multi-source data.

[0025] In other words, the core innovation of this invention lies in its hybrid document parsing module. This module combines traversing the underlying XML object model of the document with the application of hot-swappable regular expression templates, enabling it to convert documents such as those from Microsoft Word into YAML data files that retain the original hierarchical structure with high precision, serving as the system's sole input. In particular, by maintaining a stack structure to accurately restore the contextual hierarchy of semi-structured documents, it effectively solves the technical problem of insufficient parsing accuracy and fidelity of traditional templates when dealing with complex and variable formats.

[0026] Building upon this, this embodiment further integrates an adaptive multi-source data fusion and processing module, which possesses a mechanism for handling data conflicts; it also integrates a report generation module based on a template engine, preferably including a dynamic chart generation module driven by JSON configuration and a report generation module based on the Jinja2 template engine. Optionally, as... Figure 1As shown, the embodiment also includes an intelligent polishing module (i.e., AI intelligent polishing module) using an iterative feedback optimization mechanism. The embodiment completely separates content extraction, data processing, and report style, realizes high automation, high efficiency, and high accuracy of the report generation process, and has wide application value in scientific research, business, and other fields.

[0027] The hybrid document parsing module described in the embodiment is used to convert the input semi-structured document, including but not limited to Microsoft Word (.docx) document, into a completely structured YAML data file that retains the context hierarchical relationship.

[0028] More specifically, as shown, the implementation process of the hybrid document parsing module described in the embodiment includes steps A1 to A6. Figure 2

[0029] Step A1, first load the source document using the python-docx library, and parse the input semi-structured document into a document object model (DOM); then, sequentially traverse the top-level elements in the document object model, including paragraph elements (Paragraphs) and table elements (Tables), to realize document object loading and traversal.

[0030] Steps A2 to A4 are used to realize the identification and hierarchical construction of structured elements.

[0031] Among them, step A2 is used to realize title style matching and processing. During the traversal process, the system will check the built-in style attribute (paragraph.style.name) of each paragraph element. The embodiment provides a configurable style mapping file (for example, style_map.json), that is, the correspondence between the style name in the document (including word document) and the standard title level (including levels 1 to 9) is defined in advance, for example, {"My Custom Title 1": 1, "Section StyleA": 2}, which is used as a style mapping file to find the built-in style attribute of each paragraph element, and then solve the problem that the actual title does not match the preset style ('Heading 1' and the like); if the style attribute of the paragraph element does not belong to the style mapping file, that is, it is not defined in the style mapping file, it is judged as ordinary content.

[0032] ​Step A3, used to achieve the stack-based level maintenance. By maintaining a stack (Stack) data structure for dynamic tracking of the current title level, the initial state of the stack is empty, representing the root node of the document; when a title with level N is parsed, all title nodes with level greater than or equal to N in the stack are popped out first, and then the current title node with level N is pushed into the stack. Through this "pop first and then push" mechanism design, it can be ensured that the top of the stack is the latest, highest level (deepest) title node to which the current content belongs.

[0033] Step A4, used to achieve content node attribution. First, paragraphs and tables identified as non-headers are defined as content nodes, and then content nodes are added to a pre-set key (e.g., 'content') corresponding list through programming, using the list as an attribute of the title node object at the top of the stack, and associating the list with the title node.

[0034] In Step A4 described in this embodiment, for multiple consecutive content nodes, they will be sequentially added to the same key corresponding list, i.e., the same 'content' list, thereby ensuring the integrity and order of the content.

[0035] Step A5, used to achieve content extraction and pattern matching. This step extracts content and matches patterns by reading configuration files, and when the user adjusts or adds new extraction rules, it adds new objects to achieve the adjustment and addition of extraction rules. Regarding Step A3 described in this embodiment, the specific implementation process of achieving stack level maintenance is as follows: in the src / extraction / extractor.py file (Python source code file), the stack structure is implemented by combining state machines and scope chains (ScopeChain), rather than directly using a list (List) object. This implementation method is used in the sequential traversal of the Document Object Model (DOM) to dynamically track and construct the level using the "pop first and then push" mechanism. Through the cooperation and optimization of specific workflows and algorithms, it can make the efficiency higher and the logic clearer.

[0036] The logic of the stack structure is embodied in the three variables current_section, current_subsection, and current_subsubsection in the code, which constitute a dynamic context environment that expresses the hierarchical relationship. Among them, current_section can be regarded as the bottom of the stack, representing the context of the first-level title. current_subsection is the middle layer, representing the context of the second-level title, which must exist within a current_section. current_subsubsection is the top of the stack, representing the context of the third-level title, which must exist within a current_subsection. The reference relationship of the three variables naturally forms a hierarchical stack with a depth of 3.

[0037] The implementation process of the "Pop" operation is as follows: The "Pop" operation is completed by resetting the context variables of higher levels. This is also the core of the stack level maintenance mechanism. When the parser encounters a title with level N, it means that the context of the current document has been switched, and any old context deeper than N or at the same level as N has been invalidated and must be removed from the scope.

[0038] The corresponding pseudo code is as follows: When encountering a first-level title (level == 1): if level == 1: #... create a new first-level chapter... sections.append(current_section) # Reset the sub-chapter current_subsection = None current_subsubsection = None Here, current_subsection = None and current_subsubsection = None are functionally equivalent to "popping" all nodes in the stack with a level greater than or equal to 1 (i.e., the context of the second and third-level titles that may have existed before), effectively ensuring the correct convergence and opening of new branches of the document tree.

[0039] When encountering a second-level title (level == 2): elif level == 2 and current_section is not None: ... create new subsubsection... # reset subsubsection current_subsubsection = None Similarly, the operation of current_subsubsection = None is equivalent to "popping" all nodes in the stack whose level is greater than or equal to 2 (i.e., the previous possible third-level title).

[0040] The implementation of the "push" operation is as follows: the "push" operation is completed by instantiating and associating a new title node. When a new title is identified, the system creates a new OrderedDict object to represent the title node and associates it to its parent node.

[0041] The corresponding pseudo code is as follows: When a second-level title (level == 2) is encountered: current_subsection = OrderedDict() #... fill title attributes... if'subsections' not in current_section: current_section['subsections'] = [] current_section['subsections'].append(current_subsection) The above several lines of pseudo code achieve the following effects: first, a new second-level title node current_subsection is created; second, it is added (i.e., "pushed") to the subsections list of the current first-level title current_section by the.append() method, thereby constructing the parent-child relationship in the data structure.

[0042] In summary, the above embodiment effectively implements the "pop first and push later" stack operation in logic by strictly managing the context state variables, including assignment and reset, so that when traversing the flattened document element sequence, the original document's nested hierarchical structure can be accurately and robustly reconstructed.

[0043] More specifically, the step A5 described in the embodiment preferably comprises steps A501 and A502.

[0044] Step A501, for realizing hot-pluggable regular expression template. This embodiment realizes "hot-pluggable" by externalizing regular expression rules. Unlike prior art, the system reads one or more configuration files (e.g. rules.json) at startup, instead of hardcoding rules in the program. The configuration file is a JSON array, and each object represents an extraction rule, which contains at least two keys: rule name ("name") and regular expression ("pattern").

[0045] Step A502, for realizing adjustment and addition of extraction rules. When a user adjusts or adds a new extraction rule, for example, the user needs to extract a new extraction rule "project code" from the text, the user does not need to modify any Python code, but only needs to open the configuration file, i.e. the rules.json file, and then add a new object in the JSON array of the configuration file, for example {"name": "project_code", "pattern": "PJ-[0-9]{4}-[A-Z]{2}"} and save the configuration file after adding the new object; when running next time, the hybrid document parsing module will automatically load this updated file and apply this new extraction rule in its pattern matching loop, thereby realizing extraction of "project code". Through such mechanism design, this embodiment can enable the system to quickly adapt to changing extraction requirements.

[0046] Step A6, for realizing table data processing. For table elements, i.e. when traversing to table elements, the text content of the table is extracted row by row (row) and cell by cell (cell), and is converted into a two-dimensional list or dictionary list structure.

[0047] More specifically, the implementation process of Step A6 in this embodiment includes the following sub-steps: Step A601, first, check the first row of the table. If the content of the first row meets the pre-defined table header characteristics, for example, non-empty and does not contain pure numbers, then take the first row as the key (keys), and jump to Step A602, otherwise, jump to Step A603; Step A602, traverse the remaining rows of the table, and pair the data of each row with the keys of the table header to generate a dictionary (dictionary). After the traversal is completed, all dictionaries are combined into a list; Step A603, convert the entire table into a two-dimensional list composed of a list, and store it under the node of the corresponding YAML data file.

[0048] In step A602, the process of generating the data structure that meets its requirements is as follows: instead of directly generating a dictionary list at parsing time, a more primitive and flexible structure is generated, i.e. separate storage of table headers (keys) and data rows (values). This design allows downstream applications to decide whether to combine them into a dictionary list as needed.

[0049] More specifically, the implementation process of step A602 includes step A6021 and step A6022.

[0050] Step A6021 is used to extract table headers (Keys). The pseudo-code implementation process is as follows: header_row = [] if len(table.rows) > 0: for cell in table.rows[0].cells: header_row.append(cell.text.strip()) table_info['header'] = header_row The above code segment is used to extract all cell texts in the first row of the table as table headers and store them under the 'header' key of the table_info dictionary. This is exactly the "key" set required to generate a dictionary.

[0051] Step A6022 is used to extract data rows (Values). The pseudo-code implementation process is as follows: rows = [] for i, row in enumerate(table.rows): if i == 0: # skip header continue row_data = [] for cell in row.cells: row_data.append(cell.text.strip()) rows.append(row_data) table_info['rows'] = rows The above code segment traverses all rows except the header and constructs a list of data for each row, finally forming a two-dimensional list stored under the 'rows' key. In this way, all dictionaries are combined into a list.

[0052] This implementation ultimately outputs not a [{"key": "value"}] form, but rather by providing separate 'header' and 'rows', all the information needed to pair keys and values is preserved. A program consuming this YAML data file can easily complete the generation process described in claim step A602 by the logic of [dict(zip(table_info['header'], row)) for row in table_info['rows']], to achieve its decoupled design.

[0053] Step A603, the process of converting the table into a two-dimensional list, is actually the process for extracting data rows (Values), as in step S6022. Specifically, the conversion process is as follows: first, initialize an empty list; then, start the outer loop to traverse each row object of the table, ensuring that the first row is skipped in the case of a table header, and initialize an empty sub-list at the beginning of each row loop to store the data of all cells in the current row; then, start the inner loop to traverse each cell object in the current row, extract the plain text content of the cell, and add it to the row_data sub-list; finally, after the inner loop ends, the row_data sub-list full of row data is added to the rows list as a whole element; and after the outer loop ends, the data structure of the rows variable in the rows list is returned as a two-dimensional list.

[0054] The process implemented by the code is as follows: rows = []: initialize an empty list to hold all row data.

[0055] for i, row in enumerate(table.rows):: start the outer loop to traverse each row object of the table. if i == 0: continue ensures that the first row is skipped in the case of a table header.

[0056] row_data = []: initialize an empty sub-list at the beginning of each row loop to store the data of all cells in the current row.

[0057] for cell in row.cells: : start the inner loop to traverse each cell object in the current row.

[0058] row_data.append(cell.text.strip()): extract the plain text content of the cell and add it to the row_data sub-list.

[0059] rows.append(row_data): After the inner loop ends, the row_data sub-list full of data in a row is added to the outer rows list as a whole element.

[0060] After the outer loop ends, the data structure of the rows variable is a standard two-dimensional list, such as [['R2C1', 'R2C2'], ['R3C1', 'R3C2']].

[0061] When the table has no identifiable header, the header_row will be an empty list, and the data row area can also be accurately converted into a two-dimensional list, so whether there is an identifiable header or not, this embodiment can handle both cases well.

[0062] The multi-source data fusion and processing module described in this embodiment is used to enrich and expand the static data extracted from the document. The implementation process of the multi-source data fusion and processing module includes steps B1 and B2.

[0063] Step B1 is used to construct SQL queries. This construction process is not simply using existing technologies, but is designed and optimized for the technical solution of this embodiment.

[0064] Step B1 of this embodiment uses a templated method to construct SQL queries. In the configuration file (for example, db_config.json), the SQL statement template containing placeholders is preferably predefined first, for example "SELECT * FROM testsWHERE payload_id = :id". At runtime, the multi-source data fusion and processing module will find the value corresponding to the key device_id from the YAML data file generated based on the hybrid document parsing module, and then bind the value corresponding to the preset logic to the placeholder (:id) of the SQL statement template, and then execute the query. This embodiment decouples the query logic from the business code in this way, increasing flexibility, and avoids SQL injection risks through parameter binding. The preset logic refers to the pre-set query logic, which can be customized and adjusted according to actual conditions and requirements.

[0065] Step B2, for data merging and context object generation. The query result of the database (usually a list of dictionaries) is merged with the YAML data file (loaded as a Python dictionary) as a new key-value pair. The merging process is: taking the original YAML data file loaded as a Python dictionary as the base, and then adding the database query result as a new key-value pair to the dictionary. For example, yaml_data['database_results'] = query_result.

[0066] For more complex merging, such as associating query results to specific sub-nodes of the YAML data file, recursively traverse the structure of the YAML data file to find the corresponding node of the preset logic, i.e. find the matching node, for example, the node matching device_id; then add the query result as a new attribute of the node; the final object generated, i.e. the "context" data object, is a unified and structured Python object containing all source information. Preferably, in the process of merging by adding, the problem of data conflict is solved based on timestamp or priority marking rules.

[0067] The template engine-based report generation module described in this embodiment is the key to realizing the separation of content and format. Its core is the docxtpl library, which can skillfully apply the Jinja2 syntax-based template engine (i.e. Jinja2 template engine) to the.docx document format, i.e. the docxtpl library can embed placeholders and logical expressions of Jinja2 syntax in the.docx template file, and then define variables, loops and conditional judgments in the template, and then fill these placeholders dynamically through Python code to generate the final document. Docxtpl is a Python library for generating and updating Microsoft Word (.docx) files. The Jinja2 template engine refers to a Python-based template engine.

[0068] More specifically, the implementation process of the template engine-based report generation module includes step C1 and step C2.

[0069] Step C1, used to determine the insertion position of placeholders and control statements, and implement. By default, it is preferred to open a.docx file as a template, in the position where dynamic content is needed, such as in a paragraph, in a table cell, and in the header and footer, etc. Jinja2 syntax is input in the form of plain text, for example, to display the report title, simply input the string {{ report_title}} in the title position of the document; for InlineImage object placeholders, create an InlineImage instance in the Python script and assign it to the key with the same name as the placeholder in the upper and lower text dictionaries; for control statements, read the instructions for controlling the rendering logic in the Jinja2 template engine, including loop instructions ({% for %}) and conditional judgment instructions ({% if %}). For example, to dynamically generate multiple rows in a table, the user will input {% for item in items %} outside the table (or in the first row of the template table), and input {% endfor %} after the last row of the template table, and the rows in the middle of the table use {{ item.name}} and the like to refer to the loop variable.

[0070] Step C2, used to dynamically generate table rows or paragraphs. Unlike the prior art, when the Jinja2 template engine processes a document with a loop instruction ({% for %}) under the drive of the docxtpl library, it first parses the XML structure in the loop body, at which time the XML structure represents one or more table rows / paragraphs; then, according to the length of the corresponding list in the context object, the XML structure is copied and rendered N times, each time using an element in the list to fill in the placeholder, thereby dynamically generating multiple table rows or paragraphs in the final document.

[0071] Among them, the definition and use of InlineImage object placeholders are as follows: regarding the format and syntax, in the Word template, only a simple text placeholder needs to be inserted, such as {{ my_image}}; its particularity is reflected at the Python code level. Regarding the definition in the context, when constructing the "context" data object, the InlineImage class of the docxtpl library and the unit class (such as Inches) of docx.shared need to be imported. The implementation of the provision process is to create an InlineImage instance in the Python script and assign it to the key with the same name as the placeholder in the upper and lower text dictionaries, and the example code is as follows: context['my_image'] = InlineImage(template_object, 'path / to / image.png', width=Inches(4.0)).

[0072] It is worth mentioning that in the actual research and development process, it is found that there may be running error problems.

[0073] Therefore, the embodiment also preferably comprises an error handling module. In the process of providing image information, the creation of the InlineImage object placeholder and the file reading operation are wrapped using a try...except block. If a path error (FileNotFoundError) occurs, or an image file corruption exception is thrown by the Pillow library, the exception is captured, a detailed error log is recorded, the insertion of the image is selectively skipped, and a prompt text is used to prompt the image loading failure, such as "[image loading failed]", so that the report generation process is not interrupted due to a single image problem. The embodiment also preferably sets a boundary check to avoid size out-of-range problems.

[0074] The embodiment also provides an automatic report generation method based on a hybrid parsing model and a template engine, which is applied to the automatic report generation system based on the hybrid parsing model and the template engine and comprises the following steps: Step S1, used for converting an input semi-structured document into a structured YAML data file. In the conversion process, a stack data structure is maintained to construct and retain the context hierarchical relationship. In the maintenance process, when a title with a level of N is parsed, all title nodes with a level greater than or equal to N in the stack are popped out, and then the title node with the current level of N is pushed into the stack; Step S2, based on the YAML data file, extending the static data extracted from the YAML data file; and step S3, used for realizing content and format separation, inserting InlineImage object placeholders and control statements, and dynamically generating table rows or paragraphs.

[0075] Next, the automatic generation of a Web application performance test analysis report in the actual implementation and test process will be used to illustrate the technical effects of the present application. The specific embodiment is intended to demonstrate how the present application processes an IT field performance test report and automatically generates a weekly report facing the management layer containing core conclusions and trend analysis in combination with a real-time monitoring database.

[0076] 1. Input material preparation.

[0077] Source document: WebApp_Performance_Test_Report_Q3.docx. This is a typical performance test report, which contains test environment description, test scenarios (such as user login, product browsing), performance indicator tables (such as response time, TPS), and preliminary textual analysis conclusions.

[0078] Report template: templates / weekly_performance_digest_template.docx. This template includes a pre-defined weekly report title, a summary of key performance indicators (KPI Snapshot), performance trend charts for key scenarios, bottleneck analysis and recommendations, and also includes embedded Jinja2 placeholders.

[0079] External data source: An SQLite database named monitoring.db, which contains an api_metrics table that records the performance data of each online API interface in real time, such as api_name, response_time_p95 (95th percentile response time), tps_realtime, timestamp, and other fields.

[0080] Configuration files: api_rules.json defines the regular expressions used to extract API names and performance thresholds from the test report table; chart_config.json defines the style, title, and data source for the "Key API Response Time Trend" line chart to be generated; db_config.json defines the database connection information and the SQL statement template used to query performance data for the past 7 days.

[0081] 2. Execution process.

[0082] (a) Hybrid parsing stage, such as Figure 3 As shown.

[0083] First, the hybrid document parsing module is invoked. This module loads the WebApp_Performance_Test_Report_Q3.docx document, identifies section headings such as "2.1 Test Scenario" and "3.2 Performance Results" by traversing the DOM, and constructs the report's hierarchical relationship using a stack data structure. Next, it loads the extraction rules from api_rules.json, parses the tables under the "Performance Results" section, and extracts the name of each API and its agreed-upon performance threshold in the test; for example, the login interface's response time should be less than 200ms. Finally, a structured YAML data file named performance_baseline.yaml is generated.

[0084] Example of a snippet from the performance_baseline.yaml file: report_title: "Web Application Q3 Performance Test Report" report_version: "1.2" test_scenarios: - scenario_id: "SCN-01" name: "User Login" target_api: " / api / user / login" thresholds: - metric: "response_time" operator: "<" value: 200 # ms - scenario_id: "SCN-02" name: "Browse Product Detail" target_api: " / api / product / detail" thresholds: - metric: "response_time" operator: "<" value: 150 # ms (b) Multi-source data fusion stage.

[0085] Next, the multi-source data fusion and processing module is activated. Read performance_baseline.yaml and db_config.json, connect to the monitoring.db database, and for each API in the target_api list, execute the following parameterized SQL query to obtain the P95 response time sequence for the past 7 days, and the corresponding implementation process is as follows: SELECT strftime('%Y-%m-%d', timestamp), AVG(response_time_p95) FROM api_metrics WHERE api_name = ' / api / user / login' AND timestamp >= date('now', '-7days') GROUP BY 1 ORDER BY 1.

[0086] The multi-source data fusion and processing module merges the queried time series data with the baseline thresholds in the YAML data file. The implementation process of data merging is as follows: the module loads performance_baseline.yaml as a base Python dictionary. Then, each scenario in the test_scenarios list is iterated. For the "user login" scenario, the above SQL query is executed to obtain the time series data, such as [('2025-08-14', 180.5), ('2025-08-15', 190.2)], which is assigned to a new key of the scenario dictionary, such as performance_trend. In this way, dynamic database data is accurately merged into static baseline data structures to form a context data object containing baseline, actual performance, and comparative analysis.

[0087] (c) Chart generation phase.

[0088] The dynamic chart generation module generates a line chart named login_api_perf_trend.png based on the configuration of chart_config.json and the data fused in the previous step, with two lines by default: one is a red dashed line representing the performance threshold (200ms), and the other is a blue solid line showing the actual P95 response time in the past 7 days.

[0089] (d) Report rendering phase, as shown in Figure 4

[0090] Finally, the report generation module based on the template engine loads the templates / weekly_performance_digest_template.docx template and the context object containing all data.

[0091] Some Jinja2 syntax examples in the template engine are as follows: Report title: {{ report_title}} - {{ "This Week"}} Performance Digest Conditional text: {% if login_api_perf.is_degraded %} Warning: login interface performance degraded this week! {% else %} Login interface performance is stable. {% endif %} ​Chart insertion: {{ InlineImage(docx, 'login_api_perf_trend.png', width=Inches(5))}}.

[0092] The report generation module preferably replaces all placeholders with real data through the rendering engine, dynamically generates text conclusions according to performance comparison results, and inserts trend charts. Finally, a Word document named Web Application Performance Weekly Report_2025-W33.docx is output. This report accurately extracts the performance baseline from the static test report and correlates with online real-time data for analysis, automatically generating a dynamic report containing deep insights.

[0093] The above is a further detailed description of the present application in combination with specific preferred embodiments, and cannot be regarded as limiting the specific implementation of the present application to these descriptions. For ordinary skilled persons in the technical field to which the present application belongs, without departing from the concept of the present application, a number of simple deductions or substitutions can be made, which should be regarded as falling within the protection scope of the present application.

Claims

1. An automatic report generation system based on a hybrid parsing model and a template engine, characterized in that, include: The hybrid document parsing module is used to convert the input semi-structured document into a structured YAML data file. During the conversion process, the context hierarchy is built and preserved by maintaining a stack data structure. During the maintenance process, when a title of level N is parsed, all title nodes of level N or higher are popped from the stack, and then the title node of level N is pushed onto the stack. N is a natural number. The multi-source data fusion and processing module, based on YAML data files, expands the static data extracted from the YAML data files according to preset logic; Additionally, a report generation module based on a template engine is used to separate content from format, insert InlineImage object placeholders and control statements, and dynamically generate table rows or paragraphs.

2. The automatic report generation system based on a hybrid parsing model and template engine according to claim 1, characterized in that, The implementation process of the hybrid document parsing module includes the following steps: Step A1: First, use the python-docx library to load the source document, parse the input semi-structured document into a document object model, and then traverse the top-level elements in the document object model in order. The top-level elements include paragraph elements and table elements. Step A2: Predefine the correspondence between style names in the document and standard heading levels, and use this as a style mapping file. Use this style mapping file to find the built-in style attributes of each paragraph element. If the style attribute of a paragraph element does not belong to the style mapping file, it is judged as ordinary content. Step A3 involves maintaining a stack data structure to dynamically track the current heading level. The stack is initially empty, representing the root node of the document. When a heading of level N is parsed, all heading nodes of level N or higher are popped from the stack, and then the heading node of level N is pushed onto the stack to ensure that the top of the stack is the heading node of the highest level to which the current content belongs. Step A4: First, define the paragraphs and tables that are identified as non-headings as content nodes. Then, add the content nodes to a list corresponding to a pre-defined key through programming. Use this list as an attribute of the heading node at the top of the stack and associate the list with the heading node. Step A5 involves extracting content and matching patterns by reading the configuration file, and adjusting or adding extraction rules by adding new objects when the user adjusts or adds new extraction rules. Step A6: For table elements, extract the text content of the table row by row and cell by cell, and convert it into a two-dimensional list or dictionary list structure.

3. The automatic report generation system based on a hybrid parsing model and template engine according to claim 2, characterized in that, In step A4, multiple consecutive content nodes will be added sequentially to the list corresponding to the same key.

4. The automatic report generation system based on a hybrid parsing model and template engine according to claim 2, characterized in that, Step A5 includes the following sub-steps: Step A501: Read one or more configuration files. Each configuration file is a JSON array, where each object represents an extraction rule. The extraction rule includes at least two keys: rule name and regular expression. Step A502: When the user adjusts or adds new extraction rules, open the configuration file, add a new object to the JSON array in the configuration file, and save the configuration file after adding the new object; the next time it runs, read the latest saved configuration file.

5. The automatic report generation system based on a hybrid parsing model and template engine according to claim 2, characterized in that, Step A6 includes the following sub-steps: Step A601: Check the first row of the table. If the content of the first row matches the predefined header characteristics, then use the first row as the key and jump to step A602; otherwise, jump to step A603. Step A602: Traverse the remaining rows of the table, pair the data in each row with the key in the table header, generate a dictionary, and combine all the dictionaries into a list after the traversal is complete. Step A603: Convert the entire table into a two-dimensional list composed of lists and store it under the node of the corresponding YAML data file.

6. The automatic report generation system based on a hybrid parsing model and template engine according to any one of claims 1 to 5, characterized in that, The implementation process of the multi-source data fusion and processing module includes the following steps: Step B1: In the configuration file, predefine SQL statement templates containing placeholders; at runtime, based on the YAML data file generated by the hybrid document parsing module, first bind the values ​​corresponding to the preset logic to the placeholders in the SQL statement template according to the preset logic, and then execute the query. Step B2: Merge the query results as new key-value pairs with the YAML data file.

7. The automatic report generation system based on a hybrid parsing model and template engine according to claim 6, characterized in that, In step B2, when the query result needs to be associated with a specific child node of the YAML data file, the structure of the YAML data file is recursively traversed. First, the node corresponding to the preset logic is found, and then the query result is added as a new attribute of the node. During the merging process by adding data, rules based on timestamps or priority markers are used to resolve data conflicts.

8. The automatic report generation system based on a hybrid parsing model and template engine according to any one of claims 1 to 5, characterized in that, The implementation process of the template engine-based report generation module includes the following steps: Step C1: At the location where content needs to be dynamically generated, input Jinja2 syntax in plain text. For InlineImage object placeholders, create an InlineImage instance in the Python script and assign it to the key with the same name as the placeholder in the context dictionary. For control statements, read the instructions used to control the rendering logic from the Jinja2 template engine, including loop instructions and conditional judgment instructions. In step C2, when the Jinja2 template engine processes a document with loop instructions under the drive of the docxtpl library, it first parses the XML structure within the loop body. At this point, the XML structure represents one or more table rows / paragraphs. Then, based on the length of the corresponding list in the context object, this XML structure is copied and rendered N times, with each time a placeholder is filled with an element from the list, thus dynamically generating multiple rows of tables or multiple paragraphs in the final document.

9. The automatic report generation system based on a hybrid parsing model and template engine according to any one of claims 1 to 5, characterized in that, It also includes an error handling module that uses a try...except block to wrap the creation of InlineImage object placeholders and file reading operations during the provision of image information. If a path error occurs or an image file corruption exception is thrown by the Pillow library, the exception will be caught, a detailed error log will be logged, the insertion of the image will be skipped, and a prompt message will be displayed indicating that the image loading failed, so that the report generation process will not be interrupted due to a single image problem. It also sets boundary checks to avoid out-of-range sizes.

10. An automatic report generation method based on a hybrid parsing model and template engine, characterized in that, The automatic report generation system based on a hybrid parsing model and template engine, as described in any one of claims 1 to 9, includes the following steps: Step S1 is used to convert the input semi-structured document into a structured YAML data file. During the conversion process, the context hierarchy is constructed and preserved by maintaining a stack data structure. During the maintenance process, when a title of level N is parsed, all title nodes of level N or higher are popped from the stack, and then the title node of level N is pushed onto the stack. Step S2: Expand the static data extracted from the YAML data file based on the YAML data file; And step S3 is used to separate content from formatting, insert InlineImage object placeholders and control statements, and dynamically generate table rows or paragraphs.

Images