Method for quickly generating report template according to picture

Abstract

The invention discloses a method for quickly generating a report template according to a picture, which relates to the technical field of industrial reports and comprises the following steps of: performing picture information reasoning and table row and column structure analysis on the picture of a table to be extracted, and outputting an extraction result in a structured document form; analyzing a hidden hierarchical relationship in a header area of the report template to obtain text description information of all measurement points in a table; the method comprises the following steps: performing vectorization processing on an equipment model in an industrial monitoring system, storing hierarchical relationship information of the equipment model, obtaining a preset number of candidate sets, inputting the candidate sets into a multi-modal large language model to perform measurement point ID matching, and generating a corresponding data filling rule in combination with a report time level and a measurement point statistical type. According to the method, the report template meeting the requirement can be quickly obtained through a small amount of manual intervention, the report template generation efficiency is greatly improved, and the method has wide application prospects in the field of industrial monitoring reports.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial report, and in particular to a method for quickly generating a report template according to a picture. BACKGROUND

[0002] In the field of industrial monitoring, the report function plays a key role. It can read the device running status from the historical data storage to generate various production reports in a certain format and period, such as daily reports, monthly reports, etc., covering device running and production running, etc., to provide comprehensive and accurate information for the daily management of enterprises. The traditional report template configuration is generally based on manual setting of report style and data filling rules. Its core process can be divided into two major links: 1. Through the visual interface, drag and drop controls (text box, chart, table) for layout design, complete the static layout construction of the report template; 2. Define the corresponding relationship between the measurement point, statistical method and template position one by one, and need to clearly mark the data source of each cell (such as database table field, calculation formula, etc.). This method has a certain flexibility, which is convenient for engineering personnel to configure, but still has a certain learning cost for the end user.

[0003] Under the background of digital transformation, report generation technology has experienced evolution from code hardcoding to intelligent driving. The traditional report system based on manual configuration needs to manually drag and drop controls and bind data sources, which has the pain points of complex operation and poor flexibility.

[0004] In recent years, two types of innovative paths have emerged in the industry: one focuses on natural language processing (NLP) based chart generation. Users input natural language description, and the system analyzes the semantics through NLP techniques such as word segmentation, part-of-speech tagging, and dependency syntax analysis to extract keywords. According to the keywords, match the preset report template, determine the parameters according to the syntax structure analysis, automatically replace the default parameters of the template and generate the chart. Another type of report generation based on large models, users input report generation information (such as target data model and query conditions), pre-debugging large model automatically generates SQL query statement, directly queries data model, extracts data and generates report. The large model is debugged by configuring data model, index library, constraint conditions, etc. to ensure understanding of business terminology and data relationship. Its advantages are to support complex business logic, directly generate SQL and real-time data acquisition, adapt to new business scenarios, and improve the intelligence and flexibility of report generation.

[0005] As disclosed in Chinese application No. CN202510054405.1, a report generation method, device and electronic equipment based on a large model, by acquiring report generation information input by a user, accurately positioning the target data model and query condition required for query, automatically generating an SQL query statement using a large model debugged by a business model, directly querying the target data model, quickly acquiring data required for a report, and generating a target report upon receiving a user instruction. This method automatically generates an SQL query statement using a large model debugged by a business model, directly queries the target data model, quickly acquires data required for a report, dynamically generates a report, and solves the low efficiency problem of traditional report generation. However, it has the problem of SQL generation error caused by machine hallucination, although it performs error checking through SQL legality checking, it still cannot cover all errors, resulting in the generation of a problematic report.

[0006] For example, application No. CN201910426646.9 discloses a data chart generation method and related device based on natural language processing, which includes: obtaining target natural language data input by a target user, the target natural language data being natural language data related to generating a data chart; performing word segmentation and semantic analysis on the target natural language data based on natural language processing; determining at least one data chart function template corresponding to a keyword sequence; assembling the at least one data chart function template according to a syntax structure feature; and sequentially calling and executing data chart function templates in a data chart function template set. This method converts natural language requirements into an automatic chart generation process through NLP technology, significantly reducing the operation threshold of non-technical personnel. Its innovation lies in the semantic-driven template dynamic assembly mechanism, which realizes end-to-end automatic processing from requirement understanding to chart presentation. However, there is a semantic understanding deviation problem, and NLP technology has semantic analysis errors when processing complex or ambiguous user requirements, such as Chinese word segmentation deviation, which can cause keyword extraction errors and thus generate incorrect report templates. The generated chart or report does not meet the expectations and requires manual secondary adjustment. SUMMARY

[0007] Therefore, it is necessary to provide a method for quickly generating a report template according to a picture in view of the above technical problems.

[0008] The present application provides a method for quickly generating a report template according to a picture, comprising:

[0009] S1, based on a preset multi-modal large language model, performing picture information reasoning and table row and column structure analysis on a picture of a table to be extracted, and outputting an extraction result in the form of a structured document as structure and style data of a report template;

[0010] S2, by analyzing the implicit hierarchical relationship in the header area of the report template, obtaining the text description information of all measurement points in the table, and specifying the report time level and extracting the measurement point statistical type;

[0011] S3, the device model in the industrial monitoring system is vectorized, the hierarchical relationship information of the device model is saved, and a preset number of candidate sets are obtained, which are input into the multi-modal large language model for measurement point ID matching, combined with the report time level and the measurement point statistical type, to generate corresponding data filling rules.

[0012] Further, based on the preset multi-modal large language model, the picture information of the picture to be extracted is inferred and the table row and column structure is analyzed, and the extraction result in the form of structured document is output, including:

[0013] S11, obtain table image samples participating in multi-modal large language model evaluation, respectively extract structured annotation data of each table image sample, and generate a verification data set;

[0014] S12, sequentially select a plurality of multi-modal large language models to recognize the table sample image, and evaluate the table extraction capability of each multi-modal large language model based on a preset evaluation method;

[0015] S13, input the picture of the table to be extracted into the multi-modal large language model with the best table extraction capability, and obtain the row and column structure of the table and the content of each cell by multi-modal reasoning, and output the table data in a preset data format.

[0016] Further, the evaluation of the table extraction capability of each multi-modal large language model based on the preset evaluation method includes:

[0017] S121, input the verification data set into each multi-modal large language model for reasoning analysis, and calculate the text content matching degree of each cell of each multi-modal large language model on the verification data set;

[0018] S122, calculate the report template structure matching degree of each cell of each multi-modal large language model on the verification data set, to measure the ability of the multi-modal large language model in cell merging and splitting;

[0019] S123, respectively assign weights to the text content matching degree and the report template structure matching degree, and calculate the table extraction capability of each multi-modal large language model using a comprehensive capability formula.

[0020] Further, the expression of the comprehensive capability formula is:

[0021] ;

[0022] In the formula, Indicates the ability to extract data from tables; Indicates the adjustable weighting coefficient; Indicates the degree of text content matching; This indicates the degree of matching between the report template structure and the actual structure.

[0023] Furthermore, by parsing the implicit hierarchical relationships in the header area of ​​the report template, the text description information of all measurement points in the table is obtained, and the report time level and the statistical types of the extracted measurement points are specified, including:

[0024] S21. Pre-specify the header area of ​​the report template, and read the text content of the cells and their corresponding parent cells in sequence through a recursive search method. Then, merge the text content into a cell combination to extract the implicit relationship of the header of the report template and form the text description information of the measurement points.

[0025] S22. Specify the time level of the report template according to the preset time requirements;

[0026] S23. Perform semantic analysis on the text content of the cell combination, and extract the statistical type corresponding to the measurement point through keyword matching, pattern recognition and implicit inference.

[0027] Furthermore, the header area of ​​the report template is pre-defined. The text content of each cell and its corresponding parent cell is read sequentially using a recursive search method, and the text content is merged into a cell group to extract the implicit hierarchical relationship of the report template header, including:

[0028] S211. Pre-specify the header area in the report template, traverse all cells in the last row of the header area, and search upwards in the column where the cell is located, setting the first header cell that crosses the current cell as the parent cell;

[0029] S212. Prioritize reading the text content of the last cell. When a parent cell is found, read the text content of the parent cell and merge the current cell content with the parent header content into a cell combination according to a predetermined format, which will serve as the basis for matching the measurement point ID.

[0030] S213. After finding a parent cell, proceed recursively. If a parent cell exists in the previous level, continue searching upwards for a parent cell. If no parent cell exists in the previous level, stop recursion and output the final recognition result.

[0031] Furthermore, the equipment models in the industrial monitoring system are vectorized to save the hierarchical relationship information of the equipment models. A preset number of candidate sets are obtained and input into a multimodal large language model for measurement point ID matching. Combined with the report time level and measurement point statistical type, corresponding data filling rules are generated, including:

[0032] S31. Construct a directed acyclic graph using the device model. Based on the iterative depth-first algorithm of explicit stack, construct a mapping set of measurement point IDs and measurement point names. Then, add measurement point path text to the mapping set by traversing and exploring all child nodes.

[0033] S32. Use a text embedding model to convert each measurement point path text in the mapping set into a sentence vector in order to compare the semantic similarity between different measurement point path texts.

[0034] S33. Calculate and fuse the semantic similarity and path similarity between sentence vectors, input them into the multimodal large language model for analysis and reasoning, and generate corresponding data filling rules in the report template by combining the measurement point ID, report time level and measurement point statistical type.

[0035] Furthermore, a directed acyclic graph is constructed using the device model. Based on an iterative depth-first search algorithm using an explicit stack, a mapping set of measurement point IDs and measurement point names is built. By traversing and exploring all child nodes, measurement point path text is added to the mapping set, including:

[0036] S311. Based on the equipment model of the industrial monitoring system, construct a directed acyclic graph, where nodes represent equipment, locations or measurement points, and edges represent hierarchical containment relationships.

[0037] S312. Create an empty explicit stack, an empty set, and an empty mapping set respectively, and push all root nodes of the device model onto the explicit stack.

[0038] S313. If the explicit stack is not empty, pop an element from the top of the explicit stack. If the popped element is not in the collection, add the element to the collection. The element includes the current node and a list of hierarchical paths from the root node to the current node.

[0039] S314. Determine whether the current node is a leaf node. If it is a leaf node, further determine whether the current node is a measurement point. If the current node is a measurement point, connect all the node names in the hierarchical path list according to a predetermined format and append them to the base name of the current node to form the measurement point path text. Then, add the unique ID of the measurement point as the key and the measurement point path text as the value to the mapping set.

[0040] If the current node is a leaf node that is not a measurement point, then skip the current node;

[0041] S315. For each unvisited child node of the current node, construct a new path list and push the unvisited child nodes and the new path list onto the explicit stack.

[0042] S316. When the explicit stack is empty, the algorithm terminates and uses the mapping set to store all valid measurement point IDs and their corresponding measurement point path texts.

[0043] Furthermore, a text embedding model is used to convert each measurement point path text in the mapping set into a sentence vector, in order to compare the semantic similarity between different measurement point path texts, including:

[0044] S321. Input the measurement point path text into the pre-trained base model in the text embedding model, perform deep bidirectional processing on the measurement point path text, and generate word vectors for each word element in the measurement point path text.

[0045] S322. Connect a pooling layer after the sequence of the base model to perform an aggregation operation on all word vectors output by the base model, so as to convert each vector sequence into a single sentence vector of fixed length.

[0046] S323. Store all sentence vectors in the vector database, and use the sentence vectors to save the path text of the measurement point they represent and the corresponding measurement point ID together.

[0047] Furthermore, semantic similarity and path similarity between sentence vectors are calculated and fused, then input into a multimodal large language model for analysis and reasoning. Combining the measurement point ID, report time level, and measurement point statistical type, corresponding data filling rules are generated in the report template, including:

[0048] S331. Traverse all text description information in the report template and calculate the cosine similarity between each text description information and all sentence vectors to measure the semantic similarity between path texts at different measurement points.

[0049] S332. Divide the measurement point paths in the equipment model and the text description information in the report template according to the delimiter to obtain the node sequence, and set the allowed editing operations. Solve the minimum number of editing operations required from the candidate path to the report measurement point path. Combine the maximum distance between the candidate path and the report measurement point and the editing operation distance to calculate the path similarity between different measurement point path texts.

[0050] S333. The normalized path similarity and semantic similarity are fused according to weights. The fusion result is used to sort all candidate measurement points, and a preset number of candidate measurement points are selected to form a candidate set (measurement point ID set).

[0051] S334. Based on the candidate set, retrieve a preset number of candidate measurement point information from the vector database, combine it with text description information to construct the context, and obtain the model input text.

[0052] S335. Utilize a multimodal large language model to perform deep semantic analysis on the input text of the model, compare the semantic correlation between the path text of each candidate measurement point and the description to be matched, and obtain the final measurement point ID and the corresponding similarity.

[0053] S336. A similarity threshold is preset. When the similarity output by the multimodal large language model is lower than the similarity threshold, a preset number of candidate measurement IDs are visualized and debugged. When the similarity output by the multimodal large language model is higher than the similarity threshold, corresponding filling rules are generated in the report template according to the measurement point ID, the report time level, and the measurement point statistical type.

[0054] The beneficial effects of this invention are as follows:

[0055] 1. This invention utilizes a multimodal large model to extract tables from images, bypassing potential errors in the semantic understanding process and opening up a new path for report template generation. It can identify the structure and text content of tables and form a static layout for report templates. In terms of template data filling rule construction, it combines equipment models and processes the hierarchical relationships in the header data definition description to determine the measurement point ID and filling rules for each data cell. Non-technical personnel only need to upload a sample image, and with minimal manual intervention, they can quickly obtain a report template that meets their needs, greatly improving the efficiency of report template generation. It has broad application prospects in the field of industrial monitoring reports.

[0056] 2. By systematically calculating and evaluating the performance of mainstream multimodal large models on table extraction tasks, the optimal model is selected. Compared with arbitrarily selecting models, the data-driven selection method can ensure that the model is highly adapted to the table extraction task, thereby significantly improving the accuracy of table extraction, reducing the error extraction rate, and making subsequent data processing more reliable.

[0057] 3. This invention utilizes the multimodal capabilities of large models to extract tables from images, thereby enabling the rapid construction of static layouts for report templates and greatly simplifying the traditional manual drawing of report templates.

[0058] 4. By parsing the implicit hierarchical relationship of multiple table headers to obtain the measurement point description text, and combining it with retrieval enhancement technology, this invention can automatically match the measurement point ID corresponding to the cell text, and then generate data filling rules; this greatly simplifies the workload of manually configuring data filling rules in traditional report templates. Attached Figure Description

[0059] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0060] Figure 1 This is a flowchart of a method for quickly generating report templates from images according to an embodiment of the present invention;

[0061] Figure 2 This is a flowchart illustrating the determination of the optimal multimodal large language model in a method for quickly generating report templates from images according to an embodiment of the present invention.

[0062] Figure 3 This is a flowchart illustrating a method for quickly generating report templates from images according to an embodiment of the present invention, which extracts measurement point description information from the template.

[0063] Figure 4 This is a flowchart illustrating the extraction of implicit hierarchical relationships in the header of a method for quickly generating report templates from images according to an embodiment of the present invention.

[0064] Figure 5 This is a flowchart illustrating the process of obtaining a candidate set in a method for quickly generating a report template from an image according to an embodiment of the present invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0066] Please see Figure 1 This provides a method for quickly generating report templates from images, including:

[0067] S1. Based on the preset multimodal large language model, perform image information reasoning and table row and column structure parsing on the images of the table to be extracted, and output the extraction results in the form of a structured document as the structure and style data of the report template.

[0068] Specifically, Multimodal Large Language Models (MLLMs) can integrate and process multiple types of data, including text, images, audio, and video, to achieve a more comprehensive understanding and generate integrated responses. This capability enables MLLMs to simultaneously interpret text descriptions, analyze corresponding images, and generate outputs that integrate both input formats, thus demonstrating high versatility in handling complex tasks.

[0069] MLLMs directly interpret visual layouts, thus avoiding errors that may occur in the semantic understanding stage of traditional NLP. The characteristics of MLLMs, especially their unified attention mechanism and ability to handle noisy real-world data, are key factors in achieving this goal. This means that the powerful cross-modal reasoning capabilities of MLLMs are crucial for accurately identifying table structures and cell content, as they can infer the relationships between visual elements and embedded text, relationships that pure OCR-NLP pipelines may miss.

[0070] The holistic understanding makes it more robust to ambiguities or errors that may arise from imperfect text extraction by OCR. From current mainstream multimodal large models (such as OPENAI GPT-4o, Gemini 2.5, Qwen2.5-VL, Llama 3.2Vision, etc.), the performance of each model on the table extraction task was systematically calculated and evaluated, and the model with the best table extraction capability was selected.

[0071] This invention inputs an image of the table to be extracted into a selected model. Based on its multimodal analysis capabilities, the model infers the information in the input image, parses the row and column structure of the table, identifies the content of each cell, and outputs the extraction results in the form of a structured document, which serves as the structure and style data for the report template.

[0072] In the description of this invention, based on a preset multimodal large language model, image information reasoning and table row and column structure parsing are performed on the images of the table to be extracted, and the extraction results in the form of a structured document are output, including:

[0073] S11. Obtain table image samples for multimodal large language model evaluation, extract structured annotation data for each table image sample, and generate a validation dataset.

[0074] Specifically, a variety of tabular image samples are used, covering the following types:

[0075] 1. Table image base datasets (such as SciTSR scientific table structure recognition dataset, VIDoRe visual document retrieval) are used to evaluate the generalization ability of multimodal large models to recognize general tables.

[0076] 2. Typical report datasets in the field of industrial monitoring (such as typical daily and monthly operation reports in power automation monitoring) are used to evaluate the ability of multimodal large models to recognize report tables.

[0077] Each sample corresponds to a high-quality structured labeled dataset, which serves as a validation dataset for evaluating the performance of multimodal large models.

[0078] S12. Select multiple multimodal large language models in sequence to recognize the table sample images, and evaluate the table extraction capability of each multimodal large language model based on the preset evaluation method.

[0079] Specifically, large models with multimodal capabilities (OPENAI GPT-4o, Gemini 2.5, Qwen2.5-VL, Llama 3.2 Vision, etc.) were selected to identify the sample data. Since the identification results of different models vary, a pre-defined evaluation method was used to calculate the table extraction capability of each model to ensure the stability and accuracy of the final output. .

[0080] In the description of this invention, as Figure 2 As shown, the table extraction capabilities of each multimodal large language model are evaluated based on a pre-defined evaluation method, including:

[0081] S121. Input the validation dataset into each multimodal large language model for inference analysis, and calculate the degree of text content matching of each multimodal large language model in the validation dataset.

[0082] Specifically, the model (Multimodal Language Large Model) Validate the degree of cell text content matching on the dataset. The definition is as follows:

[0083] ;

[0084] In the formula, The cell content predicted by the model; This is real labeled data; This represents the number of common characters between the predicted text and the actual text. This represents the total number of characters after deduplication of the predicted text and the real text; M represents the total number of cells on the validation dataset.

[0085] The specific calculation steps are as follows:

[0086] 1. Iterate through each cell and obtain the content of each cell predicted by the model. And obtain the actual text content of the corresponding cell in the validation dataset. .

[0087] 2. Calculate the common character set of both and count the number of matching characters appearing in the same position. Obtain the total character set of both texts, remove duplicates, and calculate the total number of characters in the two texts. .

[0088] 3. Calculate the matching score according to the formula. A higher score indicates a higher degree of matching between the cell text content and the given score.

[0089] S122. Calculate the report template structure matching degree of each multimodal large language model in the cell on the validation dataset to measure the ability of the multimodal large language model in cell merging and splitting.

[0090] Specifically, the model Verify the matching degree of the report template structure on the dataset. , measurement model Its performance in cell merging and splitting is defined as follows:

[0091] ;

[0092] In the formula, This represents the set of table cells predicted by model j, including their position (row and column range) and merging status (whether they span rows / columns). A collection of actually labeled table cells, with structural information and Same format; This represents the correctly matched portion of the predicted structure and the actual structure. The calculation takes into account the cell position (row and column index) and whether the merged information is consistent. This represents the combined portion of the predicted and true table structures; P represents the total number of cell structure elements in the validation dataset used for structure matching.

[0093] While considering cell position, the model also measures the degree of matching in complex cases such as merged cells to ensure a more accurate evaluation of its structural prediction capabilities. The specific calculation steps are as follows:

[0094] 1. Iterate through each cell and check its starting row and column coordinates in the large model's predicted table and the actual table. If it is a merged cell, check its row / column span range. If the cell's starting row, starting column, row span, and column span all match, then the cell is considered a successful match.

[0095] 2. Count the number of all correctly matched cells. The model then deduplicates all cell data in both the predicted and actual tables based on their starting row and column coordinates and row and column spans, resulting in... .

[0096] 3. Calculate the matching score according to the formula. A higher score indicates a higher degree of structural matching.

[0097] S123. Assign weights to the text content matching degree and the report template structure matching degree respectively, and calculate the table extraction ability of each multimodal large language model using the comprehensive ability formula.

[0098] In the description of this invention, the expression for the comprehensive capability formula is:

[0099] ;

[0100] In the formula, Indicates the ability to extract data from tables; Indicates the degree of text content matching; Indicates the degree of matching between the report template structure and the actual content. The weighting coefficients are adjustable and can be flexibly adjusted in different scenarios.

[0101] By weighted calculation, a comprehensive score is obtained for each model, and the model with the highest comprehensive score is selected as the optimal model.

[0102] S13. Input the image of the table to be extracted into the multimodal large language model with the best table extraction capability. Obtain the row and column structure of the table and the content of each cell through multimodal reasoning, and output the table data in the preset data format.

[0103] Specifically, the image of the table to be extracted is input into the optimal multimodal large language model. Using prompt word engineering, based on the input image information, the model uses its multimodal capabilities to perform reasoning to obtain the table's row and column structure (starting coordinates, number of rows spanned, number of columns spanned, etc.) and the content of each cell (including text and format), and outputs the table data according to a certain data format (such as JSON, Markdown, etc.).

[0104] Here are some examples of prompt words:

[0105] #Role Definition#

[0106] "You are a professional table recognition assistant, skilled at parsing complex table structures and preserving original layout features."

[0107] #Task Breakdown#

[0108] 1. Identify all table areas in the image and mark the locations of merged cells (row merging / column merging);

[0109] 2. Generate structured data and mark merged cells using the rowspan / colspan attributes;

[0110] 3. Output format: Visual preview format (Markdown table with merge markers).

[0111] S2. By parsing the implicit hierarchical relationship in the header area of ​​the report template, obtain the text description information of all measurement points in the table, and specify the report time level and the statistical type of the extracted measurement points.

[0112] Specifically, the header area of ​​a report template typically contains information about the measurement points (such as name, unit, location, and associated equipment). At the same time, it usually uses multi-line headers and merged cells to describe the associated equipment of the measurement points in a hierarchical manner, with the aim of visually structuring the hierarchical relationship.

[0113] Step S2 primarily involves parsing the implicit hierarchical relationships in the table header area to obtain the textual description information of all measurement points in the table. It also provides a basis for generating data filling rules in subsequent steps by specifying the report time level and extracting the statistical types of the measurement points.

[0114] In the description of this invention, as Figure 3 As shown, by parsing the implicit hierarchical relationship in the header area of ​​the report template, the text description information of all measurement points in the table is obtained, and the report time level and the statistical types of the extracted measurement points are specified, including:

[0115] S21. Pre-specify the header area of ​​the report template, and read the text content of the cells and their corresponding parent cells in sequence through a recursive search method. Then, merge the text content into a cell group to extract the implicit layer relationship of the header of the report template and form the text description information of the measurement points.

[0116] Specifically, during the recursive upward search process in step S2, when reading the text content of the current cell and the parent cell, in addition to performing structural combination, the program will also perform semantic analysis on the content of each cell.

[0117] The header data definition area in the report template is manually specified. The program reads the text content of the specific measurement point cell in the last row, searches upwards in the same column to find the first header cell that spans that cell, and uses it as the parent cell. It then reads the text content of the parent header cell and combines the current cell's content with the parent header content according to a predetermined format. This combination serves as the basis for subsequent measurement point ID recognition steps. This process is then recursively repeated until the end. See the detailed process below. Figure 4 .

[0118] In the description of this invention, as Figure 4 As shown, the header area of ​​the report template is pre-defined. A recursive search method is used to sequentially read the text content of each cell and its corresponding parent cell, and the text content is merged into a cell group to extract the implicit hierarchy of the report template header, including:

[0119] S211. Pre-specify the header area in the report template, traverse all cells in the last row of the header area, and search upwards in the column where the cell is located, setting the first header cell that crosses the current cell as the parent cell.

[0120] S212. Prioritize reading the text content of the last cell. When a parent cell is found, read the text content of the parent cell and merge the current cell content with the parent header content into a cell combination according to a predetermined format, which will serve as the basis for matching the measurement point ID.

[0121] S213. After finding a parent cell, proceed recursively. If a parent cell exists in the previous level, continue searching upwards for a parent cell. If no parent cell exists in the previous level, stop recursion and output the final recognition result.

[0122] S22. Specify the time level of the report template according to the preset time requirements.

[0123] Specifically, the time level of the report template (such as daily, weekly, monthly, etc.) is manually specified so that data population rules can be generated later.

[0124] S23. Perform semantic analysis on the text content of the cell combination, and extract the statistical type corresponding to the measurement point through keyword matching, pattern recognition and implicit inference.

[0125] Specifically, this step performs semantic analysis on the combined content to extract the statistical types corresponding to the measurement points.

[0126] 1. Maintain an internal, scalable dictionary of commonly used statistical keywords. This dictionary will include explicit terms such as “instantaneous value”, “mean”, “maximum value”, “minimum value”, and “cumulative value”.

[0127] 2. Perform statistical type identification on the combined content in step S22.

[0128] Statistical keyword extraction includes: direct keyword matching, checking for the presence of any statistical keywords from the dictionary in the cell text; and pattern recognition and implicit inference, using regular expressions to identify implicit statistical meanings. For example, "average daily temperature" implies "mean value"; "maximum air pressure" implies "maximum value"; and "total power generation" implies "cumulative value." If a statistical keyword appears in a parent cell that directly spans the measurement point, it is highly relevant to that measurement point. Priority is assigned to statistics in cells closer to the measurement point in the hierarchy.

[0129] S3. Vectorize the equipment models in the industrial monitoring system, save the hierarchical relationship information of the equipment models, obtain a preset number of candidate sets, input them into the multimodal large language model for measurement point ID matching, and generate corresponding data filling rules by combining the report time level and measurement point statistical type.

[0130] Specifically, the equipment models in the industrial monitoring system are vectorized and their hierarchical relationship information is saved to achieve better matching results.

[0131] This invention employs Retrieval Enhanced Generation (RAG) technology to obtain information on the Top N candidate measurement points. The candidate set is then input into a large model to achieve accurate matching of measurement point IDs. Based on the matched measurement point IDs and the report time level, corresponding data filling rules are generated.

[0132] In the description of this invention, the equipment model in the industrial monitoring system is vectorized, the hierarchical relationship information of the equipment model is saved, and a preset number of candidate sets are obtained. These sets are then input into a multimodal large language model for measurement point ID matching. Combined with the report time level and measurement point statistical type, corresponding data filling rules are generated, including:

[0133] S31. Construct a directed acyclic graph using the device model. Based on an iterative depth-first algorithm using an explicit stack, construct a mapping set of measurement point IDs and measurement point names. Then, add measurement point path text to the mapping set by traversing and exploring all child nodes.

[0134] Specifically, considering the possibility of duplicate measurement point / equipment names and the complexity of industrial equipment models, an iterative depth-first algorithm with an explicit stack is used. An explicit stack data structure is used to manage the nodes to be accessed, avoiding the stack limitations of the operating system and enabling it to handle large-scale industrial equipment models.

[0135] Starting from the root node, recursively traverse all child nodes. If a leaf node is not a measurement point, remove that path. Finally, for each measurement point, generate a depth-first search path that includes all nodes. Append the complete path information to the measurement point name to obtain the measurement point path text, forming a more contextual text representation.

[0136] In the description of this invention, a directed acyclic graph is constructed using a device model. An iterative depth-first search algorithm based on an explicit stack is used to build a mapping set of measurement point IDs and measurement point names. Then, by traversing and exploring all child nodes, measurement point path text is added to the mapping set, including:

[0137] S311. Based on the equipment model of the industrial monitoring system, construct a directed acyclic graph, where nodes represent equipment, locations, or measurement points, and edges represent hierarchical containment relationships.

[0138] S312. Create an empty explicit stack, an empty set visited, and an empty map set device_map (as the final output), and push all root nodes of the device model onto the explicit stack.

[0139] Specifically, all root nodes of the device model are pushed onto the stack. To track the path during traversal, each element pushed onto the stack should contain the current node and its complete path information from the root node to that node. For example, a tuple (node, current_path_list) can be pushed onto the stack.

[0140] S313. If the explicit stack is not empty, pop an element from the top of the explicit stack. If the popped element is not already in the collection, add it to the collection. The elements include the current node and a list of hierarchical paths from the root node to the current node.

[0141] Specifically, the algorithm enters a loop, and as long as the stack is not empty, it continues to execute the following steps:

[0142] Pop a node: Pop an element (current_node, current_path_list) from the top of the stack. current_node is the current node, i.e., the device or measurement point currently being processed, and current_path_list is a list of hierarchical paths from the root node to current_node.

[0143] Access check: If current_node is not already in the visited collection, add it to the visited collection.

[0144] S314. Determine if the current node is a leaf node. If it is a leaf node, further determine if the current node is a measurement point. If the current node is a measurement point, concatenate all node names in the hierarchical path list according to a predetermined format and append them to the base name of the current node to form the measurement point path text. Add the measurement point's unique ID as the key and the measurement point path text as the value to the mapping set.

[0145] If the current node is a leaf node that is not a measurement point, then skip the current node.

[0146] Specifically, leaf node identification and pruning:

[0147] Determine if current_node is a leaf node (i.e., it has no child nodes in the device model).

[0148] If current_node is a leaf node, then it is further determined whether it is a "measurement point". This determination is usually based on predefined measurement point attributes or comparison with the measurement point master data.

[0149] If current_node is a leaf node of a measurement point:

[0150] The measurement point path text is formed by concatenating all node names (including the name of current_node) in current_path_list according to a predefined format (separated by hyphens "-") and appending them to the base name of current_node. For example: "Area A-Substation 1-Bay X-Equipment B-Current".

[0151] Use the unique ID of the measurement point as the key and the generated "measurement point path text" as the value to add it to the device_map.

[0152] If current_node is a non-measurement point leaf node, it is skipped and the path will not be added to device_map.

[0153] S315. For each unvisited child node of the current node, construct a new path list and push the unvisited child nodes and the new path list onto the explicit stack.

[0154] Specifically, for each unvisited child node of current_node:

[0155] Create a new path list: new_path_list = current_path_list + [child_node.name].

[0156] Push (child_node, new_path_list) onto the stack. To maintain the depth-first property, push the child nodes onto the stack in reverse order to ensure that the "leftmost" or "first" child node is processed first in the next loop.

[0157] Termination: When the stack becomes empty, it indicates that all reachable nodes have been visited and processed, and the algorithm terminates. At this point, device_map contains all valid measurement point IDs and their corresponding context path text.

[0158] S316. When the explicit stack is empty, the algorithm terminates and uses the mapping set to store all valid measurement point IDs and their corresponding measurement point path texts.

[0159] Here is an example of device_map:

[0160] {

[0161] “D000001”: “Area A-Substation 1-Bay X-Equipment B-Current”;

[0162] “D000002”: “Area A-Substation 1-Bag X-Equipment A-Current”;

[0163] }

[0164] S32. Use a text embedding model to convert each measurement point path text in the mapping set into a sentence vector in order to compare the semantic similarity between different measurement point path texts.

[0165] Specifically, this invention uses the text embedding model Sentence-BERT to convert the path of each measurement point in the device_map into a low-dimensional vector. The specific method is as follows:

[0166] To efficiently calculate and compare the semantic similarity between path texts at different measurement points, the Sentence-BERT (S-BERT) architecture is adopted, which converts each text string in the device_map into a fixed-dimensional sentence vector that can represent its overall semantics.

[0167] In the description of this invention, the text embedding model is used to convert each measurement point path text in the mapping set into a sentence vector, and the semantic similarity between different measurement point path texts is compared as follows:

[0168] S321. Input the measurement point path text into the pre-trained base model in the text embedding model, perform deep bidirectional processing on the measurement point path text, and generate word vectors for each word in the measurement point path text.

[0169] Specifically, the input and basic model processing involves taking a "measurement point path text" (e.g., "Area A - Substation 1 - Bay X - Equipment B - Current") as input and feeding it into the basic part of the S-BERT architecture, namely a pre-trained BERT model. The BERT model consists of multiple layers of Transformer encoders, which perform deep bidirectional processing on the input text, generating a high-dimensional word vector rich in contextual information for each token in the text. The output of this stage is a vector sequence of the same length as the input token sequence, which details the meaning of each word in the current context.

[0170] S322. A pooling layer is connected after the sequence of the base model to aggregate all word vectors output by the base model, so as to convert each vector sequence into a single sentence vector of fixed length.

[0171] Specifically, regarding pooling operations and sentence vector generation: Since BERT's direct output is a variable-length sequence of word vectors, it's not suitable for directly comparing sentence similarity. Therefore, a pooling layer is connected after this sequence. The pooling layer uses the mean pooling strategy to aggregate all the word vectors output by BERT. Through aggregation, the originally variable-length vector sequence is transformed into a single, fixed-length sentence vector.

[0172] S323. Store all sentence vectors in the vector database, and use the sentence vectors to save the path text of the measurement point they represent and the corresponding measurement point ID together.

[0173] Specifically, semantic capture and storage: The final generated sentence vector encapsulates the core semantic information of the entire measurement point path text. Thanks to the S-BERT training mechanism, semantically similar path texts (such as "current of device B" and "current-device B") are mapped to positions close to each other in the vector space. When stored in the vector database, this sentence vector is saved along with the original measurement point path text it represents and the corresponding measurement point ID, providing support for fast and accurate retrieval.

[0174] S33. Calculate and fuse the semantic similarity and path similarity between sentence vectors, input them into the multimodal large language model for analysis and reasoning, and generate corresponding data filling rules in the report template by combining the measurement point ID, report time level and measurement point statistical type.

[0175] In the description of this invention, semantic similarity and path similarity between sentence vectors are calculated and fused, then input into a multimodal large language model for analysis and reasoning. Combining the measurement point ID, report time level, and measurement point statistical type, corresponding data filling rules are generated in the report template, including:

[0176] S331. Traverse all text description information in the report template and calculate the cosine similarity between each text description information and all sentence vectors to measure the semantic similarity between path texts at different measurement points.

[0177] Specifically, the low-dimensional vectors generated in step SS32 can be directly used to calculate the cosine similarity between them, thereby measuring the degree of semantic association between path texts at different measurement points.

[0178] Iterate through all measurement point description information (text description information) in the template extracted in step SS2, calculate the cosine similarity between each description information and all measurement point path vectors (sentence vectors), and thus measure the degree of semantic association between different measurement point path texts.

[0179] ;

[0180] in, This represents specific measurement point description information. This represents the path vector of the i-th measurement point.

[0181] S332. The measurement point paths in the device model and the text description information in the report template are segmented using delimiters to obtain a node sequence. Editing operations are then allowed, and the minimum number of editing operations required to move from the candidate path to the report measurement point path is calculated. Combining the maximum distance between the candidate path and the report measurement point, and the editing operation distance, the path similarity between different measurement point path texts is calculated.

[0182] Specifically, the measurement point paths in the equipment model and the measurement point descriptions in the report template are both split using the separator "-" to obtain the node sequence.

[0183] For example, a candidate path: Area A - Substation 1 - Bay X - Equipment B - Current → [Area A, Substation 1, Bay X, Equipment B, Current];

[0184] Measurement point path in the report: Substation 1 - Bay X - Current → [Substation 1, Bay X, Current];

[0185] Define "insert" and "delete" as allowed edit operations, and solve for the minimum number of edit operations required to get from the candidate path (A) to the report measurement point path (B).

[0186] For example, [Area A, Substation 1, Bay X, Equipment B, Current];

[0187] [Substation 1, bay X, current];

[0188] Maximum distance between candidate path (A) and report measurement point path (B) To set the value to 5, simply delete "Area A" and "Device B" and edit the operation distance. =2.

[0189] To accommodate paths of varying lengths, the edit distance is normalized:

[0190]

[0191] The value ranges from [0, 1], and the larger the value, the higher the similarity.

[0192] S333. The normalized path similarity and semantic similarity are fused according to weights. The fusion result is used to sort all candidate measurement points, and a preset number of candidate measurement points are selected to form a candidate set (measurement point ID set).

[0193] Specifically, such as Figure 5As shown, the normalized path similarity and semantic similarity are fused according to their weights:

[0194] ;

[0195] Weight The values ​​are set between [0.5, 0.7] and tuned on the validation set to ensure that both semantics and path structure are taken into account.

[0196] Use the fusion results All candidate measurement points are sorted, and the top N are selected as the candidate measurement point ID set. This not only considers semantic similarity but also corrects for potential mismatches caused by differences in path structure.

[0197] Example:

[0198] Assume that the description information of the measurement point extracted by SS2 is: "Current of substation 1 bay X";

[0199] The TopN candidate measurement points retrieved from the SS32 vector database are as follows:

[0200] Candidate 1: ID: D000001, Path: "Area A-Substation 1-Bag X-Equipment B-Current" (Similarity: 0.92);

[0201] Candidate 2: ID: D000002, Path: "Area A-Substation 1-Bag Y-Equipment B-Current" (Similarity: 0.88);

[0202] Candidate 3: ID: D000003, Path: "Region B-Substation 1-Bag X-Equipment C-Current" (Similarity: 0.85);

[0203] Candidate 4: ID: D000004, Path: "Area A-Substation 1-Bay X-Equipment B-Voltage" (Similarity: 0.80);

[0204] Candidate 5: ID: D000005, Path: "Substation 1-Bag X-Current" (Similarity: 0.90).

[0205] S334. Based on the candidate set, retrieve a preset number of candidate measurement point information from the vector database, combine it with text description information to construct the context, and obtain the model input text.

[0206] Specifically, the inputs received by the large model include:

[0207] Step S2 extracts the description information of the measurement points to be matched (text description information, such as "current of substation 1 bay X" identified from the report image).

[0208] The information is retrieved from the vector database for the Top N candidate measurement points. This information typically includes the measurement point ID, its complete measurement point path text (e.g., "Area A-Substation 1-Bay X-Equipment B-Current"), and their initial cosine similarity scores with the description to be matched.

[0209] The prompt message clearly informs the large model that its task is to "select the most precisely matching measurement point ID from the candidate list based on the provided measurement point description".

[0210] Example prompt: "You are a professional power equipment management expert. I will give you a description of a measurement point in a report, as well as some candidate measurement point IDs (and similarity scores) and their complete paths from the equipment model. Please analyze the report description and select the most accurate matching measurement point ID (and similarity score) from the candidate list. Please consider your selection process."

[0211] S335. Utilize a multimodal large language model to perform deep semantic analysis on the input text of the model, compare the semantic correlation between the path text of each candidate measurement point and the description to be matched, and obtain the final measurement point ID and the corresponding similarity.

[0212] Specifically, the large model leverages its internal language model and contextual understanding to perform deep semantic analysis on the input text. It parses key information and user intent within the description to be matched. It compares the semantic relevance between the path text of each candidate measurement point and the description to be matched, identifying synonyms, hierarchical relationships, and logical consistency between different levels of information. It also ignores candidates that, while superficially similar, are semantically inconsistent. Finally, the LLM outputs the measurement point ID it deems the most precisely matched, along with its similarity score.

[0213] S336. A similarity threshold is preset. When the similarity output by the multimodal large language model is lower than the similarity threshold, a preset number of candidate measurement IDs are visualized and adjusted. When the similarity output by the multimodal large language model is higher than the similarity threshold, corresponding filling rules are generated in the report template based on the measurement point ID, the report time level, and the measurement point statistical type.

[0214] Specifically, the system sets a similarity threshold (e.g., 0.70). When the LLM value is below the threshold, the system can present the Top N candidate measurement IDs (and their similarities) to a human operator. The operator can then use their domain knowledge to make a judgment, selecting the correct ID or making corrections.

[0215] Finally, based on the measurement point ID and the report time level and measurement point statistical type obtained in step S2, the corresponding fill rules are generated in the report template.

[0216] In summary, by utilizing the above-mentioned technical solution of this invention, the present invention extracts tables from images using a multimodal large model, bypassing potential errors in the semantic understanding process, and opening up a new path for report template generation. It can identify the structure and text content of tables and form a static layout for report templates. Regarding the construction of template data filling rules, it combines equipment models and processes the hierarchical relationships in the header data definition description to determine the measurement point ID and filling rules for each data cell. Non-technical personnel only need to upload a sample image, and with minimal manual intervention, can quickly obtain a report template that meets their needs, greatly improving the efficiency of report template generation and showing broad application prospects in the field of industrial monitoring reports.

[0217] By systematically calculating and evaluating the performance of mainstream multimodal large models on table extraction tasks, the optimal model is selected. Compared with arbitrary model selection, the data-driven selection method can ensure that the model is highly adapted to the table extraction task, thereby significantly improving the accuracy of table extraction, reducing the error extraction rate, and making subsequent data processing more reliable.

[0218] This invention leverages the multimodal capabilities of large models to extract tables from images, thereby enabling the rapid construction of static layouts for report templates and greatly simplifying the traditional manual creation of report templates. By parsing the implicit hierarchical relationships of multiple table headers to obtain measurement point description text, and combining this with retrieval enhancement technology, the invention can automatically match the measurement point IDs corresponding to cell text, and then generate data filling rules; this significantly simplifies the workload of manually configuring data filling rules in traditional report templates.

[0219] This invention constructs a dual benchmark test set (general and domain-specific) and employs quantitative evaluation metrics such as cell matching degree and structural matching degree to achieve data-driven multimodal large model selection, ensuring the accuracy and domain adaptability of table extraction. Based on a recursive mechanism of "parent detection-path backtracking," it automatically extracts the table header hierarchy by detecting and merging cell attributes layer by layer. It uses a depth-first traversal of the device topology to generate measurement point path descriptions, combining semantic vectorization technology of word embedding models with a multi-level retrieval strategy based on cosine similarity, and utilizes retrieval enhancement generation technology to accurately match IDs in the device model.

[0220] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

Claims

1. A method for quickly generating a report template from a picture, characterized in that, The method comprises the following steps: S1. Based on a preset multi-modal large language model, picture information reasoning and table row-column structure analysis are performed on a picture of a table to be extracted, and an extraction result in a structured document form is output as structure and style data of a report template; S2. By analyzing the implicit hierarchical relationship in the header area of the report template, text description information of all measurement points in the table is obtained, and a report time level and a measurement point statistical type are specified; S3. The device model in the industrial monitoring system is vectorized, the hierarchical relationship information of the device model is saved, and a preset number of candidate sets are obtained and input into the multi-modal large language model for measurement point ID matching, combined with the report time level and the measurement point statistical type, to generate corresponding data filling rules.

2. The method for quickly generating a report template according to a picture of claim 1, wherein, The method comprises the following steps: S11. Obtain table image samples participating in multi-modal large language model evaluation, extract structured annotation data of each table image sample, and generate a verification data set; S12. Select a plurality of multi-modal large language models in turn to identify table sample images, and evaluate the table extraction capability of each multi-modal large language model based on a preset evaluation method; S13. Input the picture of the table to be extracted into the multi-modal large language model with the best table extraction capability, obtain the row-column structure and the content of each cell of the table through multi-modal reasoning, and output the table data in a preset data format.

3. The method for quickly generating a report template according to a picture of claim 2, characterized in that, The method comprises the following steps: S121. Input the verification data set into each multi-modal large language model for reasoning analysis, and calculate the text content matching degree of each cell of each multi-modal large language model on the verification data set; S122. Calculate the report template structure matching degree of each cell of each multi-modal large language model on the verification data set to measure the ability of the multi-modal large language model in cell merging and splitting; S123. Assign respective weights to the text content matching degree and the report template structure matching degree, and calculate the table extraction capability of each multi-modal large language model by using a comprehensive capability formula.

4. The method for quickly generating a report template according to a picture of claim 3, wherein, The expression of the comprehensive capability formula is: ； In the formula, represents the table extraction capability; represents the adjustable weight coefficient; represents the text content matching degree; represents the report template structure matching degree.

5. The method for quickly generating a report template according to a picture of claim 1, wherein, The method comprises the following steps: S21. The header area of the report template is specified in advance, the text content of the cells and the corresponding parent cells is read in turn by using a recursive search method, and the text content is combined into a cell combination to extract the implicit hierarchical relationship of the report template header, and form the text description information of the measurement points; S22. The time level of the report template is specified according to a preset time requirement; S23. The text content of the cell combination is analyzed semantically, and the statistical type corresponding to the measurement points is extracted by keyword matching, pattern recognition and implicit inference.

6. The method for quickly generating a report template according to a picture of claim 5, wherein, The table header area of the pre-design report template reads the text content of the cells and the corresponding parent cells in turn by a recursive search method, and combines the text content into a cell combination to extract the table header implicit layer relationship of the report template, including: S211, the table header area in the pre-design report template traverses all cells in the last row of the table header area, and searches upwards in the column where the cell is located to find the first table header cell that spans the cell, and sets the table header cell as the parent cell; S212, the text content of the last row cell is read first, and when the parent cell is found, the text content of the parent cell is read, and the current cell content and the parent table header content are combined into a cell combination according to a predetermined format to serve as a matching basis for the measurement point ID; S213, after the parent cell is found, recursion is performed in turn, if the parent cell exists in the previous level, the parent cell is searched upwards, if the parent cell does not exist in the previous level, the recursion is stopped, and the final recognition result is output.

7. The method for quickly generating a report template according to a picture of claim 1, wherein, The device model in the industrial monitoring system is vectorized, the hierarchical relationship information of the device model is saved, a preset number of candidate sets are obtained, and are input into a multi-modal large language model to match the measurement point ID, combined with the report time level and the measurement point statistical type, to generate corresponding data filling rules, including: S31, a directed acyclic graph is constructed using the device model, a mapping set of the measurement point ID and the measurement point name is constructed based on an explicit stack iterative depth-first algorithm, and the measurement point path text is added in the mapping set through traversal and exploration of all child nodes; S32, each measurement point path text in the mapping set is converted into a sentence vector using a text embedding model to compare the semantic similarity between different measurement point path texts; S33, the semantic similarity and the path similarity between the sentence vectors are calculated and fused, and are input into a multi-modal large language model for analysis and reasoning, combined with the measurement point ID and the report time level and the measurement point statistical type, to generate corresponding data filling rules in the report template.

8. The method for quickly generating a report template according to a picture of claim 7, wherein, The device model is used to construct a directed acyclic graph, a mapping set of the measurement point ID and the measurement point name is constructed based on an explicit stack iterative depth-first algorithm, and the measurement point path text is added in the mapping set through traversal and exploration of all child nodes, including: S311, according to the device model of the industrial monitoring system, a directed acyclic graph is constructed, nodes represent devices, locations or measurement points, and edges represent hierarchical inclusion relationships; S312, an empty explicit stack, an empty set and an empty mapping set are respectively created, and all root nodes of the device model are pushed into the explicit stack; S313, if the explicit stack is not empty, an element is popped from the top of the explicit stack, if the popped element does not appear in the set, the element is added to the set; the element includes the current node and the hierarchical path list from the root node to the current node; S314, judge whether the current node is a leaf node, if it is a leaf node, further judge whether the current node is a measurement point, if the current node is a measurement point, connect all node names in the hierarchical path list according to the predetermined format, and append to the base name of the current node to form the measurement point path text; and the unique ID of the measurement point is taken as the key, and the measurement point path text is taken as the value, and added to the mapping set; If the current node is a leaf node that is not a measurement point, skip the current node; S315, for each unvisited child node of the current node, build a new path list, and push the unvisited child node and the new path list into the explicit stack; S316, when the explicit stack is empty, the algorithm terminates, and all valid measurement point IDs and corresponding measurement point path texts are stored in the mapping set.

9. The method for quickly generating a report template according to a picture of claim 7, wherein, The use of the text embedding model to convert each measurement point path text in the mapping set into a sentence vector to compare the semantic similarity between different measurement point path texts includes: S321, input the measurement point path text into the pre-trained base model in the text embedding model, perform deep bidirectional processing on the measurement point path text, and generate a word vector for each word unit in the measurement point path text; S322, connect a pooling layer after the sequence of the base model, and perform aggregation operation on all word vectors output by the base model to convert each vector sequence into a single and fixed length sentence vector; S323, store all sentence vectors into a vector database, and save the measurement point path text represented by the sentence vector and the corresponding measurement point ID together.

10. The method for quickly generating a report template according to a picture of claim 7, wherein, The calculation of the semantic similarity and path similarity between the sentence vectors and the input into the multi-modal large language model for analysis and reasoning, combined with the measurement point ID and the report time level and the measurement point statistical type, generates the corresponding data filling rule in the report template includes: S331, traverse all text description information in the report template, calculate the cosine similarity between each text description information and all sentence vectors to measure the semantic similarity between different measurement point path texts; S332, split the measurement point path in the device model and the text description information in the report template according to the delimiter to obtain a node sequence, set the allowable editing operation, solve the minimum editing operation number required from the candidate path to the report measurement point path; combined with the maximum distance of the candidate path and the report measurement point and the editing operation distance, calculate the path similarity between different measurement point path texts; S333, fuse the normalized path similarity and semantic similarity according to the weight, sort all candidate measurement points according to the fusion result, and select a preset number of candidate measurement point combinations as a candidate set; S334, retrieve the preset number of candidate measurement point information from the vector database according to the candidate set, construct the context according to the text description information to obtain the model input text; S335, using a multi-modal large language model to perform deep semantic analysis on the model input text, comparing the semantic association between each candidate measurement point path text and the to-be-matched description, and matching the final measurement point ID and the corresponding similarity; S336, a similarity threshold is preset, when the similarity output by the multi-modal large language model is lower than the similarity threshold, a preset number of candidate measurement IDs are visualized and debugged; when the similarity output by the multi-modal large language model is higher than the similarity threshold, according to the measurement point ID, the report time level and the measurement point statistical type, the corresponding filling rule is generated in the report template.

Images