Intelligent report automatic generation method and system based on multi-modal large model

Abstract

The invention provides an intelligent report automatic generation method and system based on a multi-modal large model, and relates to the technical field of artificial intelligence, and the method comprises the steps: firstly receiving a report generation demand text, and deconstructing a demand semantic deconstruction result; constructing a multi-modal information association mapping rule based on the demand semantic deconstruction result, and obtaining and associating multi-modal information to obtain an association set; executing a task-oriented adaptive processing generation report generation task instruction; calling a large model to perform content conversion and integration on the association set according to the report generation task instruction to generate an initial report text; and finally, performing text line structure and semantic linkage optimization on the initial report to obtain a final intelligent report. According to the method, the demand can be accurately understood, the multi-modal information is fully utilized, and the high-quality intelligent report meeting the requirement is generated.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a method and system for automatically generating intelligent reports based on a multimodal large model. Background Technology

[0002] In today's era of rapid information flow, the demand for generating various reports is increasing, covering numerous fields such as business, scientific research, and education. Traditional report generation methods mainly rely on manual writing, which not only requires writers to have extensive professional knowledge and strong writing skills, but is also inefficient and unable to meet the needs of large-scale, time-sensitive report generation.

[0003] With the development of artificial intelligence technology, some report generation methods based on natural language processing have gradually emerged. However, these existing methods have many limitations. On the one hand, they often can only process single-modal information, such as generating reports based solely on text data, failing to fully utilize the rich content contained in multimodal information such as images and audio, resulting in reports that are not comprehensive or rich enough. On the other hand, existing methods lack a precise and in-depth understanding of report generation needs, making it difficult to customize reports according to specific user requirements. The generated reports may not fully meet user expectations in terms of content and format. Furthermore, the lack of an effective task-oriented adaptation mechanism during the report generation process makes it difficult to ensure that large models generate reports according to specific logic and specifications, affecting the quality and usability of the reports. Summary of the Invention

[0004] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide an automatic intelligent report generation method based on a multimodal large model, the method comprising: Receive a report and generate requirement text, perform semantic deconstruction processing on the report and generate requirement text to obtain requirement semantic deconstruction result, the requirement semantic deconstruction result includes report theme semantics, content dimension semantics and output format semantics; Based on the results of demand semantic deconstruction, a multimodal information association mapping rule is constructed. The multimodal information collection interface is called to obtain multimodal information that matches the results of demand semantic deconstruction. The multimodal information is then associated through the multimodal information association mapping rule to obtain a set of associated multimodal information. Based on the semantic deconstruction results of the requirements and the associated multimodal information set, a task-oriented adaptation process for the large model is performed to generate a report generation task instruction, which includes content generation logic and format specification requirements. The large model is invoked to perform content transformation and integration processing on the associated multimodal information set based on the report generation task instruction, and an initial report text is generated. The initial report text contains all content dimensions corresponding to the semantic deconstruction results of the requirements. The initial report text is structurally and semantically optimized to obtain the final intelligent report, which meets the output format semantics and content integrity requirements in the semantic deconstruction results.

[0005] In another aspect, embodiments of the present invention also provide an intelligent report automatic generation system based on a multimodal large model, including a processor and a machine-readable storage medium. The machine-readable storage medium is connected to the processor. The machine-readable storage medium is used to store programs, instructions, or code. The processor is used to run the programs, instructions, or code in the machine-readable storage medium to implement the above-described method.

[0006] Based on the above, this embodiment of the invention receives a report, generates a requirement text, and performs semantic deconstruction to accurately obtain key information such as the report topic, content dimensions, and output format. Based on the semantic deconstruction results, it constructs multimodal information association mapping rules, efficiently calling the multimodal information collection interface to obtain matching information. Through association processing, it forms a set of associated multimodal information, fully utilizing the richness of multimodal information. It executes task-oriented adaptation processing for the large model to generate report generation task instructions, ensuring that the large model follows specific content generation logic and format specifications when generating reports, thus improving the accuracy and standardization of report generation. The large model is then called to perform content transformation and integration processing on the associated multimodal information set to generate the initial report text. Finally, the initial report text undergoes structural optimization and semantic coherence optimization processing, ensuring that the final intelligent report not only meets output format requirements but also has complete content and semantic coherence, greatly improving the quality and readability of the report and meeting the diverse report generation needs of users. Attached Figure Description

[0007] Figure 1 This is a schematic diagram of the execution flow of the intelligent report automatic generation method based on a multimodal large model provided in the embodiments of the present invention.

[0008] Figure 2 This is a schematic diagram of the hardware architecture of the intelligent report automatic generation system based on a multimodal large model provided in an embodiment of the present invention. Detailed Implementation

[0009] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a flowchart illustrating an embodiment of the intelligent report automatic generation method based on a multimodal large model provided by the present invention. The following is a detailed description of the intelligent report automatic generation method based on a multimodal large model.

[0010] Step S110: Receive the report-generated requirement text, perform semantic deconstruction processing on the report-generated requirement text, and obtain the requirement semantic deconstruction result, which includes report topic semantics, content dimension semantics, and output format semantics.

[0011] In the financial and tax management system based on a multimodal AI intelligent enterprise management system, users submit reports and generate required text through a standardized text input interface provided by the system. This interface supports text input of up to L characters in length and provides real-time syntax checking. After receiving the text, it is first converted to UTF-8 format using Unicode encoding, and then the semantic deconstruction module is activated. This module consists of a text preprocessing submodule, a semantic unit extraction submodule, a semantic association analysis submodule, and a semantic aggregation submodule. The text preprocessing submodule is responsible for removing control characters (such as carriage returns and tabs) and repeated punctuation marks from the text, using regular expressions to filter specific pattern characters. The semantic unit extraction submodule uses a pre-trained model with a bidirectional Transformer architecture (fine-tuned based on financial and tax domain corpora) to perform sequence labeling on the text and identify core semantic boundaries through a sliding window mechanism. The semantic association analysis submodule constructs a semantic dependency graph and uses a graph attention network (GAT) to learn the association weights between nodes. The semantic aggregation submodule integrates scattered semantic information in a structured manner according to three dimensions: topic, content, and format. The final output is a requirement semantic deconstruction result containing the semantics of the report topic, the semantics of the content dimension, and the semantics of the output format. The result is stored in JSON format and contains three first-level fields and corresponding subfields.

[0012] Step S111: Receive the report and generate the requirement text. Perform semantic unit extraction processing on the report and generate the requirement text, and split it into multiple independent requirement semantic units. Each requirement semantic unit corresponds to a core expression in the report and generate the requirement text.

[0013] After receiving the report, generating the requirement text, and completing preprocessing, the semantic unit extraction stage begins. First, a pre-trained BERT model (N layers, D hidden dimensions) is loaded. This model was pre-trained on a corpus containing M texts from the finance and tax domain and fine-tuned using K requirement texts labeled with semantic unit boundaries. The pre-processed text is divided into blocks of T characters (overlapping sliding windows are used for very long texts), and the blocks are input into the BERT model to obtain the context embedding vector for each character. Then, a bidirectional LSTM network (H hidden units per layer, dropout rate P) is used to capture sequence dependencies. The LSTM output is fed into a CRF layer, and the optimal label sequence is obtained through the Viterbi algorithm. The labeling system uses the BIOES annotation method (B-UNIT indicates the start of a semantic unit, I-UNIT indicates the inside of a semantic unit, O indicates a non-semantic unit, E-UNIT indicates the end of a semantic unit, and S-UNIT indicates a single-character semantic unit). Based on the label sequence, character segments continuously labeled from B-UNIT to E-UNIT are extracted into independent semantic units for demand. Each unit is assigned a unique identifier (consisting of a unit type prefix + Q random strings) and its start and end character indices in the original text are stored.

[0014] Step S112: Perform semantic association link construction processing on multiple demand semantic units, analyze the semantic dependency relationship between each demand semantic unit, form demand semantic association links based on semantic dependency relationships, and include the sequential association order of each demand semantic unit in the demand semantic association links.

[0015] After extracting the semantic units required for the requirements, the semantic association link construction process is initiated. First, each semantic unit is vectorized using the Sentence-BERT model (pre-trained on a general domain corpus and fine-tuned using parallel sentence pairs from the finance and tax domain) to convert the unit text into an F-dimensional dense vector. The similarity value between any two unit vectors is calculated using cosine similarity, constructing an M×M similarity matrix (M being the total number of semantic units). A similarity threshold Th is set (determined through validation set experiments to maximize the F1 score), and unit pairs with similarity higher than the threshold are selected as potential association candidates. For each candidate unit pair, syntactic analysis is performed using the Stanford CoreNLP toolkit to identify the core predicates, argument structures, and dependency syntactic relations (such as nsubj, dobj, amod, etc.). Based on the syntactic analysis results, semantic dependency triples (head unit, relation type, tail unit) are constructed, with relation types including subject-verb, verb-object, attributive, and coordinate relationships, among other R subcategories. Semantic dependency triples are input into a knowledge graph building engine to generate a semantic dependency graph containing nodes (semantic units) and edges (dependency relationships). The PageRank algorithm is used to calculate the importance score of each node, and the node with the highest score is selected as the root node (usually a unit related to the report topic). A depth-first search (DFS) algorithm is used to traverse the semantic dependency graph, generating initial dependency links in the order of "root node → direct dependency node → indirect dependency node". The initial links are then checked for loops; if a circular dependency exists where A→B and B→A, the dependency relationship with the higher weight is retained. The final generated semantic dependency links are stored in a directed linked list structure, with each node containing a unit ID, semantic content, a pointer to the previous node, a pointer to the next node, and a relationship type label.

[0016] Step S1121: Perform semantic attribute annotation processing on each requirement semantic unit, and annotate the semantic type of each requirement semantic unit. The semantic type includes topic semantics, content semantics, format semantics and constraint semantics.

[0017] A semantic type classifier based on the ALBERT architecture is invoked. This classifier contains a C-layer Transformer encoder, a hidden layer dimension of D, and employs label smoothing technology (smoothing factor of S) to alleviate class imbalance. The classifier is trained on a training set containing K labeled data points, constructed according to the proportions of topic (A%), content (B%), format (C%), and constraint (D%) (A+B+C+D=100). The input layer segments the semantic unit text (using WordPiece segmentation, vocabulary size V) and adds a special classification label [CLS]. The hidden vector at the [CLS] position is obtained through the encoder, connected to a fully connected layer (activation function of GELU) and a Softmax layer, outputting a 4-dimensional probability distribution. The category with the highest probability is selected as the semantic type label, and the label is stored using one-hot encoding (topic class is [1, 0, 0, 0], content class is [0, 1, 0, 0], and so on). For units with a probability lower than the confidence threshold Tc (set via cross-validation), a manual verification process is triggered, and the notification is sent to the administrator terminal via the system notification mechanism.

[0018] Step S1122: Use semantic dependency analysis to analyze the semantic dependency relationship between any two semantic units of demand. The semantic dependency relationship includes subject-verb relationship, verb-object relationship, modifier-head relationship and coordinate relationship.

[0019] In constructing the semantic dependency graph, deep semantic dependency analysis is performed on any two semantic units of requirements. First, syntactic structure parsing is performed on each semantic unit, and a dependency parser (based on the ArcEager algorithm) is used to generate a dependency tree, identifying core predicates (such as "generate," "contain," and "adopt") and their arguments (subject, object, adverbial, etc.). For units containing core predicates (denoted as Unit A) and units acting as arguments (denoted as Unit B), if the predicate of Unit A and the core noun of Unit B form a subject-predicate relationship (e.g., "report (Unit B) - generate (Unit A predicate)"), it is marked as a subject-predicate relationship. If the predicate of Unit A directly governs the core noun of Unit B (e.g., "analysis (Unit A predicate) - data (Unit B core noun)"), it is marked as a verb-object relationship. If Unit B contains adjectives modifying nominal components of Unit A (e.g., "dynamics (Unit B) - chart (Unit A)"), it is marked as a modifier-head relationship. If two units are connected by coordinating conjunctions (such as "and", "or", "and") and have the same semantic type (such as both being content-related), they are marked as having a coordinating relationship. An attention mechanism is introduced during the relationship recognition process, and an attention weight is calculated for each candidate relationship (based on a weighted sum of word vector similarity and syntactic distance). Relationships with a weight higher than a threshold Ta are retained.

[0020] Step S1123: Construct a semantic association matrix based on semantic dependency relationships. The elements in the semantic association matrix represent the semantic dependency strength between two semantic units of demand. The semantic dependency strength is determined based on the tightness of the semantic association.

[0021] Construct an M×M semantic association matrix (M is the total number of semantic units), where the matrix element Wij represents the semantic dependency strength between the i-th and j-th semantic units. The semantic dependency strength is calculated by integrating three features: semantic vector cosine similarity (Sim_ij), dependency relation type weight (Rel_ij), and the distance between semantic units in the association link (Dist_ij). The calculation formula is: Wij=α×Sim_ij+β×Rel_ij+γ×Dist_ij, where α, β, and γ are weight coefficients (α+β+γ=1). Sim_ij is calculated using the F-dimensional vector generated in step S112; Rel_ij is assigned a value according to the dependency relation type (e.g., subject-verb relation is assigned R1, verb-object relation is assigned R2, R1>R2>R3>R4, where R3 is the weight of the symmetric relation and R4 is the weight of the parallel relation); Dist_ij is calculated using 1 / (d+1), where d is the number of interval nodes between the two units in the association link. The diagonal elements Wii of the matrix are set to 1 (autocorrelation strength), and the element pairs Wij with no dependencies are set to 0.

[0022] Step S1124: Extract semantic dependency relationships with semantic dependency strength greater than a preset association threshold from the semantic association matrix to form a core semantic dependency relationship set.

[0023] A preset association threshold Th is set (determined through validation set experiments, selecting a value that balances precision and recall). The non-zero elements Wij in the semantic association matrix are iterated. If Wij > Th, the dependency relationship between the i-th and j-th units is added to the core semantic dependency relationship set. Each element in the set is represented as a triple (i, j, Rij), where i is the source unit index, j is the target unit index, and Rij is the dependency relationship type. The set is deduplicated; if there are identical (i, j) pairs but different Rij values, the relationship type with the highest Wij value is retained.

[0024] Step S1125: Starting from the core objective of report generation requirements, construct an initial semantic association link based on the core semantic dependency relationship set. The initial semantic association link includes the requirement semantic unit corresponding to the core objective and the directly related requirement semantic unit.

[0025] Based on the semantic type annotation results in step S1121, topic-based semantic units are selected as candidate core target units. If multiple topic-based units exist (such as composite report requirements), the row sum (i.e., total dependency strength) of each unit in the semantic association matrix is ​​calculated, and the unit with the largest row sum is selected as the core target unit G. In the core semantic dependency relationship set, all triples with G as the source unit (i=G) are searched, and the target units in the above triples are taken as directly associated units. With G as the root node and directly associated units as child nodes, a tree structure of the initial semantic association link is constructed (the root node layer is level 0, and the directly associated units are level 1). The link is stored using an adjacency list, and each node records its list of child nodes and the corresponding relationship type.

[0026] Step S1126: Traverse the core semantic dependency relationship set and add the required semantic units that are not included in the initial semantic association link. Based on the semantic dependency relationship, gradually add them to the initial semantic association link to expand the coverage of the semantic association link and obtain the expanded semantic association link.

[0027] Iterate through all units (denoted as U) in the core semantic dependency relation set that are not in the initial link. For each U, find all triples in the set that have U as the source or destination. If a triple contains a unit V already in the link (called an anchor unit), add U to the child or parent node position of V according to the relation direction. If there is a relationship between U and V where i=V and j=U, then U is a child node of V; if there is a relationship where i=U and j=V, then U is a parent node of V (ensuring no cycles are formed). During the addition process, record the level of U in the link (equal to the anchor unit level ±1). Repeat this process until all units in the core semantic dependency relation set are included in the link, forming the expanded semantic association link.

[0028] Step S1127: Perform link optimization processing on the extended semantic association link to eliminate semantic loops and logical conflicts in the extended semantic association link, and adjust the association order of each required semantic unit in the extended semantic association link to obtain the optimized semantic association link.

[0029] The expanded links undergo loop detection, employing a depth-first search to traverse the links and record the access path for each node. If a node is found to be already in the access path (i.e., forming a loop), the dependency relationship with lower weight is removed from the conflicting node pair (by comparing Wij values). Logical conflict detection is then performed, checking whether content units belong to topic units, whether format units do not form subject-predicate relationships with content units, and other logical rules. Relationships violating these rules are corrected (e.g., adjusting the relationship type or removing them). Finally, the link order is adjusted according to the unit hierarchy to ensure that the path from the root node to the leaf node conforms to the logical progression of "topic → content → format / constraint," resulting in the optimized semantically related links.

[0030] Step S1128: Based on the proximity and link connection relationship between each requirement semantic unit and the core target in the optimized semantic association link, determine the sequential association order of each requirement semantic unit in the optimized semantic association link, add an association identifier to each requirement semantic unit in the optimized semantic association link, the association identifier represents the association relationship and association order between the requirement semantic unit and other requirement semantic units, and use the optimized semantic association link with added association identifiers as the requirement semantic association link.

[0031] Calculate the shortest path length L between each unit and the core target G (using the BFS algorithm to calculate the distance between nodes). A smaller L indicates a closer association. Sort units in ascending order of L value; units at the same level (with the same L) are sorted in descending order of total dependency strength (row sum) in the semantic association matrix. Assign an association identifier to each unit in the format "LXY", where L is the level, X is the order number within the same level, and Y is the parent node identifier. For example, if the core target identifier is "0-1-NULL", its first directly associated unit identifier is "1-1-0-1". Store the sorted unit sequence and association identifiers as an ordered linked list to form the final semantic association chain.

[0032] Step S113: Perform core requirement semantic focusing processing based on the requirement semantic association link, identify the core semantic nodes in the requirement semantic association link, the core semantic nodes correspond to the core objectives of the report generation requirements, extract topic-related semantic information around the core semantic nodes, and form the report topic semantics.

[0033] In the semantic association chain of requirements, the core semantic node is the core target unit G determined in step S1125. The text content of G is extracted as the core keywords of the topic, and expanded by combining it with its directly related modifier-head units (such as "2023" and "merger") to form topic phrases. The TextRank algorithm is used to extract keywords from the topic phrases, retaining the Top K keywords (K is dynamically adjusted according to the topic length). The keywords are combined according to the semantic association order to form a structured report topic semantic, containing four sub-fields: "Report Type," "Time Range," "Subject Scope," and "Core Content." For example, the topic semantic is represented as: {"Report Type": "Financial Analysis Report," "Time Range": "2023," "Subject Scope": "Group Merger Level," "Core Content": "Income Tax Declaration Status"}.

[0034] Step S114: Taking the semantics of the report topic as the core, classify and process the non-core semantic nodes in the semantic association link of the requirements. According to the semantic attributes, they are divided into content-related semantic nodes and format-related semantic nodes. Content-related semantic nodes correspond to the content direction that the report needs to cover, and format-related semantic nodes correspond to the output presentation requirements of the report.

[0035] Traverse all nodes in the semantic association link except for the core semantic nodes, and classify them according to the semantic type annotation results (content, format, constraint) in step S1121. Group content-related semantic nodes into set C, and format and constraint-related nodes into set F. For constraint-related nodes (such as data source requirements, time deadlines), if they are associated with content-related nodes through semantic dependencies, record their constraint objects simultaneously (e.g., the "data source" constraint is associated with the "revenue details" content node). Construct sub-association links in sets C and F respectively, maintaining the relative order and dependencies in the original links.

[0036] Step S115: Perform dimensional decomposition processing on the content-related semantic nodes. Each content-related semantic node is decomposed into multiple specific content sub-nodes. Each content sub-node corresponds to a specific content dimension in the report. Integrate all content sub-nodes to form content dimension semantics.

[0037] Each node in the content-related semantic node set C is decomposed into dimensions. A top-down recursive decomposition method is adopted: if the node text contains coordinating conjunctions such as "and", "with", and "and", it is split into multiple coordinating child nodes; if it contains subordinating conjunctions such as "including" and "covering", the object part is extracted as a child node; if it is a complex concept (such as "profitability analysis"), the ontology library of the finance and taxation domain is called to match the preset sub-dimensional template (such as "profitability analysis" being split into "gross profit margin analysis", "net profit margin analysis", "ROE analysis", etc.). Each child node is assigned a unique dimension identifier (formatted as "C-parent node ID-child node sequence number"), and its hierarchical relationship with the parent node is recorded. All leaf nodes (content child nodes that cannot be further decomposed) are integrated and arranged in order in the association link to form a content dimension semantic list. Each element in the list contains a dimension name, dimension description, and associated data type (such as numerical, text, and chart types).

[0038] Step S116: Perform standard extraction processing on the format-related semantic nodes, extract the output format information, layout requirement information and presentation style information from the format-related semantic nodes, and integrate the output format information, layout requirement information and presentation style information to form output format semantics.

[0039] The set of format-related semantic nodes F is categorized into: output format information (e.g., file format, page count limit), layout requirements (e.g., font, spacing, headers and footers), and presentation style information (e.g., chart type, color scheme, title style). Regular expressions are used to extract specific specification values; for example, the file format value "PDF" is extracted from "output as PDF," and the font "SimSun" and font size "small four" are extracted from "use SimSun font, size 12" in the body text. For conflicting format requirements (e.g., both "PDF format" and "Word format"), the dependency strength of each format node is calculated using a semantic association matrix, retaining the requirement with higher strength. The extracted specification values ​​are organized according to a hierarchical structure of "output format - layout requirements - presentation style" to form output format semantics, where each specification item includes specification type, specification value, and priority (used to resolve potential conflicts).

[0040] Step S117: Perform semantic consistency association processing on the report topic semantics, content dimension semantics, and output format semantics to establish the correspondence between the report topic semantics and the content dimension semantics, as well as the adaptation relationship between the content dimension semantics and the output format semantics.

[0041] Construct a topic-content association matrix: rows represent keywords of the report topic semantics, columns represent dimension names of content dimensions semantics, and matrix elements represent the association strength between keywords and dimensions (calculated using cosine similarity of word vectors). Set a threshold T1, and record association pairs with a strength higher than T1 as corresponding relationships. Construct a content-format association matrix: rows represent content dimensions, columns represent format specification items, and matrix elements indicate whether the format item is applicable to the content dimension (e.g., "trend analysis chart" is applicable to the "income change" dimension). Fill the matrix with a rule base (e.g., "numerical comparison dimension adapts to bar chart" "proportion analysis dimension adapts to pie chart") and semantic dependencies (constraints between format nodes and content nodes). Store the above association relationships as sets of triples (topic keywords, content dimensions, association strength) and (content dimensions, format specifications, adaptability) to ensure semantic consistency among topic, content, and format.

[0042] Step S118: The associated report topic semantics, content dimension semantics, and output format semantics are structurally integrated to form the requirement semantic deconstruction result. The requirement semantic deconstruction result presents the relationship between each semantic in a hierarchical structure.

[0043] The semantic information after association is structured and integrated using JSON format. The top-level structure contains three primary fields: "report_topic" (report topic semantics), "content_dimensions" (content dimension semantics), and "output_format" (output format semantics). "report_topic" includes subfields for "topic_phrase" (topic phrase), "keywords" (keyword list), and "importance" (topic importance). "content_dimensions" is an array, with each element containing subfields for "dimension_id" (dimension ID), "dimension_name" (dimension name), "description" (dimension description), "related_topics" (related topic keywords), and "data_requirements" (data requirements). "output_format" includes three secondary fields for "output_type" (output format), "layout" (layout requirements), and "style" (presentation style), each containing specific specification items. After the semantic deconstruction results are generated, a completeness check is performed to verify that all necessary fields are included and that there are no semantic conflicts. Once the check passes, the data is stored as a structured file and a version number is generated.

[0044] Step S120: Construct multimodal information association mapping rules based on the demand semantic deconstruction results, call the multimodal information acquisition interface to obtain multimodal information that matches the demand semantic deconstruction results, and perform association processing on the multimodal information through the multimodal information association mapping rules to obtain an associated multimodal information set.

[0045] The process involves analyzing the content dimension semantics and report topic semantics from the demand semantic deconstruction results to extract a core semantic keyword set K (containing topic keywords and keywords for each content dimension). Based on K, a multimodal information association mapping rule is constructed, including information type (text, image, audio, video), semantic matching threshold, and association conditions (time relevance, topic consistency, entity co-occurrence). A multimodal information collection interface is invoked, with parameters including the keyword list K, information type filter, time range (extracted from the demand semantic deconstruction results or defaulting to the most recent N accounting periods), and a data source whitelist (e.g., financial system, tax system, audit system). The collection interface returns an initial multimodal information set, containing text information (e.g., financial statements, tax returns), image information (e.g., historical trend charts, scanned invoices), and structured data (e.g., database tables, JSON returned by the API). The initial set is preprocessed: text information undergoes OCR recognition (image-to-text) and word segmentation; visual features are extracted from image information (using a pre-trained ResNet model); and structured data is converted to a unified format. The preprocessed information is filtered and associated using multimodal information association mapping rules: the semantic similarity between the information and the keyword K is calculated, and information with similarity higher than the threshold is retained; the association between different types of information is established based on timestamp and entity linking technology (such as associating the text "2023Q4 Profit Statement" with the image "2023Q4 Profit Trend Chart"); the associated multimodal information is combined into information packages, each of which corresponds to a content dimension semantic, forming a set of associated multimodal information.

[0046] Step S121: Analyze the report topic semantics and content dimension semantics in the demand semantic deconstruction results, and extract the core semantic keywords. The core semantic keywords are used to represent the core information direction required for report generation.

[0047] Extract a keyword list Kt (e.g., "corporate income tax", "tax settlement", "2023") from the "report_topic" field of the demand semantic deconstruction results. Extract keyword lists Kd1, Kd2, ..., Kdn (n is the number of content dimensions) from the "dimension_name" and "description" fields of each dimension of the "content_dimensions" array. Merge all keyword lists and remove duplicates to obtain the initial core semantic keyword set K. Calculate keyword weights for K: use the TF-IDF algorithm (based on IDF values ​​from a corpus in the financial and tax domain) and the TextRank algorithm to calculate the weights respectively, and fuse them to obtain the comprehensive weight (W=a×TF-IDF+b×TextRank, a+b=1). Retain the top M keywords by weight as the final core semantic keywords, and label each keyword with its corresponding content dimension (e.g., the keyword "income details" belongs to the "income composition analysis" dimension).

[0048] Step S122: Construct a multimodal information association dimension system based on core semantic keywords. The multimodal information association dimension system includes text information association dimension, image information association dimension, audio information association dimension and video information association dimension, and each association dimension corresponds to a type of multimodal information.

[0049] For each type of multimodal information (text, image, audio, and video), we construct association dimensions: Text information association dimensions include topic relevance (semantic similarity to keywords), entity coverage (the proportion of core entities contained in the information), and sentiment (applicable to commentary texts); Image information association dimensions include visual feature similarity (feature vectors extracted using a pre-trained CNN model), label matching (matching rate between image metadata labels and keywords), and scene consistency (e.g., financial statement image scenes); Audio information association dimensions include semantic similarity after speech-to-text conversion, speaker identification (whether it is a designated financial personnel), and timestamp matching; Video information association dimensions include keyframe image features, audio track-to-text semantics, and video title matching. Each dimension is assigned a weight coefficient and scoring standard (e.g., text topic relevance is scored using cosine similarity from 0 to 1), forming a multimodal information association dimension system.

[0050] Step S123: Develop semantic matching rules for each association dimension. The semantic matching rules determine the matching criteria between core semantic keywords and corresponding multimodal information types. The semantic matching rules for different association dimensions are set differently according to the characteristics of multimodal information types.

[0051] For the text information association dimension, the semantic matching rule is as follows: calculate the Jaccard similarity Sj between the text segmentation result and the core semantic keywords. If Sj > Th1, and the cosine similarity Sc between the text title and any topic keyword > Th2, then a match is determined. For the image information association dimension, the rule is as follows: the overlap So between the image label set and the keyword set > Th3, and the cosine similarity Sv between the visual feature vector and the keyword visual prototype vector (generated through a text-to-image model) > Th4. For audio information, the rule is as follows: the similarity Ss between the speech-to-text content and the keywords > Th5, and the audio timestamp is within the specified time range. For video information, the rule is as follows: keyframe images match image rules, and video description text matches text rules. Th1-Th5 are the matching thresholds for each dimension (determined experimentally). The rules are stored in XML format, including dimension identifiers, thresholds, calculation methods, and judgment logic.

[0052] Step S124: Integrate the semantic matching rules of each association dimension to construct multimodal information association mapping rules. The multimodal information association mapping rules include the matching relationship between core semantic keywords and each type of multimodal information, and the association conditions between multimodal information.

[0053] The multimodal information association mapping rules adopt a three-layer structure: the core layer contains a list of core semantic keywords and their weights; the matching layer contains semantic matching rules for each information type (referencing the rules in step S123); and the association layer contains association conditions for cross-type information. Association conditions are categorized as strong association (e.g., text and images sharing the same data source ID), medium association (e.g., sharing different types of information from at least three entities), and weak association (e.g., information whose timestamps differ by T days). Rules are semantically described using JSON-LD format for easy machine parsing. For example, an association rule might be expressed as: "If text information T and image information I share entities E1 and E2, and the difference between the timestamp of T and the shooting timestamp of I is less than ΔT, then T and I form a medium association."

[0054] Step S125: Based on the core semantic keywords in the demand semantic deconstruction results, configure the collection parameters of the multimodal information collection interface. The collection parameters include the information collection scope, information type filtering conditions, and information timeliness requirements.

[0055] Extract the core semantic keyword K, content dimension time range (StartDate, EndDate), and data source requirements (e.g., "Financial General Ledger System," "ERP Module") from the semantic deconstruction results. Map these parameters to standard parameters for the multimodal information collection interface: query=K, type_filter=[text, image, audio, video] (selected according to the associated dimension system), time_range=[StartDate, EndDate], source_whitelist=data source list, max_results=N (maximum number of results to collect for each dimension), sort_by="relevance" (sort by relevance). The interface supports paginated retrieval; set page_size=M, and retrieve all information that meets the conditions through a loop. Parameters are encapsulated in key-value pairs and sent to the collection interface via an HTTPS POST request.

[0056] Step S126: Call the multimodal information acquisition interface, perform multimodal information acquisition operation according to the configured acquisition parameters, and obtain the initial multimodal information set, which contains multiple types of unrelated multimodal information.

[0057] Call the RESTful multimodal data collection API at https: / / api.example.com / multimodal-collector. The request header includes an authentication token (obtained via OAuth 2.0) and content type (application / json). The request body contains the collection parameters configured in step S125. The API returns a JSON response containing a status code, a message, and a data array. Each element in the data array contains a message ID, message type, raw data (text content / image URL / audio URL / video URL), metadata (timestamp, source system, file size, format), and a relevance score (internal API calculation of keyword matching). Validate the response data: check if the message type meets the filtering criteria, if the metadata is complete, and if the raw data is accessible. Store the validated information in a local temporary directory, generating an initial multimodal data set. The set is a dictionary built with the message type as the key and the values ​​as a list of messages of that type.

[0058] Step S127: Perform semantic annotation processing on the initial multimodal information set, extract the core content of each type of multimodal information, and add corresponding semantic tags to each type of multimodal information based on the core content.

[0059] Semantic annotation is performed on different types of information in the initial multimodal information set: For text information, keyword extraction (using the RAKE algorithm combined with a stop word list for the financial and tax domain) and entity recognition (BERT-NER model to identify entities such as organizations, dates, amounts, and report subjects) are used to generate a set of text semantic tags T_tags; for image information, a pre-trained CLIP model is used to match images with core semantic keywords to generate the top K most relevant tags I_tags, and text information in the image is extracted using OCR as supplementary tags; for audio information, speech recognition (such as the Whisper model) is first converted into text, and then the same annotation process as for text information is performed to generate A_tags; for video information, keyframes are extracted (one frame every N seconds), image annotation is performed on the keyframes, audio annotation is performed on the audio tracks, and the fusion results generate V_tags. The semantic tag for each piece of information is stored as a JSON object, containing the tag name, confidence level, and associated entity.

[0060] Step S128: Match the semantic tags in the initial multimodal information set with the core semantic keywords in the multimodal information association mapping rules, and filter out the candidate multimodal information set that matches the core semantic keywords.

[0061] Iterate through each information item in the initial multimodal information set to obtain its semantic tag set, Tags. Calculate the matching degree between Tags and the core semantic keyword set K: for text and audio information, calculate the Jaccard similarity Sj = |Tags∩K| / |Tags∪K|; for image and video information, calculate the weighted sum of tag confidence Sw = Σ(confidence(t) for tinTagsiftinK). According to the matching layer rules in the multimodal information association mapping rules, if Sj > Th_j or Sw > Th_w (Th_j and Th_w are the matching thresholds for the corresponding types), then add the information item to the candidate multimodal information set. The candidate set is grouped by information type and content dimension, with each content dimension corresponding to a subset.

[0062] Step S129: Based on the multimodal information association conditions in the multimodal information association mapping rules, perform association processing on the multimodal information in the candidate multimodal information set to establish semantic association relationships between different types of multimodal information.

[0063] Entity linking is performed on the information items in the candidate multimodal information set (using a tax and finance domain entity knowledge base), and an entity list E is extracted for each information item. Based on the association layer conditions of the multimodal information association mapping rules, the association strength between any two different types of information items (e.g., text T and image I) is calculated: if E_T∩E_I contains core entities (e.g., company name, report item), the entity overlap So is calculated as So = |E_T∩E_I| / min(|E_T|, |E_I|), and the association strength Sa is calculated by combining the time difference Δt (information creation time difference) as follows: Sa = α × So + (1-α) × (1-Δt / ΔT_max), where α is the weight and ΔT_max is the maximum allowed time difference. If Sa > Th_a (association strength threshold), a semantic association is established between T and I, and the association type (strong / medium / weak) and association strength value are recorded.

[0064] Step S1291: Analyze the multimodal information association conditions in the multimodal information association mapping rules, and extract the core association elements in the association conditions. The core association elements include semantic matching elements, time association elements, and scene association elements.

[0065] Core association elements are extracted from the association layer of the multimodal information association mapping rules: semantic matching elements are defined as the overlap threshold between the semantic tags of multimodal information and the core semantic keywords (e.g., containing at least 2 common keywords); time association elements are defined as the allowable difference range of the information creation timestamp (e.g., ΔT≤7 days); and scenario association elements are defined as the consistency of the business scenario to which the information belongs (e.g., both belong to the "corporate income tax declaration" scenario). Each element includes specific matching rules and weight coefficients (0.5 for semantic matching elements, 0.3 for time association elements, and 0.2 for scenario association elements) for subsequent association strength calculation.

[0066] Step S1292: Perform core association element extraction processing on each type of multimodal information in the candidate multimodal information set, and extract the semantic matching element value, time association element value and scene association element value corresponding to each type of multimodal information.

[0067] Text information is processed by extracting semantic tags using a keyword extraction tool, and the Jaccard similarity with core semantic keywords is calculated as the semantic matching element value. Image information is processed by generating text descriptions using the CLIP model, and keywords in the descriptions are extracted to calculate semantic matching element values. Audio and video information are converted into text through speech recognition and then processed in the same way as text information. Time-related element values ​​are calculated using the creation timestamp in the information metadata; if the timestamp is within the specified range, it is 1; otherwise, it is 0. Scene-related element values ​​are determined by the matching degree between the information source system tag (such as "financial general ledger system" or "tax declaration system") and the requirement scene tag; a complete match is 1, a partial match is 0.5, and no match is 0.

[0068] Step S1293: Based on semantic matching elements, perform preliminary association screening on different types of multimodal information, and group multimodal information with the same or similar semantic matching element values ​​into the same semantic association group.

[0069] A similarity threshold Th_s (e.g., 0.6) for semantic matching element values ​​is set. All information items in the candidate multimodal information set are traversed. The absolute value of the difference between the semantic matching element values ​​of any two different types of information items is calculated. If the difference is ≤ Th_s, the two items are grouped into the same semantic association group. For example, the semantic matching element value of text information T1 is 0.85, and the semantic matching element value of image information I1 is 0.82. The difference between them is 0.03 ≤ Th_s, so they are grouped into the same semantic association group. Each semantic association group is assigned a unique group ID, and the information items within the group are stored as a list.

[0070] Step S1294: Within the same semantic association group, perform time association processing on multimodal information based on time association elements, and establish time association relationships for multimodal information whose time association element values ​​are within the same time range.

[0071] Within each semantic association group, extract the time association element value (timestamp) of the information item and calculate the timestamp difference Δt between any two information items. If Δt ≤ the maximum allowable time difference ΔT_max (e.g., 30 days) defined in the multimodal information association mapping rules, then establish a time association relationship between the two, and record the time difference Δt and the association direction (e.g., if T1 was created earlier than I1, then T1→I1). The time association relationship is stored as a triple (information item AID, information item BID, Δt).

[0072] Step S1295: Based on scene association elements, perform scene association verification on the multimodal information that establishes time association relationships, compare the scene association element values ​​corresponding to the multimodal information, and filter out multimodal information with consistent scene association element values.

[0073] For information pairs with established time associations, extract their scene association element values ​​S_A and S_B. If S_A = S_B (e.g., both are 1 or both are 0.5), the scene association verification passes. If S_A ≠ S_B, calculate the product of the scene association element values ​​S_A × S_B. If the product ≥ 0.5 (e.g., 0.5 × 1 = 0.5), it is still considered to pass verification; otherwise, the time association is discarded. For example, the scene association element value of text information T1 is 1 (matching the "corporate income tax declaration" scene), and the scene association element value of image information I1 is 0.5 (partial match). Their product 0.5 ≥ 0.5, so the verification passes.

[0074] Step S1296: Evaluate the association strength of the multimodal information after semantic, temporal and scene association processing, select multimodal information combinations that meet the preset requirements for association strength, and establish semantic association relationships between these multimodal information. The semantic association relationship includes association type and association basis.

[0075] The formula for assessing association strength is: Association Strength S = α × S_sem + β × S_time + γ × S_scene, where α, β, and γ are the weight coefficients of the core association elements (α = 0.5, β = 0.3, γ = 0.2), S_sem is the semantic matching element value, S_time is the time association element value (the reciprocal of Δt / ΔT_max), and S_scene is the scene association element value. An association strength threshold Th_a (e.g., 0.7) is set. When S ≥ Th_a, a semantic association is considered to exist between the information items. Association types are classified according to strength values: S ≥ 0.9 is a strong association, 0.7 ≤ S < 0.9 is a medium association, and 0.5 ≤ S < 0.7 is a weak association. The association criteria are recorded as "Semantic matching degree: S_sem, Time difference: Δt, Scene matching degree: S_scene".

[0076] Step S1297: Add association identifiers to the multimodal information combinations that establish semantic relationships, and organize the multimodal information combinations that establish semantic relationships into an association information list. The association information list includes multimodal information types, association identifiers, and matching status of association elements. The association identifier is used to uniquely identify the association relationship of the multimodal information combination.

[0077] Each pair of information items that passes the association strength assessment is assigned an association identifier in the format "G-ID-Association Type", where G is the semantic association group ID, ID is the association sequence number within the group, and the association type is strong / medium / weak. For example, the identifier for the third strong association combination in group G1 is "G1-3-Strong". The association information list contains the following fields: association identifier, information item type A, information item AID, information item type B, information item BID, association strength value, semantic matching degree, time difference, and scene matching degree. The list is stored in JSON format and serves as the basis for subsequent information combination and integration.

[0078] Step S1210: Combine and integrate multimodal information with semantic relationships to form a set of associated multimodal information, wherein each information combination in the set of associated multimodal information is associated with a content dimension semantic in the result of demand semantic deconstruction.

[0079] Based on the semantic list of content dimensions (generated in step S115), an information combination container is created for each content dimension D. Information items in the candidate multimodal information set are traversed, and based on the matching degree between their semantic tags and the dimension keywords of D, the information items are assigned to the corresponding containers. Inside the containers, based on the semantic relationships established in step S129, different types of information items are combined into information packages: each information package contains a main information item (usually text-based, serving as the main content) and multiple auxiliary information items (images, audio, and video, serving as supplementary descriptions). The information packages are stored using a directed graph structure, with the main information item as the root node, auxiliary information items as child nodes, and edges representing the type and strength of the association relationship. All information packages of all content dimensions are integrated to form a related multimodal information set, which is compressed and stored in ZIP format, containing an information index file (recording the correspondence between information packages and content dimensions) and a directory of original information files.

[0080] Step S130: Based on the semantic deconstruction result of the requirements and the associated multimodal information set, perform task-oriented adaptation processing on the large model to generate a report generation task instruction, which includes content generation logic and format specification requirements.

[0081] The semantic deconstruction of requirements is transformed into a large model input format: report topic semantics are transformed into task titles (TaskTitle), content dimension semantics into a task list (Tasks), and output format semantics into format constraints (FormatConstraints). The associated multimodal information set is transformed into a resource reference list (Resources), where each resource item includes an information package ID, information type, storage path, and associated dimension. A task-oriented adaptation framework is constructed, comprising task objectives (consistent with the report topic), task decomposition (each content dimension as a subtask), resource allocation (subtasks associated with information packages), and format constraints (referencing output format semantics). Through PromptEngineering, the framework is transformed into natural language instructions, which include four parts: system prompts (SystemPrompt), user queries (UserQuery), resource descriptions (ResourcesDescription), and format examples (FormatExample). The system prompts define the roles (e.g., "professional financial and tax report writing assistant") and behavioral guidelines (e.g., "generate objective and accurate reports based on provided resources"). The user requirements section reiterates the core content of the requirements semantic deconstruction results. The resource description section lists the key information packages in the associated multimodal information set. The format example section provides example report fragments that conform to the output format semantics. The generated instructions are input into the large model, which is guided to understand the task requirements through few-shot learning, and finally outputs structured report generation task instructions (JSON format).

[0082] Step S131: Transform the semantic deconstruction results of the requirements into a semantic input format that can be recognized by the large model. The semantic input format includes semantic identifiers of the report topic, semantic lists of content dimensions, and semantic descriptions of the output format, so that the large model can accurately parse the semantic deconstruction results of the requirements.

[0083] The JSON structure of the demand semantic deconstruction results is converted into a semantic input format specific to large models (using the Article type extension from Schema.org). The report topic semantic identifier is "@type":"Report", "name": topic phrase, and "description": topic keyword; the content dimension semantic list is converted into a "hasPart" array, where each element is of type "ReportSection" and contains "@id": dimension ID, "name": dimension name, and "description": dimension description; the output format semantic description is converted into "encodingFormat": output file format, and "style": {"font": font, "layout": layout}. RDFa syntax is used to annotate the semantic input format, ensuring that the large model can understand the semantic relationships between parts through entity links. For example, the relationship between content dimensions and associated multimodal information is linked through the "isBasedOn" attribute.

[0084] Step S132: Perform format conversion processing on the associated multimodal information set, converting different types of multimodal information into an input format supported by the large model, while preserving the core semantic content and association relationships of the associated multimodal information set during the format conversion process.

[0085] Text information from the associated multimodal information set is directly extracted as plain text (format markers removed); image information is converted into descriptive text (generated using an image captioning model) and visual feature vectors (stored as arrays); audio information is converted into transcribed text and audio feature vectors; video information is converted into shot-by-shot text descriptions, keyframe image descriptions, and video feature vectors. The converted information is named according to "information packet ID - information type" and stored in an input format supported by the large model (e.g., TXT for text, NPY for feature vectors). A resource index file (ResourceIndex.json) is created to record the mapping relationship and associated dimensions between the original and converted information, ensuring the large model can locate the required resources.

[0086] Step S133: Input the transformed demand semantic deconstruction result and the transformed associated multimodal information set into the pre-trained large model to start the semantic understanding process of the large model, allowing the large model to parse the associated connotations of the demand semantic deconstruction result and the associated multimodal information set.

[0087] The pre-trained large model employs a multimodal Transformer architecture (such as FLAVA), with the input layer including text embedding, image embedding, and audio embedding branches. The transformed semantic deconstruction result (text) is input into the text branch, while the image and audio feature vectors from the associated multimodal information set are input into their respective branches. Multimodal features are fused through a cross-attention mechanism. The semantic understanding process of the large model includes: word / feature embedding → context encoding → cross-modal attention → semantic role labeling → entity relation extraction → intent recognition. In the intent recognition stage, the large model outputs its understanding of the report generation requirements, including the core task (e.g., "generate income tax settlement report"), sub-tasks (e.g., "income details analysis"), resource requirements (e.g., "need profit and loss statement data"), and format preferences (e.g., "prefer to display data in tables"). The understanding results are output as logs for subsequent task adaptation.

[0088] Step S134: Based on the semantic understanding results of the large model, construct a task-oriented adaptation framework. The task-oriented adaptation framework includes the core objectives of report generation, content coverage, and format constraints.

[0089] The task-oriented adaptation framework adopts a three-tiered pyramid structure: the top tier is the core objective (CoreObjective), which generates financial and tax reports that meet the requirements; the middle tier is the content scope (ContentScope), which includes all content dimension semantics and priorities (P1-P3, with P1 being the highest); and the bottom tier is the format constraints (FormatConstraints), which includes key specifications in the output format semantics (such as file formats and chart requirements). The framework also includes a resource mapping submodule, mapping each dimension of the content scope to information packages in the associated multimodal information set. The framework's dynamic execution flow is described using a UML activity diagram: Start → Objective Resolution → Content Dimension Prioritization → Resource Allocation → Format Constraint Check → Task Generation → End. The framework is stored in YAML format for easy modification and version control.

[0090] Step S1341: Extract the core intent of the report generation requirement from the semantic understanding results of the large model. The core intent corresponds to the core goal of report generation. Based on the core goal and combined with the content dimension semantics in the requirement semantic deconstruction results, determine all content dimensions that the report needs to include and the core expression points of each dimension. The content dimensions and core expression points constitute the content coverage.

[0091] The semantic understanding results of the large model include the "core task" field output by the intent recognition module, from which the core intent (e.g., "Generate the 2023 Corporate Income Tax Settlement and Payment Analysis Report") is extracted. The core intent is matched with the "content_dimensions" array of the demand semantic deconstruction results to determine the core expression points for each content dimension: for example, the core expression points for the "revenue composition analysis" dimension are "the proportion of main business revenue to other business revenue," "regional distribution of revenue," and "year-on-year revenue growth rate." The content coverage is represented as a dictionary structure, with the dimension ID as the key and the list of core expression points as the value, ensuring that each point corresponds to at least one information package in the associated multimodal information set.

[0092] Step S1342: Extract the output format semantics from the requirement semantic deconstruction results and convert them into specific format constraints. The format constraints include the report's layout requirements, presentation style, and file format requirements.

[0093] Extract the following from the "output_format" field of the requirements semantic deconstruction results: layout requirements (e.g., "center first-level headings, left-aligned second-level headings," "paragraph first-line indentation of 2 characters"), presentation style (e.g., "chart color scheme is blue," "data labels retain two decimal places"), and file format requirements (e.g., "output as PDF," "support table of contents navigation"). Transform these requirements into executable format constraints. Each constraint includes constraint type (layout / style / format), constraint item (e.g., "header alignment," "chart color scheme"), constraint value (e.g., "center," "blue color scheme"), and priority (1-5, 1 being the highest).

[0094] Step S1343: Construct the core structure of the task-oriented adaptation framework. The core structure includes a target layer, a scope layer, and a constraint layer. The target layer corresponds to the core target of report generation, the scope layer corresponds to the content coverage, and the constraint layer corresponds to the format constraints.

[0095] The target layer stores the text description and importance score (1-10) of the core target; the scope layer stores the dictionary structure of the content coverage, including dimension IDs, key points of the core statement, and a list of related information package IDs; the constraint layer stores a list of format constraints, sorted by priority. The core structure is implemented using a three-level nested dictionary: {"Target Layer":{"Core Target":"...","Importance":9}, "Scope Layer":{"Dimension ID1":{"Key Points":[...]","Information Package":[...]}, ...}, "Constraint Layer":[{"Type":"Formatting","Constraint Items":"...","Constraint Values":"...","Priority":3}, ...]}.

[0096] Step S1344: In the target layer, the core target of report generation is broken down into multiple executable sub-targets, each sub-target corresponding to a specific report generation task; in the scope layer, the priority order of each content dimension is determined based on the degree of correlation between each content dimension in the content coverage and the core target; in the constraint layer, the format constraints are classified and organized into three categories: layout constraints, style constraints, and format constraints, and specific execution standards are set for each type of constraint.

[0097] The core objective is broken down using a verb-object structure analysis. For example, "generating a tax settlement report" is broken down into sub-objectives such as "collecting tax data," "calculating tax payable," and "writing analysis explanations." Each sub-objective corresponds to a content dimension. The content dimensions are prioritized based on the strength of their dependency on the core objective in the semantic association matrix, with higher strength indicating higher priority (e.g., P1, P2, P3). After classifying the format constraints, execution standards are set for each category: layout constraints (e.g., "first-level heading font is bold, size 2"), style constraints (e.g., "line chart line width is 2pt"), and format constraints (e.g., "PDF version is 1.7").

[0098] Step S1345: Establish the association mapping between the target layer, the scope layer, and the constraint layer, associate each sub-target with one or more content dimensions, and associate each sub-target with a set of format constraint requirements.

[0099] The association between sub-objectives and content dimensions is achieved through semantic dependencies. For example, the sub-objective "Calculate tax payable" is associated with the content dimensions "Calculate taxable income" and "Tax incentive eligibility". The association between sub-objectives and formatting constraints is achieved by matching the "applicable scenario" field of the constraint conditions. For example, the sub-objective "Write analysis description" is associated with formatting constraints such as "body text font" and "paragraph spacing". The association mapping is stored as an association table, which contains a list of sub-objective IDs, a list of associated dimension IDs, and a list of associated constraint IDs, ensuring consistency between objectives, scope, and constraints.

[0100] Step S135: Under the guidance of the task-oriented adaptation framework, adjust the generation logic of the large model to ensure that the generation logic of the large model is consistent with the core goal of report generation. The generation logic includes the content organization order and information integration method.

[0101] The generation logic of the large model is adjusted through prompt tuning. A logic adjustment prompt (LogicAdjustmentPrompt) is designed, containing instructions on the content organization order (e.g., "Organize report chapters according to content dimension priority P1→P2→P3") and information integration methods (e.g., "Text information is the main content, image information is converted into charts and inserted into corresponding chapters, and cited data must be attributed to their sources"). The prompt includes example descriptions, such as "If the information package contains a 'revenue trend chart,' then insert the chart in the 'revenue composition analysis' chapter and add the text description: 'As shown in Figure X, revenue during period XX shows the XX trend.'" The prompt is combined with a task-oriented adaptation framework and input into the large model's instruction tuning module. Backpropagation is used to adjust the model's attention weights to ensure the generation logic conforms to the framework requirements.

[0102] Step S136: Extract the output format semantics from the requirements semantic deconstruction results and convert them into format specification instructions that can be executed by the large model. The format specification instructions include the report's layout structure, font style, paragraph spacing, and chart presentation method.

[0103] The output format semantics convert the layout structure into chapter-level instructions: "The report includes an abstract (first-level heading), the main body (containing X first-level headings, each corresponding to a content dimension), and a conclusion (first-level heading). The headings in the main body adopt the numbering system of 'I,' '(I)' '1.' '(1)'"; the font style is converted to: "The main body uses font F, font size S, and line spacing L; the first-level headings use font F1, font size S1, and are bold; the second-level headings use font F2, font size S2, and are bold"; the paragraph spacing instructions are: "Parallel spacing PS, paragraph spacing PE, and first-line indent I character"; the chart presentation instructions are: "The chart title is located below the chart, using font Fc and font size Sc; the chart data labels display numerical values, and the units are clearly marked; the trend chart uses a line chart, and the structure percentage chart uses a pie chart." The format specification instructions are written in Markdown format, which is convenient for large models to parse and generate reports in the corresponding format.

[0104] Step S137: Based on the content dimension semantics in the demand semantic deconstruction results, assign corresponding content generation tasks to the large model. Each content generation task corresponds to a content dimension semantic, and the information source of each task is determined to be the corresponding information combination in the associated multimodal information set.

[0105] Each dimension D_i (i=1..n) in the content dimension semantic list is mapped to a content generation task T_i. Task T_i includes a task ID (same as the dimension ID), a task name (same as the dimension name), a task description (same as the dimension description), an information source (a list of information package IDs for the corresponding dimension in the associated multimodal information set), and generation requirements (e.g., "must include data summary, trend analysis, and anomaly explanation"). The task allocation table (TaskAllocationTable) is stored in JSON format and contains a list of tasks and task dependencies (e.g., T2 must be executed after T1 is completed). The task allocation table is communicated to the large model through a task allocation prompt (TaskAllocationPrompt), which explicitly states: "For task T_i, please prioritize the text information in information package R_i as the primary basis, use image information for generating charts, and use structured data for calculation and summarization."

[0106] Step S138: Establish the relationship between content generation tasks, determine the execution order and information interaction method of each task, and adjust the execution order and information interaction method of content generation tasks to ensure that the generated content of different content dimensions remains logically coherent.

[0107] By analyzing the semantic dependencies between content dimensions (derived from the semantic association links of requirements), the association types between tasks are determined: sequential association (e.g., T2 is executed after T1 is completed), parallel association (T3 and T4 can be executed simultaneously), and dependent association (T5 needs to reference the result of T1). A directed acyclic graph (DAG) is used to represent task associations, with nodes representing tasks and directed edges representing association types. The execution order is determined using a topological sorting algorithm, resulting in the task execution sequence S=[T1, T3, T4, T2, T5, ...]. The information interaction method is defined as follows: the output of the preceding task serves as the input reference for the following task (e.g., "The 'total revenue' data generated by T1 needs to be referenced in the 'cost and expense analysis' of T2, with the reference format 'According to the revenue analysis results, the cost and expense corresponding to XX revenue is XX'"). A task flow diagram is used to visualize the associations and execution order, serving as a logical reference during the generation of the large model.

[0108] Step S139: Integrate format specification instructions, content generation tasks and task relationships to generate initial report generation task instructions. The initial report generation task instructions include preliminary content generation logic and format specification requirements.

[0109] The initial report generation task instructions are in XML format and contain three root elements: <contentgenerationlogic> 、 <formatspecification> 、 <taskrelationships> 。 <contentgenerationlogic>It includes the task execution sequence S and the generation requirements for each task; <formatspecification>Includes the format specification instructions generated in step S136; <taskrelationships>It includes a DAG description of task relationships. The instruction also contains metadata elements, recording the instruction version, generation time, and associated requirement ID. Structural constraints are defined using XMLSchema to ensure the instruction's integrity and validity. For example, <task>The element must contain the ID, Name, Source, and Dependencies attributes.

[0110] Step S1310: Perform logical verification on the initial report generation task instruction, and check whether the execution order of the content generation task is reasonable, whether the format specification requirements are complete, and whether the semantic deconstruction results of the task and the requirements match.

[0111] The logical validation includes three checks: execution order rationality check (verifying whether the task sequence S conforms to the topological order of the DAG and has no circular dependencies); format specification completeness check (checking against the output format semantics to see if any key specifications such as fonts and chart types are missing); and task-requirement matching check (verifying whether each content generation task corresponds to the content dimensions in the requirement semantic deconstruction result, with no redundant or missing tasks). Validation is implemented through a rule engine, with rules expressed in IF-THEN form, such as "IF task T_i's Dependencies contain T_j, THEN in sequence S, T_j must precede T_i". If validation fails, an error report is output (including error type, location, and suggested repair solutions), and the process returns to step S139 to regenerate the instruction.

[0112] Step S1311: Adjust and optimize the initial report generation task instruction based on the logic verification results, correct logical conflicts and format omissions, form the final report generation task instruction, and send the final report generation task instruction to the generation module of the large model as the execution basis for the large model to generate the initial report text.

[0113] Based on the logic verification error report, make targeted adjustments to the initial instructions: if circular dependencies exist, remove or adjust the dependencies of related tasks; if format specifications are missing, extract the corresponding specifications from the output format semantics and supplement them. <formatspecification>If the task and requirements do not match, add the missing task or delete the redundant task. Re-execute the validation after adjustments until all checks pass. Finally, the report generation task instructions are converted to JSON format supported by the large model generation module and sent to the generation module via an RPC interface. The interface call includes the instruction content, timeout (T seconds), and priority (high) parameters. The generation module returns an ACK confirmation, indicating that report generation has begun.

[0114] Step S140: Invoke the large model to perform content transformation and integration processing on the associated multimodal information set based on the report generation task instruction, and generate an initial report text. The initial report text contains all content dimensions corresponding to the semantic deconstruction results of the requirements.

[0115] After receiving the report generation task instruction, the large model generation module executes the following process: instruction parsing → resource loading → content generation → integration and layout. In the instruction parsing stage, the task sequence S, format specifications, and resource references are extracted. In the resource loading stage, information packets are read from the associated multimodal information set based on resource references, and text content, image descriptions, and structured data are parsed. In the content generation stage, each task T_i is processed sequentially according to the task sequence S: using the text information in the information packet as the basic content, it is expanded and refined using the large model's text generation capabilities (e.g., converting data tables into analytical text), converting image descriptions into chart insertion suggestions, and converting structured data into statistical analysis results. In the integration and layout stage, text styles and paragraph spacing are adjusted according to the format specifications, charts are inserted (calling the large model's chart generation API to generate charts that meet the requirements based on image information), and headers, footers, and a table of contents are added. During the generation process, the large model generates text segment by segment using an autoregressive method, performing internal validation (e.g., data consistency, semantic coherence) after each generated chapter. The final generated initial report text is in rich text format (e.g., DOCX), containing all content dimensions, chapters, charts, and references.

[0116] Step S141: Call the generation module of the large model to receive the report generation task instruction, parse the content generation logic and format specification requirements in the report generation task instruction, and determine the specific requirements and execution steps for report generation.

[0117] The instruction parser of the large model generation module performs syntactic analysis and semantic understanding on the received report generation task instructions. Syntactic analysis uses an LR parser to verify the JSON format validity of the instructions; semantic understanding extracts key information through Named Entity Recognition (NER) and Relation Extraction (RE) techniques: the task sequence S in the content generation logic, the information source and generation requirements for each task; and the layout structure, font style, and chart parameters in the format specification requirements. The parsing results are stored in a TaskContext object, containing the current task index, the generated content cache, a resource reference table, and a format state machine (tracking the current font, font size, and paragraph style). The execution step planner converts the parsing results into a list of operation steps, such as "Step 1: Load the information package of task T1; Step 2: Generate the chapter text of T1; Step 3: Apply first-level heading format; Step 4: Insert the related chart of T1; Step 5: Execute task T2...".

[0118] Step S142: Based on the content organization order in the content generation logic, extract the multimodal information combination of the corresponding content dimension from the associated multimodal information set, and use the multimodal information combination as the generation material of the current content dimension.

[0119] Based on the current task T_i in the task sequence S, the associated information packet R_i is retrieved from the associated multimodal information set. Information packet R_i contains text information packets (T1, T2, ..., Tm), image information packets (I1, I2, ..., In), and structured data (D1, D2, ..., Dk). The extraction process includes: text information merging (sorting the text content in Tm according to time order or importance and then concatenating it); image information filtering (sorting the images according to their relevance to task T_i and selecting the Top P images); and structured data transformation (converting Dk to DataFrame format for statistical analysis). The generated materials are organized as material objects, containing a list of text segments, a list of images, and a list of data tables. Each list item contains the original content, source identifier, and confidence level.

[0120] Step S143: Perform content transformation processing on the extracted multimodal information combination, converting image information into text description content, audio information into text transcription content, video information into shot-by-shot text description content, and directly extracting the core expression content from the text information.

[0121] Image information conversion uses image captioning models (such as BLIP) to generate text descriptions, which include the image theme (e.g., "2023 annual revenue trend line chart"), key data points (e.g., "The highest point occurred in Q4, at XXX million yuan"), and trend analysis (e.g., "The overall trend shows a decline followed by an increase"). Audio information conversion first generates transcribed text through speech recognition (ASR), and then extracts the core content through semantic compression (e.g., "Meeting minutes core conclusion: XX expenses need to be adjusted to increase taxable income"). Video information conversion extracts keyframes (one frame every N seconds), generates image descriptions for each frame, and then combines them into shot-by-shot text descriptions in chronological order (e.g., "Shot 1: Showing a summary of the balance sheet, total assets XXX million yuan; Shot 2: Highlighting the proportion of current assets at 60%..."). Text information extraction uses the TextRank algorithm to extract core sentences and keywords, removing redundant expressions and duplicate content. The converted content is marked with source information (e.g., "[Source: Financial Statements 2023Q4]").

[0122] Step S144: According to the information integration method in the content generation logic, perform preliminary integration processing on the transformed text content to form a preliminary text fragment of the current content dimension. The preliminary text fragment contains the core information of the content dimension.

[0123] The information integration method is executed according to the generation requirements of task T_i: if "data summary + trend analysis" is required, the integration process is "data extraction (extracting key indicators from structured data Dk) → trend calculation (calculating month-on-month / year-on-year growth rate) → text description (converting data and trends into natural language) → chart suggestion (line chart is recommended to display trends)"; if "structural analysis" is required, the process is "entity identification (extracting constituent elements from text) → percentage calculation (calculating the percentage of each element) → text description → chart suggestion (pie chart or donut chart is recommended)". Template-based generation technology is used during the integration process. The template contains placeholders (such as "{indicator name} has an average value of {value} over {time range}, and {percentage} of {growth / decline} compared to the previous period"). The text is generated by filling the placeholders with information extracted from the material. The initial text fragment includes a title, body paragraphs, and chart reference marks (such as "[CHART:I1]" indicating that the chart generated by image I1 should be inserted here).

[0124] Step S145: Perform semantic coherence processing on the preliminary text fragment, identify the logical relationships between sentences in the preliminary text fragment, adjust the order of sentences according to the logical relationships, and insert corresponding connecting words between sentences.

[0125] The coherence submodule optimizes the initial text fragments: Logical relationship identification is achieved through dependency parsing and discourse relationship identification (such as models trained on the PDTB corpus), identifying causal relationships (cause → effect), progressive relationships (shallow → deep), and adversative relationships (positive → negative) between sentences; Sentence ordering uses a greedy algorithm, starting with the first sentence and selecting the next sentence with the strongest logical relationship as the subsequent sentence, until all sentences are ordered; Connective words are inserted according to the type of logical relationship, such as inserting "therefore" or "because" for causal relationships, inserting "in addition" or "furthermore" for progressive relationships, and inserting "however" or "but" for adversative relationships. For example, the original sentence "Revenue increased. Costs decreased." was identified as a parallel relationship, and after inserting the connective words, it becomes "Revenue increased, and costs decreased at the same time." Step S146: Repeat the extraction, transformation, integration, and coherence processing steps to generate text fragments for all content dimensions in sequence, resulting in multiple independent text fragments for each content dimension.

[0126] Each task T_i (i=1ton) is processed sequentially according to task sequence S: step S142 is called to extract information package R_i, step S143 converts multimodal information, step S144 integrates it into preliminary text fragments, and step S145 optimizes semantic coherence. Each content dimension text fragment is stored as an independent temporary file, with the filename containing the task ID and timestamp. The fragment contains metadata (such as word count, number of charts, source information package ID) and content data (rich text format). After completing all fragments, an integrity check is performed to ensure that each content dimension has a corresponding text fragment, and that the fragment word count meets the expected range (set according to the importance of the content dimensions: P1 dimension > P2 dimension > P3 dimension).

[0127] Step S147: Based on the task association in the content generation logic, determine the logical connection order of text fragments in each content dimension; arrange the text fragments in each content dimension according to the logical connection order, and insert or adjust transitional content between adjacent text fragments to form a preliminary report text framework.

[0128] Based on the Directed Acyclic Graph (DAG) of task relationships, determine the logical connection order of the text fragments in the content dimension (consistent with the task sequence S). Transitional content is inserted between adjacent fragments, generated according to the relationship type between the preceding and following fragments: if it's a progressive relationship (e.g., "Revenue Analysis" → "Cost Analysis"), the transition sentence is "After analyzing the revenue situation, we will now discuss the cost and expense structure in detail"; if it's a causal relationship (e.g., "Tax Adjustment Items" → "Tax Payable Calculation"), the transition sentence is "Based on the above tax adjustment items, the final tax payable is calculated as follows"; if it's a parallel relationship (e.g., "Operating Revenue Analysis" → "Other Business Revenue Analysis"), the transition sentence is "Besides the main business revenue, the other business revenue situation is as follows." The transitional content is automatically generated by the transition sentence generator of the large model (based on a fine-tuned BART model), referencing the core themes and keywords of the preceding and following fragments during generation. The preliminary report text framework includes a title page, table of contents (automatically generated), chapters for each content dimension (arranged in order), and a conclusion chapter (initially empty, to be generated later).

[0129] For example, step S1471: parse the task association relationship in the content generation logic, extract the logical relationship type between each content generation task, and the logical relationship type includes progressive relationship, parallel relationship, causal relationship and temporal relationship.

[0130] Extract the attribute labels of the edges from the DAG of task relationships to determine the logical relationship type between tasks: progressive relationship (e.g., "Basic Data Summary" → "In-depth Trend Analysis", indicating a deeper level of analysis), parallel relationship (e.g., "Operating Revenue Analysis" → "Operating Cost Analysis", indicating different aspects at the same level), causal relationship (e.g., "Tax Adjustment Items" → "Tax Payable Calculation", indicating that the former is a prerequisite for the latter), and temporal relationship (e.g., "Quarterly Data Summary" → "Annual Data Summary", indicating chronological order). The logical relationship type is stored in the task relationship table, and each record contains the source task ID, target task ID, relationship type, and relationship strength (0-1).

[0131] Step S1472: Perform core information extraction processing on each content dimension text fragment, extract the core viewpoints and key information in the text fragment, and form a core information summary of the text fragment.

[0132] An extractive summarization model based on BERT is used to extract core information from text fragments for each content dimension. The model input is a sequence of sentences from the text fragment, and the output is an importance score for each sentence. The top three sentences with the highest scores are selected as the core viewpoints. Key information is extracted using an entity recognition model, including data entities (such as "taxable income of XXX million yuan"), time entities (such as "2023 year"), and attribute entities (such as "year-on-year growth of 15%)". The core information summary format is: {"core viewpoint":[sentence 1, sentence 2, sentence 3], "key information":[{"entity type":"data", "value":"XXX million yuan"},...]}.

[0133] Step S1473: Based on the logical relationship type in the task association relationship, determine the arrangement order of text fragments in each content dimension. For progressive relationships, arrange them according to the order of deepening viewpoints; for parallel relationships, arrange them according to the order of content importance; for causal relationships, arrange them according to the order of cause before effect; and for temporal relationships, arrange them according to the order of time sequence.

[0134] Progressive tasks are arranged in the order of "basic → advanced" (e.g., "data summary" → "trend analysis" → "anomaly diagnosis"); parallel tasks are arranged according to content dimension priority (P1 → P2 → P3); causal tasks are arranged in the order of "cause → effect" (e.g., "tax adjustment" → "tax calculation"); and temporal tasks are arranged in ascending order of timestamps (e.g., "Q1 data" → "Q2 data" → "Q3 data" → "Q4 data"). After the arrangement order is generated, it is verified to be conflict-free using a topological sorting algorithm to ensure that it conforms to the logical flow.

[0135] Step S1474: According to the determined arrangement order, the text fragments of each content dimension are initially spliced ​​together to form an initial text framework, which contains the initial arrangement structure of each text fragment.

[0136] The text fragments are concatenated sequentially according to their arrangement, with temporary separators (such as "[SEGMENT_SEPARATOR]") added between them. The initial text frame contains a title page (temporarily stored), a table of contents (to be generated), the main text (arranged in fragment order), and a conclusion (temporarily stored). Each text fragment retains its metadata (such as word count and figure / table reference markers) within the frame, facilitating subsequent insertion of transitional content and formatting adjustments.

[0137] Step S1475: Analyze the semantic connection between adjacent content dimension text fragments, identify the locations where the semantic connection is not smooth in the initial text framework, and generate corresponding transition sentences based on the core information summary of adjacent text fragments.

[0138] Calculate the semantic similarity (cosine similarity) of the core information summaries of adjacent segments. If the similarity is less than the threshold Th_c (e.g., 0.4), it is considered that the semantic connection is not smooth. Call the transition statement generation model (based on BART fine-tuning), input the core viewpoints and key information of the preceding and following segments, and generate transition statements. For example, if the core viewpoint of the preceding segment is "revenue increased by 10% year-on-year" and the core viewpoint of the following segment is "costs decreased by 5% year-on-year", the generated transition statement is: "While revenue has increased, costs have also shown a downward trend, as detailed below:".

[0139] Step S1476: Insert the generated transition statement into the position where the semantic connection is not smooth in the initial text frame to obtain the associated text frame. Add a title level to the associated text frame. The title level is determined based on the division of content dimensions. The first-level title corresponds to the main content dimension, and the second-level title corresponds to the sub-dimension under the content dimension. The text frame with added titles is obtained.

[0140] Transitional statements are inserted at the delimiter positions in the initial text frame to replace the delimiters. The heading hierarchy is generated according to the hierarchical relationship of the content dimensions: the first-level heading corresponds to the P1 priority dimension (e.g., "I. Revenue Composition Analysis"), the second-level heading corresponds to the sub-dimensions under the P1 dimension (e.g., "(I) Main Business Revenue Analysis"), and the third-level heading corresponds to the sub-items under the sub-dimensions (e.g., "1. Product Revenue Analysis"). The heading numbering adopts the "I. (I), 1., (1)" system to ensure clear hierarchy.

[0141] Step S1477: Perform logical coherence analysis on the text framework after adding titles, compare the actual logical relationship between each text segment with the task association relationship, and compare the title level with the content dimension division. Based on the comparison results of the logical relationship, adjust the logical connection order and transitional content between text segments; based on the comparison results of the title level and content dimension division, adjust the title level classification; and form a preliminary report text framework.

[0142] Logical coherence analysis is achieved through a coherence scoring model. The model scores each pair of adjacent segments in the text framework. If the score is less than a threshold Th_l (e.g., 0.6), the segment order is adjusted or transitional statements are modified based on task relevance. A heading hierarchy comparison checks for inconsistencies between heading levels and content dimensions (e.g., a sub-dimension being labeled a first-level heading). If such inconsistencies exist, the heading hierarchy is adjusted (e.g., downgrading a sub-dimension heading to a second-level heading). After adjustment, a preliminary report text framework containing the correct logical order and heading hierarchy is generated.

[0143] Step S148: Adjust the format of the preliminary report text framework according to the format specification requirements, and set paragraph divisions, heading levels and text alignment.

[0144] Formatting adjustments are performed according to formatting guidelines: paragraphs are automatically divided based on punctuation (., !, ?) and semantic pauses (such as the first-line keywords "firstly" and "secondly"); style sheets are applied to heading levels: first-level headings use "bold, size 2, centered"; second-level headings use "bold, size 3, left-aligned"; and body text uses "SimSun, size 4, justified"; text alignment is set according to content type: headings are centered, body text is justified, chart titles are centered, and table content is centered. Document Object Model (DOM) manipulation tools are used for formatting adjustments, such as setting paragraph style attributes (font-family, font-size, text-align) and heading outline-level attributes. The adjusted text frame is saved as a DOCX file, containing style definitions (Styles.xml) and content (Document.xml).

[0145] Step S149: Insert the image information and video frame information from the associated multimodal information set into the corresponding positions in the report text according to the format specifications to form an initial report text containing text and images.

[0146] Iterate through the chart reference markers (e.g., "[CHART:I1]") in the preliminary report text framework, and retrieve the corresponding image information I1 from the associated multimodal information set based on the marker ID. Call the large model's chart generation API to convert the image information I1 into a chart conforming to the format specifications: if the original image is a trend chart, maintain the trend chart type, and adjust the color scheme (e.g., change to blue tones), font (SimSun), and axis labels (add units); if the original image is a scanned document (e.g., an invoice), convert it to a table format (extract key information such as invoice number, amount, and date). After the chart is generated, insert it into the report text at the marked position and update the chart number (e.g., "Figure 1 2023 Annual Revenue Trend Chart"). Convert video frame information into a keyframe image sequence, insert it into the corresponding chapter, and add text descriptions (e.g., "Video keyframe display: XX business process includes the following steps..."). After insertion, update the chart index and page numbers in the table of contents.

[0147] Step S1410: Verify the content dimensions contained in the initial report text, compare the verification results with the content dimension semantics in the requirement semantic deconstruction results, and supplement the information corresponding to the missing content dimensions in the initial report text.

[0148] Content dimension verification employs a two-way comparison method: chapter titles are extracted from the initial report text and compared with the content dimension semantic list from the requirement semantic deconstruction result to generate a discrepancy report. If any content dimensions (MissedDimensions) are missing in the report, the supplementary content generator of the large model is invoked to regenerate the text fragments for that dimension based on the corresponding information packages in the associated multimodal information set, and inserts them into the corresponding positions in the report (the insertion order is determined according to the content dimension priority). If any extra chapters (ExtraChapters) exist in the report and are not in the content dimension semantic list, the user is prompted whether to retain them (implemented through system prompts); if the user does not specify, they are deleted by default. After supplementation, the table of contents and chart index are regenerated to ensure consistency with the report content.

[0149] Step S150: Perform structural optimization and semantic connection optimization on the initial report text to obtain the final intelligent report, which meets the output format semantics and content integrity requirements in the semantic deconstruction results of the requirements.

[0150] The initial report text optimization comprises two stages: structural optimization and semantic coherence optimization. The structural optimization stage analyzes whether the report's chapter distribution is even (standard deviation of chapter word count < σ), whether the heading hierarchy is clear (no more than 4 levels), and whether the chart placement is reasonable (proximity principle, interval between charts and cited text < 3 paragraphs). The semantic coherence optimization stage uses a coherence scoring model (a BERT-based classification model, outputting a coherence score of 0-1) to score the entire report, providing manual intervention or automatic rewriting for paragraphs scoring below the threshold Th. Optimization also includes data consistency checks (ensuring consistent values ​​for the same indicator throughout the report), terminology consistency checks (ensuring consistent use of professional terms such as "taxable income" throughout the text), and format standardization checks (ensuring continuous chart numbering and correct citations). The final intelligent report output is in PDF format (meeting output format semantic requirements), including the report text, charts, appendices (data source descriptions), and an index (keywords and corresponding page numbers). A checksum for the final report is generated for integrity verification.

[0151] Step S151: Perform structural parsing on the initial report text, extract the title hierarchy, paragraph distribution, and content module division of the initial report text, and form the initial structural analysis results.

[0152] The structural parsing process uses a document structure analyzer (fine-tuned based on the LayoutLM model): the title hierarchy is identified by font size, weight, and style, generating a title tree (TitleTree), where each node contains the title text, level, page number, and start / end character position; paragraph distribution analysis calculates the number of paragraphs, average paragraph word count, and longest / shortest paragraph word count for each chapter, generating a paragraph statistics histogram; content module division aggregates consecutive paragraphs into content modules (such as "Data Summary Module", "Analysis and Discussion Module", "Conclusion and Recommendation Module") based on topic similarity (using cosine similarity to calculate the topic distance between paragraphs). The initial structural analysis results are output in JSON format, containing three fields: "title_hierarchy", "paragraph_distribution", and "content_modules".

[0153] Step S152: Compare the initial structural analysis results with the output format semantics in the requirement semantic deconstruction results, output the comparison difference results, and perform structural optimization processing on the initial report text based on the comparison difference results. The structural optimization processing includes: adjusting the title level according to the output format semantics, correcting the paragraph division according to the output format semantics, and optimizing the arrangement order of the content modules according to the output format semantics, to obtain the structurally optimized report text.

[0154] The comparison process uses a structured difference algorithm (such as the tree edit distance algorithm) to compare the initial structure analysis results with the structural requirements in the output format semantics: the title hierarchy is compared to check whether it conforms to the hierarchical system of "I, (I), 1, (1)". If the hierarchy is found to be disordered (such as a first-level title appearing directly after a third-level title), it is adjusted to the correct hierarchy; the paragraph distribution is compared to check whether it conforms to the requirement of "the number of paragraphs in each chapter is controlled between MN". Excessively long paragraphs are split (according to semantic pause points), and excessively short paragraphs are merged (merging adjacent and topic-related paragraphs); the content module arrangement order is compared to check whether it conforms to the logical order of "data → analysis → conclusion". If the module order is reversed (such as the "conclusion" module appearing before the "analysis" module), it is adjusted to the correct order. The structure-optimized report text regenerates the title tree and paragraph distribution statistics to ensure consistency with the output format semantics.

[0155] Step S153: Perform semantic fluency analysis on the optimized report text to identify semantic gaps, logical disjointness, and redundant segments in the optimized report text.

[0156] Semantic fluency analysis uses a fluency assessment model. The model takes a sequence of sentences from the report text as input and outputs a fluency score and problem type for each sentence (no problem, gaps in cohesion, logical inconsistencies, and redundancy). Gaps in cohesion refer to a lack of semantic connection between adjacent sentences (e.g., the previous sentence discusses revenue, and the next sentence suddenly discusses costs without transition); logical inconsistencies refer to logical contradictions between sentences (e.g., "revenue decreased by 5%" and "profit increased by 10%" are not explained); redundancy refers to repetitive statements (e.g., the same data is mentioned repeatedly in adjacent paragraphs). During the analysis, the model also checks for referential consistency (e.g., whether "this data" clearly refers to data mentioned earlier) and tense consistency (the tense throughout the text is consistently past or present). Problem identification results are displayed in the report editing interface with highlights and annotations.

[0157] Step S154: Adjust the expression of the preceding and following sentences for the identified semantic connection gaps; add logical connectors or transition sentences for the identified logical disconnects in the sentences; delete or merge the identified redundant segments of expression; and obtain the semantically optimized report text.

[0158] Semantic cohesion optimization employs targeted measures: For semantic gaps, keywords in sentences are adjusted to ensure consistency with the preceding and following text (e.g., if "revenue" is mentioned earlier, "revenue" is changed to "operating revenue" later), or transitional phrases are added (e.g., "related to this is" or "from another perspective"). For logical disjointness, logical connectors are added (e.g., "because," "therefore," "despite this") or explanatory statements are provided (e.g., "profit growth is mainly due to a 15% cost reduction"). For redundant passages, pronouns are used to replace repeated nouns (e.g., "the indicator" replaces the repeated "operating profit margin"), or repeated paragraphs are merged (retaining core information and removing redundant descriptions). Optimization is automatically performed by the large model's text polisher (based on GPT-4 fine-tuning), preserving the original style and terminology during polishing, adjusting only sentence structure and connection methods.

[0159] Step S155: Perform content consistency processing on the semantically optimized report text, identify different expressions of the same core concept in the report text, replace them with a unified preset expression, and adjust the discussion angle and level of detail of the core concept between different content modules to obtain the report text after content consistency processing.

[0160] Content consistency processing encompasses two aspects: terminology standardization and discourse coordination. Terminology standardization utilizes a tax and finance terminology dictionary to identify different expressions of the same concept in the report (e.g., "taxable income," "taxable amount," "taxable income") and replace them with pre-defined standard terms (e.g., "taxable income"). Discourse coordination analyzes the different perspectives (policy basis, calculation method, actual impact) and levels of detail in different content modules regarding the same core concept (e.g., "R&D expense deduction"). This ensures that important concepts are discussed in detail in key modules and briefly mentioned in other modules without contradiction. For example, the "Tax Incentive Enjoyment" module details the calculation process for R&D expense deduction, while the "Tax Payable Calculation" module only cites the result, stating, "The R&D expense deduction amount has been calculated using the aforementioned method and will not be repeated here."

[0161] Step S156: Extract the content dimension semantics from the requirement semantic deconstruction results, check the corresponding content dimensions in the report text after content consistency processing one by one, and check whether there are any missing or distorted content segments in the report text after content consistency processing.

[0162] Content dimension verification employs a dimension mapping method: the semantic list of content dimensions from the requirement semantic deconstruction results is mapped to the chapters of the report text, with each dimension corresponding to one or more chapters. Verification personnel or automated verification tools check one by one whether each dimension has corresponding content in the report and whether the content covers all key points in the dimension description (e.g., the "Revenue Composition Analysis" dimension needs to cover key points such as "Revenue Share of Main Business," "Revenue Share of Other Businesses," and "Regional Distribution of Revenue"). Statement deviation checks are achieved through keyword matching and semantic similarity calculation. If the content of a certain dimension in the report deviates from the information in the associated multimodal information set by more than a threshold (e.g., data difference > 5%), it is marked as a statement deviation. The verification results generate a content completeness report, including covered dimensions, uncovered dimensions, deviation dimensions, and suggested modification schemes.

[0163] Step S157: Based on the verification results of missing content or expression deviation, supplement the corresponding content information in the report text after content consistency processing, and modify the content with expression deviation to obtain the supplemented and corrected report text.

[0164] Content supplementation and correction are based on the content integrity report: For missing content, unused information (such as omitted charts and data tables) is extracted from the corresponding information package of the associated multimodal information set, supplementary text is generated from the large model, and inserted into the corresponding chapter; for discrepancies in expression, the information source and calculation process are re-examined. If the data reference is incorrect, the data is corrected; if the analysis is incorrect, the analysis text is regenerated. During the supplementation and correction process, the consistency of the report format (e.g., supplementary charts must conform to chart format specifications) and the continuity of page numbers must be maintained. After correction, content consistency processing and verification are re-executed until all content dimensions are covered and there are no expression discrepancies.

[0165] Step S158: Based on the font style, paragraph spacing, and chart presentation method in the output format semantics of the demand semantic deconstruction results, perform final format optimization processing on the supplemented and corrected report text to make the presentation effect of the supplemented and corrected report text meet the requirements of the output format semantics, obtain the format-optimized report text, convert the format-optimized report text into the preset output file format, and generate the final intelligent report.

[0166] The final formatting optimization includes: consistent font styles (applying style sheets to ensure uniform font, size, and color throughout the text), paragraph spacing adjustments (20-point spacing before paragraphs, 10-point spacing after paragraphs, and 1.5-fold line spacing), optimized chart presentation (chart titles are located below the chart, in SimSun font, size 5, with data labels displayed to two decimal places), and header / footer settings (report title on the left side of the header, chapter name on the right; page number centered in the footer, formatted as "Page X of Y"). A professional document conversion tool is used to convert the optimized report text to a preset output file format (such as PDF). During the conversion process, fonts are embedded (to avoid formatting errors due to missing fonts) and images are compressed (to maintain clarity while reducing file size). After the final intelligent report is generated, it is digitally signed (to ensure the report has not been tampered with) and marked with a version number (V1.0.0), stored in the system's report library, and a completion notification is sent to the user.

[0167] Figure 2 The following is a schematic diagram of the hardware structure of an intelligent report automatic generation system 100 based on a multimodal large model, provided by an embodiment of the present invention, for implementing the above-described intelligent report automatic generation method based on a multimodal large model. Figure 2 As shown, the intelligent report automatic generation system 100 based on a multimodal large model may include a processor 110, a machine-readable storage medium 120, a bus 130, and a communication unit 140.

[0168] Machine-readable storage medium 120 can store data and / or instructions. In some embodiments, machine-readable storage medium 120 can store data acquired from an external terminal. In some embodiments, machine-readable storage medium 120 can store data and / or instructions used by the intelligent report automatic generation system 100 based on a multimodal large model to execute or use in order to complete the exemplary methods described in this invention. In a specific implementation, one or more processors 110 execute the computer-executable instructions stored in machine-readable storage medium 120, enabling processor 110 to execute the intelligent report automatic generation method based on a multimodal large model as described in the above method embodiments. Processor 110, machine-readable storage medium 120, and communication unit 140 are connected via bus 130, and processor 110 can be used to control the transmission and reception actions of communication unit 140. The specific implementation process of processor 110 can be found in the various method embodiments executed by the intelligent report automatic generation system 100 based on a multimodal large model described above, and their implementation principles and technical effects are similar, so they will not be repeated here.

[0169] Furthermore, this embodiment of the invention also provides a readable storage medium containing computer-executable instructions. When the processor executes the computer-executable instructions, the above-mentioned intelligent report automatic generation method based on a multimodal large model is implemented.

[0170] It should be noted that, in order to simplify the description of this invention and thus aid in the understanding of one or more embodiments, the foregoing description of the embodiments of this invention sometimes combines multiple features into a single embodiment, drawing, or description thereof. Similarly, it should be noted that, in order to simplify the description of this invention and thus aid in the understanding of one or more embodiments, the foregoing description of the embodiments of this invention sometimes combines multiple features into a single embodiment, drawing, or description thereof.< / formatspecification> < / task> < / taskrelationships> < / formatspecification> < / contentgenerationlogic> < / taskrelationships> < / formatspecification> < / contentgenerationlogic>

Claims

1. A method for automatically generating intelligent reports based on a multimodal large model, characterized in that, The method includes: Receive a report and generate requirement text, perform semantic deconstruction processing on the report and generate requirement text to obtain requirement semantic deconstruction result, the requirement semantic deconstruction result includes report theme semantics, content dimension semantics and output format semantics; Based on the results of demand semantic deconstruction, multimodal information association mapping rules are constructed. Multimodal information acquisition interface is called to obtain multimodal information that matches the results of demand semantic deconstruction. Multimodal information is then associated through the multimodal information association mapping rules to obtain a set of associated multimodal information. Based on the semantic deconstruction results of the requirements and the associated multimodal information set, a task-oriented adaptation process for the large model is performed to generate a report generation task instruction, which includes content generation logic and format specification requirements. The large model is invoked to perform content transformation and integration processing on the associated multimodal information set based on the report generation task instruction, and an initial report text is generated. The initial report text contains all content dimensions corresponding to the semantic deconstruction results of the requirements. The initial report text is structurally and semantically optimized to obtain the final intelligent report, which meets the output format semantics and content integrity requirements in the semantic deconstruction results.

2. The intelligent report automatic generation method based on a multimodal large model according to claim 1, characterized in that, The received report generates a requirement text, and the requirement text is semantically deconstructed to obtain a requirement semantic deconstruction result, including: The system receives a report and generates a requirement text. It then performs semantic unit extraction on the generated requirement text, breaking it down into multiple independent requirement semantic units. Each requirement semantic unit corresponds to a core expression in the generated requirement text. Semantic association links are constructed for multiple semantic units of demand, the semantic dependency relationships between each semantic unit of demand are analyzed, and semantic association links of demand are formed based on the semantic dependency relationships. The semantic association links of demand contain the sequential association order of each semantic unit of demand. Based on the semantic association link of requirements, core requirement semantic focusing processing is performed to identify the core semantic nodes in the semantic association link of requirements. The core semantic nodes correspond to the core objectives of report generation requirements. Thematic semantic information related to the core semantic nodes is extracted to form the semantics of the report theme. Taking the semantics of the report topic as the core, non-core semantic nodes in the semantic association link of requirements are classified and processed according to semantic attributes into content-related semantic nodes and format-related semantic nodes. Content-related semantic nodes correspond to the content direction that the report needs to cover, while format-related semantic nodes correspond to the output presentation requirements of the report. The content-related semantic nodes are decomposed into multiple specific content sub-nodes. Each content sub-node corresponds to a specific content dimension in the report. All content sub-nodes are integrated to form the content dimension semantics. The format-related semantic nodes are extracted and processed in a standardized manner. The output format information, typesetting requirements information and presentation style information are extracted from the format-related semantic nodes, and the output format information, typesetting requirements information and presentation style information are integrated to form the output format semantics. Semantic consistency association processing is performed on the semantics of the report topic, the semantics of the content dimension, and the semantics of the output format to establish the correspondence between the semantics of the report topic and the semantics of the content dimension, as well as the adaptation relationship between the semantics of the content dimension and the semantics of the output format; The semantics of the associated report topic, content dimension, and output format are structurally integrated to form a semantic deconstruction result of the requirements. The semantic deconstruction result of the requirements adopts a hierarchical structure to present the relationship between the semantics.

3. The intelligent report automatic generation method based on a multimodal large model according to claim 1, characterized in that, The process involves constructing multimodal information association mapping rules based on the demand semantic deconstruction results, calling the multimodal information acquisition interface to obtain multimodal information matching the demand semantic deconstruction results, and then performing association processing on the multimodal information through the multimodal information association mapping rules to obtain an associated multimodal information set, including: The report topic semantics and content dimension semantics in the semantic deconstruction results of the requirements are analyzed, and the core semantic keywords are extracted. The core semantic keywords are used to represent the core information direction required for report generation. A multimodal information association dimension system is constructed based on core semantic keywords. The multimodal information association dimension system includes text information association dimension, image information association dimension, audio information association dimension and video information association dimension, and each association dimension corresponds to a type of multimodal information. Semantic matching rules are formulated for each association dimension. The semantic matching rules determine the matching standard between core semantic keywords and corresponding multimodal information types. The semantic matching rules for different association dimensions are set differently according to the characteristics of multimodal information types. By integrating semantic matching rules from various association dimensions, a multimodal information association mapping rule is constructed. This rule includes the matching relationship between core semantic keywords and various multimodal information types, as well as the association conditions between multimodal information. Based on the core semantic keywords in the demand semantic deconstruction results, configure the collection parameters of the multimodal information collection interface. The collection parameters include the information collection scope, information type filtering conditions and information timeliness requirements. Call the multimodal information acquisition interface, perform multimodal information acquisition operations according to the configured acquisition parameters, and obtain an initial multimodal information set. The initial multimodal information set contains multiple types of unrelated multimodal information. Semantic annotation is performed on the initial multimodal information set to extract the core content of each type of multimodal information, and corresponding semantic tags are added to each type of multimodal information based on the core content; The semantic tags in the initial multimodal information set are matched with the core semantic keywords in the multimodal information association mapping rules to filter out the candidate multimodal information set that matches the core semantic keywords; Based on the multimodal information association conditions in the multimodal information association mapping rules, the multimodal information in the candidate multimodal information set is associated to establish semantic association relationships between different types of multimodal information; Multimodal information with semantic relationships is combined and integrated to form a set of associated multimodal information, wherein each information combination in the set of associated multimodal information is associated with a content dimension semantic in the result of demand semantic deconstruction.

4. The intelligent report automatic generation method based on a multimodal large model according to claim 1, characterized in that, Based on the semantic deconstruction results of the requirements and the associated multimodal information set, the task-oriented adaptation processing of the large model is performed to generate report generation task instructions, including: The semantic deconstruction results of the requirements are transformed into a semantic input format that can be recognized by the large model. The semantic input format includes semantic identifiers of the report topic, semantic lists of content dimensions, and semantic descriptions of the output format, enabling the large model to accurately parse the semantic deconstruction results of the requirements. The associated multimodal information set is format-converted to convert different types of multimodal information into an input format supported by the large model. The core semantic content and relationships of the associated multimodal information set are preserved during the format conversion process. The transformed demand semantic deconstruction results and the transformed set of associated multimodal information are input into the pre-trained large model to start the semantic understanding process of the large model, allowing the large model to parse the associated connotations of the demand semantic deconstruction results and the set of associated multimodal information. Based on the semantic understanding results of the large model, a task-oriented adaptation framework is constructed, which includes the core objectives of report generation, content coverage, and format constraints. Guided by the task-oriented adaptation framework, the generation logic of the large model is adjusted to ensure that the generation logic of the large model is consistent with the core goal of report generation. The generation logic includes the content organization order and information integration method. Extract the output format semantics from the demand semantic deconstruction results and transform them into format specification instructions that can be executed by the large model. The format specification instructions include the report's layout structure, font style, paragraph spacing, and chart presentation method. Based on the content dimension semantics in the demand semantic deconstruction results, corresponding content generation tasks are assigned to the large model. Each content generation task corresponds to a content dimension semantic, and the information source of each task is determined to be the corresponding information combination in the associated multimodal information set. Establish the relationships between content generation tasks, determine the execution order and information interaction method of each task, and adjust the execution order and information interaction method of content generation tasks to ensure that the generated content of different content dimensions remains logically coherent. Integrate format specification instructions, content generation tasks, and task relationships to generate initial report generation task instructions, which include preliminary content generation logic and format specification requirements. Perform logical validation on the initial report generation task instructions to check whether the execution order of the content generation tasks is reasonable, whether the format specification requirements are complete, and whether the task and requirement semantic deconstruction results match. Based on the logical verification results, the initial report generation task instructions are adjusted and optimized to correct logical conflicts and format omissions, forming the final report generation task instructions. These final instructions are then sent to the generation module of the large model as the basis for the large model to generate the initial report text.

5. The intelligent report automatic generation method based on a multimodal large model according to claim 1, characterized in that, The process of calling the large model to perform content transformation and integration on the associated multimodal information set based on the report generation task instruction, and generating an initial report text, includes: The large model's generation module is invoked to receive the report generation task instruction, parse the content generation logic and format specification requirements in the report generation task instruction, and determine the specific requirements and execution steps for report generation. Based on the content organization order in the content generation logic, the multimodal information combination corresponding to the content dimension is extracted from the associated multimodal information set, and the multimodal information combination is used as the generation material for the current content dimension. The extracted multimodal information is combined and processed for content transformation, converting image information into text descriptions, audio information into transcribed text, video information into shot-by-shot text descriptions, and extracting core content directly from the text information. According to the information integration method in the content generation logic, the transformed text content is initially integrated to form a preliminary text fragment for the current content dimension. The preliminary text fragment contains the core information of the content dimension. Semantic coherence processing is performed on the initial text fragments to identify the logical relationships between sentences in the initial text fragments, the order of sentences is adjusted according to the logical relationships, and corresponding connecting words are inserted between sentences. Repeated extraction, transformation, integration, and coherence processing steps are performed sequentially to generate text fragments for all content dimensions, resulting in multiple independent text fragments for each content dimension. Based on the task associations in the content generation logic, the logical connection order of text fragments in each content dimension is determined; the text fragments in each content dimension are arranged according to the logical connection order, and transitional content is inserted or adjusted between adjacent text fragments to form a preliminary report text framework; According to the formatting specifications, the initial report text framework was formatted and adjusted, including paragraph divisions, heading levels, and text alignment. The image information and video frame information from the associated multimodal information set are inserted into the corresponding positions in the report text according to the format specifications to form the initial report text containing text and images; Verify the content dimensions contained in the initial report text, compare the verification results with the content dimension semantics in the requirement semantic deconstruction results, and supplement the information corresponding to the missing content dimensions in the initial report text.

6. The intelligent report automatic generation method based on a multimodal large model according to claim 1, characterized in that, The process of optimizing the structure and semantic coherence of the initial report text to obtain the final intelligent report includes: The initial report text is subjected to structural parsing to extract the title hierarchy, paragraph distribution, and content module division, forming the initial structural analysis results; The initial structural analysis results are compared with the output format semantics in the requirement semantic deconstruction results to output the comparison difference results. Based on the comparison difference results, the initial report text is structurally optimized. The structural optimization process includes: adjusting the title level according to the output format semantics, correcting the paragraph division according to the output format semantics, and optimizing the arrangement order of the content modules according to the output format semantics to obtain the structurally optimized report text. Semantic fluency analysis is performed on the optimized report text to identify semantic gaps, logical disjointness, and redundant fragments in the optimized report text. For the identified semantic gaps, adjust the wording of the preceding and following sentences; for the identified logical disconnects, add logical connectors or transitional sentences; for the identified redundant segments, delete or merge them; and obtain the semantically optimized report text. The report text after semantic coherence optimization is processed for content consistency. Different expressions of the same core concept in the report text are identified and replaced with a unified preset expression. The perspective and level of detail of the discussion on the core concept between different content modules are adjusted to obtain the report text after content consistency processing. Extract the content dimension semantics from the demand semantic deconstruction results, check the corresponding content dimensions in the report text after content consistency processing one by one, and check whether there are missing or deviated content segments in the report text after content consistency processing. Based on the verification results of missing or misrepresented content, the corresponding content information is added to the report text after content consistency processing, and the content with misrepresentation is modified to obtain the supplemented and corrected report text. Based on the font style, paragraph spacing, and chart presentation method in the output format semantics of the demand semantic deconstruction results, the supplemented and corrected report text is finally optimized to make the presentation effect of the supplemented and corrected report text meet the requirements of the output format semantics, resulting in a format-optimized report text. The format-optimized report text is then converted into a preset output file format to generate the final intelligent report.

7. The method for automatic intelligent report generation based on a multimodal large model according to claim 2, characterized in that, The process of constructing semantic association links for multiple demand semantic units, analyzing the semantic dependency relationships between each demand semantic unit, and forming demand semantic association links based on the semantic dependency relationships includes: Each requirement semantic unit is annotated with semantic attributes to indicate its semantic type. The semantic types include topic semantics, content semantics, format semantics, and constraint semantics. The semantic dependency analysis method is used to analyze the semantic dependency relationship between any two semantic units of demand. The semantic dependency relationship includes subject-verb relationship, verb-object relationship, modifier-head relationship and coordinate relationship. A semantic association matrix is ​​constructed based on semantic dependency relationships. The elements in the semantic association matrix represent the semantic dependency strength between two semantic units of demand. The semantic dependency strength is determined based on the tightness of the semantic association. Extract semantic dependencies with semantic dependency strength greater than a preset association threshold from the semantic association matrix to form a core set of semantic dependencies; Starting with the core objective of report generation requirements, an initial semantic association link is constructed based on the core semantic dependency relationship set. The initial semantic association link includes the requirement semantic unit corresponding to the core objective and the directly related requirement semantic unit. Traverse the core semantic dependency relationship set and add the required semantic units that are not included in the initial semantic association link step by step to the initial semantic association link based on the semantic dependency relationship. Expand the coverage of the semantic association link to obtain the expanded semantic association link. Perform link optimization processing on the expanded semantic association link to eliminate semantic loops and logical conflicts in the expanded semantic association link and adjust the association order of each required semantic unit in the expanded semantic association link to obtain the optimized semantic association link. Based on the proximity and link connection relationship between each requirement semantic unit and the core target in the optimized semantic association link, the sequential association order of each requirement semantic unit in the optimized semantic association link is determined. An association identifier is added to each requirement semantic unit in the optimized semantic association link. The association identifier represents the association relationship and association order between the requirement semantic unit and other requirement semantic units. The optimized semantic association link with the added association identifier is used as the requirement semantic association link.

8. The method for automatic intelligent report generation based on a multimodal large model according to claim 3, characterized in that, The multimodal information association conditions in the multimodal information association mapping rules are used to perform association processing on multimodal information in the candidate multimodal information set, establishing semantic association relationships between different types of multimodal information, including: The multimodal information association conditions in the multimodal information association mapping rules are analyzed, and the core association elements in the association conditions are extracted. The core association elements include semantic matching elements, temporal association elements, and scene association elements. For each type of multimodal information in the candidate multimodal information set, core related element extraction processing is performed to extract the semantic matching element value, time-related element value, and scene-related element value corresponding to each type of multimodal information; Based on semantic matching elements, preliminary association screening is performed on different types of multimodal information, and multimodal information with the same or similar semantic matching element values ​​are grouped into the same semantic association group. Within the same semantic association group, multimodal information is processed for temporal association based on temporal association elements, and temporal association relationships are established for multimodal information whose temporal association element values ​​are within the same time range; Based on scene association elements, scene association verification is performed on multimodal information that establishes time association relationships. The scene association element values ​​corresponding to the multimodal information are compared, and multimodal information with consistent scene association element values ​​is selected. The association strength of multimodal information that has undergone semantic, temporal and scene association processing is evaluated, and multimodal information combinations that meet the preset requirements are selected. Semantic association relationships between these multimodal information are established, including association type and association basis. To establish semantic relationships, a correlation identifier is added to the multimodal information combination. The multimodal information combination that establishes semantic relationships is organized into a correlation information list. The correlation information list includes the multimodal information type, correlation identifier, and correlation element matching status. The correlation identifier is used to uniquely identify the correlation relationship of the multimodal information combination.

9. The intelligent report automatic generation method based on a multimodal large model according to claim 4, characterized in that, Based on the semantic understanding results of the large model, a task-oriented adaptation framework is constructed. This framework includes the core objectives of report generation, content coverage, and format constraints, including: Extract the core intent of the report generation requirement from the semantic understanding results of the large model. The core intent corresponds to the core goal of the report generation. Based on the core goal and combined with the content dimension semantics in the semantic deconstruction results of the requirement, determine all content dimensions that the report needs to include and the core expression points of each dimension. The content dimensions and core expression points constitute the content coverage. Extract the output format semantics from the demand semantic deconstruction results and transform them into specific format constraints, which include the report's layout requirements, presentation style, and file format requirements; The core structure of the task-oriented adaptation framework includes an objective layer, a scope layer, and a constraint layer. The objective layer corresponds to the core objective of report generation, the scope layer corresponds to the content coverage, and the constraint layer corresponds to the format constraints. In the target layer, the core target of report generation is broken down into multiple executable sub-targets, each corresponding to a specific report generation task; in the scope layer, the priority order of each content dimension is determined based on the degree of correlation between each content dimension in the content coverage and the core target; in the constraint layer, the format constraints are classified and organized into three categories: layout constraints, style constraints, and format constraints, and specific execution standards are set for each type of constraint. Establish a mapping between the target layer, scope layer, and constraint layer, associating each sub-target with one or more content dimensions, and associating each sub-target with a set of format constraints.

10. An intelligent report automatic generation system based on a multimodal large model, characterized in that, The intelligent report automatic generation system based on a multimodal large model includes a processor and a memory, the memory and the processor are connected, the memory is used to store programs, instructions or code, and the processor is used to run the programs, instructions or code in the memory to implement the intelligent report automatic generation method based on a multimodal large model as described in any one of claims 1-9.

Images