A report generation method based on natural language processing and multi-modal intelligent agent

Abstract

The application discloses a report generation method based on natural language processing and a multi-modal intelligent agent, relates to the technical field of natural language processing, and comprises the following steps: S1, outputting a multi-modal environment perception tensor; S2, outputting a hierarchical task planning tree; S3, outputting a multi-modal intelligent agent action scheduling sequence; S4, constructing a double-flow fiber plexus based on an improved ViLT model, projecting to generate a bottom manifold feature, performing coordinate transformation according to the scheduling sequence, constructing a fusion tensor and decoding and outputting a picture-text pair; S5, outputting a consistency check feature vector; S6, outputting a layout topology correction layout; and S7, outputting a multi-modal intelligent report. The application overcomes the limitations of the prior art, such as insufficient multi-modal fusion depth, weak intelligent agent planning capability and rigid page generation, and provides an efficient solution for multi-modal intelligent report generation.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a report generation method based on natural language processing and multimodal intelligent agents. Background Technology

[0002] As report generation applications become increasingly complex, the scale and dimensionality of multi-source heterogeneous input data are growing exponentially. Existing report generation methods typically rely on statistical language models or pre-trained generative models. While these can generate fluent text paragraphs, they primarily depend on simple feature concatenation or attention mechanisms for fusion when processing multimodal information. This purely probabilistic approach ignores the complex topological structures and high-dimensional geometric features implicit in multi-source data, failing to capture the inherent curvature and global relationships on the data manifold. This results in a lack of logical coherence in the generated content, making it difficult to accurately reflect the deep semantic relationships between data. Furthermore, most existing technologies employ linear pipelines or simple sequence generation strategies, lacking proactive planning and decision-making capabilities based on an intelligent agent architecture. When faced with complex report generation tasks, they struggle to achieve autonomous construction of chapter structures and efficient scheduling of tool interfaces, limiting the automation level and cross-modal generalization capabilities of the generation system.

[0003] Therefore, how to provide a report generation method based on natural language processing and multimodal intelligent agents is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] This invention proposes a report generation method based on natural language processing and multimodal agents. By improving the ViLT model, it models the semantic topological invariant tensor and the sequence of sliced ​​images into a two-stream fiber bundle full space. It utilizes a local section projection layer to generate local coordinate features of the bottom manifold and performs coordinate transformation operations based on a structure group transformation alignment layer to construct a fused feature tensor. A hierarchical task planning tree is generated by combining fractal iterative function systems, and a scheduling tool interface generates a multimodal agent action scheduling sequence to guide the decoding and output of multimodal content memory units. Furthermore, it calculates consistency verification feature vectors based on covariant derivatives and uses persistent homology barcodes to identify holes and fragmented areas on the page, outputting a topologically corrected layout. This invention effectively solves the problems of inaccurate feature alignment and missing structural associations in traditional models when processing multimodal data by establishing a complete process from "differential homeomorphic feature extraction" to "fiber bundle geometric fusion" and then to "topological data analysis and layout," achieving the technical effect of optimizing the report layout while maintaining semantic consistency. This invention overcomes the limitations of existing technologies, such as insufficient multimodal fusion depth, weak intelligent agent planning capabilities, and rigid layout generation, providing an efficient solution for multimodal intelligent report generation.

[0005] A report generation method based on natural language processing and multimodal intelligent agents according to an embodiment of the present invention specifically includes:

[0006] S1. Acquire multi-source heterogeneous data, construct a high-dimensional topological space for multimodal data, extract the tangent space vector and curvature features of the local neighborhood based on differential homeomorphism, and output the multimodal environment perception tensor.

[0007] S2. Read the multimodal environment perception tensor as the seed vector, define fractal element rules and dimension parameters, perform self-similar iteration and dimension reduction mapping through the iterative function system, and output a hierarchical task planning tree.

[0008] S3. Read the hierarchical task planning tree to construct hypergraph constraints and parallel time slots, match the tool interface and encode to generate graph image slice sequences and multimodal environment perception tensors, define tensor quantum block extraction rules, and output multimodal agent action scheduling sequences;

[0009] S4. Based on the improved ViLT model, the multimodal environment perception tensor and the sequence of chart image slices are modeled as a two-stream fiber bundle full space. The projection generates local coordinate features of the bottom manifold. According to the tool interface parameters in the action scheduling sequence of the multimodal agent, the coordinate transformation operation is performed on the local coordinate features of the bottom manifold. The fused feature tensor is constructed and decoded to output the text paragraph description and the chart image pair.

[0010] S5. Read the hierarchical task planning tree as the base space, text paragraph descriptions and graph image pairs as fibers, construct a semantic fiber bundle, calculate the covariant derivative and curvature tensor, quantify the geodesic deviation, and output the consistency verification feature vector.

[0011] S6. Read the consistency verification feature vector and map it into a two-dimensional page point cloud. Use topological data analysis to calculate the persistent coherent barcode, identify holes and fragmented areas and adjust the space occupancy, and output the page topology correction layout.

[0012] S7. Read the layout of the page topology, correct the layout, define the information bottleneck constraint target, calculate the lower bound of mutual information and remove redundant style parameters, and output a multimodal intelligent report.

[0013] Optionally, S1 specifically includes:

[0014] S11. Extract raw text data, chart and image data and structured table data from multi-source heterogeneous data sources. Perform word segmentation and stop word removal on the raw text data. Perform size normalization and color space conversion on the chart and image data. Extract key-value pairs from the structured table data and output the preprocessed standardized multimodal data.

[0015] S12. Read the preprocessed standardized multimodal data, construct the word vector space of text data, the pixel feature space of image data, and the numerical embedding space of table data respectively, calculate the intrinsic dimension of each modal data through the manifold learning algorithm, map each modal feature space to a unified high-dimensional Hilbert space, and output the high-dimensional topological space of multimodal data.

[0016] S13. Read the high-dimensional topological space of multimodal data, construct the local neighborhood graph of data points based on Euclidean distance, calculate the tangent space basis vector of each node in the local neighborhood graph using the local principal component analysis method, and output the tangent space vector of the local neighborhood.

[0017] S14. Read the local neighborhood graph and the tangent space vector of the local neighborhood, calculate the Riemann curvature tensor and Ricci curvature tensor at each node, analyze the distribution characteristics of the curvature tensor in different dimensions, and extract the geometric features describing the local curvature of the data manifold.

[0018] S15. Read the tangent space vector of the local neighborhood and the geometric features describing the local curvature of the data manifold. Perform tensor product operation on the tangent space vector and the curvature tensor. Eliminate redundant information through orthogonal projection to generate a multimodal environment perception tensor that remains invariant under differential homeomorphic transformation.

[0019] Optionally, S2 specifically includes:

[0020] S21. Read the multimodal environment perception tensor and calculate the principal eigenvalues ​​and principal eigenvectors. Project the principal eigenvectors into low-dimensional seed vectors. Calculate the intrinsic dimension of the data manifold based on the numerical distribution of the principal eigenvalues ​​and map the intrinsic dimension to the fractal dimension parameter.

[0021] S22. Determine the initial attractor coordinates of the iterative function system based on the low-dimensional seed vector, calculate the compression ratio and rotation angle of the affine transformation matrix in the iterative function system according to the fractal dimension parameter, and generate a customized iterative function system.

[0022] S23. Input the low-dimensional seed vector into the customized iterative function system, perform self-similar iterative operation according to the number of iterations limited by the fractal dimension parameter, and output a high-dimensional fractal point cloud set.

[0023] S24. Perform principal component analysis to reduce the dimensionality of the high-dimensional fractal point cloud set, project the point cloud onto a two-dimensional plane coordinate system, and generate a planar tree-like topology with nested hierarchical relationships.

[0024] S25. Traverse the planar tree topology, map the topology nodes to report chapter titles, map the topology connecting edges to chapter hierarchical relationships, and output a hierarchical task planning tree.

[0025] Optionally, S3 specifically includes:

[0026] S31. Read the hierarchical task planning tree, map each chapter node in the structure tree to a hypergraph vertex, and map the data reference relationship and hierarchical membership relationship between nodes to hyperedge constraints that connect multiple hypergraph vertices.

[0027] S32. Perform graph coloring algorithm on the hypergraph vertices, mark the hypergraph vertices that are not connected by hyperedges as the same color, map each color to an independent parallel time slot, and output the parallel time slot allocation scheme.

[0028] S33. Based on the semantic type of nodes in the hierarchical task planning tree, match the corresponding data visualization tool interface and natural language generation interface; read the original chart image data, perform image slicing and feature encoding operations, and output the chart image slice sequence; read the semantic text data, extract semantic features and construct tensor structure, output multimodal environment-aware tensor, and define the dimension parameters and data format of tensor slices.

[0029] S34. Read the parallel time slot allocation scheme, divide the tasks corresponding to the hypergraph vertices belonging to the same time slot into concurrent execution groups, divide the vertex tasks with hyperedge constraints into serial dependency groups, and generate a preliminary task sequence.

[0030] S35. Inject the interface parameters, chart image slice sequence and tensor block extraction rules into the initial task sequence, verify the integrity of the data flow direction of the hyperedge constraint, and output a multimodal agent action scheduling sequence containing the execution order, parallel time slots, input data entities, tool interface parameters and tool call instructions.

[0031] Optionally, the improved ViLT model includes a two-stream fiber bundle structure embedding layer, a local section projection layer, a structure group transformation alignment layer, a bottom manifold unified representation layer, and a parallel decoding output layer:

[0032] The dual-stream fiber bundle structure embedding layer is used to receive multimodal environment perception tensors and chart image slice sequences, model the multimodal environment perception tensors as semantic fiber bundle full space, model the chart image slice sequences as visual fiber bundle full space, define the fiber types of the semantic fiber bundle full space and the structure groups of the visual fiber bundle full space respectively, and output the semantic fiber bundle full space structure and the visual fiber bundle full space structure.

[0033] The local section projection layer is used to receive the semantic fiber bundle full-space structure and the visual fiber bundle full-space structure, construct a smooth mapping operator, select non-singular neighborhoods and generate local sections pointing to the bottom manifold, project the high-dimensional heterogeneous fiber features to the low-dimensional isomorphic bottom manifold coordinate system, and output the bottom manifold local coordinate features.

[0034] The structure group transformation alignment layer is used to receive the local coordinate features of the bottom manifold and the tool interface parameters in the action scheduling sequence of the multimodal agent, construct the structure group action matrix based on the tool interface parameters, perform coordinate transformation operation on the local coordinate features of the bottom manifold, map the semantic bottom manifold features and the visual bottom manifold features to a unified bottom space coordinate system, and output the aligned semantic bottom manifold features and the visual bottom manifold features.

[0035] The unified representation layer of the bottom manifold is used to receive the aligned semantic bottom manifold features and visual bottom manifold features, perform tangent vector field concatenation and Riemann integral operation, construct a globally unified fusion feature tensor, and output the fusion feature tensor.

[0036] The parallel decoding output layer is used to receive the fused feature tensor, map the fused feature tensor to the text generation space and the image pixel space respectively, generate natural language description text through autoregressive calculation, and reconstruct the optimized chart image through deconvolution operation, and output the text paragraph description and chart image pair.

[0037] Optionally, S5 specifically includes:

[0038] S51. Read the hierarchical task planning tree as the base space, text paragraph descriptions and graph / image pairs as fibers, establish the Cartesian product space of the base space and fibers, and output the semantic fiber bundle full space data structure.

[0039] S52. Define the connection coefficients on the semantic fiber bundle full-space data structure, calculate the covariant derivatives on the semantic manifold, and output the path-wise rate of change of the tangent vector field.

[0040] S53. Calculate the curvature tensor based on the path change rate of the tangent vector field and output the curvature tensor of the semantic fiber bundle space.

[0041] S54. Solve the geodesic equation based on the curvature tensor, calculate the geodesic deviation between adjacent fiber paths, and output the geodesic deviation value.

[0042] S55. Perform norm normalization on the geodesic deviation values, generate a consistency verification feature vector, and output it.

[0043] Optionally, S6 specifically includes:

[0044] S61. Read the consistency verification feature vector, and map the feature dimension to the plane coordinate dimension through linear transformation to generate two-dimensional page point cloud data;

[0045] S62. Calculate the Vietoris-Rips complex corresponding to the two-dimensional point cloud data, and construct the simple complex topology based on the set neighborhood radius parameter.

[0046] S63. Perform boundary operator operations and chain group homology calculations based on the simple complex topology, and output a persistent homology barcode;

[0047] S64. Analyze the survival range of persistent coherent barcodes, identify the hole and segmented regions in the point cloud of the page, and calculate the geometric center coordinates and bounding boxes of the hole and segmented regions.

[0048] S65. Based on the geometric center coordinates and bounding boxes of the empty and fragmented regions, perform spatial filling and connectivity repair operations on the two-dimensional point cloud to generate a layout with corrected topology.

[0049] Optionally, S7 specifically includes:

[0050] S71. Read the layout topology and correct the layout, extract the pixel matrix of the chart image and the text paragraph description vector in the layout;

[0051] S72. Construct an information bottleneck constraint objective function based on the pixel matrix of the chart image and the description vector of the text paragraph, and calculate the lower bound of mutual information between the input variables and the potential representation variables;

[0052] S73. Perform variational inference operations based on the lower bound of mutual information to calculate the gradient values ​​corresponding to redundant style parameters;

[0053] S74. Based on the gradient values, perform pruning and stripping operations on redundant style parameters to generate key semantic feature parameters;

[0054] S75. Reorganize the text structure and image rendering attributes based on key semantic feature parameters to generate multimodal intelligent reports.

[0055] The beneficial effects of this invention are:

[0056] (1) This invention constructs a high-dimensional topological space and a differential homeomorphism, extracts manifold features using Riemann curvature and generates a semantic topological invariant tensor, and maps it to a report hierarchy structure using fractal theory. This mechanism achieves deep coupling of heterogeneous data, accurately captures inherent topological characteristics, and provides mathematically rigorous semantic guidance for report generation.

[0057] (2) This invention establishes a geometry-oriented multimodal feature fusion and agent action scheduling system by employing an improved ViLT model and full-space modeling of dual-stream fiber bundles. The dual-stream fiber bundle structure embedding layer models semantic and visual features as independent full-space structures. A smooth mapping operator is constructed through a local section projection layer to project high-dimensional heterogeneous fiber features onto a low-dimensional isomorphic bottom manifold coordinate system. The structure group transformation alignment layer constructs an action matrix based on the tool interface parameters and performs coordinate transformation operations to achieve unified alignment of semantic and visual features. This system eliminates the heterogeneity barrier of multimodal data through fiber bundle section projection, structure group transformation alignment, and unified bottom manifold representation, ensuring that the generated graphic content has accurate cross-modal consistency. Attached Figure Description

[0058] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0059] Figure 1 This is an overall flowchart of a report generation method based on natural language processing and multimodal intelligent agents proposed in this invention;

[0060] Figure 2 This is a flowchart illustrating the working principle of an improved ViLT model based on natural language processing and multimodal intelligent agents, as proposed in this invention. Detailed Implementation

[0061] The invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0062] refer to Figure 1 and Figure 2 A report generation method based on natural language processing and multimodal intelligent agents, specifically including:

[0063] S1. Acquire multi-source heterogeneous data, construct a high-dimensional topological space for multimodal data, extract the tangent space vector and curvature features of the local neighborhood based on differential homeomorphism, and output the multimodal environment perception tensor.

[0064] S2. Read the multimodal environment perception tensor as the seed vector, define fractal element rules and dimension parameters, perform self-similar iteration and dimension reduction mapping through the iterative function system, and output a hierarchical task planning tree.

[0065] S3. Read the hierarchical task planning tree to construct hypergraph constraints and parallel time slots, match the tool interface and encode to generate graph image slice sequences and multimodal environment perception tensors, define tensor quantum block extraction rules, and output multimodal agent action scheduling sequences;

[0066] S4. Based on the improved ViLT model, the multimodal environment perception tensor and the sequence of chart image slices are modeled as a two-stream fiber bundle full space. The projection generates local coordinate features of the bottom manifold. According to the tool interface parameters in the action scheduling sequence of the multimodal agent, the coordinate transformation operation is performed on the local coordinate features of the bottom manifold. The fused feature tensor is constructed and decoded to output the text paragraph description and the chart image pair.

[0067] S5. Read the hierarchical task planning tree as the base space, text paragraph descriptions and graph image pairs as fibers, construct a semantic fiber bundle, calculate the covariant derivative and curvature tensor, quantify the geodesic deviation, and output the consistency verification feature vector.

[0068] S6. Read the consistency verification feature vector and map it into a two-dimensional page point cloud. Use topological data analysis to calculate the persistent coherent barcode, identify holes and fragmented areas and adjust the space occupancy, and output the page topology correction layout.

[0069] S7. Read the layout of the page topology, correct the layout, define the information bottleneck constraint target, calculate the lower bound of mutual information and remove redundant style parameters, and output a multimodal intelligent report.

[0070] In this embodiment, S1 specifically includes:

[0071] S11. Extract raw text data, chart and image data, and structured table data from multi-source heterogeneous data sources. Construct a prefix dictionary tree for the raw text data and read the character sequence one by one to match the longest path and remove stop words. Calculate the fractional part weight of the target pixel coordinates for the chart and image data and perform bilinear weighted summation on the values ​​of the four surrounding neighboring pixels to obtain the scaling value. Traverse the cells of the structured table data row by row and read the value of the first column as the key and read the values ​​of the subsequent columns as the value. Output the preprocessed standardized multimodal data.

[0072] S12. Read the preprocessed standardized multimodal data, scan the text data to count the co-occurrence frequency of words in a window of size 5 to construct a co-occurrence matrix, perform singular value decomposition on the co-occurrence matrix and extract the first 128 left singular vectors as the word vector space; initialize the convolution kernel parameters and slide to cover the image pixel region, calculate the sum of the multiplication of the convolution kernel elements and the corresponding covered pixels to obtain the feature map, perform max pooling to reduce the dimensionality of all feature maps and then concatenate them to obtain the pixel feature space; calculate the hash integer index for each key-value pair of the tabular data, find the corresponding row vector of the preset embedding matrix and sum and average it to obtain the numerical embedding space; construct the 5 nearest neighbor graph of each modality data and calculate the shortest path distance matrix, perform centering processing on the distance matrix and calculate the feature vectors corresponding to the first 3 largest eigenvalues, construct the radial basis function kernel matrix to map the feature vectors to the 256-dimensional regenerating kernel Hilbert space, calculate the value of each column of the kernel matrix divided by the Euclidean norm of the value of that column, concatenate the normalized feature vectors of each modality column by column, and output the high-dimensional topological space of the multimodal data.

[0073] S13. Read the high-dimensional topological space of multimodal data, calculate the sum of the squared differences of coordinates between any two data points in the space as the squared Euclidean distance, select the 5 data points with the smallest squared distance to construct a local neighborhood graph, calculate the centralized matrix after subtracting the mean coordinates from the coordinates of each point in the neighborhood, calculate the product of the centralized matrix and its transpose to obtain the covariance matrix, solve for the eigenvectors corresponding to the first 3 largest eigenvalues ​​of the covariance matrix as the basis vectors of the tangent space, and combine the basis vectors of each node to output the tangent space vector of the local neighborhood.

[0074] S14. Read the local neighborhood graph and the tangent space vector of the local neighborhood. Calculate the metric matrix formed by the basis vectors of the tangent space. Calculate the partial derivative value of each element of the metric matrix with respect to each coordinate axis. Multiply the partial derivative value matrix on the left and right using the inverse matrix of the metric matrix respectively. Add the product results and divide by 2 to obtain the Christofel symbol value. Calculate the partial derivative values ​​of the Christofel symbol with respect to different coordinate components. Subtract the partial derivative values ​​from the product terms of the Christofel symbol to obtain the component values ​​of the Riemann curvature tensor. Multiply the components corresponding to the first and third indices of the Riemann curvature tensor and sum them to obtain the Ricci curvature tensor. Calculate the square root of the sum of the squares of all components of the Riemann curvature tensor and the Ricci curvature tensor as the modulus. Extract the modulus as a geometric feature describing the local curvature of the data manifold.

[0075] S15. Read the tangent space vector of the local neighborhood and the geometric features describing the local curvature of the data manifold. Read each element in the tangent space vector one by one and multiply it with each element in the Riemann curvature tensor. Arrange the product results according to the position order of the elements in the original tensor to construct a high-dimensional joint tensor. Flatten the joint tensor into a two-dimensional matrix and calculate the dot product of the matrix and its transpose to obtain the Gram matrix. Solve for the eigenvector matrix of the Gram matrix. Multiply the joint tensor with the transpose of the eigenvector matrix to perform an orthogonal projection transformation. Extract the principal components with eigenvalues ​​greater than 0.01 after projection to generate a multimodal environment-aware tensor that remains invariant under the differential homeomorphic transformation.

[0076] In this embodiment, S2 specifically includes:

[0077] S21. Read the multimodal environment perception tensor, calculate all eigenvalues ​​of the tensor matrix and sort them in descending order of value. Select the first 10 eigenvalues ​​with values ​​greater than 0.1 as principal eigenvalues. Calculate the number of principal eigenvalues ​​to be 10 and output them as intrinsic dimension values. Calculate the proportion of the first 10 principal eigenvalues ​​to the sum of all eigenvalues ​​as the probability distribution. Calculate the logarithm of each probability and the negative of the product of the probabilities to obtain the information entropy value. Map the information entropy value to the fractal dimension parameter 1.8. Calculate the dot product of the principal eigenvectors and the preset projection matrix. Use the projection result as the coordinate components of the low-dimensional seed vector and output the low-dimensional seed vector and the fractal dimension parameter.

[0078] S22. Read the low-dimensional seed vector and the fractal dimension parameter 1.8, and set the coordinate values ​​of the seed vector as the coordinate positions of the initial attractor of the iterative function system; read the fractal dimension parameter value 1.8, divide it by the total number of affine transformations 4 to obtain a quotient value of 0.45, and calculate the result of the operation with the natural constant e raised to the power of 0.45, 0.637, as the basic compression ratio value; calculate the product of the value of pi, 3.14, and the fractal dimension parameter 1.8, 5.652, as the basic rotation angle value; construct a set containing 4 affine transformations, with the first transformation setting the rotation angle to 5.652 radians and the scaling ratio to 0.637, the second transformation setting the rotation angle to 11.304 radians and the scaling ratio to 0.637, the third transformation setting the rotation angle to 16.956 radians and the scaling ratio to 0.637, and the fourth transformation setting the rotation angle to 22.608 radians and the scaling ratio to 0.637, and output a customized iterative function system.

[0079] S23. Read the low-dimensional seed vector and the customized iterative function system, initialize an empty coordinate list as a high-dimensional fractal point cloud set, and use the coordinates of the low-dimensional seed vector as the current iteration point; execute the loop operation: sequentially read each affine transformation in the iterative function system, calculate the product of the current iteration point coordinates and the affine transformation matrix to obtain the new coordinates, and add the new coordinates to the high-dimensional fractal point cloud set; set the number of iterations to an integer multiple of the fractal dimension parameter, i.e., 10,000 times, and stop the loop when the number of points in the set reaches 10,000, and output the high-dimensional fractal point cloud set.

[0080] S24. Read the high-dimensional fractal point cloud set, calculate the mean coordinates of all points in the point cloud set, subtract the mean coordinates from the coordinates of each point to obtain the centered coordinate matrix, calculate the product of the transpose of the centered coordinate matrix and the centered coordinate matrix to obtain a covariance matrix with a dimension of 3 rows and 3 columns; solve for the eigenvectors corresponding to the first two largest eigenvalues ​​of the covariance matrix to construct a projection transformation matrix with a dimension of 3 rows and 2 columns, calculate the product of the point cloud set (dimension of 10000 rows and 3 columns) and the projection transformation matrix to obtain a two-dimensional plane coordinate set with a dimension of 10000 rows and 2 columns; use the Kruskal algorithm to sort the Euclidean distances of all point pairs in the two-dimensional plane in ascending order, select the edge with the smallest distance in turn, if the two nodes connected by the edge are currently in different connected components, keep them, otherwise discard them, until all nodes are connected, and output a planar tree topology structure without loops.

[0081] S25. Read the planar tree topology, select the node with the most connected edges as the root node and map it to the overall report title, calculate the path distance from each remaining node to the root node, map the nodes to first-level or multi-level chapter titles based on the path distance values, calculate the average distance between adjacent level nodes and map it to the level of detail parameters of the chapter content, determine the membership logic of parent and child nodes based on the node connection relationship, and output the hierarchical task planning tree.

[0082] The task planning tree generation process based on fractal iterative function system proposed in this step is similar to traditional hierarchical task networks or chain-like thinking planning in that it is based on structured task decomposition theory. That is, by decomposing complex global task objectives into local sub-task nodes with inclusion relationships, the tree topology is used to represent the hierarchical membership and execution order between tasks, and the feature mapping method is used to compress high-dimensional environmental information into the input basis for planning decisions.

[0083] The difference lies in that this invention breaks away from the limitations of traditional methods that rely on logical reasoning or symbol matching. It innovatively adds fractal geometric mapping, determines fractal parameters based on intrinsic dimensions, and replaces the fixed levels set by humans. It uses affine transformation of iterative function systems to replace logical discrimination branches, and generates nested levels iteratively through parameter control, thus getting rid of the constraints of predefined templates.

[0084] The beneficial effects of the improvements are that, by introducing fractal mapping, the environmental topological characteristics are transformed into a self-similar task structure, breaking the limitations of the traditional planning structure's rigidity and weak generalization, and realizing a leap from discrete logical planning to continuous geometric generation; it significantly enhances the task tree's adaptability to multi-scale features and improves the structural rationality and efficiency of complex report generation.

[0085] In this embodiment, S3 specifically includes:

[0086] S31. The dual-stream fiber bundle embedding layer reads the multimodal environment perception tensor and the image slice sequence, constructs a 1024-row, 1024-column identity matrix as the basis coordinate matrix of the semantic fiber bundle full space, inputs each row of the multimodal environment perception tensor's 256-dimensional vector into a neural network containing 4 fully connected layers, calculates the dot product of the weight matrix of each neural network layer with the input vector and processes it through the ReLU activation function, outputting a 1024-dimensional vector as the fiber coordinates in the semantic fiber space; constructs a 512-row, 512-column diagonal matrix as the metric matrix of the visual fiber bundle full space, inputs each slice of the image slice sequence into a convolutional neural network, calculates the weighted sum of the convolution kernel and the slice pixels to obtain a 512-dimensional feature vector, calculates the product of this feature vector and the inverse matrix of the metric matrix as the tangent vector coordinates of the visual fiber, and sets the structure group transformation rule to matrix orthogonalization operation; outputs a semantic fiber bundle full space structure containing 1024-dimensional fiber coordinates and a visual fiber bundle full space structure containing 512-dimensional fiber coordinates.

[0087] S32. The local section projection layer reads the full-space structure of the semantic fiber bundle and the full-space structure of the visual fiber bundle, constructs a 128-dimensional Euclidean space as the bottom manifold, initializes a 1024-row 128-dimensional projection matrix and a 512-row 128-dimensional projection matrix; calculates the matrix multiplication of the semantic fiber coordinates with the 1024-row 128-dimensional projection matrix to obtain a 128-dimensional vector as the semantic bottom manifold coordinates; calculates the matrix multiplication of the visual fiber coordinates with the 512-row 128-dimensional projection matrix to obtain a 128-dimensional vector as the visual bottom manifold coordinates; calculates the eigenvalues ​​of the projection vector covariance matrix, selects regions with eigenvalues ​​greater than 0.01 as non-singular neighborhoods, retains the coordinate points located in the non-singular neighborhoods, and outputs the 128-dimensional local coordinate features of the bottom manifold.

[0088] S33. The structure group transformation alignment layer reads the local coordinate features of the bottom manifold and the tool interface parameters in the multimodal agent action scheduling sequence, extracts the scaling factor value of 0.8 and the rotation angle value of 0.5 radians from the tool interface parameters; constructs a 128-row, 128-dimensional orthogonal transformation matrix as the structure group action matrix, and uses the SVD singular value decomposition algorithm to solve for the optimal rotation matrix parameters between the two bottom manifold feature distributions; calculates the product of the semantic bottom manifold feature vector and the orthogonal transformation matrix, and calculates the product of the visual bottom manifold feature vector and the inverse of the orthogonal transformation matrix; calculates the Euclidean distance between the two product results, adjusts the matrix parameters using the Adam algorithm until the distance value is less than 0.1, maps the feature vectors to a unified bottom space coordinate system, and outputs the aligned semantic bottom manifold features and visual bottom manifold features.

[0089] S34. The bottom manifold unified representation layer reads the aligned semantic bottom manifold features and visual bottom manifold features, calculates the Hadamard product of the two 128-dimensional feature vectors to obtain a 128-dimensional interaction feature vector, and concatenates the semantic features, visual features and interaction features to construct a 384-dimensional combined vector; constructs a 384-row, 256-column fully connected weight matrix, calculates the product of the combined vector and the weight matrix; inputs the product result into the ReLU activation function for nonlinear transformation, and outputs a 256-dimensional global unified tensor as the fused feature tensor.

[0090] S35. The parallel decoding output layer reads the fused feature tensor and constructs a word embedding matrix containing 5000 words with dimensions of 5000 rows and 256 columns. It calculates the dot product of the fused feature tensor and the transpose of the word embedding matrix to obtain 5000 score values. It selects the word index corresponding to the top 50 scores and outputs the vocabulary as a natural language description text. It constructs a deconvolution neural network module. First, it calculates the product of the fused feature tensor and the 256-row 1024-dimensional fully connected matrix, reshaping the 256-dimensional feature into a 16-row 16-column four-channel three-dimensional feature map. It constructs a deconvolution kernel with a size of 4 rows and 4 columns and a stride of 2. It performs two deconvolution operations to output 32-row 32-column and 64-row 64-column feature maps in turn. Finally, it outputs a 64-row 64-column image matrix, outputting text paragraph descriptions and chart image pairs.

[0091] In this embodiment, the improved ViLT model includes a dual-stream fiber bundle structure embedding layer, a local section projection layer, a structure group transformation alignment layer, a bottom manifold unified representation layer, and a parallel decoding output layer:

[0092] A dual-stream fiber bundle embedding layer reads the multimodal environment-aware tensor and the image slice sequence, constructing a 1024x1024 identity matrix as the basis coordinate matrix of the semantic fiber bundle full space. Each row of the multimodal environment-aware tensor's 256-dimensional vector is input into a neural network containing four fully connected layers. The dot product of the weight matrix of each neural network layer and the input vector is calculated and processed by the ReLU activation function, outputting a 1024-dimensional vector as the fiber coordinates in the semantic fiber space. A 512x512 diagonal matrix is ​​constructed as the metric matrix of the visual fiber bundle full space. Each slice of the image slice sequence is input into a convolutional neural network, and the weighted sum of the convolution kernel and the slice pixels is calculated to obtain a 512-dimensional feature vector. The product of this feature vector and the inverse of the metric matrix is ​​calculated as the tangent vector coordinates of the visual fiber, and the structure group transformation rule is set to matrix orthogonalization operation. The output is a semantic fiber bundle full space structure containing 1024-dimensional fiber coordinates and a visual fiber bundle full space structure containing 512-dimensional fiber coordinates.

[0093] The local section projection layer reads the full-space structure of the semantic fiber bundle and the full-space structure of the visual fiber bundle, constructs a 128-dimensional Euclidean space as the bottom manifold, and initializes a 1024-row 128-dimensional projection matrix and a 512-row 128-dimensional projection matrix. The matrix multiplication of the semantic fiber coordinates with the 1024-row 128-dimensional projection matrix is ​​calculated to obtain a 128-dimensional vector as the semantic bottom manifold coordinates. The matrix multiplication of the visual fiber coordinates with the 512-row 128-dimensional projection matrix is ​​calculated to obtain a 128-dimensional vector as the visual bottom manifold coordinates. The eigenvalues ​​of the projection vector covariance matrix are calculated, and regions with eigenvalues ​​greater than 0.01 are selected as non-singular neighborhoods. The coordinate points located in the non-singular neighborhoods are retained, and the local coordinate features of the bottom manifold with a dimension of 128 are output.

[0094] The structure group transformation alignment layer reads the local coordinate features of the bottom manifold and the tool interface parameters from the multimodal agent action scheduling sequence, extracting the scaling factor value of 0.8 and the rotation angle value of 0.5 radians from the tool interface parameters; constructs a 128-row, 128-dimensional orthogonal transformation matrix as the structure group action matrix, and uses the SVD singular value decomposition algorithm to solve for the optimal rotation matrix parameters between the two bottom manifold feature distributions; calculates the product of the semantic bottom manifold feature vector and the orthogonal transformation matrix, and calculates the product of the visual bottom manifold feature vector and the inverse of the orthogonal transformation matrix; calculates the Euclidean distance between the two product results, and adjusts the matrix parameters using the Adam algorithm until the distance value is less than 0.1, mapping the feature vectors to a unified bottom space coordinate system, and outputting the aligned semantic bottom manifold features and visual bottom manifold features.

[0095] The bottom manifold unified representation layer reads the aligned semantic bottom manifold features and visual bottom manifold features, calculates the Hadamard product of the two 128-dimensional feature vectors to obtain a 128-dimensional interaction feature vector, and concatenates the semantic features, visual features and interaction features to construct a 384-dimensional combined vector; constructs a 384-row 256-column fully connected weight matrix, calculates the product of the combined vector and the weight matrix, inputs the product result into the ReLU activation function for nonlinear transformation, and outputs a 256-dimensional global unified tensor as the fused feature tensor.

[0096] The parallel decoding output layer reads the fused feature tensor and constructs a word embedding matrix containing 5000 words with dimensions of 5000 rows and 256 columns. It calculates the dot product of the fused feature tensor and the transpose of the word embedding matrix to obtain 5000 score values. The word indices corresponding to the top 50 highest scores are selected, and the vocabulary is searched to output natural language description text. A deconvolutional neural network module is constructed. First, the product of the fused feature tensor and a 256-row, 1024-dimensional fully connected matrix is ​​calculated, reshaping the 256-dimensional feature into a 16-row, 16-column four-channel 3D feature map. A deconvolution kernel with a size of 4 rows and 4 columns and a stride of 2 is constructed. Two deconvolution operations are performed, outputting 32-row, 32-column and 64-row, 64-column feature maps respectively. Finally, a 64-row, 64-column image matrix is ​​output, providing text paragraph descriptions and chart image pairs.

[0097] The improved ViLT model proposed in this step is similar to the traditional ViLT model in that it is based on the visual language Transformer architecture and multimodal interaction theory. That is, it extracts feature representations of image slices and text tokens through deep neural networks, captures semantic associations between modalities using self-attention and cross-attention mechanisms, and uses a parallel decoding structure to output multimodal generation results.

[0098] The difference lies in that this invention breaks away from the limitations of traditional ViLT models, which only perform vector concatenation or simple interactions in Euclidean space and ignore the inherent geometric differences and topological heterogeneity of data from different modalities. Building upon the traditional model's direct fusion of feature vectors, this invention adds a dual-stream fiber bundle structure embedding step, modeling the multimodal environment-aware tensor and image slices as semantic and visual fiber bundles in their respective full spaces, rather than as a single sequence of feature vectors. In the feature alignment step, a smoothing mapping operator is constructed using a local section projection layer to project the high-dimensional heterogeneous fiber features onto a low-dimensional isomorphic bottom manifold coordinate system, instead of directly calculating attention in the original high-dimensional space. Finally, in the fusion representation step, a coordinate transformation is performed based on the structure group transformation alignment layer according to the tool interface parameters, and a globally unified fusion feature tensor is constructed through Riemann integration, rather than a simple weighted summation of features.

[0099] The beneficial effects of this improvement are that by introducing fiber bundle geometry theory, this invention can decouple the local detailed features (fibers) of different modal data from the global task structure (bottom manifold) for modeling. This breaks the limitation of the traditional ViLT model in handling topologically heterogeneous data, where feature mixing leads to semantic drift. It achieves a precise mapping from Euclidean space vector fusion to manifold space geometric alignment. This design significantly enhances the model's ability to perceive and align the inherent topological structure of multimodal features, and can more accurately capture the geometric relationship between task and content in the bottom manifold space through structure group transformation. The unified representation based on section projection and Riemann integral effectively improves the mathematical rigor of multimodal fusion and enhances the consistency and generation quality of cross-modal semantics in complex report generation tasks.

[0100] In this embodiment, S5 specifically includes:

[0101] S51. Read the hierarchical task planning tree, traverse each node in the tree structure, and extract the 128-dimensional coordinate vector of the node in the base manifold as the base space coordinate; read the text paragraph description and the chart image pair, extract the 768-dimensional text semantic feature vector through the pre-trained BERT model, extract the 2048-dimensional chart visual feature vector through the convolutional neural network, and concatenate the two to obtain the 2816-dimensional fiber vector; calculate the direct product of the base space coordinate vector and the fiber vector, construct a Cartesian product space with a dimension of 2944, generate a triplet data structure containing node index, base space coordinate and fiber vector, and output the semantic fiber bundle full space data structure.

[0102] S52. Construct an affine connection on the semantic fiber bundle full-space data structure, initialize a 2944-row, 2944-column connection coefficient matrix, set the diagonal elements to 0 and the off-diagonal elements to 0.01; select the direction of the connection between the parent node and the child node in the hierarchical task planning tree as the integral path of the tangent vector direction, and map the 128-dimensional base space coordinate change to the initial value of the 2944-dimensional full-space tangent vector; according to the definition of covariant derivative, calculate the numerical differential of the tangent vector along the path as the partial derivative term, calculate the matrix product of the connection coefficient matrix and the tangent vector as the connection term, add the two values, and output the numerical value of the tangent vector field along the path.

[0103] S53. Read the path change rate of the tangent vector field, select any two different child node branch directions under the same parent node in the hierarchical task planning tree as the calculation direction; construct the curvature operator calculation logic, calculate the numerical differential difference of the connection coefficient matrix along two different directions as the differential term result; calculate the difference of the product of the connection coefficient matrices in the two directions as the nonlinear row term result; add the differential term result and the nonlinear row term result to obtain the curvature tensor characterizing the degree of spatial curvature, and output the four-dimensional curvature tensor data volume.

[0104] S54. Construct the solution logic for the geodesic deviation equation based on the curvature tensor data volume. Set the initial deviation distance between two adjacent fiber paths to 0.001 and the initial relative velocity to 0 at the initial moment. Set the iteration step size to 0.01 and read the deviation vector coordinates and velocity vector coordinates at the current moment. First, calculate the tensor shrinkage operation result of the curvature tensor and the current deviation vector, and then multiply it by -1 to obtain the acceleration vector prediction value K1. Second, add the velocity vector multiplied by half the step size to the current deviation vector, add K1 multiplied by half the step size to the current velocity vector, and calculate the new curvature shrinkage result to obtain the acceleration vector prediction value K2. Third, add the velocity vector multiplied by half the step size to the current deviation vector. First, add K2 multiplied by half the step size to the current velocity vector, calculate the new curvature shrinkage result to obtain the acceleration vector prediction value K3; second, add the velocity vector multiplied by the step size to the current deviation vector, add K3 multiplied by the step size to the current velocity vector, calculate the new curvature shrinkage result to obtain the acceleration vector prediction value K4; third, calculate the weighted sum of the four acceleration prediction values, with weights of 1, 2, 2, and 1 respectively, divide by 6 to obtain the average acceleration, update the velocity vector using the average acceleration, and update the deviation vector using the updated velocity vector; repeat the above steps until the number of iterations reaches 1000, extract the coordinates of the final updated deviation vector, calculate the Euclidean norm of the vector, and output the geodesic deviation value.

[0105] S55. Read the geodesic deviation values, set the standard deviation threshold to 1.0, and calculate the ratio of the deviation value to the threshold. If the ratio is less than 1, retain the original value; if the ratio is greater than 1, truncate it to 1.0. Construct a 64-dimensional consistency check feature vector, and fill the first dimension of the vector with the normalized deviation ratio. Extract the mean and variance statistics of the geodesic trajectory coordinate sequence, and fill them into the second to 65th dimensions of the vector, respectively. Map the 65-dimensional features to the standard 64-dimensional output through a fully connected layer, and output the consistency check feature vector.

[0106] The geodesic deviation consistency verification process based on semantic fiber bundles proposed in this step is similar to the traditional multimodal consistency verification method in that it is based on the feature space distance measurement and correlation analysis theory. That is, semantic consistency is evaluated by comparing the relative positions or distribution differences of different modal features in the latent space, and both use vector operations and numerical statistical methods to output verification indicators.

[0107] The difference lies in that this invention breaks through the limitations of traditional methods that rely solely on static matching based on Euclidean distance. It innovatively constructs a semantic fiber bundle model, modeling the task planning tree and generated content as the base space and fibers, respectively. It uses covariant derivatives and curvature tensors to calculate geodesic deviations, replacing the traditional fixed threshold judgment, and achieving a leap from calculating local point-pair similarity to the geometric quantization of global manifold trajectory evolution.

[0108] The beneficial effect of the improvement is that, by introducing fiber bundle geometry, the present invention integrates task structure constraints into the verification process, breaking the limitation of traditional methods that ignore curvature and cause distortion, and realizing accurate quantification from static point pair matching to dynamic trajectory evolution, significantly enhancing the ability to perceive complex semantic drift and the mathematical interpretability of the verification process.

[0109] In this embodiment, S6 specifically includes:

[0110] S61. Read the consistency check feature vector and construct a linear transformation weight matrix W with dimensions of 64 rows and 2 columns. Initialize the matrix elements to random values ​​that follow a standard normal distribution. Calculate the matrix product of the consistency check feature vector and the weight matrix W to obtain a row vector containing two elements. Construct a bias vector b containing two elements, setting the two elements of the bias vector to 100 and 50 respectively. Add the row vector and the bias vector, and set the negative values ​​in the result to 0 for non-linear activation. Normalize the output coordinate values ​​by calculating the ratio of the coordinate value to the maximum absolute value of the coordinate (100) and multiplying it by a scaling factor of 1000 to map the coordinates to the interval between 0 and 1000. Repeat the above calculation for the 1000 feature vectors in the input batch to output two-dimensional layout point cloud data containing 1000 two-dimensional coordinate points.

[0111] S62. Read the 2D point cloud data and initialize an empty simplex structure container. Set the neighborhood radius parameter epsilon to 10.0. Construct a 1000-row, 1000-column distance matrix. Traverse each pair of points in the point cloud, calculate the square root of the sum of the squares of the differences between the two point coordinates to obtain the Euclidean distance, and fill it into the corresponding position in the distance matrix. Traverse the distance matrix. If the value in the i-th row and j-th column is less than 10.0 and i is not equal to j, add the combination of point i and point j to the simplex structure as a 1D simplex. Traverse all 1D simplex combinations. Find any two edges sharing a vertex. If the distance values ​​of the two vertices and the other two edges formed by the third vertex are all less than 10.0, add the triangular facet formed by these three vertices to the simplex structure as a 2D simplex. Output the simplex topology containing points, edges, and triangular facets.

[0112] S63. Construct a boundary operator matrix based on the simplicium topology. Set the number of rows in the matrix to the number of edges in a 1-dimensional simplicium and the number of columns to the number of points in a 0-dimensional simplicium. Traverse each edge, find the indices of its two endpoints, and in the corresponding rows of the matrix, set the element at the starting column position to -1, the element at the ending column position to 1, and the remaining elements to 0. Calculate the rank of the boundary operator matrix, transform the matrix into its row-minimum echelon form using Gaussian elimination, and count the number of non-zero rows as the dimension of the image space. Calculate... The null space dimension of the matrix is ​​calculated by subtracting the rank of the matrix from the number of columns, which serves as the dimension of the kernel space. The neighborhood radius parameter is set to start from 1.0 and increase in increments of 1.0 to 50.0. The simple complex construction and rank calculation operations described above are performed at each radius value. The radius value at the time of generation of each homology class is recorded as the birth time, and the radius value at the time of its extinction is recorded as the death time, generating a set of intervals. Each interval is mapped to a horizontal line segment, with the radius parameter value on the horizontal axis and the index number of the homology class on the vertical axis, and a persistent homology barcode is output.

[0113] S64. Read the persistent homology barcode, traverse each horizontal line segment in the barcode, and calculate the difference between the x-coordinate value of the end point and the x-coordinate value of the start point of the line segment to obtain the survival interval length; set the length threshold to 20.0. If the survival interval length is less than 20.0, it is considered noise and ignored; for line segments with a survival interval length greater than 20.0, check the dimension of their corresponding homology group. If the dimension is 1, it is identified as a hole region. If the dimension is 0 and it is not connected to other components under the current radius, it is identified as a cut region; for hole regions, find the closed chain structure corresponding to the homology class, extract the coordinates of all vertices in the chain structure, calculate the arithmetic mean of the x-coordinates of all vertices as the x-coordinate of the geometric center, and calculate the arithmetic mean of the y-coordinates as the y-coordinate of the geometric center; construct a bounding box, traverse all vertex coordinates, extract the maximum and minimum values ​​of the x-coordinates and the maximum and minimum values ​​of the y-coordinates, add and subtract the boundary expansion coefficient of 5.0 respectively, and construct a rectangular bounding box; output the geometric center coordinates and bounding boxes of hole regions and cut regions.

[0114] S65. Read the geometric center coordinates and bounding boxes of the hollow and fragmented regions. For the hollow region, extract all edge points within the bounding box, calculate the arithmetic mean of the feature vectors of the edge points to obtain the filling feature vector, combine the geometric center coordinates and the filling feature vector to generate a new data point, and insert this point into the 2D layout point cloud data. For the fragmented region, read the two independent connected components, calculate the Euclidean distance between the geometric centers of the two connected components, traverse all point pairs in the two connected components, and find the pair of points with the smallest distance value as the optimal connection point. Construct a linear interpolation path between the optimal connection points, set the interpolation step size to 5.0, and calculate the coordinate value of each interpolation point on the path. Assign a feature vector to each interpolation point, calculate the linear weighted average of the feature vectors of the two connection points, with the weight inversely proportional to the distance from the interpolation point to the two endpoints. Add the generated interpolation points and their feature vectors to the 2D layout point cloud data, update the simple complex topology, and output the layout topology correction layout.

[0115] In this embodiment, S7 specifically includes:

[0116] S71. Read the layout topology correction layout, traverse each node in the layout data structure, extract the chart image pixel matrix data in the node, set the pixel matrix dimension to 256 rows, 256 columns, 3 channels, and the value range to integers from 0 to 255; perform normalization operation, divide the pixel value by 255.0, and map it to the floating-point range from 0 to 1; extract the text paragraph description vector in the node, set the vector dimension to 2816 dimensions; flatten the chart pixel matrix in row-major order to generate a one-dimensional image vector with a dimension of 196608; calculate the concatenation operation of the text vector and the one-dimensional image vector to generate a fused feature vector sequence with a dimension of 199424, and output the fused feature vector sequence.

[0117] S72. Read the fused feature vector sequence, construct an encoder network, and set the network to contain 3 fully connected hidden layers with dimensions of 1024, 512, and 256 respectively; calculate the matrix product of the fused feature vector and the first layer weight matrix, add the bias vector, and pass it through the ReLU activation function to obtain 1024-dimensional hidden features; repeat the above forward propagation steps to obtain a 256-dimensional latent representation vector; construct the information bottleneck objective function, and calculate the lower bound of mutual information between the latent representation vector Z and the input vector X. The specific calculation process is as follows: construct a discriminator network, with sample pairs as input, positive samples being real samples, and negative samples being randomly sampled from the marginal distribution; calculate the binary classification cross-entropy loss function value output by the discriminator, and take the negative sign as the lower bound estimate of mutual information; set the compression term weight coefficient to 0.1, calculate the weighted sum of the lower bound of mutual information and the compression term, and output the objective function value.

[0118] S73. Read the objective function value and latent representation vector, construct a variational inference model, and assume that the latent representation variables follow a Gaussian distribution; the encoder network outputs a 256-dimensional mean vector and a 256-dimensional variance vector, and adds a constant 1e-8 to each element of the variance vector to prevent division by zero errors; using a reparameterization technique, sample from the standard normal distribution to generate a 256-dimensional noise vector, calculate the dot product of the noise vector and the standard deviation vector, and add it to the mean vector to obtain the reparameterized latent representation variables; construct a pattern parameter matrix with dimensions of 256 rows and 256 columns; calculate the matrix product of the latent representation variables and the pattern parameter matrix to obtain the patterned features; construct a decoder network to map the patterned features back to the high-dimensional space, calculate the reconstruction error, and use the mean squared error function to calculate the average of the sum of squares of the differences between the reconstructed vector and the original fused vector; calculate the KL divergence loss, which is the sum of the sum of squares of the mean and the logarithm of the variance, minus half the vector dimension; add the reconstruction error and the KL divergence loss, and use the backpropagation algorithm to calculate the gradient values ​​of the pattern parameter matrix, and output the gradient values ​​corresponding to the redundant pattern parameters.

[0119] S74. Read the style parameter matrix and its gradient values, construct a parameter importance evaluation index, calculate the Hadamard product of the gradient value matrix and the style parameter matrix to obtain the importance weight matrix; calculate the absolute value of the importance weight matrix, sum by row to obtain the row weight of each row, and sum by column to obtain the column weight of each column; set the row pruning ratio and column pruning ratio to 20%, sort the row weights in descending order, mark the 20% of row indices with the smallest weight values ​​as redundant rows, and mark the 20% of column indices with the smallest weight values ​​as redundant columns; construct a pruning mask matrix with the same dimensions as the style parameter matrix, initialize all elements to 1, and set the elements corresponding to redundant rows and redundant columns to 0; calculate the element-wise product of the style parameter matrix and the pruning mask matrix, concatenate the pruned parameter matrix with the remaining non-zero parameters to generate key semantic feature parameters, and output the key semantic feature parameters.

[0120] S75. Read the key semantic feature parameters and split the parameter dimensions into text feature parts and image feature parts; for the text feature part, construct a fully connected mapping layer to output a 100-dimensional text style vector, calculate the Euclidean distance between the text style vector and the vectors in the predefined style template library, and select the template index with the smallest distance; retrieve the corresponding text structure parameters based on the template index, including font size, line spacing, and indentation characters, and generate a text structure description file; for the image feature part, construct a deconvolutional neural network, input the key semantic feature parameters, perform transposed convolution operation, with a convolution kernel size of 4 rows and 4 columns, a stride of 2 rows and 2 columns, and padding of 1, and output a 256-dimensional feature map; pass through an upsampling layer and activation function in sequence to generate the final RGB three-channel chart image data; read the coordinate information in the layout topology correction, fill the generated text structure description file and chart image data into the corresponding coordinate positions, generate a multimodal intelligent report file, and output the multimodal intelligent report.

[0121] Example 1: To verify the feasibility of this invention in generating financial investment analysis reports, the method of this invention was applied to the intelligent research report automatic generation platform of a securities company research institute (hereinafter referred to as "Institution Z"). In the traditional financial report writing process, analysts usually need to manually collect a large amount of market data, listed company financial reports, and charts, and rely on experience for subjective judgment and layout. This method is not only time-consuming and labor-intensive, but also difficult to accurately capture the implicit nonlinear relationships between data when faced with massive amounts of high-frequency data, resulting in monotonous report structures, fragmented text and graphics logic, and an inability to meet the timeliness and in-depth analysis requirements of high-frequency trading scenarios. To solve the above problems, Institution Z decided to adopt the report generation method based on natural language processing and multimodal intelligent agents proposed in this invention.

[0122] During implementation, Institution Z first used a data acquisition engine to crawl raw market data sequences, financial statements, and candlestick charts from exchange interfaces, financial news sources, and corporate disclosure systems. After preprocessing operations such as word segmentation and noise reduction, size normalization, and key-value extraction, a standardized input dataset containing text, numerical, and image modalities was constructed. Simultaneously, Institution Z's senior analysts provided structured annotations of core viewpoints and logical chains from historical research reports, serving as a benchmark for evaluating the quality of the generated model.

[0123] Organization Z constructs a high-dimensional topological space for multimodal data and uses local principal component analysis to extract tangent space vectors and curvature features describing the bending degree of the market sentiment manifold, generating a multimodal environment perception tensor. Next, it uses an iterative function system to perform self-similar iteration and dimensionality reduction mapping on the perception tensor, outputting a hierarchical task planning tree containing core viewpoints, sector analysis, and individual stock commentary. In the content generation stage, the system reads the planning tree to construct hypergraph constraints, matches them with the data visualization interface, and outputs a multimodal agent action scheduling sequence. Based on an improved ViLT model, the system models the market semantic tensor and K-line image slices as a two-stream fiber bundle full space. Through local cross-sectional projection and structure group transformation alignment, it constructs a fusion feature tensor and decodes and outputs text paragraph descriptions and chart image pairs. Subsequently, the system reads the planning tree and generated content to construct a semantic fiber bundle. By calculating covariant derivatives and geodesic deviations, it quantifies the consistency between text logic and chart form, outputting a consistency verification feature vector. Finally, by using persistent coherent barcodes to identify gaps in the layout and perform space-filling repairs, combined with information bottleneck removal to eliminate redundant styles, a richly illustrated in-depth investment analysis report was generated.

[0124] During implementation, the research team at Institution Z discovered that, compared to traditional template-filling generation methods, the method of this invention significantly improves the logical depth and layout quality of research reports. Reports generated by traditional methods often exhibit discrepancies between text and figures or disjointed chapters, while the method of this invention, through differential homeomorphism mapping and fiber bundle verification, effectively ensures a deep fit between the textual logic and data features, achieving end-to-end intelligent processing from data perception to report output.

[0125] To further verify the actual performance of the method of the present invention, organization Z conducted a detailed comparative test with the method of the present invention, manual writing, and traditional generation methods. The specific performance data is shown in Table 1:

[0126] Table 1 Performance Comparison of Institution Z Investment Analysis Report Generation Methods

[0127] index Traditional generation methods Method of the present invention Increase Average time (in hours) to generate the report 0.5 0.15 -70.0% Key data association accuracy (%) 82.5 98.2 +15.8% Graphical and textual logical consistency score (points) 6.5 9.5 +46.2% Page layout rationality score (points) 7.0 9.6 +37.1% In-depth perspective mining coverage (%) 45.0 92.0 +104.4% Manual revision rate (%) 65.0 8.5 -86.9% Labor cost per report (RMB) 200 50 -75.0% Analyst job satisfaction (%) 60.0 96.0 +60.0% Completeness of risk warning (%) 70.0 99.5 +42.1%

[0128] As shown in Table 1, the performance of the intelligent research report generation platform was comprehensively improved after applying the method of this invention. The average time for report generation was reduced from 0.5 hours using traditional methods to 0.15 hours, significantly improving the timeliness of research report output. The accuracy rate of key data association increased from 82.5% to 98.2%, effectively avoiding data citation errors. The consistency score between text and graphics significantly improved from 6.5 to 9.5, solving the pain point of semantic fragmentation between text and graphics in traditional generation methods. The manual revision rate was greatly reduced from 65.0% to 8.5%, requiring analysts to conduct only a small amount of review before publication, significantly reducing labor costs. In addition, the coverage rate of in-depth insight mining reached 92.0%, proving that this invention can uncover the implicit logic behind the data like a seasoned analyst.

[0129] Through the method of this invention, Institution Z has successfully achieved automated and intelligent generation of financial research reports, effectively solving the bottlenecks of rigid structure, shallow logic and difficulty in multimodal integration of traditional methods, greatly improving research efficiency and report quality, enhancing the support for investment decisions, and providing strong technical support for the intelligent transformation of the financial technology field.

[0130] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A report generation method based on natural language processing and multimodal intelligent agents, characterized in that, Includes the following steps: S1. Acquire multi-source heterogeneous data, construct a high-dimensional topological space for multimodal data, extract the tangent space vector and curvature features of the local neighborhood based on differential homeomorphism, and output the multimodal environment perception tensor. S2. Read the multimodal environment perception tensor as the seed vector, define fractal element rules and dimension parameters, perform self-similar iteration and dimension reduction mapping through the iterative function system, and output a hierarchical task planning tree. S3. Read the hierarchical task planning tree to construct hypergraph constraints and parallel time slots, match the tool interface and encode to generate graph image slice sequences and multimodal environment perception tensors, define tensor quantum block extraction rules, and output multimodal agent action scheduling sequences; S4. Based on the improved ViLT model, the multimodal environment perception tensor and the sequence of chart image slices are modeled as a two-stream fiber bundle full space. The projection generates local coordinate features of the bottom manifold. According to the tool interface parameters in the action scheduling sequence of the multimodal agent, the coordinate transformation operation is performed on the local coordinate features of the bottom manifold. The fused feature tensor is constructed and decoded to output the text paragraph description and the chart image pair. S5. Read the hierarchical task planning tree as the base space, text paragraph descriptions and graph image pairs as fibers, construct a semantic fiber bundle, calculate the covariant derivative and curvature tensor, quantify the geodesic deviation, and output the consistency verification feature vector. S6. Read the consistency verification feature vector and map it into a two-dimensional page point cloud. Use topological data analysis to calculate the persistent coherent barcode, identify holes and fragmented areas and adjust the space occupancy, and output the page topology correction layout. S7. Read the layout of the page topology, correct the layout, define the information bottleneck constraint target, calculate the lower bound of mutual information and remove redundant style parameters, and output a multimodal intelligent report.

2. The report generation method based on natural language processing and multimodal intelligent agents according to claim 1, characterized in that, S1 specifically includes: S11. Extract raw text data, chart and image data and structured table data from multi-source heterogeneous data sources. Perform word segmentation and stop word removal on the raw text data. Perform size normalization and color space conversion on the chart and image data. Extract key-value pairs from the structured table data and output the preprocessed standardized multimodal data. S12. Read the preprocessed standardized multimodal data, construct the word vector space of text data, the pixel feature space of image data, and the numerical embedding space of table data respectively, calculate the intrinsic dimension of each modal data through the manifold learning algorithm, map each modal feature space to a unified high-dimensional Hilbert space, and output the high-dimensional topological space of multimodal data. S13. Read the high-dimensional topological space of multimodal data, construct the local neighborhood graph of data points based on Euclidean distance, calculate the tangent space basis vector of each node in the local neighborhood graph using the local principal component analysis method, and output the tangent space vector of the local neighborhood. S14. Read the local neighborhood graph and the tangent space vector of the local neighborhood, calculate the Riemann curvature tensor and Ricci curvature tensor at each node, analyze the distribution characteristics of the curvature tensor in different dimensions, and extract the geometric features describing the local curvature of the data manifold. S15. Read the tangent space vector of the local neighborhood and the geometric features describing the local curvature of the data manifold. Perform tensor product operation on the tangent space vector and the curvature tensor. Eliminate redundant information through orthogonal projection to generate a multimodal environment perception tensor that remains invariant under differential homeomorphic transformation.

3. The report generation method based on natural language processing and multimodal intelligent agents according to claim 1, characterized in that, S2 specifically includes: S21. Read the multimodal environment perception tensor and calculate the principal eigenvalues ​​and principal eigenvectors. Project the principal eigenvectors into low-dimensional seed vectors. Calculate the intrinsic dimension of the data manifold based on the numerical distribution of the principal eigenvalues ​​and map the intrinsic dimension to the fractal dimension parameter. S22. Determine the initial attractor coordinates of the iterative function system based on the low-dimensional seed vector, calculate the compression ratio and rotation angle of the affine transformation matrix in the iterative function system according to the fractal dimension parameter, and generate a customized iterative function system. S23. Input the low-dimensional seed vector into the customized iterative function system, perform self-similar iterative operation according to the number of iterations limited by the fractal dimension parameter, and output a high-dimensional fractal point cloud set. S24. Perform principal component analysis to reduce the dimensionality of the high-dimensional fractal point cloud set, project the point cloud onto a two-dimensional plane coordinate system, and generate a planar tree-like topology with nested hierarchical relationships. S25. Traverse the planar tree topology, map the topology nodes to report chapter titles, map the topology connecting edges to chapter hierarchical relationships, and output a hierarchical task planning tree.

4. The report generation method based on natural language processing and multimodal intelligent agents according to claim 1, characterized in that, S3 specifically includes: S31. Read the hierarchical task planning tree, map each chapter node in the structure tree to a hypergraph vertex, and map the data reference relationship and hierarchical membership relationship between nodes to hyperedge constraints that connect multiple hypergraph vertices. S32. Perform graph coloring algorithm on the hypergraph vertices, mark the hypergraph vertices that are not connected by hyperedges as the same color, map each color to an independent parallel time slot, and output the parallel time slot allocation scheme. S33. Based on the semantic type of nodes in the hierarchical task planning tree, match the corresponding data visualization tool interface and natural language generation interface; read the original chart image data, perform image slicing and feature encoding operations, and output the chart image slice sequence; read the semantic text data, extract semantic features and construct tensor structure, output multimodal environment-aware tensor, and define the dimension parameters and data format of tensor slices. S34. Read the parallel time slot allocation scheme, divide the tasks corresponding to the hypergraph vertices belonging to the same time slot into concurrent execution groups, divide the vertex tasks with hyperedge constraints into serial dependency groups, and generate a preliminary task sequence. S35. Inject the interface parameters, chart image slice sequence and tensor block extraction rules into the initial task sequence, verify the integrity of the data flow direction of the hyperedge constraint, and output a multimodal agent action scheduling sequence containing the execution order, parallel time slots, input data entities, tool interface parameters and tool call instructions.

5. The report generation method based on natural language processing and multimodal intelligent agents according to claim 1, characterized in that, The improved ViLT model includes a dual-stream fiber bundle structure embedding layer, a local section projection layer, a structure group transformation alignment layer, a bottom manifold unified representation layer, and a parallel decoding output layer: The dual-stream fiber bundle structure embedding layer is used to receive multimodal environment perception tensors and chart image slice sequences, model the multimodal environment perception tensors as semantic fiber bundle full space, model the chart image slice sequences as visual fiber bundle full space, define the fiber types of the semantic fiber bundle full space and the structure groups of the visual fiber bundle full space respectively, and output the semantic fiber bundle full space structure and the visual fiber bundle full space structure. The local section projection layer is used to receive the semantic fiber bundle full-space structure and the visual fiber bundle full-space structure, construct a smooth mapping operator, select non-singular neighborhoods and generate local sections pointing to the bottom manifold, project the high-dimensional heterogeneous fiber features to the low-dimensional isomorphic bottom manifold coordinate system, and output the bottom manifold local coordinate features. The structure group transformation alignment layer is used to receive the local coordinate features of the bottom manifold and the tool interface parameters in the action scheduling sequence of the multimodal agent, construct the structure group action matrix based on the tool interface parameters, perform coordinate transformation operation on the local coordinate features of the bottom manifold, map the semantic bottom manifold features and the visual bottom manifold features to a unified bottom space coordinate system, and output the aligned semantic bottom manifold features and the visual bottom manifold features. The unified representation layer of the bottom manifold is used to receive the aligned semantic bottom manifold features and visual bottom manifold features, perform tangent vector field concatenation and Riemann integral operation, construct a globally unified fusion feature tensor, and output the fusion feature tensor. The parallel decoding output layer is used to receive the fused feature tensor, map the fused feature tensor to the text generation space and the image pixel space respectively, generate natural language description text through autoregressive calculation, and reconstruct the optimized chart image through deconvolution operation, and output the text paragraph description and chart image pair.

6. The report generation method based on natural language processing and multimodal intelligent agents according to claim 1, characterized in that, S5 specifically includes: S51. Read the hierarchical task planning tree as the base space, text paragraph descriptions and graph / image pairs as fibers, establish the Cartesian product space of the base space and fibers, and output the semantic fiber bundle full space data structure. S52. Define the connection coefficients on the semantic fiber bundle full-space data structure, calculate the covariant derivatives on the semantic manifold, and output the path-wise rate of change of the tangent vector field. S53. Calculate the curvature tensor based on the path change rate of the tangent vector field and output the curvature tensor of the semantic fiber bundle space. S54. Solve the geodesic equation based on the curvature tensor, calculate the geodesic deviation between adjacent fiber paths, and output the geodesic deviation value. S55. Perform norm normalization on the geodesic deviation values, generate a consistency verification feature vector, and output it.

7. The report generation method based on natural language processing and multimodal intelligent agents according to claim 1, characterized in that, S6 specifically includes: S61. Read the consistency verification feature vector, and map the feature dimension to the plane coordinate dimension through linear transformation to generate two-dimensional page point cloud data; S62. Calculate the Vietoris-Rips complex corresponding to the two-dimensional point cloud data, and construct the simple complex topology based on the set neighborhood radius parameter. S63. Perform boundary operator operations and chain group homology calculations based on the simple complex topology, and output a persistent homology barcode; S64. Analyze the survival range of persistent coherent barcodes, identify the hole and segmented regions in the point cloud of the page, and calculate the geometric center coordinates and bounding boxes of the hole and segmented regions. S65. Based on the geometric center coordinates and bounding boxes of the empty and fragmented regions, perform spatial filling and connectivity repair operations on the two-dimensional point cloud to generate a layout with corrected topology.

8. The report generation method based on natural language processing and multimodal intelligent agents according to claim 1, characterized in that, Specifically, S7 includes: S71. Read the layout topology and correct the layout, extract the pixel matrix of the chart image and the text paragraph description vector in the layout; S72. Construct an information bottleneck constraint objective function based on the pixel matrix of the chart image and the description vector of the text paragraph, and calculate the lower bound of the mutual information between the input variables and the potential representation variables; S73. Perform variational inference operations based on the lower bound of mutual information to calculate the gradient values ​​corresponding to redundant style parameters; S74. Based on the gradient values, perform pruning and stripping operations on redundant style parameters to generate key semantic feature parameters; S75. Reorganize the text structure and image rendering attributes based on key semantic feature parameters to generate multimodal intelligent reports.

Images