Document processing method and apparatus based on intent perception

Abstract

The application discloses a document processing method and device based on intention perception, and relates to the technical field of artificial intelligence. The method comprises the following steps: converting text data, image data and behavior data into text features, image features and behavior features through corresponding pre-training models respectively; mapping the features to a shared feature space through a full connection layer after splicing to obtain a shared feature after document-behavior-visual fusion; preliminarily predicting the shared feature through an intention prediction module, a content recommendation module and a chart analysis module to obtain three corresponding preliminary prediction results; and dynamically optimizing the preliminary prediction results according to a pre-set interaction rule to obtain an optimized target text and an optimized chart abstract to generate a target document. According to the embodiment of the application, the preliminary results of the three modules are optimized through a cooperative interaction mechanism, the error of independent prediction is reduced, and the efficiency and accuracy of intelligent document processing are significantly improved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and in particular relates to a document processing method and apparatus based on intent awareness. Background Technology

[0002] In today's office environment, document editing faces the dual challenges of efficiency and accuracy. With the increasing complexity of document types such as government documents, research papers, and project reports, users need to perform numerous operations during editing, such as paragraph formatting, inserting charts and graphs, and writing content. Traditional document editing tools rely primarily on manual user operation, lacking anticipation of user intent and intelligent assistance. The emergence of intelligent document processing platforms provides a fast and convenient solution to these problems.

[0003] The field of intelligent document processing is currently at a critical stage of evolution from static analysis to dynamic interaction. With the popularization of multimodal fusion technology, mainstream systems can achieve basic semantic understanding by jointly learning text, layout, and visual features (such as LayoutLMv3 models), achieving high accuracy in locating key fields in scenarios such as government documents. Some document platforms have further introduced user behavior analysis functions, generating document recommendation lists based on editing history and collaboration relationships, initially shortening the information retrieval path. At the same time, automation frameworks such as LangChain connect modules such as OCR and text segmentation through the Chain / Agent mechanism, providing a technical foundation for end-to-end task processing.

[0004] However, while related technologies are relatively mature in processing single-modal information such as text and icons independently, they struggle to establish semantic associations and collaborative reasoning mechanisms across modal elements. For example, chart information is parsed as isolated fragments, failing to form logical bindings with adjacent text descriptions; document structure information is not dynamically incorporated into the decision-making process, leading to conflicts between content recommendation and the chapter logic framework. Summary of the Invention

[0005] This application provides an intent-aware document processing method and apparatus to at least solve the problem in related technologies of the difficulty in establishing semantic association and collaborative reasoning mechanisms for cross-modal elements.

[0006] In a first aspect, embodiments of this application provide an intent-aware document processing method, including:

[0007] Text data, image data, and behavioral data are transformed into text features, image features, and behavioral features respectively through corresponding pre-trained models;

[0008] The text features, image features, and behavioral features are concatenated and then mapped to a shared feature space through a fully connected layer to obtain the document-behavior-visual fusion shared features.

[0009] The shared features are initially predicted by the intent prediction module, the content recommendation module, and the chart parsing module, and the initial prediction results of each of the three modules are obtained. The initial prediction result of the intent prediction module is the probability distribution of the user's next operation, the initial prediction result of the content recommendation module is the target text, and the initial prediction result of the chart parsing module is the chart summary.

[0010] The preliminary prediction results are dynamically optimized according to pre-set interaction rules to obtain optimized target text and optimized chart summary to generate target document;

[0011] The interaction rules include: when the probability distribution of the user's next operation is a chart operation, the target text is chart-related text, and the semantic similarity between the target text and the chart summary is greater than a preset threshold.

[0012] The behavioral data includes user cursor trajectory information, operation event information, and document context information, wherein the operation event information includes short-term high-frequency operation events and long-term low-frequency operation events; the behavioral features include short-term features and long-term features.

[0013] The process of converting text data, image data, and behavioral data into text features, image features, and behavioral features respectively through corresponding pre-trained models includes:

[0014] The cursor trajectory information and short-term high-frequency operation event information are extracted using a gated recurrent unit (GRU) to obtain short-term features; the long-term low-frequency operation events are extracted using a sliding window and a multilayer perceptron (MLP) to obtain long-term features.

[0015] Optionally, the preliminary prediction of the shared features through the intent prediction module, content recommendation module, and chart parsing module yields preliminary prediction results for each of the three modules, including:

[0016] The intent prediction module constructs a dynamic heterogeneous graph based on the short-term and long-term features. The nodes of the dynamic heterogeneous graph are obtained by splicing and normalizing the short-term and long-term features. The edge weights of the dynamic heterogeneous graph are calculated by combining the semantic correlation between nodes and the time decay factor.

[0017] The node information of the dynamic heterogeneous graph is aggregated based on the graph attention mechanism, and the aggregated node features are fused with the document context semantic vector to output the probability distribution of the user's next operation.

[0018] Optionally, constructing a dynamic heterogeneous graph based on the short-term and long-term features includes:

[0019] The short-time and long-time features are concatenated and normalized according to the following formula to obtain the vector corresponding to node i of the dynamic heterogeneous graph. :

[0020] ;

[0021] Where LayerNorm represents layer normalization; This represents a learnable weight matrix used to concatenate short-term features. and long-term features Mapped to node space;

[0022] The edge weights of a dynamic heterogeneous graph are calculated using the following formula. :

[0023] ;

[0024] in, This represents the first weight matrix. This represents the second weight matrix. Let d represent the vector corresponding to node j, and let d represent the dimension of the attention head. This represents the time interval between node i and node j. Indicates the time decay coefficient. The scale constant represents the rate of decay of the time-dependent correlation of the control operation, and T represents the transpose.

[0025] Optionally, the step of aggregating node information of the dynamic heterogeneous graph based on the graph attention mechanism, and fusing the aggregated node features with the document context semantic vector to output the probability distribution of the user's next operation includes:

[0026] The features of the aggregated nodes are fused with the document context semantic vector according to the following formula to output the user's next action. probability distribution :

[0027] ;

[0028] in, This represents the aggregated node features at the current time. Represents a document context semantic vector. This represents the output weight matrix. This represents the context mapping matrix, and GELU is the activation function.

[0029] Optionally, the preliminary prediction of the shared features through the intent prediction module, content recommendation module, and chart parsing module yields preliminary prediction results for each of the three modules, including:

[0030] The content recommendation module deconstructs the document into title nodes, paragraph nodes, and chart nodes, and establishes hierarchical relationship edges, reference relationship edges, and semantic similarity edges between the title nodes, paragraph nodes, and chart nodes to obtain the document structure graph. The hierarchical relationship edge represents the parent-child relationship between title nodes, the reference relationship edge represents the paragraph node referencing the chart node, and the semantic similarity edge represents the semantic association between paragraph nodes.

[0031] Using the user's operation intent vector as the query, retrieve the corresponding candidate cases from the pre-accessed external knowledge base, and calculate the association weight between the candidate cases and the nodes in the document structure graph;

[0032] Based on the association weights and the document structure graph node features updated by the graph neural network (GNN), graph constraint features are generated.

[0033] The graph constraint features are input into the generative model, and the generative model is decoded to obtain the target text.

[0034] Optionally, calculating the association weights between the candidate cases and nodes in the document structure graph includes:

[0035] The association weights between the candidate cases and the nodes in the document structure graph are calculated using the following formula. :

[0036] ;

[0037] in, This represents the user's action intent vector. This represents the j-th node in the document structure graph. Represents a node The document structure graph node features are updated using a graph neural network (GNN). This represents the attention weight matrix, where k represents the loop index during summation. This represents the k-th node in the document structure graph. Document structure graph node features updated using graph neural network (GNN);

[0038] The step of generating graph-constrained features based on the association weights and the document structure graph node features updated by the graph neural network (GNN) includes:

[0039] Generate graphical constraint features based on the following formula. :

[0040] ;

[0041] Where j represents the node index.

[0042] Optionally, the preliminary prediction of the shared features through the intent prediction module, content recommendation module, and chart parsing module yields preliminary prediction results for each of the three modules, including:

[0043] The chart parsing module performs syntactic parsing on the chart-related text to obtain semantic role labels, and maps the semantic role labels to semantic role vectors;

[0044] Visual features of chart images are extracted using ResNet;

[0045] The semantic role vector is used to drive deformable convolution to adjust the receptive field of the visual feature, so that the semantic role vector and the visual feature are aligned.

[0046] Based on the aligned semantic role vectors and visual features, an inference graph is constructed for the data points in the chart. The node features represent the data points, and the edge features integrate the Euclidean distance, relative angle, and color similarity between the data points. The node features are propagated and updated through a multi-layer graph neural network.

[0047] Based on the updated node features of the inference graph, structured data is extracted, and a template engine is used to generate a chart summary that conforms to the document context. The chart summary includes multi-granular chart descriptions of the data layer, analysis layer, and decision layer.

[0048] Secondly, embodiments of this application provide an intent-aware document processing apparatus, the apparatus comprising:

[0049] The transformation module is used to convert text data, image data, and behavioral data into text features, image features, and behavioral features respectively through the corresponding pre-trained models.

[0050] The splicing module is used to splice the text features, image features and behavioral features and then map them to the shared feature space through a fully connected layer to obtain the document-behavior-visual fusion shared features;

[0051] The intent prediction module, content recommendation module, and chart parsing module are used to perform preliminary predictions on the shared features respectively, and obtain the preliminary prediction results corresponding to each of the three modules. The preliminary prediction result of the intent prediction module is the probability distribution of the user's next operation, the preliminary prediction result of the content recommendation module is the target text, and the preliminary prediction result of the chart parsing module is the chart summary.

[0052] The optimization module is used to dynamically optimize the preliminary prediction results according to the pre-set interaction rules, so as to obtain the optimized target text and the optimized chart summary to generate the target document;

[0053] The interaction rules include: when the probability distribution of the user's next operation is a chart operation, the target text is chart-related text, and the semantic similarity between the target text and the chart summary is greater than a preset threshold.

[0054] The behavioral data includes user cursor trajectory information, operation event information, and document context information, wherein the operation event information includes short-term high-frequency operation events and long-term low-frequency operation events; the behavioral features include short-term features and long-term features.

[0055] The process of converting text data, image data, and behavioral data into text features, image features, and behavioral features respectively through corresponding pre-trained models includes:

[0056] The cursor trajectory information and short-term high-frequency operation event information are extracted using a gated recurrent unit (GRU) to obtain short-term features; the long-term low-frequency operation events are extracted using a sliding window and a multilayer perceptron (MLP) to obtain long-term features.

[0057] Thirdly, embodiments of this application provide an electronic device, which includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the steps of the intent-aware document processing method as described in any embodiment of the first aspect.

[0058] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the steps of the intent-aware document processing method as described in any embodiment of the first aspect.

[0059] Fifthly, embodiments of this application provide a computer program product, which is stored in a storage medium and executed by at least one processor to implement the steps of the intent-aware document processing method provided in the first aspect of embodiments of this application.

[0060] The document processing method and apparatus based on intent awareness in this application integrates spatiotemporal behavior awareness, structured semantic collaboration, and cross-modal deep parsing. It constructs an end-to-end process from three aspects: data processing, shared feature extraction, and multi-task interaction. Simultaneously, it performs task decoding of the three modules and outputs their respective preliminary results. The preliminary results of the three modules are mutually optimized through a collaborative interaction mechanism to reduce the error of independent prediction, thereby significantly improving the efficiency and accuracy of intelligent document processing. Attached Figure Description

[0061] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a flowchart illustrating an intent-aware document processing method provided in an embodiment of this application.

[0063] Figure 2 This is a schematic diagram of the process of obtaining preliminary prediction results through the intent prediction module provided in the embodiments of this application;

[0064] Figure 3 This is a schematic diagram of the process for obtaining preliminary prediction results through a content recommendation module, as provided in an embodiment of this application.

[0065] Figure 4 This is a schematic diagram of the process for obtaining preliminary prediction results through a chart analysis module, as provided in an embodiment of this application.

[0066] Figure 5 This is a flowchart illustrating another intent-aware document processing method provided in an embodiment of this application;

[0067] Figure 6 This is a schematic diagram of the structure of an intent-aware document processing device provided in an embodiment of this application;

[0068] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0069] Figure label:

[0070] Conversion module 601, stitching module 602, intent prediction module 603, content recommendation module 604, chart parsing module 605, optimization module 606.

[0071] Electronic device 700, processor 701, memory 702, communication interface 703, bus 710. Detailed Implementation

[0072] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0073] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0074] In today's office environment, document editing faces the dual challenges of efficiency and accuracy. With the increasing complexity of document types such as government documents, research papers, and project reports, users need to perform numerous operations during editing, such as paragraph formatting, inserting charts and graphs, and writing content. Traditional document editing tools rely primarily on manual user operation, lacking anticipation of user intent and intelligent assistance. The emergence of intelligent document processing platforms provides a fast and convenient solution to these problems.

[0075] The field of intelligent document processing is currently at a critical stage of evolution from static analysis to dynamic interaction. With the popularization of multimodal fusion technology, mainstream systems can achieve basic semantic understanding by jointly learning text, layout, and visual features (such as LayoutLMv3 models), achieving high accuracy in locating key fields in scenarios such as government documents. Some document platforms have further introduced user behavior analysis functions, generating document recommendation lists based on editing history and collaboration relationships, initially shortening the information retrieval path. At the same time, automation frameworks such as LangChain connect modules such as OCR and text segmentation through the Chain / Agent mechanism, providing a technical foundation for end-to-end task processing.

[0076] Currently, while the ability to independently process single-modal information such as text and icons is relatively mature, it is difficult to establish semantic associations and collaborative reasoning mechanisms across modal elements. For example, chart information is parsed as isolated fragments and not logically bound to adjacent text descriptions; document structure information is not dynamically incorporated into the decision-making process, leading to conflicts between content recommendation and the chapter logic framework.

[0077] While some related technologies attempt to provide simple recommendations by analyzing user operation sequences or document context, they suffer from shortcomings in areas such as multimodal data fusion, accuracy of intent prediction, relevance of content recommendations, and precision of chart information extraction. For example, in intent prediction, judgments are based solely on single-dimensional operation data. Some traditional systems only respond to explicit instructions and cannot predict potential intents based on continuous behavior (cursor trajectory, editing frequency), ignoring document semantics. Furthermore, they lack a continuous learning mechanism for user operation habits, leading to prediction results that deviate from actual needs. In terms of content recommendation, they are mostly limited to the text level and have weak connections with contextual logic. Recommendation engines rely solely on keyword matching, without integrating document topology, and lack dynamic adaptability, failing to respond to actual editing. Chart information extraction often only recognizes simple chart types, supporting only basic rendering of charts embedded in documents, and cannot extract structured data for collaborative analysis with text descriptions.

[0078] Specifically, regarding the static nature of intent prediction, traditional solutions only respond to explicit operation commands, failing to capture the spatiotemporal coupling characteristics of user behavior, such as changes in cursor movement speed and time interval patterns of operation events. They also lack the ability to learn long-term operating habits, resulting in delayed prediction results that deviate from actual needs. Regarding the disconnect between content recommendation and document structure, recommendation engines rely excessively on text keyword matching, ignoring the dynamic constraints of document topology, causing logical misalignment between recommended content and the current editing position. Regarding the shallow nature of chart parsing, existing technologies only achieve basic visual rendering and text extraction, lacking in-depth analysis of the structural relationships between chart elements and failing to coordinate scattered text and image descriptions, making it difficult to effectively release data value.

[0079] It is evident that these defects form a systemic bottleneck: inaccurate intention prediction forces users to frequently perform redundant operations; unstructured recommended content causes logical confusion; and superficial chart analysis leads to a decline in data value.

[0080] To address the problems in the related technologies, embodiments of this application provide an intent-aware document processing method and apparatus.

[0081] The intent-aware document processing method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0082] Figure 1 A flowchart illustrating an embodiment of the intent-aware document processing method of this application is shown. Figure 1 As shown, this intent-aware document processing method may specifically include the following steps:

[0083] S101. Transform text data, image data, and behavioral data into text features, image features, and behavioral features respectively using the corresponding pre-trained models;

[0084] S102. The text features, image features and behavioral features are concatenated and then mapped to the shared feature space through a fully connected layer to obtain the shared features after document-behavior-visual fusion.

[0085] S103. The shared features are initially predicted by the intent prediction module, the content recommendation module, and the chart parsing module to obtain the initial prediction results of each of the three modules. The initial prediction result of the intent prediction module is the probability distribution of the user's next operation, the initial prediction result of the content recommendation module is the target text, and the initial prediction result of the chart parsing module is the chart summary.

[0086] S104. Dynamically optimize the preliminary prediction results according to the pre-set interaction rules to obtain the optimized target text and optimized chart summary to generate the target document;

[0087] The interaction rules include: when the probability distribution of the user's next operation is a chart operation, the target text is chart-related text, and the semantic similarity between the target text and the chart summary is greater than a preset threshold.

[0088] Therefore, by integrating spatiotemporal behavior perception, structured semantic collaboration, and cross-modal deep parsing, an end-to-end process is constructed from three aspects: data processing, shared feature extraction, and multi-task interaction. Simultaneously, the tasks of the three modules are decoded, and their respective preliminary results are output. The preliminary results of the three modules are mutually optimized through a collaborative interaction mechanism to reduce the error of independent prediction, thereby significantly improving the efficiency and accuracy of intelligent document processing.

[0089] The specific implementation methods for each of the above steps are described below.

[0090] In some embodiments, prior to S101, data preprocessing is performed on the text data, image data, and behavioral data respectively. Specifically, the text data is processed by word segmentation and stop word removal, converted into a token sequence, mapped to an integer ID, and a special marker is added; for the image data, the chart is resized to a uniform size and standardized; for the behavioral data, continuous values ​​such as cursor coordinates and action time intervals are discretized and encoded into a fixed-length vector.

[0091] In some embodiments, in S101, the text token sequence is processed using a pre-trained BERT model to output text features, including lexical semantics, sentence structure, and other information; the chart image is processed using a ViT (Vision Transformer) model, segmenting the image into patches and outputting image features, including chart type, color distribution, geometric shape, and other information; and the user behavior sequence is processed through a GRU network to output behavioral features, reflecting cursor movement trends, operating habits, etc. In this way, text, image, and behavioral data are transformed into a unified semantic feature space, achieving cross-modal information sharing.

[0092] In specific implementation, behavioral data includes user cursor trajectory information, operation event information, and document context information. The operation event information includes short-term high-frequency operation events and long-term low-frequency operation events. The behavioral features include short-term features and long-term features. Therefore, a gated recurrent unit (GRU) is used to extract micro-features from the cursor trajectory information and short-term high-frequency operation event information to obtain short-term features; a sliding window and a multilayer perceptron (MLP) are used to extract macro-features from the long-term low-frequency operation events to obtain long-term features.

[0093] In some embodiments, in S102, text features, image features, and behavioral features are concatenated and mapped to a shared feature space through a fully connected layer to obtain a unified "document-behavior-visual" fusion feature, denoted as F. Then, based on the shared feature F, the intent prediction module, content recommendation module, and chart parsing module simultaneously perform task decoding and output their respective preliminary prediction results.

[0094] refer to Figure 2 This is a flowchart illustrating the process of obtaining preliminary prediction results through the intent prediction module.

[0095] It's important to note that traditional intent prediction methods suffer from two core flaws: first, they focus solely on explicit actions (such as clicking a button), neglecting to model implicit behavioral signals (cursor trajectories, operation intervals, etc.), thus failing to capture potential spatiotemporally coupled intents such as "the cursor rapidly moving to a table area, implying formatting adjustments"; second, they cannot simultaneously handle short-term high-frequency behaviors and long-term low-frequency patterns, resulting in high prediction latency. Therefore, this algorithm utilizes "multi-scale spatiotemporal coding + dynamic graph structure learning" to achieve real-time parsing of implicit behaviors and unified modeling of long and short-period patterns, solving the problem of low-latency, high-precision intent prediction.

[0096] like Figure 2 As shown, obtaining preliminary prediction results through the intent prediction module specifically includes the following steps:

[0097] First, to collect user behavior data, it is necessary to capture implicit behavioral signals during the user's editing process in real time, including cursor trajectory information, operation event information, and document context information. Specifically, cursor trajectory information refers to the implicit behavioral signals generated within a short, continuous time frame. Internal record of one coordinate change ,in Represents the displacement change on the horizontal axis. This represents the displacement change on the vertical axis; operation events refer to the types of operations such as deletion, insertion, and formatting adjustments, as well as the time interval between operations; document context refers to the semantic vector of the paragraph where the cursor is currently located. The document's context semantic vector is generated through encoding using a pre-trained language model.

[0098] After data acquisition, the raw behavioral data needs to be transformed into computable feature vectors. Specifically, this step is divided into two scales: micro and macro. At the micro scale, GRU (Gated Recurrent Unit) is used to process short-term, high-frequency cursor trajectories and operation events, capturing the patterns of instantaneous behaviors such as "cursor rapidly jumping between paragraphs," and outputting short-term features. It reflects behavioral patterns within a recently defined time period; at a macro scale, it uses a sliding window to statistically analyze long-term low-frequency patterns (such as the frequency of deletion operations and the variance of format adjustment intervals), and generates long-term features through an MLP (Multilayer Perceptron). .

[0099] Furthermore, through the intent prediction module, a dynamic heterogeneous graph is constructed based on the short-term and long-term features. The nodes of the dynamic heterogeneous graph are obtained by concatenating and normalizing the short-term and long-term features, and the edge weights of the dynamic heterogeneous graph are calculated by combining the semantic correlation between nodes and the time decay factor. Based on the graph attention mechanism, the node information of the dynamic heterogeneous graph is aggregated, and the aggregated node features are fused with the document context semantic vector to output the probability distribution of the user's next operation.

[0100] In practice, the short-term and long-term features are concatenated and normalized according to the following formula to obtain the vector corresponding to node i of the dynamic heterogeneous graph. :

[0101] ;

[0102] Where LayerNorm represents layer normalization; This represents a learnable weight matrix used to concatenate short-term features. and long-term features Mapped to the node space.

[0103] In practice, the edge weights of the dynamic heterogeneous graph are calculated according to the following formula. :

[0104] ;

[0105] in, This represents the first weight matrix; This represents the second weight matrix; d represents the vector corresponding to node j; d represents the attention head dimension to ensure numerical stability. This represents the time interval between node i and node j; This represents the time decay coefficient, used to control the weight of the time factor; The scale constant represents the rate at which the correlation between operation time decays, used to adjust the decay rate and ensure that recent behavior is given higher weight; T represents the transpose.

[0106] In practice, the features of the aggregated nodes are fused with the document context semantic vector according to the following formula to output the user's next action. probability distribution :

[0107] ;

[0108] in, This represents the aggregated node features at the current time. Represents a document context semantic vector. This represents the output weight matrix. This represents the context mapping matrix, and GELU is the activation function.

[0109] In other words, after obtaining the output short-term and long-term features, a dynamic heterogeneous graph is constructed to transform the features into a graph structure, enabling explicit modeling of behavioral relationships. First, nodes are defined, and the features at each time step are normalized, converting the short-term features... and long-term features After concatenation, the node vectors are obtained by standardizing using LayerNorm (layer normalization). Then, based on the semantic relationships and time decay between nodes, the edge weights are dynamically calculated. Semantic association is calculated using query-keys attention. Next, a multi-attention mechanism is used to aggregate neighbor node information to strengthen the influence of key behaviors. Finally, the document context is fused, specifically the aggregated node features. Document context semantic vector (Generated by encoding the current paragraph using BERT) and fused together, the final output is the predicted probability of the next operation. .

[0110] refer to Figure 3This is a flowchart illustrating the process of obtaining preliminary prediction results through the content recommendation module.

[0111] It should be noted that traditional content recommendation suffers from three problems: first, recommended content is isolated and ignores the document's topological structure; second, semantic coherence is poor; and third, it cannot adapt to the logical framework of real-time user editing. Therefore, this algorithm generates a document structure graph and graph constraints to ensure that the recommended content is deeply aligned with the document structure and semantic logic.

[0112] like Figure 3 As shown, obtaining preliminary prediction results through the content recommendation module specifically includes the following steps:

[0113] The content recommendation module deconstructs the document into title nodes, paragraph nodes, and chart nodes, and establishes hierarchical relationship edges, referencing relationship edges, and semantic similarity edges among these nodes to obtain a document structure graph. The hierarchical relationship edges represent parent-child relationships between title nodes, the referencing relationship edges represent paragraph nodes referencing chart nodes, and the semantic similarity edges represent semantic associations between paragraph nodes. Using the user's action intent vector as the query, corresponding candidate cases are retrieved from a pre-accessed external knowledge base, and the association weights between these candidate cases and nodes in the document structure graph are calculated. Based on these association weights and the updated document structure graph node features using a graph neural network (GNN), graph constraint features are generated. These graph constraint features are input into the generation model, and the model is decoded to obtain the target text.

[0114] In practice, to parse the hierarchical relationships within a document, a document structure graph is constructed. Specifically, headings are defined. ,paragraph ,chart Three types of nodes, and three types of edge relationships are constructed: hierarchical relationships (such as...) ), indicating the parent-child relationship between headings; citation relationship (e.g. (), indicating paragraph citation chart; semantic similarity, indicating semantic relationships between paragraphs.

[0115] Then, enhanced retrieval and fusion are performed, using the user's current intent vector generated by the intent prediction module. For the query, retrieve Top-K related cases from the case library (such as similar papers and template libraries). We extract key sentences from the cases as recommendation candidates and calculate the association weights between the candidate cases and the nodes of the document structure graph.

[0116] In practice, the association weight between the candidate case and the node in the document structure graph is calculated according to the following formula. :

[0117] ;

[0118] in, This represents the user's action intent vector. This represents the j-th node in the document structure graph. Represents a node The document structure graph node features are updated using a graph neural network (GNN). This represents the attention weight matrix, where k represents the loop index during summation. This represents the k-th node in the document structure graph. The document structure graph node features are updated using a graph neural network (GNN).

[0119] In practice, graph constraint features are generated according to the following formula. :

[0120] ;

[0121] Where j represents the node index.

[0122] In practical implementation, graph constraint features After injecting the generative model and obtaining the relevant data, the transformer is used for hierarchical decoding control. The encoder input of the transformer is [the current paragraph vector; ; The decoder initially uses the prefix of the currently input text. In each decoding step, a vocabulary mask is dynamically generated to prevent words that semantically conflict with the current node. Finally, recommended content that conforms to the document structure and semantic logic is generated.

[0123] refer to Figure 4 This is a flowchart illustrating the process of obtaining preliminary prediction results through the chart analysis module.

[0124] It should be noted that traditional chart analysis has two limitations: First, it only extracts the text and values ​​from the chart, failing to establish a semantic relationship between "value-label-unit," such as whether "38%" represents "growth rate" or "market share." Second, it cannot handle visual overlap (such as obscuring the value labels in a bar chart) and scattered text and image descriptions (such as "see..."). Figure 1 (In the main text, "Data Source" is in the footnote). Therefore, this algorithm achieves structured parsing and semantic understanding of graphs through cross-modal alignment and graph reasoning.

[0125] like Figure 4 As shown, obtaining preliminary prediction results through the chart analysis module includes the following steps:

[0126] The chart parsing module performs syntactic parsing on the chart-related text to obtain semantic role labels, and maps these labels to semantic role vectors. Visual features of the chart image are extracted using ResNet. The semantic role vectors are used to drive deformable convolutions to adjust the receptive field of the visual features, aligning them. Based on the aligned semantic role vectors and visual features, an inference graph is constructed for the data points in the chart. Node features represent data points, and edge features integrate Euclidean distance, relative angle, and color similarity between data points. Node features are propagated and updated using a multi-layer graph neural network. Based on the updated node features of the inference graph, structured data is extracted, and a template engine is used to generate chart summaries that conform to the document context. These summaries include multi-granularity chart descriptions at the data, analysis, and decision layers.

[0127] In some optional embodiments, a mapping relationship between the visual features of the chart and the semantic roles of the text is first established. Specifically, the text associated with the chart (such as "...") is first mapped. Figure 1 The code displays "Q2 sales increased by 38%", performs syntactic parsing to generate semantic role tags, and maps each role to a vector. Simultaneously, visual features are extracted from the chart images using ResNet. Then, by combining the semantic role vector, the offset of each pixel in the (x,y) direction is calculated, and the receptive field of the visual features is adjusted by deformable convolution, thereby achieving accurate alignment between visual features and semantic roles.

[0128] Next, a geometric and semantic relationship graph is constructed based on the aligned features to address visual occlusion and relationship reasoning issues. Specifically, a fully connected graph is first constructed for the data points in the chart (such as bars in a bar chart and points in a line chart). Nodes are represented by feature vectors, and edge features comprehensively consider Euclidean distance, relative angle, and color similarity. Then, feature propagation is performed through a multi-layer graph neural network. When updating node features at each layer, the evolution of the node's own features is considered, as well as the aggregation of information from neighboring nodes. The formula is as follows:

[0129] ;

[0130] in It is the node feature of the i-th node at the l-th layer. It is a node self-updating weight. It is the node feature of the (l-1)th layer of the i-th node. It is the set of neighboring nodes of the i-th node. It is an edge feature. It is an edge mapping matrix. It is an edge filter. It is the node feature of the (l-1)th layer of the j-th neighbor node in the neighbor node set.

[0131] It should be understood that this reasoning mechanism allows inferences to be made based on the features of adjacent nodes even when some data points are occluded.

[0132] Finally, based on the inferred node features, a structured data table containing fields such as numerical values, tags, units, and sources is extracted. On this basis, a template engine is used to generate multi-granular chart descriptions: from numerical details at the data layer, to trend interpretation at the analysis layer, and then to in-depth analysis at the decision-making layer, ultimately outputting chart summaries that conform to the document context. This hierarchical description mechanism allows the generated summaries to meet the needs of different users, from basic data browsing to in-depth business analysis.

[0133] The above is based on Figure 2 , Figure 3 , Figure 4 The specific implementation of S103 is described. First, a spatiotemporal convolutional dynamic decision algorithm is used to break through the traditional menu triggering mode. By fusing micro-features of cursor displacement with macro-statistics of operational habits (such as the variance of deletion frequency), a dynamic heterogeneous graph is constructed to predict intent in real time, reducing latency. Then, the document is deconstructed into a semantic graph of title-paragraph-chart, and structural constraints are injected through a graph attention mechanism to ensure a deep fit between the recommended content and the document's logical framework, thus constructing a graph constraint condition generation architecture. Finally, based on semantic role labels driving deformable convolution to align text and image features, and combining geometric relationships and visual attributes to construct a data point reasoning graph, an end-to-end mapping from chart structured parsing to summary generation is achieved, designing cross-modal reasoning relationships.

[0134] Furthermore, in some embodiments, in S104, the interaction rules include: when the probability distribution of the user's next operation is a chart operation, the target text is chart-related text, and the semantic similarity between the target text and the chart summary is greater than a preset threshold. Specifically, the chart parsing module extracts information such as data, trends, and tags from the chart, and the text generated by the content recommendation module should accurately reflect this information without contradictions or inconsistencies.

[0135] In other words, the initial results from the three modules are mutually optimized through a collaborative interaction mechanism to reduce the error of independent predictions. For example, if the probability of the intent prediction result being "insert chart" is greater than a certain threshold, the content recommendation module will prioritize generating text related to the chart and increase the chart parsing module's focus on data visualization features; if the chart parsing extracts changes in the data, the intent prediction module will increase the probability of related operations such as "insert data description" and "adjust chart title"; the text generated by the content recommendation must be logically consistent with the chart parsing result, and if they are inconsistent, regeneration will be triggered.

[0136] In addition, in some optional embodiments, a collaborative constraint loss is added to the model unification. During the model optimization process, the model parameters are updated based on the total loss (the weighted sum of the losses of each task, with the weights dynamically adjusted) to minimize the loss value, thereby improving the performance of the three modules simultaneously.

[0137] In this way, through the system collaboration mechanism, the three modules no longer work in isolation, but form an organic whole by sharing features and interaction mechanisms, which significantly improves the efficiency and accuracy of intelligent document processing.

[0138] In addition, refer to Figure 5 This application also provides another intent-aware document processing method.

[0139] like Figure 5 As shown in the embodiments of this application, another intent-aware document processing method may include the following steps:

[0140] To achieve deep integration of the three modules of intent prediction, content recommendation, and chart parsing, this case study constructs an end-to-end process from three aspects: data processing, shared feature extraction, and multi-task interaction, thus unifying the entire model. Through multi-module collaboration and deep integration, leveraging the "shared feature extraction + module collaborative interaction" mechanism, the bottleneck of cross-modal semantic association is overcome, achieving deep coupling and collaborative reasoning of text, charts, and user behavior. The intent prediction module, through multi-scale spatiotemporal coding and dynamic heterogeneous graph design, achieves accurate capture of implicit behavior, significantly reducing prediction latency, and continuously learns user operating habits to improve personalized prediction accuracy. The content recommendation module, based on document structure graph and graph constraint generation mechanism, ensures that recommended content is deeply aligned with the document topology, dynamically adapting to structural changes during document editing, and improving the logical consistency and user adoption rate of recommended content. The chart parsing module, through cross-modal alignment and graph reasoning, achieves deep parsing of chart data and multi-granularity summary generation, effectively solving the collaborative analysis problem of visual overlap and scattered text-image descriptions, significantly releasing data value. The end-to-end collaboration of the three modules further reduces user operating costs, reduces redundant operations and rework rates, and comprehensively improves the overall efficiency and accuracy of document editing, especially suitable for scenarios with high rigor requirements such as government documents and scientific research papers.

[0141] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0142] Based on the same technical concept, corresponding to any of the above embodiments, this application also provides an intent-aware document processing device 600.

[0143] like Figure 6 As shown, the intent-aware document processing device 600 may include:

[0144] The conversion module 601 is used to convert text data, image data and behavior data into text features, image features and behavior features respectively through the corresponding pre-trained models;

[0145] The splicing module 602 is used to splice the text features, image features and behavioral features and then map them to a shared feature space through a fully connected layer to obtain the document-behavior-visual fusion shared features;

[0146] The intent prediction module 603, the content recommendation module 604, and the chart parsing module 605 are used to perform preliminary predictions on the shared features respectively, and obtain the preliminary prediction results corresponding to each of the three modules. The preliminary prediction result of the intent prediction module is the probability distribution of the user's next operation, the preliminary prediction result of the content recommendation module is the target text, and the preliminary prediction result of the chart parsing module is the chart summary.

[0147] The optimization module 606 is used to dynamically optimize the preliminary prediction results according to the preset interaction rules, so as to obtain the optimized target text and the optimized chart summary to generate the target document.

[0148] The interaction rules include: when the probability distribution of the user's next operation is a chart operation, the target text is chart-related text, and the semantic similarity between the target text and the chart summary is greater than a preset threshold.

[0149] Optionally, the behavioral data includes the user's cursor trajectory information, operation event information, and document context information, wherein the operation event information includes short-term high-frequency operation events and long-term low-frequency operation events; and the behavioral features include short-term features and long-term features.

[0150] Optionally, the step of converting text data, image data, and behavioral data into text features, image features, and behavioral features respectively through corresponding pre-trained models includes: using a gated recurrent unit (GRU) to extract micro-features from the cursor trajectory information and short-term high-frequency operation event information to obtain short-term features; and using a sliding window and a multilayer perceptron (MLP) to extract macro-features from the long-term low-frequency operation events to obtain long-term features.

[0151] It should be noted that, for ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0152] The apparatus of the above embodiments is used to implement the corresponding intent-aware document processing method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0153] Based on the same technical concept, corresponding to any of the above embodiments, this application also provides an electronic device.

[0154] Figure 7 A schematic diagram of a more specific electronic device hardware structure provided in this embodiment is shown.

[0155] The electronic device 700 may include a processor 701 and a memory 702 storing computer program instructions.

[0156] Specifically, the processor 701 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0157] Memory 702 may include mass storage for data or instructions. For example, and not limitingly, memory 702 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 702 may include removable or non-removable (or fixed) media. Where appropriate, memory 702 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 702 is non-volatile solid-state memory.

[0158] In certain embodiments, the memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Thus, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this application.

[0159] The processor 701 reads and executes computer program instructions stored in the memory 702 to implement any of the intent-aware document processing methods described above.

[0160] In some examples, the electronic device 700 may also include a communication interface 703 and a bus 710. For example, Figure 7 As shown, the processor 701, memory 702, and communication interface 703 are connected through bus 710 and complete communication with each other.

[0161] The communication interface 703 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0162] Bus 710 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not as a limitation, bus 710 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 710 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0163] For example, the electronic device 700 can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc.

[0164] Based on the same technical concept, corresponding to any of the methods in the above embodiments, this application also provides a non-transitory computer-readable storage medium. This computer-readable storage medium stores computer program instructions; when executed by a processor, these computer program instructions implement any of the intent-aware document processing methods in the above embodiments. Examples of computer-readable storage media include non-transitory computer-readable storage media such as portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, etc.

[0165] Based on the same technical concept, corresponding to any of the above embodiments, this application also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processor to perform the intent-aware document processing method. Corresponding to the execution entity for each step in each embodiment of the intent-aware document processing method, the processor executing the corresponding step may belong to the corresponding execution entity.

[0166] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0167] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0168] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0169] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0170] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. An intent-aware document processing method, characterized in that, include: Text data, image data, and behavioral data are transformed into text features, image features, and behavioral features respectively through corresponding pre-trained models; The text features, image features, and behavioral features are concatenated and then mapped to a shared feature space through a fully connected layer to obtain the document-behavior-visual fusion shared features. The shared features are initially predicted by the intent prediction module, the content recommendation module, and the chart parsing module, and the initial prediction results of each of the three modules are obtained. The initial prediction result of the intent prediction module is the probability distribution of the user's next operation, the initial prediction result of the content recommendation module is the target text, and the initial prediction result of the chart parsing module is the chart summary. The preliminary prediction results are dynamically optimized according to pre-set interaction rules to obtain optimized target text and optimized chart summary to generate target document; The interaction rules include: when the probability distribution of the user's next operation is a chart operation, the target text is chart-related text, and the semantic similarity between the target text and the chart summary is greater than a preset threshold. The behavioral data includes user cursor trajectory information, operation event information, and document context information, wherein the operation event information includes short-term high-frequency operation events and long-term low-frequency operation events; the behavioral features include short-term features and long-term features. The process of converting text data, image data, and behavioral data into text features, image features, and behavioral features respectively through corresponding pre-trained models includes: The cursor trajectory information and short-term high-frequency operation event information are extracted using a gated recurrent unit (GRU) to obtain short-term features; the long-term low-frequency operation events are extracted using a sliding window and a multilayer perceptron (MLP) to obtain long-term features.

2. The method according to claim 1, characterized in that, The shared features are initially predicted using the intent prediction module, content recommendation module, and chart parsing module, yielding preliminary prediction results for each of the three modules, including: The intent prediction module constructs a dynamic heterogeneous graph based on the short-term and long-term features. The nodes of the dynamic heterogeneous graph are obtained by splicing and normalizing the short-term and long-term features. The edge weights of the dynamic heterogeneous graph are calculated by combining the semantic correlation between nodes and the time decay factor. The node information of the dynamic heterogeneous graph is aggregated based on the graph attention mechanism, and the aggregated node features are fused with the document context semantic vector to output the probability distribution of the user's next operation.

3. The method according to claim 2, characterized in that, The construction of the dynamic heterogeneous graph based on the short-term and long-term features includes: The short-time and long-time features are concatenated and normalized according to the following formula to obtain the vector corresponding to node i of the dynamic heterogeneous graph. : ； Where LayerNorm represents layer normalization; This represents a learnable weight matrix used to concatenate short-term features. and long-term features Mapped to node space; The edge weights of a dynamic heterogeneous graph are calculated using the following formula. : ； in, This represents the first weight matrix. This represents the second weight matrix. Let d represent the vector corresponding to node j, and let d represent the dimension of the attention head. This represents the time interval between node i and node j. Indicates the time decay coefficient. The scale constant represents the rate of decay of the time-dependent correlation of the control operation, and T represents the transpose.

4. The method according to claim 3, characterized in that, The process involves aggregating node information from the dynamic heterogeneous graph using a graph attention mechanism, fusing the aggregated node features with the document context semantic vector, and outputting the probability distribution of the user's next operation. This includes: The features of the aggregated nodes are fused with the document context semantic vector according to the following formula to output the user's next action. probability distribution : ； in, This represents the aggregated node features at the current time. Represents a document context semantic vector. This represents the output weight matrix. This represents the context mapping matrix, and GELU is the activation function.

5. The method according to claim 1, characterized in that, The shared features are initially predicted using the intent prediction module, content recommendation module, and chart parsing module, yielding preliminary prediction results for each of the three modules, including: The content recommendation module deconstructs the document into title nodes, paragraph nodes, and chart nodes, and establishes hierarchical relationship edges, reference relationship edges, and semantic similarity edges between the title nodes, paragraph nodes, and chart nodes to obtain the document structure graph. The hierarchical relationship edge represents the parent-child relationship between title nodes, the reference relationship edge represents the paragraph node referencing the chart node, and the semantic similarity edge represents the semantic association between paragraph nodes. Using the user's operation intent vector as the query, retrieve the corresponding candidate cases from the pre-accessed external knowledge base, and calculate the association weight between the candidate cases and the nodes in the document structure graph; Based on the association weights and the document structure graph node features updated by the graph neural network (GNN), graph constraint features are generated. The graph constraint features are input into the generative model, and the generative model is decoded to obtain the target text.

6. The method according to claim 5, characterized in that, The calculation of the association weights between the candidate cases and the nodes in the document structure graph includes: The association weights between the candidate cases and the nodes in the document structure graph are calculated using the following formula. : ； in, Represents the user's action intent vector. This represents the j-th node in the document structure graph. Represents a node The document structure graph node features are updated using a graph neural network (GNN). This represents the attention weight matrix, where k represents the loop index during summation. This represents the k-th node in the document structure graph. Document structure graph node features updated using graph neural network (GNN); The step of generating graph-constrained features based on the association weights and the document structure graph node features updated by the graph neural network (GNN) includes: Generate graphical constraint features based on the following formula. : ； Where j represents the node index.

7. The method according to claim 1, characterized in that, The shared features are initially predicted using the intent prediction module, content recommendation module, and chart parsing module, yielding preliminary prediction results for each of the three modules, including: The chart parsing module performs syntactic parsing on the chart-related text to obtain semantic role labels, and maps the semantic role labels to semantic role vectors; Visual features of chart images are extracted using ResNet; The receptive field of the visual feature is adjusted by using the semantic role vector to drive deformable convolution, so that the semantic role vector and the visual feature are aligned. Based on the aligned semantic role vectors and visual features, an inference graph is constructed for the data points in the chart. The node features represent the data points, and the edge features integrate the Euclidean distance, relative angle, and color similarity between the data points. The node features are propagated and updated through a multi-layer graph neural network. Based on the updated node features of the inference graph, structured data is extracted, and a template engine is used to generate a chart summary that conforms to the document context. The chart summary includes multi-granular chart descriptions of the data layer, analysis layer, and decision layer.

8. An intent-aware document processing device, characterized in that, The device includes: The transformation module is used to convert text data, image data, and behavioral data into text features, image features, and behavioral features respectively through the corresponding pre-trained models. The splicing module is used to splice the text features, image features and behavioral features and then map them to the shared feature space through a fully connected layer to obtain the document-behavior-visual fusion shared features; The intent prediction module, content recommendation module, and chart parsing module are used to perform preliminary predictions on the shared features respectively, and obtain the preliminary prediction results corresponding to each of the three modules. The preliminary prediction result of the intent prediction module is the probability distribution of the user's next operation, the preliminary prediction result of the content recommendation module is the target text, and the preliminary prediction result of the chart parsing module is the chart summary. The optimization module is used to dynamically optimize the preliminary prediction results according to the pre-set interaction rules, so as to obtain the optimized target text and the optimized chart summary to generate the target document; The interaction rules include: when the probability distribution of the user's next operation is a chart operation, the target text is chart-related text, and the semantic similarity between the target text and the chart summary is greater than a preset threshold. The behavioral data includes user cursor trajectory information, operation event information, and document context information, wherein the operation event information includes short-term high-frequency operation events and long-term low-frequency operation events; the behavioral features include short-term features and long-term features. The process of converting text data, image data, and behavioral data into text features, image features, and behavioral features respectively through corresponding pre-trained models includes: The cursor trajectory information and short-term high-frequency operation event information are extracted using a gated recurrent unit (GRU) to obtain short-term features; the long-term low-frequency operation events are extracted using a sliding window and a multilayer perceptron (MLP) to obtain long-term features.

9. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; the processor, when invoking the computer program instructions, implements the intent-aware document processing method as described in any one of claims 1-7.

10. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the intent-aware document processing method as described in any one of claims 1-7.

Images