Document generation method and system integrating document analysis and cognitive reasoning

Abstract

The invention provides an official document generation method and system fusing document analysis and cognitive reasoning, and the method comprises the steps: carrying out the multi-modal feature extraction of an input original document through employing an analysis engine combining a convolutional neural network and a graph attention network, and obtaining the multi-modal data in the original document; based on the multi-modal data, generating a classification decision result of the multi-modal data by using a hierarchical classifier enhanced by a knowledge graph; based on the multi-modal data and the classification decision result, generating a task decomposition result by combining a cognitive inference engine; and based on the classification decision result and the task decomposition result of the multi-modal data, generating an official document by combining compliance constraint with an official document format template, so that multi-modal accurate analysis of a complex document can be realized, and dynamic semantic understanding and classification are realized by utilizing a hierarchical classifier enhanced by a knowledge graph. The classification accuracy in the fuzzy semantic scene can be improved, so that the recognition error rate can be reduced, and the accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent document processing, and in particular to a method and system for generating official documents by fusing document analysis and cognitive reasoning. BACKGROUND

[0002] Official document generation is widely used in various scenarios. For example, when important information needs to be conveyed to subordinate units or employees, it can be done by making notice, announcement and other documents. In the process of project implementation, it may be necessary to report project progress regularly, and at this time, official documents can be used to create project reports and progress reports. These scenarios demonstrate the practicality of official document generation in different fields, which not only saves time, but also improves work efficiency and document quality.

[0003] In the traditional scheme, a process for generating official documents based on OCR (Optical Character Recognition) character recognition to extract official document elements is proposed. First, a scanner is used to convert paper official documents into electronic files. The coordinate region positioning method of pre-defined rectangular region coordinates is used to extract element information, and the text rule positioning method based on keyword rules is used to position the requirements. Finally, the extracted elements are stored in the system form field. However, this method only supports static templates such as pre-defined coordinate regions or fixed keyword rules, and cannot handle dynamic content such as handwritten annotations and nested tables. In addition, fixed keywords cannot handle semantically ambiguous content, and semantic understanding is rigid, resulting in high error rate and low accuracy. SUMMARY

[0004] In order to overcome the defects of the above-mentioned traditional scheme, such as the inability to handle dynamic content such as handwritten annotations and nested tables using static templates, and the inability to handle semantically ambiguous content using fixed keywords, and the rigidity of semantic understanding, resulting in high error rate and low accuracy, the present application provides a method for generating official documents by fusing document analysis and cognitive reasoning, comprising:

[0005] An analysis engine combining convolutional neural network and graph attention network is used to extract multi-modal features from the input original document, and obtain multi-modal data in the original document;

[0006] Based on the multi-modal data, a hierarchical classifier enhanced by a knowledge graph is used to generate a classification decision result of the multi-modal data; based on the multi-modal data and the classification decision result, a task decomposition result is generated by combining a cognitive reasoning engine;

[0007] Based on the classification decision result and the task decomposition result of the multi-modal data, an official document is generated by combining compliance constraints and an official document format template.

[0008] Optionally, the analysis engine combining convolutional neural network and graph attention network extracts multi-modal features of the input original document to obtain multi-modal data in the original document, including:

[0009] The language representation model pre-trained extracts structured data from the input original document to obtain structured data;

[0010] The improved YOLOv5 model and layout analysis algorithm extract unstructured data from the original document to obtain unstructured data;

[0011] The feature alignment module maps the structured data and the unstructured data in space to obtain multi-modal data.

[0012] Optionally, the improved YOLOv5 model and layout analysis algorithm extract unstructured data from the original document to obtain unstructured data, including:

[0013] The improved YOLOv5 model detects the table frame, handwritten area and seal contour in the original document, and outputs a visual feature vector with coordinates;

[0014] The OpenCV-based contour detection algorithm analyzes the layout of the original document to generate a version topology graph;

[0015] The multi-modal data satisfies the following formula:

[0016] F fuision =Softmax(W v V i +W t T e +W l L a )

[0017] Wherein, F fuision represents the multi-modal data, V i represents the visual feature vector, T e represents the text embedding result corresponding to the structured data, L a represents the version topology graph, W v , w t , W l is the weight matrix obtained by dynamic optimization through back propagation, and Softmax() represents the space mapping algorithm.

[0018] Optionally, the hierarchical classifier enhanced by the knowledge graph generates a classification decision result of the multi-modal data based on the multi-modal data, including:

[0019] The hierarchical classifier enhanced by the knowledge graph is used to perform context-aware feature transmission based on the multi-modal data, and attention coefficients of each node in the official document specification knowledge graph are calculated.

[0020] The hierarchical classifier enhanced by the knowledge graph is used to calculate semantic similarity of the multi-modal data based on the multi-modal data and the attention coefficients of each node, in combination with a dynamic threshold.

[0021] Based on the semantic similarity, a hierarchical result with a confidence score is output.

[0022] According to the semantic similarity, a directory level expansion is triggered to obtain a decision path graph.

[0023] According to the hierarchical result and the decision path graph, a classification decision result is determined.

[0024] Optionally, based on the multi-modal data and the classification decision result, a task decomposition result is generated by using a cognitive reasoning engine in combination.

[0025] Based on the multi-modal data and the classification decision result, a probability distribution of a task decomposition path is predicted by using a cognitive reasoning engine in combination.

[0026] Based on the probability of the task decomposition path, a reinforcement learning algorithm is used to optimize a strategy, and recursive task decomposition is performed until an atomic task granularity meets a termination condition, and a current decomposition result is taken as a task decomposition result.

[0027] Optionally, based on the classification decision result and the task decomposition result of the multi-modal data, an official document is generated by using a compliance constraint in combination with an official document format template.

[0028] Based on the classification decision result and the task decomposition result of the multi-modal data, a format syntax tree is verified by using a compliance constraint.

[0029] If the verification result is compliant, an official document is generated by using an official document format template based on the classification decision result and the task decomposition result of the multi-modal data.

[0030] Optionally, if the verification result is compliant, before the official document is generated by using the official document format template based on the classification decision result and the task decomposition result of the multi-modal data, the following steps are further included.

[0031] An improved practical Byzantine fault tolerance (PBFT) consensus algorithm is used, and the classification decision result and the task decomposition result are taken as a storage record to be chained, and a storage record index hash value with a timestamp is generated for the storage record.

[0032] Optionally, the storage record index hash value with a timestamp is generated for the storage record, including:

[0033] generating a timestamped evidence index hash value for the evidence based on the task identification, the timestamp and the node information corresponding to the classification decision result in the task decomposition result;

[0034] The evidence index hash value satisfies the following formula:

[0035] H index =SHA3-256(TaskID||Timestamp||MerkleRoot)

[0036] wherein H index represents the evidence index hash value, TaskID represents the task identification, Timestamp represents the timestamp, MerkleRoot represents the node information corresponding to the classification decision result, and SHA3 represents a hash algorithm.

[0037] In another aspect, the present application also provides a document generation system combining document analysis and cognitive reasoning, comprising:

[0038] a multi-modal analysis module, configured to use a convolutional neural network combined with a graph attention network to extract multi-modal features from an input original document, to obtain multi-modal data in the original document;

[0039] a hierarchical reasoning module, configured to use a knowledge graph enhanced hierarchical classifier to generate a classification decision result of the multi-modal data based on the multi-modal data, and use a cognitive reasoning engine to generate a task decomposition result based on the multi-modal data and the classification decision result;

[0040] a document generation module, configured to use a compliance constraint combined with a document format template to generate a document based on the classification decision result and the task decomposition result of the multi-modal data.

[0041] Optionally, the multi-modal analysis module comprises:

[0042] a structured data extraction unit, configured to use a pre-trained language representation model to extract structured data from the input original document;

[0043] an unstructured data extraction unit, configured to use an improved YOLOv5 model and a layout analysis algorithm to extract unstructured data from the original document;

[0044] a mapping unit, configured to use a feature alignment module to spatially map the structured data and the unstructured data to obtain multi-modal data.

[0045] Optionally, the unstructured data extraction unit is specifically configured to:

[0046] The YOLOv5 model is improved to detect table lines, handwritten regions and seal contours in the original document, and output a visual feature vector with coordinates;

[0047] The original document is analyzed by an OpenCV-based contour detection algorithm to generate a version topology graph;

[0048] The multi-modal data satisfies the following formula:

[0049] F fuision =Softmax(W v V i +W t T e +W l L a )

[0050] Wherein, F fuision represents the multi-modal data, V i represents the visual feature vector, T e represents the text embedding result corresponding to the structured data, L a represents the version topology graph, W v , W t , W l are weight matrices obtained by dynamic optimization through back propagation, and Softmax() represents a spatial mapping algorithm.

[0051] Optionally, the hierarchical reasoning module is specifically configured to:

[0052] A hierarchical classifier enhanced by a knowledge graph is used to perform context-aware feature transmission based on the multi-modal data, and calculate attention coefficients of each node in the official document specification knowledge graph;

[0053] The hierarchical classifier enhanced by the knowledge graph is used to calculate the semantic similarity of the multi-modal data based on the multi-modal data and the attention coefficients of each node, combined with a dynamic threshold;

[0054] Based on the semantic similarity, a hierarchical result with a confidence score is output;

[0055] According to the semantic similarity, a directory level expansion is triggered to obtain a decision path graph;

[0056] According to the hierarchical result and the decision path graph, a classification decision result is determined.

[0057] Optionally, the hierarchical reasoning module is specifically configured to:

[0058] Based on the multi-modal data and the classification decision result, a probability distribution of a task decomposition path is predicted by combining a cognitive reasoning engine;

[0059] Based on the probability of the task decomposition path, a reinforcement learning algorithm is combined to optimize a strategy, recursive task decomposition is performed until an atomic task granularity meets a termination condition, and a current decomposition result is taken as a task decomposition result.

[0060] Optionally, the official document generation module is specifically configured to:

[0061] Based on the classification decision result and the task decomposition result of the multi-modal data, a format syntax tree is verified by using a compliance constraint.

[0062] If the verification result is compliant, an official document is generated based on the classification decision result and the task decomposition result of the multi-modal data by using an official document format template.

[0063] Optionally, the method further comprises:

[0064] The blockchain storage is used to store the classification decision result and the task decomposition result as a storage chain by using an improved practical Byzantine fault tolerance (PBFT) consensus algorithm, and a timestamped storage index hash value of the storage is generated.

[0065] Optionally, the blockchain storage is specifically configured to:

[0066] Based on the task identification, the timestamp and the node information corresponding to the classification decision result in the task decomposition result, a timestamped storage index hash value of the storage is generated.

[0067] The storage index hash value satisfies the following formula:

[0068] H index =SHA3-256(TaskID||Timestamp||MerkleRoot)

[0069] Wherein, H index represents the storage index hash value, TaskID represents the task identification, Timestamp represents the timestamp, MerkleRoot represents the node information corresponding to the classification decision result, and SHA3 represents a hash algorithm.

[0070] In another aspect, the present application also provides a computer device, comprising: one or more processors;

[0071] The processor is configured to store one or more programs.

[0072] When the one or more programs are executed by the one or more processors, the official document generation method of fusing document analysis and cognitive reasoning is realized.

[0073] In another aspect, the application also provides a computer readable storage medium, which has a computer program stored thereon, and the computer program, when executed, realizes the official document generation method of fusing document analysis and cognitive reasoning.

[0074] Compared with the prior art, the application has the following beneficial effects:

[0075] The application provides an official document generation method and system of fusing document analysis and cognitive reasoning. BRIEF DESCRIPTION OF DRAWINGS

[0076] Figure 1 FIG. 1 is a flowchart of the official document generation method of fusing document analysis and cognitive reasoning according to the application;

[0077] Figure 2 FIG. 2 is a flowchart of the automatic official document generation method of fusing document analysis and cognitive reasoning according to the application;

[0078] Figure 3 FIG. 3 is a content structure diagram of the organization management work expense use approval form according to the application;

[0079] Figure 4 FIG. 4 is a diagram of the YOLOv5 model detecting the signature region according to the application;

[0080] Figure 5 FIG. 5 is a diagram of the standardized directory structure according to the application;

[0081] Figure 6 FIG. 6 is a diagram of the reading process of the blockchain storage according to the application;

[0082] Figure 7A structural schematic diagram of the official document generation system fusing document analysis and cognitive reasoning of the application;

[0083] Figure 8 A structural schematic diagram of the electronic device of the application. DETAILED DESCRIPTION

[0084] The specific embodiments of the application will be further described in detail below with reference to the accompanying drawings.

[0085] Embodiment 1:

[0086] The application provides an official document generation method fusing document analysis and cognitive reasoning, as shown in the flowchart, which comprises the following steps: Figure 1 As shown in the flowchart, it comprises the following steps:

[0087] Step 101: using a parsing engine combining convolutional neural network and graph attention network, multi-modal feature extraction is performed on the input original document to obtain multi-modal data in the original document;

[0088] Step 102: based on the multi-modal data, a knowledge graph enhanced hierarchical classifier is used to generate a classification decision result of the multi-modal data; based on the multi-modal data and the classification decision result, a task decomposition result is generated by using a cognitive reasoning engine;

[0089] Step 103: based on the classification decision result and the task decomposition result of the multi-modal data, a compliance constraint is combined with a document format template to generate a document.

[0090] In the embodiment of the application, the parsing engine combining convolutional neural network and graph attention network is used to extract multi-modal features, which can realize multi-modal accurate analysis of complex documents, and the knowledge graph enhanced hierarchical classifier is used to realize dynamic semantic understanding and hierarchical classification, which can improve the classification accuracy in fuzzy semantic scenarios, thereby reducing the recognition error rate and improving the accuracy.

[0091] The parsing engine in step 101 adopts a hybrid architecture of convolutional neural network and graph attention network, which is used for multi-modal feature extraction of the input original document. The multi-modal features include structured data and unstructured data, for example, the structured information in the document includes but is not limited to text, and the unstructured information includes but is not limited to table frame, handwritten area, seal contour and document layout contour, etc. In this step, the parsing engine combines convolutional neural network and graph attention network to synchronously analyze text, table frame and handwritten visual features, and realizes multi-modal feature fusion analysis.

[0092] In an implementation manner, in the step 101, the pre-trained language representation model can be used to extract structured data from the input original document; the improved YOLOv5 model and the layout analysis algorithm can be used to extract unstructured data from the original document; and the feature alignment module can be used to spatially map the structured data and the unstructured data to obtain the multi-modal data.

[0093] The pre-trained language representation model can be a pre-trained BERT (Bidirectional Encoder Representations from Transformers) model, which can extract text semantic embeddings from the original document to obtain text embeddings T e The text embeddings are the extracted structured data.

[0094] For the non-structured information such as table line frames, handwritten regions, and seal contours, the improved YOLOv5 model can be used for monitoring, that is, when the improved YOLOv5 model and the layout analysis algorithm are used to extract unstructured data from the original document, the improved YOLOv5 model can be used to detect the table line frames, the handwritten regions, and the seal contours in the original document to output the coordinate-based visual feature vector V i The coordinate-based visual feature vector is the extracted unstructured data.

[0095] For the non-structured information such as document layout contours, the layout analysis algorithm can be used for detection, for example, but not limited to, an OpenCV-based contour detection algorithm. When the improved YOLOv5 model and the layout analysis algorithm are used to extract unstructured data from the original document, the OpenCV-based contour detection algorithm can be used to analyze the layout of the original document to generate a version topology graph L a The version topology graph is the extracted unstructured data.

[0096] In addition to integrating the above-mentioned various detection algorithms, the multi-modal document parsing engine can further include a feature alignment module, which can spatially map the structured data such as the text embeddings, the unstructured data such as the coordinate-based visual feature vector and the version topology graph, and the multi-modal data. For example, the multi-modal data satisfies the following formula:

[0097] F fuision = Softmax (W v V i + W t T e + W l L a )

[0098] wherein F fuision represents multi-modal data, V i represents visual feature vector, T e represents text embedding result corresponding to structured data, L a represents version topology graph, W v , W t , W l is a weight matrix obtained through back propagation dynamic optimization, and Softmax() represents a spatial mapping algorithm.

[0099] The multi-modal analysis can be realized through the multi-modal feature fusion mathematical modeling of the feature alignment function and the weight optimization mechanism in the implementation mode.

[0100] The above step 102 includes dynamic hierarchical classification and cognitive reasoning.

[0101] The dynamic hierarchical classification adopts a knowledge graph enhanced hierarchical classifier, which can be, for example, a graph neural network classifier such as a GAT (Graph Attention Network) model. In the above step 102, based on the multi-modal data, the knowledge graph enhanced hierarchical classifier is used to generate the classification decision result of the multi-modal data. The knowledge graph enhanced hierarchical classifier can be used to perform context-aware feature transmission based on the multi-modal data, calculate the attention coefficients of each node in the document specification knowledge graph, calculate the semantic similarity of the multi-modal data based on the multi-modal data and the attention coefficients of each node in combination with a dynamic threshold, output a hierarchical result with a confidence score based on the semantic similarity, trigger directory level expansion according to the semantic similarity to obtain a decision path graph, and determine the classification decision result according to the hierarchical result and the decision path graph.

[0102] The knowledge graph in the hierarchical classifier can be a document specification knowledge graph constructed based on GB / T 9704-2012 and other standards. The nodes of the knowledge graph contain entities such as document types and classification levels, and the edge relationship of the knowledge graph defines hierarchical constraints. The hierarchical classifier, such as the GAT model, can perform context-aware feature propagation and calculate node attention coefficients. For example, the node attention coefficients satisfy the following formula:

[0103]

[0104] wherein α ij represents the attention coefficient of node i to neighbor node j, h i , h j , h kRepresent the original feature vectors of node i, node j and node k respectively, W represents the learnable weight matrix in the graph attention network, Wh i Indicates that h is represented by the weight matrix i Do linear transformation, [Wh i ||Wh j ] represents the feature Wh of node i after linear transformation of weight matrix W i and node j feature Wh j Perform splicing operation (“||” is the splicing symbol), α T Represents the learnable parameter vector in the attention mechanism, which is used to do the inner product with the transformed node feature vector to calculate the basic value of the attention score. exp() represents the exponential function, and LeakyReLU() represents the activation function. It represents the sum of all neighbor nodes k of node i, and Ni represents the neighbor set of node i.

[0105] Furthermore, the hierarchical classifier incorporates a dynamic threshold mechanism that automatically triggers catalog expansion based on semantic similarity. Semantic similarity can be characterized by cosine similarity, with an example, but not limited to, a cosine similarity ≥ 0.7, automatically triggering catalog expansion. Based on this process, the hierarchical classifier outputs a classification result for confidence scoring and a decision path map, thereby determining the final classification decision.

[0106] The cognitive reasoning process in the above step 102 can be based on task decomposition by a cognitive reasoning engine, and the cognitive reasoning engine can be a reasoning system that develops rules and collaborates with deep learning. The cognitive reasoning engine can be pre-installed with a business rule template library, and can also be integrated with an LSTM (Long Short-Term Memory) network and a reinforcement learning algorithm, such as a PPO (Proximal Policy Optimization) algorithm. The LSTM network is used to predict the task decomposition sequence, and the rule triggering strategy is optimized through the reinforcement learning algorithm, and recursive task decomposition is performed until the atomic task granularity meets the termination condition. In this step, the dynamic cognitive reasoning engine constructs a collaborative decision-making mechanism between the business rule template library and the deep learning model, supporting dynamic expansion of the directory hierarchy and tracing of the reasoning path.

[0107] For example, in the above step 102, when the task decomposition result is generated based on the multimodal data and the classification decision result, the probability distribution of the task decomposition path can be predicted based on the multimodal data and the classification decision result, and the reinforcement learning algorithm optimization strategy is combined to perform recursive task decomposition until the atomic task granularity meets the termination condition, and the current decomposition result is used as the task decomposition result.

[0108] In an implementation manner, the termination condition can be that the atomic task granularity satisfies the following formula:

[0109] Complexity(Task i )<θ(θ=0.35)

[0110] wherein, Complexity(Task i ) denotes the complexity of the i-th task Task i , which is calculated by the number of dependent edges and resource demand weighting, θ denotes the complexity threshold, which is set to 0.35 in this case, when the complexity of the task is lower than the threshold, it is considered that the task is simple enough and does not need to be decomposed, and the threshold is pre-set, and it can be understood that in some cases, the complexity threshold θ can also be set to other values.

[0111] In this implementation manner, the dynamic hierarchical classification and collaborative decision-making architecture triggered by the knowledge graph + GAT + dynamic threshold can realize dynamic hierarchical classification and task decomposition.

[0112] In an implementation manner, in the step 103, the compliance constraint can be used for format syntax tree verification based on the classification decision result and the task decomposition result of the multi-modal data; if the verification result is compliant, the document format template is generated based on the classification decision result and the task decomposition result of the multi-modal data.

[0113] The compliance constraint can be generated based on a compliance constraint generation system to realize multi-dimensional generation quality control. The compliance constraint generation system can include a constructed format syntax tree verifier, which is used to define the constraint rules of the XML (Extensible Markup Language) format. The compliance constraint generation system can also include a developed hybrid generation strategy controller, which is used to enable template filling when the template matching degree is greater than 85%; switch to LLM (Large Language Model) free generation when the innovation demand weight is greater than 0.6; and start a hybrid rewriting mode for conflict content, and use an edit distance algorithm to optimize the transition coherence. The compliance constraint generation system designs a format syntax tree verification algorithm, which can detect 200+ format elements such as title level and seal position in real time.

[0114] In this implementation manner, the full-process control system for compliance generation includes: format syntax tree + hybrid generation strategy + semantic detection, which can realize compliance generation control.

[0115] In some scenarios, a blockchain storage module can be deployed to record key nodes of the smart contract generation process to achieve audit traceability. For example, a trusted storage chain can be built based on Hyperledger Fabric (a design of a blockchain network), a storage data structure can be designed, an improved PBFT (Practical Byzantine Fault Tolerance) consensus algorithm can be used, key nodes such as classification decisions and rule triggering records can be chained, and a storage index hash value with a timestamp can be generated. In an example, before the above-mentioned document is generated based on the classification decision result and the task decomposition result of the multi-modal data, the improved PBFT consensus algorithm can be used to chain the classification decision result and the task decomposition result as storage, and a storage index hash value with a timestamp can be generated for the storage.

[0116] In an implementation, when generating a storage index hash value with a timestamp for the storage, the storage index hash value with a timestamp can be generated for the storage based on the task identification in the task decomposition result, the timestamp, and the node information corresponding to the classification decision result. In an example, the storage index hash value satisfies the following formula:

[0117] H index =SHA3-256(TaskID∣∣Timestamp∣∣MerkleRoot)

[0118] wherein H index represents the storage index hash value, TaskID represents the task identification, Timestamp represents the timestamp, MerkleRoot represents the node information corresponding to the classification decision result, and SHA3 represents a hash algorithm.

[0119] The embodiment of the application also provides a cross-scenario migration adapter for cross-scenario migration of the document generation method provided by the embodiment of the application to achieve dynamic loading of domain knowledge. The cross-scenario migration adapter is constructed with a domain adaptation rule mapping table, for example, the mapping relationship between the official business domain and the medical domain shown in Table 1.

[0120] Table 1

[0121] Public Service Area Medical Area "Request" "Medical Plan Application" "Financial Budget" "Medical Equipment Procurement"

[0122] A more specific mapping relationship can be shown in Table 2.

[0123] Table 2

[0124] Public Service Element Medical Element Conversion Rule Work Conference Resolution Ethical Review Opinion Approval Agency Mapping Union Dues Collection Details Medical Expense List Financial Field Conversion Work Group Member Information Table Patient Medical Record Number ID Encryption Conversion

[0125] The transfer learning fine-tuning module can be integrated in the cross-scene transfer adapter, and the module can perform local parameter update based on a domain corpus (>=100,000 pieces):

[0126]

[0127] Where, theta new represents the new parameters of the model after adaptation in the target scene, theta base represents the basic parameters of the pre-trained model, and eta represents the learning rate, represents the gradient operator of the calculation parameter theta, that is, the "direction" of parameter update, D target represents the corpus data of the target scene, L(theta;D target ) represents a loss function, which is used to calculate the error of the basic model in the target scene data D target . The learning rate eta is dynamically adjusted by using the cosine annealing strategy.

[0128] Here, through the blockchain data structure + domain adaptation rule mapping, trusted traceability and migration can be realized.

[0129] The embodiment of the application breaks through the linear processing paradigm of the traditional scheme and constructs a full-chain intelligent system of "analysis-cognition-reasoning-generation", that is, a full-process system architecture: a multi-modal analysis engine, a dynamic hierarchical classification module, a cognitive reasoning engine, a compliance generation system, and a blockchain storage module. In the organization and management of the scene, the processing efficiency of complex documents is improved by 3 times, and the format compliance rate reaches 98.5%, which is significantly better than the traditional technology.

[0130] Next, taking the organization and management of the scene as an example, the method for generating the document according to the embodiment of the application is described, as shown in Figure 2 .

[0131] Step 1: multi-modal document analysis engine processing.

[0132] 1.1 Input the original document: the input document is 《Organization and Management of Working Expenses Approval Form.docx》(Word format). The content structure is as shown in Figure 3 .

[0133] Structured data: application time (year / month / day), application department / organization unit name, activity content (budget details, person in charge information, etc.).

[0134] Unstructured data: 5 signature areas (person in charge, department head, organization head, etc.), attachment documents (activity plan.docx, personnel list.xlsx, etc.).

[0135] 1.2 Multi-modal analysis, that is, obtaining structured data and unstructured data through the analysis engine:

[0136] Structured data extraction includes: (1) Using a pre-trained BERT model to parse text fields (such as "application time" and "budget details"), extract semantic information, and convert it into a structured data table; (2) Parsing budget tables (amounts, expenditure items) to generate standardized financial data formats.

[0137] Unstructured data extraction includes: (1) Signature area positioning: Based on the improved YOLOv5 model, detect the signature box coordinates, and combine OpenCV contour analysis to accurately crop the signature area (positioning error ≤ 3px), such as Figure 4 YOLOv5 model detects table areas, handwritten notes, and organization-specific seal radii in the original document; (2) Attachment analysis: Joint analysis of associated attachments (activity plan.docx, personnel list.xlsx) to extract key information (such as activity time, participant list).

[0138] Step 2: Multi-modal document parsing engine processing.

[0139] 2.1 Knowledge graph construction:

[0140] Signature flow knowledge graph: (1) Node definition: application department, approval role (person in charge → responsible person), budget item, attachment type. (2) Edge relationship: Define the dependency relationship of the approval process (such as "department head signature" is a prerequisite for "organization head signature").

[0141] Budget rule base: Embed relevant provisions of the "Organization Activity Fund Management Measures" to constrain budget allocation logic.

[0142] 2.2 Task decomposition and rule triggering:

[0143] Signature integrity verification: Detect whether the five signature areas are complete, and if missing, trigger a re-signing reminder task.

[0144] Budget specification verification: Check whether the total budget meets the department standard (such as single activity ≤ 100,000 yuan), and trigger budget adjustment task if exceeded.

[0145] Attachment relevance verification: Verify the matching of the activity plan and the personnel list (such as whether the number of participants exceeds the plan's preset size).

[0146] Third step: Compliance constraint generation and verification.

[0147] 3.1 Generation module:

[0148] Generate signature flow: According to the signature order rules (person in charge → responsible person), automatically generate an electronic signature flowchart, and mark the status of each node (signed / to be signed).

[0149] Generate budget statement: call Jinja2 template engine to fill in the budget details table, dynamically bind the amount of data, and generate a statement document that meets the financial specifications.

[0150] Generate attachment directory: extract attachment metadata (filename, page number, key content summary), generate a standardized directory structure, such as Figure 5 The target agencies of the subsequent consultation files include activity profiles, expense details, and security plan requirements.

[0151] 3.2 Format syntax tree verification:

[0152] Signature sequence compliance: Check if the signature process meets the level sequence of "person in charge → department head → organization head".

[0153] Budget table format: Verify table title font (small Song Simplified Chinese), amount unit (RMB symbol "￥"), and number alignment (right-aligned).

[0154] Attachment association rules: Check if the attachment page numbers are consecutive and the file types meet the requirements (such as prohibiting the upload of non-PDF attachments).

[0155] Step 4: Blockchain notarization and output.

[0156] 4.1 Notarization data packaging:

[0157] The protobuf of the notarization data structure is as follows:

[0158]

[0159]

[0160] Notarization process: Use the improved PBFT consensus algorithm to chain key data such as the signature process and budget verification results, generating an immutable notarization record.

[0161] For example Figure 6 After the notarization is chained, during subsequent reading, the decision path can be queried in the blockchain by inputting the hash, returning the format verification log based on the decision path, outputting the audit report, and achieving traceability.

[0162] 4.2 Final document generation:

[0163] Integrate the signature process, budget statement, and attachment directory to generate a "Organizational Special Work Expense Approval Result Letter.docx" according to GB / T 9704 standards, which includes: (1) standard document header (issue number, classified level), (2) main text (approval conclusion, budget reply details), (3) signature page (electronic signature and seal position), and (4) attachment directory (hyperlink to original attachments).

[0164] Fifth step: cross-scene migration adaptation.

[0165] 5.1 Rule mapping and adaptation: for example, organization management -> official scene migration as shown in Table 3:

[0166] Table 3

[0167]

[0168] When the technology is implemented: load the official field knowledge graph, dynamically replace the approval role and financial terms; parameter fine-tuning (learning rate η = 0.001) is performed on the generated template to adapt to the official document format (such as adding a "copy unit" field).

[0169] In the embodiment of the application, a multi-modal feature fusion analysis engine is constructed, a convolution attention mechanism is used to realize joint analysis of table line frame detection and handwriting recognition; a knowledge graph enhanced dynamic hierarchical classification model is developed, the directory hierarchy is reconstructed through context perception; a scalable rule template library and a cognitive reasoning engine based on deep learning are designed to realize task decomposition that fits business; an algorithm for verifying format syntax trees and detecting semantic consistency is innovated to ensure that the generated text format compliance rate reaches the industry leading level; a blockchain storage module is integrated synchronously to build a whole-process traceability system. Compared with the traditional scheme, the present technology breaks through the whole-chain bottleneck of multi-modal cognition to business generation, and the processing efficiency is improved by more than 3 times, providing a high-reliability solution for intelligent document processing in the fields of organization management, official business, etc.

[0170] The technical effects of the official document generation method provided by the embodiment of the application will be illustrated by comparing with the traditional technical scheme. The closest technical scheme to the present application in the traditional technical scheme is CN112149679A, a method and device for extracting official document elements based on OCR text recognition, which includes the following steps: (1) document scanning and electrization: converting paper official documents into electronic files through a scanner; (2) dynamic algorithm selection: generating a selection box of coordinate region positioning method or text rule positioning method: coordinate region positioning method: predefining rectangular region coordinates (such as title located at 35mm on the top of the page), extracting element information; text rule positioning method: positioning elements based on keyword rules (such as unit name after "main send:"); (3) element extraction and storage: storing the extracted "issuing" and "handling unit" elements into the system form field.

[0171] The traditional technical scheme is mainly based on the text filling technology of fixed templates, which is difficult to cope with the challenges of complex document analysis, dynamic business adaptation and format compliance guarantee. It mainly relies on the simple concatenation of OCR text extraction, rule engine and pre-set template library, and has the following defects in the systematic technology of multi-modal document understanding, dynamic task decomposition and business compliance guarantee, including:

[0172] (1) Rely on static templates, complex document element analysis is not complete, multi-modal analysis ability is insufficient: only support pre-defined coordinate area or fixed keyword rules, cannot process handwritten notes, nested tables and other dynamic content, the detection rate is high, the analysis ability of nested tables, handwritten notes, seal superposition and other composite elements in Excel is insufficient, the single text modal processing strategy is difficult to capture the layout characteristics and semantic association, and the error of layout structure restoration easily leads to the loss of key information, resulting in distortion of document structure restoration, affecting the reliability of subsequent processing.

[0173] Compared with problem (1), the official document generation method provided by the embodiment of the application can realize multi-modal accurate analysis of complex documents: in view of the element missing problem of nested tables, handwritten notes and the like caused by the dependence of the prior art on static templates, the application constructs a multi-modal feature fusion analysis engine, extracts text semantics, table line frame and handwritten visual features through a convolution attention network (CNN+Attention), establishes the topological association between the layout and the content by using a spatial alignment algorithm, the target restores the nested table structure with an accuracy of ≥96%, the handwritten note recognition rate is increased to 89%, and the defect that the missing rate of the traditional OCR scheme is greater than 40% is overcome.

[0174] (2) Semantic understanding and task decomposition are rigid and limited: the conventional RAG (Retrieval-augmented Generation, retrieval-augmented generation) system relies on a static rule base when classifying directories, keyword matching cannot handle semantic ambiguity content (such as confusion between “funds-related matters” and “budget execution clauses”), and the self-adaptive ability is missing when facing dynamic changes in business scenarios, it is difficult to dynamically adapt to changes in business scenarios, the classification accuracy drops sharply in a fuzzy semantic matching scenario, and there is no interpretable reasoning path; the traditional CoT (Chain of Thought, chain of thought) method does not fuse business rule constraints, and the task decomposition result deviates from the requirements of official documents.

[0175] Compared with problem (2), the official document generation method provided by the embodiment of the application can construct a dynamic semantic understanding and hierarchical system: to solve the semantic classification errors caused by rigid rule matching, a dynamic hierarchical model enhanced by a knowledge graph is designed. Based on a graph neural network (GNN), a knowledge graph of official documents (including GB / T 9704 standard entities and relationships) is fused, and a context perception mechanism is used to realize adaptive expansion of directory levels (supporting 6 levels of nesting), the classification accuracy in a fuzzy semantic scenario is improved to 88%.

[0176] (3) Generation quality and format compliance out of control, out of line with standard specifications: mainstream LLM generation schemes often have problems such as format misalignment and logical discontinuity, lack of closed-loop verification mechanism, and generally lower format compliance rate than industry requirements. Template filling leads to format misalignment (such as incorrect title font, incorrect page number, incorrect title level), and when processing unstructured supplementary information, key information is easily missed and logical discontinuity occurs. Lack of closed-loop quality verification mechanism, unable to meet the strict format specifications of organizational management, legal and other scenarios, and lack of dynamic adjustment capability.

[0177] Compared with problem (3), the official document generation method provided by the embodiment of the application can establish a full-process compliance generation constraint mechanism: for the problems of title misalignment and seal deviation caused by the lack of format verification, a format syntax tree verification algorithm is developed, 200+ structured format rules (such as title font, seal positioning tolerance ±5px) are defined, combined with semantic consistency detection model, three-level control of pre-generation verification, generation correction and post-generation review is realized, and the format compliance rate is increased to 98%.

[0178] Moreover, the official document generation method provided by the embodiment of the application can optimize the task decomposition and generation logic coherence: the information discontinuity problem of linear process is solved by recursive atomization processing, LSTM is used to predict task decomposition sequence, combined with reinforcement learning to optimize rule triggering strategy, to ensure that the extraction completeness of unstructured supplementary information reaches 95%, the logical discontinuity rate is reduced to 5%, and the single document processing time is shortened to 5 minutes.

[0179] (4) Lack of cross-scene migration and process traceability: the demand generation process lacks credible evidence, existing systems lack domain knowledge guidance mechanism, the professional degree of the generated results is insufficient, and a credible traceability chain is not established, making it difficult to adapt to the audit compliance needs of organizational management, public affairs, medical and other professional scenarios. And because the decision path is not recorded in the generation process, the efficiency of post-audit is low.

[0180] Compared with problem (4), the official document generation method provided by the embodiment of the application can build a credible audit traceability technology base: design a blockchain evidence storage module to solve the problem of process data loss. Based on the improved PBFT consensus algorithm, record the key nodes such as classification decision path and rule triggering log, generate a timestamped evidence index hash, and the target is to compress the audit traceability response time to 0.5 seconds, and the error troubleshooting efficiency is increased to 98%.

[0181] After the above comparison, compared with the traditional technical scheme, the technical means shown in Table 4 is adopted to solve the defects of the traditional technical scheme.

[0182] Table 4

[0183]

[0184]

[0185] It can be seen that the above defects are all derived from the technical faults of the traditional technical solutions in the aspects of multi-modal cognition, dynamic reasoning and closed-loop verification, and the embodiments of the present application realize a systematic breakthrough of these defects by constructing a collaborative technology system, thereby improving the document processing accuracy, efficiency and compliance rate.

[0186] Embodiment 2:

[0187] Based on the same inventive concept, the present application also provides a document generation system integrating document parsing and cognitive reasoning, a structural schematic diagram of which is shown in Figure 7 , and includes:

[0188] A multi-modal parsing module is configured to use a parsing engine combining a convolutional neural network and a graph attention network to perform multi-modal feature extraction on an input original document, so as to obtain multi-modal data in the original document.

[0189] A hierarchical reasoning module is configured to use a hierarchical classifier enhanced by a knowledge graph to generate a classification decision result of the multi-modal data based on the multi-modal data, and use a cognitive reasoning engine to generate a task decomposition result based on the multi-modal data and the classification decision result.

[0190] A document generation module is configured to use a compliance constraint in combination with a document format template to generate a document based on the classification decision result and the task decomposition result of the multi-modal data.

[0191] Optionally, the multi-modal parsing module includes:

[0192] A structured data extraction unit is configured to use a pre-trained language representation model to perform structured data extraction on the input original document, so as to obtain structured data.

[0193] An unstructured data extraction unit is configured to use an improved YOLOv5 model and a layout analysis algorithm to perform unstructured data extraction on the original document, so as to obtain unstructured data.

[0194] A mapping unit is configured to use a feature alignment module to perform spatial mapping on the structured data and the unstructured data, so as to obtain multi-modal data.

[0195] Optionally, the unstructured data extraction unit is specifically configured to:

[0196] Detect the table frame, handwritten area and seal contour in the original document by using the improved YOLOv5 model, and output a visual feature vector with coordinates;

[0197] Perform layout analysis on the original document based on an OpenCV-based contour detection algorithm, and generate a version topology graph.

[0198] The multi-modal data satisfies the following formula:

[0199] F fuision =Softmax(W v V i +W t T e +W l L a )

[0200] Wherein, F fuision represents the multi-modal data, V i represents the visual feature vector, T e represents the text embedding result corresponding to the structured data, L a represents the version topology graph, W v , W t , W l are weight matrices obtained by dynamic optimization through back propagation.

[0201] Optionally, the hierarchical reasoning module is specifically configured to:

[0202] use the hierarchical classifier enhanced by the knowledge graph to perform context-aware feature transmission based on the multi-modal data, and calculate attention coefficients of each node in the official document specification knowledge graph;

[0203] use the hierarchical classifier enhanced by the knowledge graph to calculate the semantic similarity of the multi-modal data based on the multi-modal data and the attention coefficients of each node, in combination with a dynamic threshold;

[0204] based on the semantic similarity, output hierarchical results with confidence scores;

[0205] trigger directory level expansion according to the semantic similarity to obtain a decision path graph;

[0206] determine a classification decision result according to the hierarchical results and the decision path graph.

[0207] Optionally, the hierarchical reasoning module is specifically configured to:

[0208] based on the multi-modal data and the classification decision result, use the cognitive reasoning engine to predict the probability distribution of the task decomposition path;

[0209] based on the probability of the task decomposition path, combine a reinforcement learning algorithm optimization strategy to perform recursive task decomposition until the atomic task granularity meets the termination condition, and take the current decomposition result as the task decomposition result.

[0210] Optionally, the official document generation module is specifically configured to:

[0211] based on the classification decision result and the task decomposition result of the multi-modal data, use the compliance constraints to perform format syntax tree verification;

[0212] If the verification result is compliant, a document is generated based on the classification decision result and the task decomposition result of the multi-modal data and the document format template.

[0213] Optionally, the method further comprises:

[0214] Optionally, the method further comprises:

[0215] Optionally, the method further comprises:

[0216] Optionally, the method further comprises:

[0217] Optionally, the method further comprises:

[0218] H index =SHA3-256(TaskID||Timestamp||MerkleRoot)

[0219] wherein H index represents the storage evidence index hash value, TaskID represents the task identifier, Timestamp represents the timestamp, and MerkleRoot represents the node information corresponding to the classification decision result.

[0220] Embodiment 3

[0221] As shown in FIG. Figure 8 The electronic device in this embodiment can include a processor, a memory, a transceiver component, and the like. The memory, the processor, and the transceiver component are connected through a bus; the memory can be used to store an execution program, and the example execution program can include instructions; the processor is used to execute the instructions stored in the memory. The memory can also be used to store data, which can be called and / or modified when the instructions are executed.

[0222] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, and the like, which are a computing core and a control core of the terminal, and are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in the storage medium to implement a corresponding method flow or a corresponding function, so as to implement the steps of the document generation method of fusing document parsing and cognitive reasoning in the above embodiment.

[0223] Embodiment 4:

[0224] Based on the same inventive concept, the application further provides a readable storage medium, specifically an electronic device readable storage medium (Memory). The electronic device readable storage medium is a memory device in the electronic device, and is used for storing programs and data. It can be understood that the storage medium herein can include a built-in storage medium in the electronic device, and of course can also include an expansion storage medium supported by the electronic device. The storage medium provides a storage space, and the storage space stores an operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more execution programs (including program codes). It should be noted that the storage medium herein can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory. The processor loads and executes one or more instructions stored in the storage medium, and the steps of the document generation method of fusing document parsing and cognitive reasoning in the above embodiment can be implemented.

[0225] Those skilled in the art should understand that the embodiments of the application can be provided as a method, a system, or a computer program product. Therefore, the application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the application can adopt the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0226] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart Figure 1 one or more flows and / or blocks. Figure 1 one or more flows and / or blocks.

[0227] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart Figure 1 one or more flows and / or blocks. Figure 1 one or more flows and / or blocks.

[0228] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart Figure 1 one or more flows and / or blocks. Figure 1 one or more flows and / or blocks.

[0229] Finally, it should be noted that the above embodiments are merely used to illustrate the technical solutions of the present application, rather than limiting the scope of protection of the present application. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that: after reading the present application, those skilled in the art can make various changes, modifications or equivalent replacements to the specific embodiments of the application, but these changes, modifications or equivalent replacements are all within the scope of protection of the claims of the present application.

Claims

1. A document generation method integrating document parsing and cognitive reasoning, characterized in that: The method comprises the following steps: An analysis engine combining a convolutional neural network and a graph attention network is used to extract multi-modal features of an input original document to obtain multi-modal data in the original document; Based on the multi-modal data, a hierarchical classifier enhanced by a knowledge graph is used to generate a classification decision result of the multi-modal data; Based on the multi-modal data and the classification decision result, a task decomposition result is generated by combining a cognitive reasoning engine; Based on the classification decision result and the task decomposition result of the multi-modal data, a document is generated by combining compliance constraints with a document format template.

2. The method of claim 1, wherein, The analysis engine combining the convolutional neural network and the graph attention network is used to extract multi-modal features of the input original document to obtain multi-modal data in the original document, which comprises the following steps: The input original document is subjected to structured data extraction by a pre-trained language representation model to obtain structured data; The original document is subjected to unstructured data extraction by an improved YOLOv5 model and a layout analysis algorithm to obtain unstructured data; The structured data and the unstructured data are subjected to spatial mapping by a feature alignment module to obtain multi-modal data.

3. The method of claim 2, wherein, The original document is subjected to unstructured data extraction by an improved YOLOv5 model and a layout analysis algorithm to obtain unstructured data, which comprises the following steps: The table lines, handwritten regions and seal contours in the original document are detected by the improved YOLOv5 model to output a visual feature vector with coordinates; The original document is subjected to layout analysis based on an OpenCV contour detection algorithm to generate a version topology graph; The multi-modal data satisfies the following formula: F fuision = Softmax(W v V i + W t T e + W l L a ) wherein F fuision represents the multi-modal data, V i represents the visual feature vector, T e represents the text embedding result corresponding to the structured data, L a represents the version topology graph, W v , W t , W l is a weight matrix obtained by dynamic optimization through back propagation, and Softmax() represents a spatial mapping algorithm.

4. The method according to any one of claims 1 to 3, characterized in that, The hierarchical classifier enhanced by the knowledge graph is used to perform context-aware feature transmission based on the multi-modal data to calculate attention coefficients of each node in the knowledge graph of the document specification; The hierarchical classifier enhanced by the knowledge graph is used to calculate the semantic similarity of the multi-modal data based on the multi-modal data and the attention coefficients of each node in combination with a dynamic threshold; Based on the semantic similarity, a hierarchical result with a confidence score is outputted; According to the semantic similarity, a directory level expansion is triggered to obtain a decision path graph; According to the hierarchical result and the decision path graph, a classification decision result is determined. Based on the multi-modal data and the classification decision result, a task decomposition result is generated by combining a cognitive reasoning engine, which comprises the following steps:

5. The method according to any one of claims 1 to 3, wherein Based on the multi-modal data and the classification decision result, a probability distribution of a task decomposition path is predicted by combining the cognitive reasoning engine; Based on the probability of the task decomposition path, a reinforcement learning algorithm is used to optimize the strategy to perform recursive task decomposition until the atomic task granularity meets the termination condition, and the current decomposition result is taken as the task decomposition result. Based on the classification decision result and the task decomposition result of the multi-modal data, a document is generated by combining compliance constraints with a document format template, which comprises the following steps:

6. The method according to any one of claims 1 to 3, wherein ​ Based on the classification decision result and the task decomposition result of the multi-modal data, a format syntax tree is verified by using compliance constraints; If the verification result is compliant, a document is generated based on the classification decision result and the task decomposition result of the multi-modal data by using a document format template.

7. The method of claim 6, wherein, If the verification result is compliant, before the document is generated based on the classification decision result and the task decomposition result of the multi-modal data by using the document format template, the method further includes: The classification decision result and the task decomposition result are stored in a blockchain by using an improved practical Byzantine fault tolerance (PBFT) consensus algorithm, and a timestamped storage index hash value of the storage is generated.

8. The method of claim 7, wherein, The timestamped storage index hash value of the storage includes: Based on the task identifier, the timestamp, and the node information corresponding to the classification decision result in the task decomposition result, the timestamped storage index hash value of the storage is generated. The storage index hash value satisfies the following formula: H index = H3-256(TaskId || Timestamp || MerkleRoot) wherein H index represents the evidence storage index hash value, TaskID represents a task identification, Timestamp represents a timestamp, MerkleRoot represents node information corresponding to the classification decision result, and H3 represents a hash algorithm.

9. A system for generating a document by fusing document parsing and cognitive reasoning, characterized in that, The method includes: A multi-modal analysis module is configured to perform multi-modal feature extraction on an input original document by using a convolutional neural network combined with a graph attention network to obtain multi-modal data in the original document; A hierarchical reasoning module is configured to generate a classification decision result of the multi-modal data by using a knowledge graph enhanced hierarchical classifier based on the multi-modal data; Based on the multi-modal data and the classification decision result, a task decomposition result is generated by using a cognitive reasoning engine; A document generation module is configured to generate a document by using a compliance constraint combined with a document format template based on the classification decision result and the task decomposition result of the multi-modal data.

10. An electronic device, comprising: The method includes: At least one processor and a memory; The memory and the processor are connected through a bus; The memory is configured to store one or more programs; When the one or more programs are executed by the at least one processor, the method for generating a document by fusing document analysis and cognitive reasoning according to any one of claims 1-8 is implemented.

Images