OFD file intelligent abstract generation method fusing OCR and deep learning

Abstract

The application discloses an OFD file intelligent abstract generation method fusing OCR and deep learning, relates to the technical field of electronic file processing and management, and preliminarily scans text and tables through image recognition technology to identify fuzzy areas; for the areas containing handwriting, image enhancement is combined with semantic analysis to restore the clarity; then, seal and signature key elements are extracted, layout analysis is used to divide the document area and judge the relevance; if the preset condition is met, the information is integrated through a computer vision model to determine the overall logical structure; then, a machine learning model is applied to analyze the semantic relationship to form a structured information set; finally, a concise abstract containing the writing date and the year is generated by fusing the visual model. The application ensures efficient extraction and structured presentation of complex document information, and significantly improves the accuracy and practicability of document processing.

Description

Technical Field

[0001] This invention relates to the field of electronic document processing and management technology, and in particular discloses a method for generating intelligent summaries of OFD documents that integrates OCR and deep learning. Background Technology

[0002] In the information age, the processing and management of electronic documents has become an indispensable part of the information society. Current technologies often fall short when dealing with complex electronic document formats. Many methods are prone to information gaps or omissions when processing documents containing text, tables, seals, or even handwritten content. Especially when the document image quality is low or the content is diverse, traditional recognition methods struggle to fully capture the overall logic and details of the document, resulting in incomplete information extraction and impacting subsequent application effectiveness.

[0003] A deeper challenge lies in understanding the document's layout and accurately locating key information. As the core organizational form of document content, the layout directly determines the relevance and importance ranking of information. Without accurately analyzing the layout patterns, it's difficult to distinguish between core content and secondary information. This lack of analytical ability means that even if some text is identified, the document's logical structure cannot be truly reconstructed. For example, in a scanned official document, key elements such as the title, document number, and date of issuance may be distributed in different locations and may be obscured by blurring or handwriting. Without considering the overall layout to determine the relationships between these elements, a complete understanding of the document's content is impossible.

[0004] Therefore, how to accurately analyze the layout structure of a document under conditions of complex layout and poor image quality, and extract logically clear and complete key information based on this, has become a critical problem that urgently needs to be solved. Summary of the Invention

[0005] This invention provides a method for intelligent summarization of OFD documents that integrates OCR and deep learning, aiming to solve the problem of processing complex layout content in electronic documents.

[0006] This invention relates to a method for intelligent summarization of OFD documents that integrates OCR and deep learning, comprising the following steps: S100. Obtain complex layout content from electronic documents, and perform preliminary scanning of text and tables in complex layout content using image recognition technology to identify blurred areas. Complex layout content includes multi-level text layouts and mixed graphic elements. S200. If the identified blurred area contains handwritten characters, then image enhancement processing combined with semantic analysis is used to restore the clarity of the blurred area, and the enhanced image data is obtained. S300. Extract key elements of seal and signature from the enhanced image data, use layout analysis technology to divide the document regions in the enhanced image data, and determine the correlation between document regions. S400. If the correlation between document regions meets the preset conditions, the key elements and document region information extracted are integrated through a computer vision model to determine the overall logical structure of the document. S500. Based on the determined overall logical structure, apply a machine learning model to analyze the semantic relationship between the title and document number elements in the overall logical structure, and obtain a set of structured key information. S600: For a set of structured key information, a visual model is integrated to generate a concise text summary, and the output is a complete description including the date and year of writing.

[0007] Further, step S100 includes: S110. Extract complex layout content from electronic documents to generate an initial layout image, wherein the initial layout image contains multi-level text layouts and mixed graphic elements. S120. Perform a preliminary scan on the initial layout image to obtain the first feature set; S130. Generate a contrast feature matrix based on the first feature set; S140. If the value in the contrast feature matrix is ​​lower than the preset threshold, then construct the fuzzy region coordinate matrix and determine the identified fuzzy region through the fuzzy region coordinate matrix.

[0008] Further, step S200 includes: S210. If the identified blurred area contains handwritten content, then the blurred area is denoised to obtain a handwritten binarized image. S220. Extract the skeleton from the handwritten binary image and construct the character structure feature vector; S230. Input the character structure feature vector into the semantic analysis model to generate a semantic correction mask; S240. Use a semantic correction mask to perform pixel-level reconstruction of the handwritten binary image to obtain enhanced image data.

[0009] Further, step S300 includes: S310. Acquire the enhanced image data and separate the red component region and the high curvature stroke connected region to determine the set of key elements of the seal and signature. S320. Map the set of key elements of the seal and signature onto the page segmentation diagram to obtain multiple independent document areas; S330. Extract the geometric centroid coordinates and high-dimensional semantic vector of the document region, and construct the association probability matrix based on the geometric centroid coordinates and high-dimensional semantic vector; S340. Traverse the correlation probability matrix. If the element value in the correlation probability matrix is ​​greater than the preset threshold, output the correlation determination result between document regions.

[0010] Further, step S400 includes: S410. Obtain the document region correlation determination result that meets the preset conditions, and extract the text flow characteristics and spatial constraint information within the document region. S420. Construct a document attribute map based on text flow characteristics and spatial constraint information, and identify logical anchor points in the document attribute map that contain key elements of seals and signatures. S430. Perform semantic topological analysis on logical anchors to map document regions into a topological sequence with a sequential order. S440. Analyze the topological sequence and divide it into semantic block clusters using a computer vision model. If the association weights of the semantic block clusters meet the preset logical hierarchy, determine the overall logical structure of the document based on the entity relationships between the semantic block clusters.

[0011] Further, step S500 includes: S510. Locate the title entity and document number entity based on the determined overall logical structure, and extract the contextual embedding representation of the title entity and document number entity. S520. The context embedding representation is used to generate entity feature vectors containing semantic dependency paths from the input sequence labeling model, and an association probability matrix reflecting the degree of semantic alignment is constructed for the entity feature vectors. S530. If the correlation probability matrix satisfies the constraints of the preset logical level labels, then the text content of the title entity and the document number entity is aggregated according to the mapped structured field definition to obtain a set of structured key information.

[0012] Further, step S600 includes: S610. Obtain the original document image associated with the structured key information set, and extract the local visual feature map covering the key area; S620. Obtain the text string of the date of writing and the year value based on the parsing of local visual feature maps; S630. Combine the text string of the date of writing, the annual value, and the semantic information of the title in the structured key information set to construct a comprehensive feature vector; S640. Input the comprehensive feature vector into the decoder to generate an initial summary text sequence, and embed the text string of the writing date and the year value into the initial summary text sequence, and output a complete description containing the writing date and year.

[0013] The beneficial effects achieved by this invention are as follows: 1. This invention provides an intelligent OFD document summarization method integrating OCR and deep learning. Addressing the challenge of processing complex layouts in electronic documents, it integrates multi-level text layout, mixed graphic element recognition, and blurred region recovery, forming a logically coherent overall solution—that is, how to extract key information from complex document layouts and generate structured summaries. This invention initially scans text and tables using image recognition technology to identify blurred areas; for areas containing handwriting, image enhancement combined with semantic analysis is used to restore clarity; subsequently, key elements such as seals and signatures are extracted, and layout analysis is used to divide document areas and determine their relevance; if preset conditions are met, information is integrated using a computer vision model to determine the overall logical structure; then, a machine learning model is applied to analyze semantic relationships, forming a structured information set; finally, a concise summary containing the date and year of creation is generated by integrating the visual model.

[0014] 2. The most prominent innovation of this invention lies in the integration of multiple technologies, ensuring efficient extraction and structured presentation of complex document information, significantly improving the accuracy and practicality of document processing. The entire process technical indicators are verifiable. Test conditions: A test set containing 1000 OFD files, covering official documents (400), contracts (300), and archives (300), of which 30% are fuzzy documents (clarity index 0.3~0.6), 20% contain handwritten text, and 50% have complex layouts (mixed text and images / multi-level headings). Evaluation indicators: Fuzzy region recognition accuracy ≥92%, handwritten text recognition accuracy ≥88%, region segmentation accuracy ≥93%, key information completeness ≥94%, summary accuracy ≥88% (cosine similarity between generated and manually generated summaries ≥0.8), and format standardization rate 100%. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating an embodiment of the OFD document intelligent summarization method integrating OCR and deep learning according to the present invention. Detailed Implementation

[0016] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0017] like Figure 1 As shown, the first embodiment of the present invention proposes a method for intelligent summarization of OFD documents that integrates OCR and deep learning, including the following steps: Step S100: Obtain complex layout content from the electronic document. Use image recognition technology to perform a preliminary scan of the text and tables in the complex layout content to identify the blurred areas. The complex layout content includes multi-level text layouts and mixed graphic elements.

[0018] This step involves scanning complex layouts and locating blurry areas in OFD (Open Fixed-layout Document) files. The process involves acquiring the OFD electronic file to be processed (a layout file supporting multi-level layouts, embedded graphics, etc.), and extracting the complex layout content. This content specifically includes multi-level text layouts (such as heading levels: first-level heading, second-level heading, body text, with 3-8 levels) and mixed graphic elements (such as illustrations, stamps, signatures, tables, flowcharts, etc.).

[0019] The multi-level text layout is divided into 3 to 8 levels, based on font attributes and layout metadata. The core judgment indicators are "font size + line spacing + bold / italic": Level 1 (heading): font size ≥ 20, line spacing ≥ 1.5, bold; Levels 2-3 (subheading): font size 16-18, line spacing 1.2-1.5, optional bold; Levels 4-8 (body text / footnotes): font size ≤ 14, line spacing 1.0-1.2, no special formatting.

[0020] OCR (Optical Character Recognition) image recognition technology (such as an OCR recognition model based on CNN (Convolutional Neural Network)) is used to perform a comprehensive preliminary scan of text (including printed text and potentially handwritten text) and tables in complex layout content. The scanning resolution is set to 300~600 dpi (to ensure detailed recognition accuracy), and the scanning speed is 1~5 pages / second (dynamically adjusted according to the file size). Through pixel grayscale value analysis and sharpness detection algorithms (such as the variance method), the sharpness index of the scanned area is calculated. When the sharpness index is <0.6 (the value range is 0~1, with 1 being the sharpest), it is determined to be a blurry area. Finally, all identified blurry areas are determined, and the accuracy rate of blurry area recognition is ≥92%, providing targeted objects for subsequent sharpening processing.

[0021] OFD files, or Layout File Format, are electronic document format standards independently developed in my country. They support multi-level text layouts and the embedding of mixed graphic elements (seals, signatures, tables, etc.). They are characterized by stable format, immutability, and cross-platform compatibility, and are widely used in official documents, contracts, archives, and other scenarios.

[0022] Complex layout content refers to a collection of structurally complex content contained in an OFD file, specifically encompassing multi-level text layouts (multi-level headings, body text) and mixed graphic elements (illustrations, stamps, signatures, tables, etc.), which is different from simple files with a single text layout.

[0023] Step S200: If the identified blurred area contains handwritten characters, then image enhancement processing combined with semantic analysis is used to restore the clarity of the blurred area, and the enhanced image data is obtained.

[0024] This step involves handwritten text blurring and image enhancement. The blurry areas identified in step S100 are analyzed for content, categorizing them as handwritten text blur, printed text blur, or graphic blur. If the blurry area contains handwritten text (handwritten text recognition accuracy ≥ 88%), image enhancement techniques combined with semantic analysis are used to restore the clarity of the blurry area. 1. Image enhancement processing: Adaptive histogram equalization, Gaussian filtering for noise reduction, and sharpening (sharpening intensity parameter value is 0.3~0.8) are used to eliminate blur and noise and improve the clarity of handwritten strokes; 2. Semantic analysis method: Combining the semantics of the context text and a database of common handwritten characters, blurry strokes are completed and corrected (completion accuracy ≥ 85%).

[0025] After processing, enhanced image data is obtained. The sharpness index of the blurred area after enhancement is ≥0.8, ensuring the accuracy of subsequent key element extraction and region segmentation. The resolution of the enhanced image remains unchanged at 300~600dpi.

[0026] Blurred areas are regions in the OCR scanning process where recognition accuracy decreases (clarity index < 0.6) due to insufficient image clarity, noise interference, illegible handwriting, etc. They are divided into three categories: handwritten blur, printed blur, and graphic blur.

[0027] Step S300: Extract key elements of seal and signature from the enhanced image data, use layout analysis technology to divide the document regions in the enhanced image data, and determine the correlation between document regions.

[0028] This step involves key element extraction and document region segmentation. From the enhanced image data obtained in step S200, an object detection algorithm (such as the YOLO model) is used to accurately extract two types of key elements: seals and signatures, with an extraction accuracy of ≥90%. 1. Seal elements: Identify the outline, text content, and color of the seal (red is commonly used, recognition threshold: red channel grayscale value ≥ 200), and record the seal position coordinates (accuracy ≤ 2 pixels); 2. Signature elements: Identify the stroke outline and writing trajectory of the signature, distinguish between handwritten and printed signatures, and record the signature position and size (size error ≤ 5%).

[0029] Simultaneously, layout analysis techniques (such as a deep learning-based document layout analysis model) are employed to divide the enhanced image data into multiple independent document regions, including title regions, body text regions, table regions, seal / signature regions, and graphic regions, with a region segmentation accuracy of ≥93%. Through text semantic similarity calculation (similarity threshold ≥0.7) and positional correlation analysis, the logical relationships between each document region are determined (e.g., the relationship between the title region and its corresponding body text region, and the relationship between the table region and its surrounding text).

[0030] Layout analysis technology is a document processing technique based on computer vision and deep learning. It is used to divide the image data of OFD files into multiple independent document regions (title, body text, tables, etc.) and analyze the position and logical relationship of each region to provide support for the construction of the document's logical structure.

[0031] Step S400: If the correlation between document regions meets the preset conditions, the information of the extracted key elements and document regions is integrated through a computer vision model to determine the overall logical structure of the document.

[0032] This step determines the overall logical structure of the document. Preset conditions for the relationships between document regions are defined, with the following specific values: 1. Semantic similarity threshold: ≥0.7 (for determining semantic association between regions); 2. Location distance threshold: ≤50 pixels (for determining location correlation between regions); 3. Threshold for the number of associated regions: A single core region (such as the title region) is associated with ≥2 subordinate regions.

[0033] If the correlation between document regions meets any of the above preset conditions (the core region must meet at least two conditions), then the computer vision model (such as Faster R-CNN (Faster Region-based Convolutional Neural Network) model) is activated. The key elements of the seal and signature extracted in step S300, as well as the content information, location information, and semantic information of each document region, are integrated to construct the hierarchical structure and logical relationship graph of the document. The subordinate relationship of each region is clarified (such as the first-level heading belonging to the main text, and the table belonging to the corresponding paragraph of the main text) and the distribution of core content. Finally, the overall logical structure of the document is determined (such as the logical order of "heading → main text → table → seal and signature"). The accuracy rate of logical structure determination is ≥91%.

[0034] The overall logical structure is based on the correlation between different areas of the document and the distribution of key elements. It constructs the hierarchical relationship and logical order of the content, clarifies the subordinate relationship of each area and the distribution of core content, and serves as the core basis for extracting key information and generating summaries.

[0035] Step S500: Based on the determined overall logical structure, apply a machine learning model to analyze the semantic relationship between the title and document number elements in the overall logical structure to obtain a structured key information set.

[0036] This step is the structured key information extraction stage. Based on the overall logical structure determined in step S400, the title elements (such as document title, chapter title) and document number elements (such as document number, document number code) in the document are located, and the text content and location information of these two types of elements are extracted.

[0037] The title and document number elements are input into a machine learning model (such as BERT (Bidirectional Encoder Representations from Transformers) or LSTM (Long Short-Term Memory)). The machine learning model is trained on a large number of OFD document samples (≥10,000 samples). It analyzes the semantic relationships between the two types of elements (synonymous, subordinate, and corresponding relationships), filters redundant information, retains core content, and generates a structured key information set. This structured key information set includes: title, document number, core chapter summary, seal information, and signature information. The completeness of key information in the set is ≥94%, ensuring the relevance and accuracy of subsequent summary generation.

[0038] A structured key information set is a collection of core information with clear semantics and standardized format extracted from the overall logical structure of a document. It mainly includes information such as title, document number, core chapter summary, seal, and signature, and has the characteristics of completeness and structure.

[0039] Step S600: For the structured key information set, integrate the visual model to generate a concise text summary, and output a complete description including the date of writing and the year.

[0040] This step is the intelligent summary generation and complete output stage. Based on the structured key information set obtained in step S500, the computer vision model used in step S300 is integrated to extract the core semantics from the key information. A text generation algorithm (such as the Transformer generation model) is then used to generate a concise and coherent text summary. The summary length is controlled between 50 and 300 words (dynamically adjusted according to the length of the original document: original document ≤ 10 pages, summary 50-100 words; original document 11-50 pages, summary 100-200 words; original document > 50 pages, summary 200-300 words).

[0041] Text summaries are concise and coherent text descriptions generated by text generation algorithms, integrating structured key information. The length is controlled between 50 and 300 words. The core is to extract the core content of the document, while also including the date of writing and year information to ensure traceability.

[0042] The generated text summary is required to include the date of creation (format YYYY-MM-DD, recognition accuracy ≥95%) and year information (extracted from the date of creation, error ≤1 year) to ensure the completeness and traceability of the summary. The final output is a complete description containing the text summary, date of creation, and year information, with a text summary accuracy ≥88% and a format compliance rate of 100%, meeting the practical requirements for OFD document summarization.

[0043] Furthermore, the OFD document intelligent summarization generation method integrating OCR and deep learning provided in this embodiment includes step S100 as follows: S110. Extract complex layout content from electronic documents to generate an initial layout image, wherein the initial layout image contains multi-level text layouts and mixed graphic elements.

[0044] The initial layout image is derived using the following formula: (1) In formula (1), The initial layout image (a two-dimensional pixel matrix, with a size of...) ); This is an operator for extracting content from OFD files. It is used to parse the layout and graphic data of OFD files. Based on the parsing operators of the national standard GB / T 33190-2016 for OFD files, it parses the original binary data stream of OFD files, extracts layout metadata (text level, graphic coordinates, font attributes) and pixel data, and outputs an RGB three-channel two-dimensional pixel matrix with the same resolution as the original OFD file. It supports multi-layer fusion parsing. The raw data stream (binary format) of the OFD electronic document; The format metadata of the OFD file (including text hierarchy, graphic coordinates, etc.); The initial layout image height (in pixels). 3 represents the initial layout image width (unit: pixels); 3 represents the RGB three channels. The control logic of formula (1) is the core logic of OFD file layout visualization, and its core value is: 1. Complete layout restoration: It integrates the original data stream and layout metadata to accurately restore multi-level text layout and mixed graphic elements, avoiding the loss of structural information in pure text extraction; 2. Standardized output: It is converted into a unified RGB pixel matrix, which is compatible with the input format of general OCR and deep learning models; 3. Strong compatibility: The operator adapts to different OFD file encoding rules and supports the parsing of complex layouts (multiple columns, nested tables, and mixed text and images).

[0045] For electronic documents such as PDF contract documents, whose complex layouts include nested tables and illustrations, it is necessary to extract and generate an initial layout image. For example, parsing a PDF file into an initial layout image in bitmap format may involve a multi-level text layout with hierarchical structures of headings, body text, and footnotes, interspersed with graphic elements such as flowcharts and icons.

[0046] S120. Perform a preliminary scan on the initial layout image to obtain the first feature set.

[0047] The first feature set is obtained by the following formula: (2) In formula (2), This is the first feature set (convolutional feature map); This is a 3×3 convolution operator (the core operation for the initial scan). A 3×3 convolution kernel (weight matrix, dimensionless); For the feature space ( The height of the convolutional feature map; The width of the convolutional feature map; (This is the number of feature channels, which is 64 by default). The control logic of formula (2) is the underlying feature scanning logic of OFD complex layout images. Its core value is: 1. Preliminary scanning and feature extraction: Using lightweight 3×3 convolution, the full-domain scanning of the initial layout image is completed quickly to extract the most basic visual primitives; 2. Multi-channel feature expression: The single-channel grayscale / RGB image is converted into a multi-channel feature map, which enriches the feature dimension and provides more sufficient information carrier for subsequent semantic understanding; 3. Layout structure perception: The edge information of text and graphics is preserved, which lays the foundation for distinguishing the title level and recognizing the mixed layout of text and graphics.

[0048] The initial layout image is scanned using an optical character recognition engine to extract a first set of features, including pixel grayscale values ​​and edge contour points. Specifically, this scanning process involves traversing the image pixels line by line, calculating local gradients to identify text blocks and graphic boundaries, thereby obtaining a feature set as the basis for subsequent analysis.

[0049] S130. Generate a contrast feature matrix based on the first feature set.

[0050] The contrast feature matrix is ​​generated using the following formula: (3) In formula (3), The contrast feature matrix (a two-dimensional matrix, with dimensions equal to...) Consistent); Image pixel coordinates (unit: pixels); for The pixel value at the coordinates (RGB channel value ranges from 0 to 255); for The average pixel value (dimensionless) of the neighborhood (5×5). for The standard deviation of pixels in the neighborhood (dimensionless). The minimum value is used (default is 10−6) to avoid the denominator being 0. The control logic of formula (3) is the local contrast enhancement logic of OFD complex layout images. Its core value is: 1. Region adaptive normalization: Based on the mean and standard deviation of the 5×5 neighborhood, it is standardized to adapt to document backgrounds with different brightness and eliminate the influence of uneven lighting; 2. Key feature enhancement: It amplifies the feature values ​​of high contrast areas such as text and lines, weakens flat background areas, and improves the signal-to-noise ratio of subsequent feature extraction; 3. Robustness guarantee: Through the minimum value To avoid zero denominator, ensure the stability of the algorithm in extremely uniform regions such as all white / all black.

[0051] The process of generating a contrast feature matrix based on the first feature set first maps the feature set to a two-dimensional grid, for example, dividing the image into multiple sub-blocks, calculating the average contrast for each sub-block, which is the statistical value of the gray level difference between adjacent pixels, to form a contrast feature matrix, where the matrix elements represent the sharpness index of each region.

[0052] S140. If the value in the contrast feature matrix is ​​lower than the preset threshold, then construct the fuzzy region coordinate matrix and determine the identified fuzzy region through the fuzzy region coordinate matrix.

[0053] The coordinate matrix of the fuzzy region is constructed using the following formula: (4) In formula (4), This is the coordinate matrix of the blurred region (containing the coordinates of all blurred pixels); The mask matrix for the blurred region is a Boolean type, where 1 = blurred and 0 = clear. The control logic of formula (4) is based on the contrast feature matrix. It uses a threshold to determine the location of the blurred region and uses the mask matrix to generate a coordinate matrix, thus achieving accurate spatial positioning of the blurred region and providing standardized mathematical support for subsequent layout analysis and model optimization.

[0054] The fuzzy region mask matrix is ​​obtained using the following formula: (5) In formula (5), The contrast threshold is dimensionless and defaults to 0.2. The threshold value is based on the test of 500 OFD files with different degrees of fuzziness. When the contrast index is <0.6, the OCR recognition accuracy is <60%, and enhancement processing is required. The scene adjustment range is: official documents (handwriting standard): 0.5~0.6; contracts (including handwriting): 0.6~0.7; archives (scanned and aged): 0.7~0.8. The control logic of formula (5) completes the accurate transformation from "contrast feature" to "fuzzy area" through thresholding. It is the core control switch for realizing fuzzy area recognition and ensures the feasibility of special processing strategies (such as reducing weight and focusing on repair) for low-quality areas in subsequent summary generation / format analysis.

[0055] If the values ​​in the contrast feature matrix are below a preset threshold, such as 0.5, it indicates potential blurring, requiring the construction of a coordinate matrix. Specifically, the coordinate matrix is ​​constructed by recording the center coordinates and boundary points of low-contrast sub-blocks. For example, for a blurred table region, the contrast feature matrix lists its x and y coordinate ranges. These coordinates are then aggregated using clustering algorithms such as k-means to identify the blurred regions, such as watermark interference areas in a document. This method effectively improves the accuracy of image processing, thereby reducing recognition errors in document digitization. For instance, in intelligent summary generation systems, this process optimizes the localization of blurred regions, achieving efficient layout restoration.

[0056] Preferably, the OFD document intelligent summarization method integrating OCR and deep learning provided in this embodiment includes step S200 as follows: S210. If the identified blurred area contains handwritten content, then the blurred area is denoised to obtain a handwritten binarized image.

[0057] The handwritten binary image is obtained using the following formula: (6) In formula (6), Binarized image of handwritten characters (single channel, pixel value 0 / 255); For denoising operators (such as nonlocal mean denoising); The original image of the blurred area (from) (Excerpt from the middle) The threshold value is 128 (default value, pixel value). The control logic of formula (6) is "local denoising + threshold segmentation", which solves the problem of noise interference and unclear strokes in handwritten images within the blurred area. Through standardized binary output, it provides clean and standardized input data for subsequent handwritten processing, which is the key control link to ensure the overall processing accuracy.

[0058] For the blurred areas in electronic documents, if handwritten content is detected, denoising processing is first performed. Specifically, noise points are removed by applying a median filter. For example, in a scanned contract document, the handwritten signature area may be mixed with background interference. The pixel values of this area are adjusted to a binary form to obtain a clear binary image of the handwritten text, where the foreground characters are black and the background is white, providing a basis for subsequent extraction.

[0059] S220. Extract the skeleton of the binary handwritten image and construct a character structure feature vector.

[0060] The character structure feature vector is constructed by the following formula: (7) In formula (7), is the character structure feature vector (a 4-dimensional vector); is the ratio of the skeleton length to the bounding rectangle of the character (dimensionless), used to characterize the stroke compactness; is the aspect ratio of the width to the height of the character bounding rectangle (dimensionless), used to characterize the character shape; is the number of intersection points of the character skeleton (dimensionless), used to characterize the structural complexity; is the number of holes in the character area (dimensionless), used to characterize the closed structure feature; is the 4-dimensional real number space. The control logic of formula (7) is to extract 4 key structural features from the binary image and aggregate them into a standardized vector. Core values: 1. Structural representation: This character structure feature vector breaks through the traditional features that only rely on pixel grayscale and delves into the topological structure level of the character, capable of distinguishing characters with similar glyphs but different structures (such as "日" and "目", "O" and "D"). 2. Enhanced robustness: For deformations such as uneven stroke thickness and offsets in handwritten text, the features based on the skeleton and geometric ratios have better robustness. 3. Classification basis: It can be used as an auxiliary feature for deep learning models or the core feature vector of traditional classifiers, significantly improving the accuracy of handwritten text recognition.

[0061] When processing the binary handwritten image, skeleton extraction is performed to construct the character structure feature vector. Specifically, the thinning algorithm is used in skeleton extraction to gradually peel off the pixel layer until the character centerline is retained. For example, for the handwritten "signature" character, the algorithm iteratively deletes non-skeleton points starting from the edge pixels, generating a one-dimensional skeleton path, and then calculates features such as the curvature, length, and branch points on the path to form a multi-dimensional vector. For example, the character structure feature vector includes 10 elements representing different structural attributes. These character structure feature vectors capture the topological structure of the character and prepare data for semantic analysis.

[0062] S230. Input the character structure feature vector into the semantic analysis model to generate a semantic correction mask.

[0063] The semantic correction mask is generated using the following formula: (8) In formula (8), For semantic correction mask (and) (pixel-level masks of consistent size) For semantic analysis models (such as CNN-LSTM hybrid models), the CNN layer is a 2-layer 3×3 convolution (output 64 dimensions), the LSTM layer is a single layer with 128 dimensions, the training dataset is a blurred region sample containing 50,000 handwritten OFD files, the learning rate is 1e-4, the batch size is 32, and the number of iterations is 100 rounds; the input is a 4-dimensional character structure feature vector, and the output is a semantic correction mask (pixel value 0~1) of the same size as the handwritten binarized image. The dictionary contains 3500 commonly used characters in OFD files (3000 first-level Chinese characters + 500 numbers / letters / symbols). Each character corresponds to a 4-dimensional structural feature vector. The average value is taken from 100 handwritten samples from different writers. Euclidean distance matching is used. When the Euclidean distance between the feature vector of the character to be identified and the character in the dictionary is <0.2, it is considered a successful match. The control logic of formula (8) is "based on the dictionary template, analyze the character structure through the semantic model, and generate a directional repair mask". Formula (8) solves the problems of "similar characters but different semantics" and "blind repair of fuzzy areas" in handwritten character recognition. It is the core algorithm for accurate handwritten character processing in intelligent summary generation.

[0064] Character structure feature vectors are input into a semantic analysis model to generate a semantic correction mask. Specifically, the semantic analysis model is based on a neural network, such as a recurrent neural network combined with an attention mechanism. After inputting the character structure feature vectors, semantic relevance is calculated through hidden layers. For example, in handwritten annotations in a contract document, the semantic analysis model analyzes the semantic context of the word "agree" and outputs a mask matrix. The matrix elements have values ​​from 0 to 1, representing the semantic confidence of a pixel. The mask covers the image area to highlight the effective character parts, thereby guiding the reconstruction process.

[0065] S240. Use a semantic correction mask to perform pixel-level reconstruction of the handwritten binary image to obtain enhanced image data.

[0066] The enhanced image data is obtained using the following formula: (9) In formula (9), The image data is enhanced; ⊙ represents the pixel-by-pixel multiplication operator. The inverse mask (dimensionless) is used for the mask; the clarity index of the blurred area after enhancement is ≥0.8, and the threshold value is based on the following: when the enhanced index is ≥0.8, the OCR recognition accuracy is ≥90%, which meets the requirements for subsequent key element extraction; the scene-based adjustment range is increased by 0.2 in sync with the blur judgment threshold. The control logic of formula (9) is "regional differentiation enhancement", which realizes the accurate reconstruction of handwritten images through pixel-by-pixel operation.

[0067] A semantic correction mask is used to reconstruct the handwritten binary image pixel by pixel, resulting in enhanced image data. Specifically, the reconstruction process applies mask multiplication pixel by pixel, retaining high-confidence pixels and filling low-confidence areas. For example, in the handwritten date area, the mask corrects blurred handwriting, and interpolation algorithms such as bilinear interpolation are used to smooth edges, generating clear image data. This method effectively enhances the handwritten content.

[0068] Furthermore, the OFD document intelligent summarization method integrating OCR and deep learning provided in this embodiment includes step S300 as follows: S310. Acquire the enhanced image data and separate the red component region and the high curvature stroke connected region to determine the set of key elements of the seal and signature.

[0069] The set of key elements for a seal and signature is determined using the following formula: (10) In formula (10), This is a collection of key elements for seals and signatures, used to store the pixel coordinates of the red seal area and the high-curvature stroke area; for The red component value of a pixel (0~255); The threshold for the red component (default value is 200) is determined based on the following: In the RGB color space, the R channel pixel values ​​of red in the stamp are mostly distributed between 200 and 255. When the value is ≥200, the stamp extraction accuracy is ≥90%. The scene-specific adjustment range is: Color scanned files: 210~220; Black and white scanned files: 180~200. for The curvature value of the pixel (dimensionless) is obtained using the discrete curve curvature calculation method; The curvature threshold (default value is 0.8) is determined based on the following: the average curvature of handwritten signature strokes is 0.7~1.0. When the curvature is ≥0.8, it can effectively distinguish between handwritten signatures and printed text, with an extraction accuracy of ≥88%. The scene-based adjustment range is: 0.7~0.8 for messy handwritten signatures; 0.8~0.9 for neat handwritten signatures. ∪ is the set merging operator, used to merge the red area and the high curvature area set. The control logic of formula (10) is "dual feature collaborative localization", which accurately separates the stamp and signature areas in the document through dual screening of color features and geometric features.

[0070] For the processed contract scan image, the enhanced image data is first acquired. The red component region is extracted using color channel separation techniques, such as isolating regions with R channel pixel values ​​greater than a threshold in the RGB image to identify the red ink marks of the seal. Specifically, the separation of connected regions of high-curvature strokes involves edge detection and connected component analysis. For example, the Canny operator is applied to detect the edge curves of the signature strokes, and then curvature is calculated as a measure of the degree of local bending. Curvature is estimated by the difference in tangent direction between adjacent pixels. High-curvature parts, such as sharp corners or loops in the signature, are marked as connected regions, thereby determining the set of key elements of the seal and signature. For example, in a lease contract, the red seal region and the high-curvature strokes of the handwritten signature are combined into an element set, including the seal outline and signature path points. These elements are refined through morphological operations such as dilation and erosion to ensure the integrity of the element set to support subsequent mapping.

[0071] S320. Map the set of key elements of the seal and signature to the page layout diagram to obtain multiple independent document areas.

[0072] The set of independent document regions is derived using the following formula: (11) In formula (11), For a collection of independent document regions ( (total number of regions), used to store multiple independent document regions after page layout; For layout analysis segmentation operators (such as contour-based segmentation), used for augmented images And combine the key element set Perform contour detection and connected component segmentation; For the first Each document region (pixel coordinate range) represents the pixel coordinate range of a single independent block. The control logic of formula (11) is "anchor point guidance + global layout segmentation". 1. It decomposes complex mixed documents (containing text, images, seals, and signatures) into standardized independent blocks, solving the problem of information extraction difficulties caused by "text and image mixing" in OFD / documents; 2. Through segmentation, it physically separates elements with different attributes such as seals, signatures, body text, and annotations; 3. Independent document region set It is the basic unit for building knowledge graphs.

[0073] The set of key elements of the seal and signature is mapped to the page segmentation map, which is a document layout map generated by a semantic segmentation model such as U-Net, in which each pixel is classified as text, table or blank area. For example, the bounding box of the element set is projected onto the page segmentation map to obtain multiple independent document areas such as signature area and seal area.

[0074] S330. Extract the geometric centroid coordinates and high-dimensional semantic vector of the document region, and construct the association probability matrix based on the geometric centroid coordinates and high-dimensional semantic vector.

[0075] The correlation probability matrix is ​​constructed using the following formula: (12) In formula (12), For the correlation probability matrix ( (order, value 0~1). For the first Geometric centroid coordinates (in pixels) of a document region; For the first Geometric centroid coordinates (in pixels) of a document region; Euclidean distance to the centroid (unit: pixels); For the first High-dimensional semantic vectors of each region (the default dimension is 256); For the first High-dimensional semantic vectors of each region (the default dimension is 256); For cosine similarity operator (representing semantic relevance); Weighting coefficients ( , , The control logic of formula (12) is the key bridge to realize the semantic association from independent regions. Formula (12) not only considers whether the images look similar (semantics), but also whether they are close to each other (space), to ensure that in complex layout documents, entity elements that "should be together" (such as official seal and date) are accurately identified.

[0076] The geometric centroid coordinates and high-dimensional semantic vectors of the document regions are extracted. The geometric centroid coordinates are calculated as a weighted average of the region's pixel coordinates; for example, for a signature region, the centroid is the mean of the x and y coordinates of all pixels. The high-dimensional semantic vectors are generated from the region text by a pre-trained BERT model, capturing semantic features such as the meaning of contract terms. Then, an association probability matrix is ​​constructed based on these geometric centroid coordinates and high-dimensional semantic vectors. The matrix elements are calculated using cosine similarity combined with Euclidean distance; for example, the matrix rows represent regions. , column represents region The value represents the joint probability of semantics and location.

[0077] S340. Traverse the correlation probability matrix. If the element value in the correlation probability matrix is ​​greater than the preset threshold, output the correlation determination result between document regions.

[0078] The document region correlation determination result is output using the following formula: (13) In formula (13), The result of the regional correlation determination (Boolean type); The threshold value is 0.7 (default). The value is based on the layout analysis test of 1000 OFD documents. When the value is ≥0.7, the accuracy of the determination of regional semantics + location association is ≥91%. The scenario adjustment range is: 0.65~0.7 for mixed text and image documents; 0.7~0.75 for plain text documents. The control logic of formula (13) is the output gate of the entire document region association analysis process. Formula (13) transforms the complex association probability matrix into a clear entity relationship network through threshold truncation. This not only completes the automatic identification of logical relationships such as "seal-date" and "title-body text", but also provides the most core structured relationship data for subsequent intelligent summary generation and knowledge graph construction.

[0079] Traverse the association probability matrix. If an element value is greater than a preset association probability threshold such as 0.7, output the association determination result between document regions. For example, in a financial report, determine the association between the signature area and the seal area to verify the integrity of the document.

[0080] Preferably, the OFD document intelligent summarization method integrating OCR and deep learning provided in this embodiment includes step S400 as follows: S410. Obtain the document region correlation determination result that meets the preset conditions, and extract the text flow characteristics and spatial constraint information within the document region.

[0081] Text flow characteristics are derived using the following formula: (14) In formula (14), Text flow features (including direction, line length, and line spacing); The main direction angle of the text line (unit: °); Average row length (in pixels); The average line spacing is expressed in pixels. The control logic of formula (14) completes the second layer of abstraction from "image pixels" to "layout semantics". Formula (14) no longer focuses on the pixel details of individual characters, but describes the writing order of the document area in a macroscopic way. This step is a key prerequisite for ensuring that the intelligent summary generation "conforms to reading habits" and "adapts to the page structure".

[0082] Spatial constraint information is obtained through the following formula: (15) In formula (15), Spatial constraint information (including overlap rate and alignment distance); The region overlap rate is dimensionless. The region alignment deviation is expressed in pixels. The control logic of formula (15) is a precise sketch of the "page layout". Formula (15) no longer describes what a single region looks like, but rather how close or aligned the regions are. This is the "spatial sense" that intelligent summary generation must have, ensuring that the generated summary is not only accurate in content, but also conforms to the reading spatial logic of the original document.

[0083] For the processed contract document, the first step is to obtain the document region correlation determination results that meet preset conditions. For example, region pairs with a value greater than 0.8 are filtered out through the correlation probability matrix to identify the close relationship between the seal area and the signature area, thus providing a basis for subsequent extraction. In one embodiment, text flow features and spatial constraint information within the document region are extracted. Specifically, the text flow features analyze the direction and order of text lines using optical character recognition technology. For example, in a rental contract, the Tesseract engine is used to detect the reading path of the text from left to right and calculate the line spacing as the flow vector. The spatial constraint information involves the geometric properties of the region bounding box, such as using the OpenCV library to calculate the coordinates of the rectangular bounding box of the region, thereby capturing the vertical alignment constraint of the text block above the seal area. This extraction process first preprocesses the document image, including binarization and noise removal, and then applies the directional gradient histogram to calculate the text tilt angle to ensure the accuracy of the flow features. Next, the spatial constraints are quantified by calculating the overlap, intersection, and union ratio between regions. For example, if the overlap rate of two regions exceeds 50%, they are marked as spatially adjacent, thereby constructing a feature set containing direction vectors and constraint matrices to support the structured representation of the document.

[0084] S420. Construct a document attribute graph based on text flow characteristics and spatial constraint information, and identify logical anchor points in the document attribute graph that contain key elements of seals and signatures.

[0085] Construct a document attribute graph using the following formula: (16) In formula (16), For document attribute graphs; For graph nodes (corresponding document regions) ); For graph edges (corresponding to regional correlations) The control logic of formula (16) is the "structured output hub" of the entire technical solution. Formula (16) packages all the previous pixel processing, feature extraction, and correlation analysis into a "document attribute map". This document attribute map is the object of subsequent "logic anchor point recognition", "map reasoning" and "intelligent summary generation", ensuring the interpretability of the processing process and the structure of the results.

[0086] The set of logical anchor points is derived using the following formula: (17) In formula (17), A set of logical anchor points (containing graph nodes of seals / signatures); For nodes, For set intersection operators; The non-empty condition is used for judgment. The control logic of formula (17) is the "precise focus switch" of the entire process. Formula (17) directly maps the previously painstakingly identified "stamp and signature" pixels to the core entity nodes in the graph network. This step is the final key closed loop to ensure that the intelligent summary generation is not off-target, not redundant, and deeply fits the document business logic.

[0087] Based on the text flow characteristics and spatial constraint information, a document attribute graph is constructed to identify logical anchor points containing key elements such as seals and signatures within the document attribute graph. Specifically, the document attribute graph employs a graph neural network model such as GraphSAGE, using regions as nodes, flow characteristics as edge weights, and spatial constraints as node attributes. For example, in a financial report, nodes represent text blocks, and edges represent the reading order from top to bottom, thus forming a directed graph. Logical anchor point identification identifies nodes with high connectivity through centrality calculation; for example, a seal node is marked as an anchor point because it connects multiple signature and clause nodes.

[0088] S430. Perform semantic topological analysis on logical anchors to map document regions into a topological sequence with a sequential order.

[0089] The topological sequence is derived using the following formula: (18) In formula (18), A topological sequence (a document region arranged in logical order); This is a topological sorting operator (sorting of directed acyclic graphs based on logical anchors). The control logic of formula (18) is the last mile from "structure" to "sequence". It takes the previously constructed static graph... This process dynamically transforms into a summary generation roadmap that aligns with reading habits. This is a crucial step in ensuring that intelligent summary generation is not only "content-correct" but also "sequentially correct" and "compliant with the original document's business logic."

[0090] Semantic topology analysis is performed on the logical anchors to map the document regions into a topological sequence with a chronological order. Specifically, the semantic topology analysis uses a topological sorting algorithm in a deep learning framework such as PyTorch, combined with semantic embedding vectors generated by BERT, to sort the anchors. For example, the opening clause of the contract is placed at the beginning of the sequence, and the seal anchor is placed at the end, forming a linear topological sequence.

[0091] S440. Analyze the topological sequence and divide it into semantic block clusters using a computer vision model. If the association weights of the semantic block clusters meet the preset logical hierarchy, determine the overall logical structure of the document based on the entity relationships between the semantic block clusters.

[0092] The overall logical structure of a document can be determined using the following formula: (19) In formula (19), The overall logical structure of the document (hierarchical semantic block clusters); The computer vision model (Vision Transformer (ViT-B / 16) takes a 224×224 document region image as input and outputs a 128-dimensional semantic block cluster feature vector. The training dataset consists of layout samples of 80,000 complex OFD files, with a learning rate of 3e-4 and a batch size of 8). The threshold for the association weight of semantic block clusters (default value is 0.8) is determined based on the following: 800 complex OFD files were tested, and the accuracy of the logical hierarchy determination of semantic block clusters was ≥93% when the threshold was ≥0.8. The scenario-based adjustment range is: multi-level title files: 0.75~0.8; simple format files: 0.8~0.85. The control logic of formula (19) is the final implementation from "image / text features" to "business logic structure". Formula (19) not only completes clustering, but more importantly, it forces the construction of a document skeleton that conforms to business logic by setting the weight threshold. This skeleton is the final blueprint for generating high-quality, highly compliant OFD intelligent summaries.

[0093] The topological sequence is analyzed and semantic block clusters are divided using a computer vision model. If the association weights of the semantic block clusters conform to a preset logical hierarchy, the overall logical structure of the document is determined based on the entity relationships between the semantic block clusters. Specifically, a computer vision model such as ResNet combined with an attention mechanism analyzes the sequence, and clusters are divided using a clustering algorithm such as K-means based on semantic similarity, for example, associating related clause clusters with signature clusters. The association weight is calculated as the average of the edges between clusters. If it is greater than the association weight threshold of 0.8, the hierarchy is confirmed, such as clause clusters at the upper level and seal clusters at the lower level, thereby determining the logical structure of the contract and supporting automated verification. Through the above method, accurate document structure parsing is achieved.

[0094] Furthermore, the OFD document intelligent summarization method integrating OCR and deep learning provided in this embodiment includes step S500 as follows: S510. Locate the title entity and document number entity based on the determined overall logical structure, and extract the contextual embedding representation of the title entity and document number entity.

[0095] The contextual embedding representation of the title entity and the document number entity is derived using the following formula: (20) In formula (20), The contextual embedding representation of the title entity (768-dimensional vector); The contextual embedding representation of the document number entity (768-dimensional vector); For pre-trained language models (semantic embedding operators); The text content of the title entity; The text content of the document number entity. The control logic of formula (20) is the final language bridge from "image structure" to "semantic understanding". It transforms the text recognized in the layout into a semantic vector that AI can truly "understand". This is the last piece of the puzzle to achieve accurate title / document number extraction and subsequent structured summary generation.

[0096] For a legal contract document, the first step is to locate the title entity and document number entity based on the previously determined overall logical structure. For example, identify the contract title such as "Lease Agreement" and the document number such as "Contract Number: 2023-001". These entities are usually located in the header area of ​​the document, thus providing a basis for subsequent extraction. In one embodiment, the contextual embedding representation of the title entity and the document number entity is extracted. Specifically, the contextual embedding representation is a vector representation generated by natural language processing techniques, such as using a BERT model to encode the text surrounding the entity. The process includes inputting the entity text into the BERT model, which captures contextual dependencies through multiple Transformer layers. For example, in a contract, the date and party information surrounding the title "Lease Agreement" are embedded as a high-dimensional vector, capturing semantic relationships such as the topic association between "agreement" and "lease". Then, positional encoding is combined to ensure that the embedding reflects the relative position of the entity in the document, thereby generating a vector representation containing semantic and positional information to support subsequent sequential processing.

[0097] S520. The context embedding representation is used to generate entity feature vectors containing semantic dependency paths from the input sequence labeling model, and an association probability matrix reflecting the degree of semantic alignment is constructed for the entity feature vectors.

[0098] The entity feature vector containing semantic dependency paths is generated using the following formula: (twenty one) In formula (21), The entity feature vector (256 dimensions); For sequence labeling models (such as BiLSTM-CRF, where the BiLSTM layer is a two-layer 128-dimensional structure, the CRF layer is a linear chain conditional random field, the training dataset contains 100,000 OFD document / contract title-document number samples, the learning rate is 5e-5, the batch size is 16; the input is a 768-dimensional contextual embedding representation, and the output is a 256-dimensional entity feature vector containing semantic dependency paths). For vector fusion operators; This is a semantic dependency path (representing the semantic association between title and document number). The control logic of formula (21) is a sublimation from "content fusion" to "logical understanding". Formula (21) truly "connects" the previously independent "title vector" and "document number vector" through the semantic dependency path. This entity feature vector It is the final version that establishes the "core identity identifier" of a document during intelligent summary generation.

[0099] The correlation probability matrix is ​​constructed using the following formula: (twenty two) In formula (22), This is the semantic alignment association probability matrix (values ​​range from 0 to 1). Transpose the entity feature vector; Let L2 be the norm of the vector. The control logic of formula (22) is a converter from "semantic features" to "association logic". Formula (22) transforms the abstract vector features generated by formula (21) into an association probability matrix that machine learning can directly use to judge "correctness" and "similarity". This is the decisive mathematical step in realizing "precise locking of core document entities" in intelligent summarization.

[0100] The context embedding method is used to generate entity feature vectors containing semantic dependency paths from the input sequence labeling model. Specifically, the sequence labeling model, such as BiLSTM-CRF, first receives the embedding vector as input and processes the sequence through a bidirectional long short-term memory network to capture dependencies. For example, in the document number entity "2023-001", the sequence labeling model labels the path as the numerical dependency from "2023" to "001". Then, a conditional random field layer optimizes the label sequence to generate entity feature vectors, where the semantic dependency paths represent the grammatical relationships between entities, such as the title entity depending on the numbering logic of the document number, thus forming a vector reflecting the paths between entities.

[0101] An association probability matrix reflecting the degree of semantic alignment is constructed based on the entity feature vectors. Specifically, the association probability matrix is ​​constructed by calculating the cosine similarity between vectors. For example, the dot product operation is performed on the feature vectors of the title and document number to obtain a similarity score, forming an association probability matrix where rows and columns correspond to different entities, and values ​​represent the degree of alignment, such as 0.9 indicating high alignment. The process includes normalization to ensure that the sum of probabilities is 1, thereby quantifying semantic matching and supporting constraint judgment.

[0102] S530. If the correlation probability matrix satisfies the constraints of the preset logical level labels, then the text content of the title entity and the document number entity is aggregated according to the mapped structured field definition to obtain a set of structured key information.

[0103] The structured key information set is derived using the following formula: (twenty three) In formula (23), A structured collection of key information; For structured aggregation operators; Preset logical hierarchy labels (such as "title-document number-text"). The control logic of formula (23) is the "structured final step" of the entire technical solution. It transforms all the previous image recognition, semantic analysis, and vector calculation into a set of standard structured data that conforms to business logic. This is the decisive closed loop for realizing OFD files from "image parsing" to "intelligent summary generation".

[0104] If the association probability matrix satisfies the constraints of the preset logical hierarchy labels, then the text content of the title entity and the document number entity is aggregated according to the mapped structured field definitions to obtain a structured key information set. Specifically, the constraints include satisfying the hierarchy labels when the matrix value is greater than 0.8, for example, the title is at the upper level and the document number is at the lower level; the aggregation process uses field definitions such as JSON schema to map text to fields, for example, aggregating "rental agreement" and "2023-001" into {"title": "rental agreement", "number": "2023-001"}, thereby forming a structured key information set.

[0105] Preferably, the OFD document intelligent summarization method integrating OCR and deep learning provided in this embodiment includes step S600 as follows: S610. Obtain the original document image associated with the structured key information set, and extract the local visual feature map covering the key area.

[0106] Local visual feature maps are extracted using the following formula: (twenty four) In formula (24), This is a local visual feature map (2048-dimensional feature vector). For visual feature extraction model (ResNet50 as the backbone network, the last fully connected layer is removed, the output is a 2048-dimensional local visual feature vector, the input is a cropped key region image (size 224×224, normalized to [0,1])); The original document image; Coordinates of the key area (from) The mapping is a set of pixel coordinates of core business areas such as seals, signatures, titles, and document numbers obtained by mapping the structured key information set. The control logic of formula (24), "region cropping + deep feature embedding", is to extract differentiated visual features for the "core areas" that the machine learning model is most concerned with. It is the last physical mapping from "text structure" to "image features". Formula (24) transforms the previously abstracted "key area coordinates" into visual feature vectors that AI can truly "understand". This is the core visual foundation for generating the final OFD intelligent summary, which is "rich in both text and images".

[0107] For a financial report document, the process begins by acquiring an original document image associated with a set of structured key information, such as a PDF file image captured by a scanner. This key information set includes a title such as "Annual Financial Report." In one embodiment, local visual feature maps covering key regions are extracted. Specifically, this process utilizes optical character recognition (OCR) techniques to locate the document header region, for example, by detecting rectangular bounding boxes in the image using the OpenCV library, and cropping sub-images for the date portion near the title. These sub-images are then processed using a convolutional neural network such as a ResNet model to generate local visual feature maps. Each pixel captures texture and color gradients; for example, in the report image, the contrast between the black text surrounding "December 31, 2023" and the white background is converted into a high-dimensional tensor, providing a visual basis for subsequent parsing. This extraction method ensures the locality of the feature maps, reduces global noise interference, and supports accurate text recognition.

[0108] S620. Obtain the text string of the date of writing and the year value based on the parsing of local visual feature maps.

[0109] The date text string and year value are obtained using the following formula: (25) In formula (25), The date of the document is a text string (e.g., "2026-03-22"). For annual values ​​(e.g., 2026, an integer); For accurate OCR recognition operators (an end-to-end OCR recognition operator based on CRNN+CTC, the input is a local visual feature map of the date region, the output is a character sequence, and it supports the recognition of multiple date formats such as "year / month / day". The character set includes 0-9, "-", " / ", "year", "month", and "day"); The visual feature template for the date area includes visual features (texture, character spacing, font) of horizontal / vertical and printed / handwritten dates. The template matching algorithm used is normalized cross-correlation (NCC), with a matching threshold ≥ 0.8. The year extraction operator (a character extraction operator based on regular expressions, the regular expression being "\d{4}", extracts 4 digits from the date text string recognized by OCR as the year value; if there are multiple 4 digits, the value that matches the date semantics is taken). The control logic of formula (25) is used to extract and parse the written date information in the document based on OCR technology and visual feature templates, and generate a calculable date text string and the corresponding year value. 1. Ensure the standardization and calculability of date information extraction, and avoid format errors or numerical deviations caused by manual parsing; 2. Combine visual feature templates and accurate OCR to adapt to date extraction of complex layouts and multi-format documents, and have strong robustness; 3. The generated year value It can be directly used in subsequent calculations, process control, data archiving and other business scenarios to achieve standardization and efficiency in document date processing.

[0110] The text string containing the date and the year value are obtained through parsing local visual feature maps. Specifically, the parsing uses a sequence-to-sequence model such as the Tesseract engine combined with an attention mechanism. First, the local visual feature map is input into the sequence-to-sequence model. The Tesseract engine recognizes the text sequence through a trained character classifier. For example, it decodes "December 31, 2023" from the local visual feature map as a complete date string. Then, regular expressions are applied to extract the year value, such as "2023". The process includes string matching and numerical conversion to ensure accuracy, thereby obtaining structured data that can be used for embedding.

[0111] S630. Combine the text string of the date of writing, the annual value, and the semantic information of the title in the structured key information set to construct a comprehensive feature vector.

[0112] The comprehensive features are constructed using the following formula: (26) In formula (26), The composite feature vector (1024 dimensions); For normalization operator Normalization maps each feature vector to the interval [0, 1], as shown in the formula. ,in, These are the original data points (values ​​to be transformed) in the feature vector. It is the minimum value among all elements of the feature vector. It is the maximum value among all elements of the feature vector. The result after normalization This is a vector fusion operator. The control logic of formula (26) is "unified mapping of multimodal heterogeneous features", which integrates scattered text, image, and date information into a unified "document ID card". It possesses multimodal fault tolerance, ensuring that correct summaries can be generated under various disturbances.

[0113] By combining the text string of the date, the annual numerical value, and the semantic information of the title from the structured key information set, a comprehensive feature vector is constructed. Specifically, the Word2Vec model is first used to generate a semantic vector for the title, such as "Annual Financial Report," by decomposing words into embeddings to capture the relationship between "financial" and "report." Then, the date string and the annual numerical value are converted into numerical embeddings, such as mapping the year "2023" to a one-hot vector. These are then fused with the semantic vector through concatenation to form a multimodal comprehensive vector. For example, a vector with a dimension of 512 integrates textual semantics and time information, thereby improving the vector's expressive power and supporting summary generation.

[0114] S640. Input the comprehensive feature vector into the decoder to generate an initial summary text sequence, and embed the text string of the writing date and the year value into the initial summary text sequence, and output a complete description containing the writing date and year.

[0115] The following formula outputs a complete description including the date and year of writing: (27) In formula (27), A complete summary text including the date and year of writing; The text generation decoder is a 6-layer Transformer decoder with 512 hidden layers, 8 multi-head attention heads, and a vocabulary size of 8000 (covering commonly used words in OFD files). The input is a comprehensive feature vector, and the output is a word-level text sequence. A beam search (beam width = 5) is used to generate a summary. As a vector fusion operator, the date of writing and the year are embedded into a specified position in the summary text. The control logic of formula (27) is "generation-driven + entity forced insertion", that is, the content is generated by using a deep learning model and the precise date entity is forcibly injected to ensure the integrity and timeliness of the summary.

[0116] The comprehensive feature vector is input into the decoder to generate an initial summary text sequence. The date of issue and the year are then embedded into this initial summary text sequence, outputting a complete description containing the date of issue and the year. Specifically, the decoder employs a Transformer-based model, such as a variant of GPT. After receiving the comprehensive feature vector, it generates an initial summary text sequence through a self-attention layer, for example, "This report outlines the financial situation." Then, the embedding process uses positional encoding to insert "December 31, 2023" and "2023" into the initial summary text sequence, forming a complete description such as "This annual financial report was issued on December 31, 2023, and covers data for the year 2023," thus ensuring the timeliness and accuracy of the summary.

[0117] Abstract length rules: The abstract length should be controlled between 50 and 300 words, dynamically adjusted according to the length of the original document; the number of pages in the original document is counted according to the standard pages of OFD files, with a single page character threshold of 500-800 characters (Chinese). If the single page character count is <500, it is merged into the next page for counting; if it is >800, it is split into two pages for counting; specific adjustment standards: original document ≤10 pages, abstract 50-100 words; original document 11-50 pages, abstract 100-200 words; original document >50 pages, abstract 200-300 words.

[0118] The OFD document intelligent summarization method integrating OCR and deep learning provided in this embodiment achieves the following beneficial effects compared with existing technologies: 1. The core logic of this embodiment is "OFD complex layout scanning → fuzzy region recovery → key element extraction and region division → logical structure construction → structured information extraction → intelligent summary generation", which integrates OCR recognition and deep learning technology to solve the pain points of complex layout, fuzzy content and difficulty in extracting key elements of OFD files; 2. Key technological highlights include the enhancement and restoration of blurred areas in handwritten text (combining image enhancement and semantic analysis to improve recognition accuracy), document region correlation determination and logical structure construction (ensuring that the summary fits the core of the document), and the fusion of structured information and visual models to generate summaries (balancing accuracy and conciseness). 3. The entire process uses clearly defined thresholds and numerical ranges (such as clarity index 0.6 / 0.8, summary length 50~300 characters, and recognition accuracy ≥88%) to ensure the engineering feasibility of the method. It is suitable for intelligent summary generation of various OFD documents such as official documents, contracts, and archives, improving document processing efficiency and accuracy.

[0119] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A method for intelligent summarization of OFD documents integrating OCR and deep learning, characterized in that, Includes the following steps: S100. Obtain complex layout content from electronic documents, and perform preliminary scanning of the text and tables in the complex layout content using image recognition technology to determine the identified blurred areas, wherein the complex layout content includes multi-level text layouts and mixed graphic elements. S200. If the identified blurred area contains handwritten characters, then image enhancement processing combined with semantic analysis is used to restore the clarity of the blurred area to obtain enhanced image data. S300: Extract key elements of seal and signature from the enhanced image data, divide the document regions in the enhanced image data using layout analysis technology, and determine the correlation between the document regions; S400. If the correlation between the document regions meets the preset conditions, the overall logical structure of the document is determined by integrating the extracted key elements and the information of the document regions through a computer vision model. S500. Based on the determined overall logical structure, apply a machine learning model to analyze the semantic relationship between the title and document number elements in the overall logical structure to obtain a set of structured key information. S600. For the structured key information set, a concise text summary is generated by fusing a visual model, and a complete description including the date of writing and year is output.

2. The method for intelligent summarizing OFD documents by integrating OCR and deep learning according to claim 1, characterized in that, Step S100 includes: S110. Extract complex layout content from electronic documents to generate an initial layout image, wherein the initial layout image includes multi-level text layouts and mixed graphic elements. S120. Perform a preliminary scan on the initial layout image to obtain a first feature set; S130. Generate a contrast feature matrix based on the first feature set; S140. If the value in the contrast feature matrix is ​​lower than a preset threshold, a fuzzy region coordinate matrix is ​​constructed, and the identified fuzzy region is determined by the fuzzy region coordinate matrix.

3. The method for intelligent summarizing OFD documents by integrating OCR and deep learning according to claim 1, characterized in that, Step S200 includes: S210. If the identified blurred area contains handwritten content, then the blurred area is denoised to obtain a handwritten binarized image. S220. Extract the skeleton from the handwritten binary image and construct a character structure feature vector; S230. Input the character structure feature vector into the semantic analysis model to generate a semantic correction mask; S240. The handwritten binary image is reconstructed pixel-level using the semantic correction mask to obtain enhanced image data.

4. The method for intelligent summarizing OFD documents by integrating OCR and deep learning according to claim 1, characterized in that, Step S300 includes: S310. Acquire the enhanced image data and separate the red component region and the high curvature stroke connected region to determine the set of key elements of the seal and signature. S320. Map the set of key elements of the seal and signature onto the page segmentation diagram to obtain multiple independent document areas; S330. Extract the geometric centroid coordinates and high-dimensional semantic vector of the document region, and construct an association probability matrix based on the geometric centroid coordinates and the high-dimensional semantic vector; S340. Traverse the association probability matrix. If the element value in the association probability matrix is ​​greater than a preset threshold, output the association determination result between the document regions.

5. The method for intelligent summarizing OFD documents by integrating OCR and deep learning according to claim 4, characterized in that, Step S400 includes: S410. Obtain the document region correlation determination result that meets the preset conditions, and extract the text flow characteristics and spatial constraint information within the document region; S420. Construct a document attribute graph based on the text flow characteristics and spatial constraint information, and identify logical anchor points in the document attribute graph that contain key elements of seals and signatures. S430. Perform semantic topological analysis on the logical anchor point to map the document region into a topological sequence with a sequential order; S440. The topological sequence is analyzed and semantic block clusters are divided using a computer vision model. If the association weights of the semantic block clusters meet the preset logical hierarchy, the overall logical structure of the document is determined based on the entity relationships between the semantic block clusters.

6. The method for intelligent summarizing OFD documents by integrating OCR and deep learning according to claim 1, characterized in that, Step S500 includes: S510. Locate the title entity and document number entity according to the determined overall logical structure, and extract the contextual embedding representation of the title entity and the document number entity; S520. The context embedding representation input sequence labeling model generates entity feature vectors containing semantic dependency paths, and an association probability matrix reflecting the degree of semantic alignment is constructed for the entity feature vectors. S530. If the correlation probability matrix satisfies the constraints of the preset logical level label, then the text content of the title entity and the document number entity is aggregated according to the mapped structured field definition to obtain a set of structured key information.

7. The method for intelligent summarizing OFD documents by integrating OCR and deep learning according to claim 1, characterized in that, Step S600 includes: S610. Obtain the original document image associated with the structured key information set, and extract the local visual feature map covering the key area; Local visual feature maps are extracted using the following formula: ； in, This is a local visual feature map. For visual feature extraction models, The original document image, Coordinates of the key area; S620. Based on the local visual feature map parsing, obtain the text string of the document date and the year value; S630. Combine the text string of the date of writing, the annual value, and the title semantic information in the structured key information set to construct a comprehensive feature vector; S640. Input the comprehensive feature vector into the decoder to generate an initial summary text sequence, and embed the text string of the writing date and the year value into the initial summary text sequence, and output a complete description containing the writing date and year.

8. The method for intelligent summarizing OFD documents by integrating OCR and deep learning according to claim 7, characterized in that, In step S620, the document date text string and the year value are obtained using the following formula: ； in, The date text string is the date the document was written. For annual values, To achieve accurate OCR recognition operators, A visual feature template for the date area; For annual extraction operators.

9. The method for intelligent summarizing OFD documents by integrating OCR and deep learning according to claim 8, characterized in that, In step S630, the comprehensive features are constructed using the following formula: ； in, For the comprehensive feature vector, For the normalization operator, Contextual embedding representation for title entities, Contextual embedding representation of document number entities, This is a vector fusion operator.

10. The method for intelligent summarizing OFD documents by integrating OCR and deep learning according to claim 9, characterized in that, In step S640, a complete description including the date and year of writing is output using the following formula: ； in, For a complete summary text including the date and year of writing, Generate a decoder for the text. This is a vector fusion operator.

Images