A layout semantic node-based cross-format document semantic content extraction method

Abstract

The application discloses a layout semantic node-based cross-format document semantic content extraction method, and belongs to the technical field of computer application, which comprises the following steps: S1, keyword dictionary construction; S2, defining a semantic content extraction template, which is represented by a local relationship direction matrix; OCR recognizing a document, and constructing a global relationship direction matrix according to the keyword dictionary; S3, global-local matrix matching positioning a target area; S4, semantic extraction position determination, text block screening and structured output. The layout semantic node-based cross-format document semantic content extraction method has the advantages that fault tolerance is improved, low-cost fast deployment is realized without large-scale labeled data, auditability is guaranteed and misfit rate is reduced through explicit modeling, the whole text processing flow is covered through a parameterized strategy, complex scenes such as cross-column and broken-line are efficiently dealt with, and the blank of weak complex layout adaptation capability is filled.

Description

Technical Field

[0001] This invention belongs to the field of computer application technology, specifically relating to a method for extracting semantic content from cross-format documents based on layout semantic nodes. Background Technology

[0002] Cross-format document semantic content extraction technology is an automated information extraction technology for multi-format documents. Its core is to solve the problem that traditional extraction methods fail due to format differences in the same type of documents (such as invoices and customs declarations) generated in different regions and systems.

[0003] However, existing technologies have the following shortcomings: traditional fixed template / single-point positioning methods rely on absolute coordinates or keyword single-point matching, which has extremely poor universality, no fault tolerance mechanism, and poor scalability. New layouts require redefining the rules; large model methods have strong data dependence, high deployment and inference costs, and slow speed. They also have black-box and unauditable issues and are not robust enough when facing document version differences; most technologies lack the ability to adapt to complex layouts, can only handle simple documents, and have weak processing capabilities for cross-column fields, broken text, and low-confidence OCR text blocks.

[0004] Therefore, a new method is urgently needed. Summary of the Invention

[0005] The purpose of this invention is to provide a method for extracting semantic content from cross-format documents based on layout semantic nodes. This method achieves cross-format universality by combining semantic nodes with a global relational direction matrix, improves fault tolerance through parameterization strategies, and enhances scalability through a unified semantic node index. It does not require large-scale labeled data, adopts a non-large model architecture to achieve low-cost and fast deployment, and ensures auditability and reduces mismatch rate through explicit modeling. It covers the entire text processing process through parameterization strategies, efficiently handles complex scenarios such as cross-column and cross-row, and fills the gap in the ability to adapt to complex layouts.

[0006] To achieve the above objectives, this invention provides a method for extracting semantic content from cross-format documents based on layout semantic nodes, comprising the following steps:

[0007] S1. Collect target documents of various types and formats, traverse all collected document samples, extract keywords that appear consistently, and obtain a standardized keyword dictionary by standardizing keyword index and filtering semantic nodes.

[0008] S2 receives the standardized keyword dictionary from S1, performs format and quality checks on the document image to be processed, recognizes the text information in the image through OCR technology, and outputs the OCR result in structured JSON format.

[0009] Semantic matching is performed on OCR text blocks to locate the keywords that actually appear in the document and obtain a set of effective keywords; and the document layout is transformed into a structured semantic matrix based on the set of effective keywords to obtain a global relational direction matrix.

[0010] The target keywords corresponding to the target semantic block to be extracted are determined, and related keywords that have a fixed positional association with the target keywords are screened. Based on the unified index number of the S1 standardized keyword dictionary, the dimensions and row and column indices of the local relation direction matrix are fully aligned with the global relation direction matrix. The relative positional relationship between the target keywords and adjacent keywords is modeled through the chain code method, and then the elements of the local relation direction matrix are assigned values ​​according to the 4-chain code rule to finally complete the local relation direction matrix of the target semantic block to be extracted.

[0011] S3: Receive the global relation direction matrix, the set of effective keywords, and the local relation direction matrix from S2; by calculating the similarity between the global relation direction matrix and the local relation direction matrix, find the actual region in the document that matches the target semantics, and obtain the target region coordinate range of the semantic block to be extracted;

[0012] S4 receives the structured JSON format OCR result from S2, the set of valid keywords, and the target area coordinate range from S3; processes the OCR text block to generate structured semantic content.

[0013] Preferably, in S1, the standardized keyword dictionary includes keyword content, unique ID, and semantic category.

[0014] Preferably, S2 also includes:

[0015] The document is scanned or photographed to generate RGB / JPG / PNG format images, with varying image resolutions. Inclination angle ;

[0016] The OCR process yields structured JSON data, with each record containing text content, text block coordinates, and recognition confidence score. The text block is marked as low quality.

[0017] Preferably, in S2, the global relation direction matrix is: 3D matrix The number of keywords in the keyword dictionary is represented by the matrix elements, which represent the relative directional relationship between two corresponding semantic nodes. The directional relationship is encoded using 4-chain code.

[0018] Preferably, in S3, the similarity calculation adopts any one or a combination of the following methods:

[0019] The matrix structure dissimilarity is calculated based on mean squared error or mean absolute difference, and then converted into a range of values. The smaller the similarity value, the smaller the structural differences.

[0020] The direction consistency score is obtained by statistically analyzing the proportion of the direction signs of the sub-regions of the global relation direction matrix and the effective elements in the local matrix that are consistent with each other. The higher the proportion, the stronger the semantic consistency.

[0021] Calculate the weighted sum of node weights and the similarity function. The closer the weighted similarity is to 1, the higher the structural similarity. The node weight is the average of the confidence scores of the two corresponding semantic nodes. The similarity function can be a Gaussian kernel, cosine similarity, or structural difference function.

[0022] Preferably, in S3, the process of determining the actual region in the document that matches the target semantics is as follows:

[0023] Traverse all possible sub-regions of the global relation direction matrix according to the effective element range of the local matrix, calculate the similarity score between each sub-region and the local matrix and sort them, and select the sub-region with the highest score as the optimal matching sub-region.

[0024] Associate the sub-region index with the semantic node, obtain the spatial coordinates of the corresponding semantic node, and use these coordinates as the boundary to generate a minimum bounding rectangle or irregular region as the semantic region to be extracted.

[0025] Preferably, the processing of the OCR text block in S4 specifically includes:

[0026] Filter text blocks that are completely within the target area and within the tolerance offset range, and filter out low-confidence and short text invalid blocks according to confidence weight;

[0027] The filtered text blocks are sorted by coordinates according to the predefined main extraction direction; if there are missing text contents after sorting, the extraction range is expanded according to the auxiliary expansion direction to supplement adjacent valid text blocks.

[0028] Calculate the spatial distance or overlap rate of adjacent text blocks. If the distance is less than or equal to the merging threshold or the overlap rate is greater than the merging threshold, then merge the text blocks.

[0029] Detect cross-column fields and concatenate them according to the cross-column merging strategy; handle line breaks of multi-line fields according to the paragraph connection strategy.

[0030] Invalid information is removed based on filtering criteria, and the optimal content is selected from multiple text blocks with the same semantic meaning according to their credibility weights. Finally, they are spliced ​​together to form complete structured semantic content.

[0031] Therefore, the present invention employs the above-mentioned method for extracting semantic content from cross-page documents based on layout semantic nodes. Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0032] (1) Semantic nodes combined with global relation direction matrix realize cross-format universality, improve fault tolerance by parametric strategy, and enhance scalability by unified semantic node index, thus solving the problems of "poor universality and weak fault tolerance" of traditional methods.

[0033] (2) It does not require large-scale labeled data, adopts a non-large model architecture to achieve low-cost and fast deployment, and ensures auditability and reduces mismatch rate through explicit modeling, thus solving the problem of defects in large models;

[0034] (3) By using parameterized strategies to cover the entire text processing process, it can efficiently handle complex scenarios such as cross-column and line breaks, fill the gap in the ability to adapt to complex layouts, and overcome the problem of weak ability to adapt to complex layouts.

[0035] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating an embodiment of a cross-format document semantic content extraction method based on layout semantic nodes according to the present invention.

[0037] Figure 2 This is a schematic diagram of document keyword relationship direction matrix modeling, representing an embodiment of a cross-format document semantic content extraction method based on layout semantic nodes according to the present invention; wherein, Figure 2 (a) in the document represents a standard document used in the power industry to record information related to user electricity consumption; Figure 2 In the diagram, (b) represents the power grid meter reading with key information items identified by OCR, and the red box represents the text block of the OCR recognition result; Figure 2 (c) in the figure represents the semantic node matching result of power grid meter reading, which is indicated by the blue box; Figure 2 In the table, (d) represents the global relationship direction matrix of the power grid meter reading form; the corresponding relationship is represented by arrows in different directions. The actual matrix element values ​​and the correspondence between the arrows are: "right arrow" is 0, "up arrow" is 1, "left arrow" is 2, and "down arrow" is 3.

[0038] Figure 3 This is a schematic diagram of the semantic region to be extracted, representing an embodiment of a cross-format document semantic content extraction method based on layout semantic nodes according to the present invention. Figure 3 The meaning of the arrow in the middle is the same as Figure 2 ;

[0039] Figure 4 This is a diagram showing the global and local matching results of an embodiment of a cross-format document semantic content extraction method based on layout semantic nodes according to the present invention; where successfully matched areas are indicated in red. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Unless otherwise defined, the technical or scientific terms used in the present invention should have the ordinary meaning understood by those skilled in the art.

[0041] Example 1

[0042] like Figures 1-4 As shown, this embodiment provides a method for extracting semantic content from cross-format documents based on layout semantic nodes. It should be understood that the specific parameters, models, and protocols mentioned in this embodiment are merely examples to help those skilled in the art understand the present invention, and are not intended to limit the present invention.

[0043] The present invention provides a method for extracting semantic content from cross-format documents based on layout semantic nodes, comprising the following steps:

[0044] S1. Collect target documents of various types and formats, traverse all collected document samples, extract keywords that appear consistently, and obtain a standardized keyword dictionary by standardizing keyword index and filtering semantic nodes.

[0045] In this step, documents (such as invoices, customs declarations, electricity bills, etc.) essentially consist of two parts: keywords and content. Keywords belong to semantic types (such as "user ID" or "electricity address"), and their position in the document is relatively stable. Content is the actual value corresponding to this semantic type (such as "BM1688" or "Yuzhong County, Lanzhou City, Gansu Province"). Stable keywords in the document are defined as semantic nodes, each node having a standardized coding structure. A semantic node contains:

[0046] Unique Identifier: A unique ID that distinguishes different nodes;

[0047] Semantic category: The business semantics corresponding to the node (such as user identity identifier, electricity location information);

[0048] Fixed index: A unified numbering system designed for cross-format alignment (ensuring consistency of indexes for the same type of keywords in documents of different formats);

[0049] Text content: The original text of the keywords (such as user ID, meter reading date).

[0050] Based on sample data of similar documents (such as invoices, bills, and electricity bills), all appearing keywords are counted, and each keyword is assigned a "unique identifier + fixed index + semantic category" according to semantic node attributes to form a standardized keyword dictionary;

[0051] A standardized keyword dictionary ensures that similar documents generated across different regions, versions, and systems have consistent, fixed indexes for keywords with the same semantic meaning (such as "form number" in all data transfer forms), resolving the "identity alignment" issue across different formats. For example, in the power grid data transfer form dictionary, the fixed index for "user number" is 1, and the fixed index for "electricity address" is 6; these indexes remain unchanged regardless of document format changes.

[0052] S2 receives the standardized keyword dictionary from S1, performs format and quality checks on the document image to be processed, identifies the text information in the image through OCR technology, and outputs a structured JSON format OCR result; performs semantic matching on the OCR text block, locates the keywords that actually appear in the document, and obtains a set of effective keywords; and transforms the document layout into a structured semantic matrix based on the set of effective keywords to obtain a global relational direction matrix.

[0053] In this step, the format and quality of the document image to be processed are checked to ensure that the image format is correct. Format, resolution (To ensure OCR recognition accuracy), tilt angle ;

[0054] This technology uses OCR to recognize text information in images and outputs structured JSON-formatted OCR results. Each result corresponds to an independent text block and contains three core attributes:

[0055] (Original recognized text of the text block) (The spatial location of the text block in the image, in pixels) (The confidence level of OCR recognition ranges from 0 to 1.) (Text blocks marked as low quality); covering similar documents from different regions / versions. (Ensure keyword coverage) );

[0056] Semantic matching is performed on the OCR text blocks to locate the keywords that actually appear in the document and obtain a set of valid keywords, specifically:

[0057] Based on a standardized keyword dictionary and structured JSON format OCR results, all OCR text blocks are traversed. By combining exact matching and fuzzy matching, text blocks that match the keywords in the keyword dictionary are selected. For successfully matched text blocks, their corresponding core information is recorded, including: the unique ID of the keyword in the dictionary, the coordinate information and recognition confidence of the text block, and the semantic category of the keyword.

[0058] Based on the effective keyword set, the document layout is transformed into a structured semantic matrix, specifically:

[0059] Let the number of keywords in the set of valid keywords be . The semantic node set is Then the global relation direction matrix for Dimensions: Row index corresponds to a unique ID for a specific keyword (fixed index); column index corresponds to a unique ID for another keyword (fixed index); matrix elements. This represents the relative directional relationship between the two keywords;

[0060] That is, the global relational direction matrix Each element Indicates the first i Line number j The elements of the column represent unique IDs. i The keywords and unique ID are j The relative directional relationship of keywords in the document image is represented by a 4-chain code (e.g., 0 represents "right", 1 represents "up", 2 represents "left", and 3 represents "down"). The direction is determined and assigned a value through coordinate calculation (e.g., the x / y difference between the center coordinates of two keywords). The final result is a relational direction matrix. It can serve as a compact, structured description of document layout patterns, exhibiting significant layout independence and semantic consistency.

[0061] In subsequent key steps such as similarity measurement and cross-format matching, this matrix can be directly used to calculate the structural similarity between different documents, effectively supporting core tasks such as form classification and semantic information extraction. Furthermore, since the relational direction matrix uses the "relative relationship between semantic nodes in the layout" as its core feature, rather than relying on the absolute coordinates of nodes, it naturally possesses cross-format migration capabilities. Even if documents have layout differences (such as font or font size adjustments, or changes in column layout), it can still maintain a consistent expression of structural semantics, laying a core foundation for the stability of subsequent cross-format information extraction.

[0062] Identify the target keywords corresponding to the target semantic block to be extracted, and filter the related keywords that have a fixed positional association with the target keywords. Align the dimensions of the local relation direction matrix with the dimensions of the global relation direction matrix, and align the row and column indices. Assign values ​​to the elements of the local relation direction matrix to obtain the local relation direction matrix.

[0063] In this step, the target keywords corresponding to the target semantic block to be extracted are identified. For example, when extracting "electricity address", the target keyword is "electricity address". Related keywords that have a fixed positional association with the target keyword are also selected, such as "user ID" and "table number". These are usually keywords that appear stably around the target keyword.

[0064] Local relation direction matrix Global Relationship Direction Matrix Same dimension Ensure that the row and column indices are consistent with the global relational direction matrix. Completely identical (i.e.) The Line number Column correspondence The Line number Columns, corresponding to the same pair of semantic nodes and This ensures that the indices can be directly aligned during subsequent matching;

[0065] Local relation direction matrix elements (corresponding to the first in the global relation direction matrix) line, number The node relationships of the columns are defined according to the following rules:

[0066] ;

[0067] In the formula, Local relation direction matrix The Line number Column elements; Global relational direction matrix The Line number Column elements; , For the first The, the One semantic node; The set of associated nodes for the target semantic node; This is an invalid relationship; This is an undefined relation.

[0068] S3: Receive the global relation direction matrix, the set of effective keywords, and the local relation direction matrix from S2; by calculating the similarity between the global relation direction matrix and the local relation direction matrix, find the actual region in the document that matches the target semantics, and obtain the target region coordinate range of the semantic block to be extracted;

[0069] In this step, the global relational direction matrix is ​​calculated. With local matrix Based on the similarity, the actual region in the document that matches the target semantics is found, specifically:

[0070] Calculate the global relational direction matrix With local matrix The structural similarity can be measured using one of the following three methods (select the appropriate method based on the document complexity). The higher the similarity score, the more consistent the structures of the two documents are, and the more likely the corresponding region is to be the target semantic region.

[0071] By comparing the differences of matrix elements item by item, the structural deviation between the global relation direction matrix sub-region and the local matrix is ​​quantified. The smaller the deviation, the higher the similarity. The mean squared error (MSE) and mean absolute difference (MAE) are used to calculate the difference-to-similarity ratio. The formula is as follows:

[0072] ;

[0073] In the formula, Mean squared error; Mean absolute difference; Convert matrix structure difference to similarity, with a value range of... , The smaller the value, the smaller the structural difference and the higher the similarity; when the value is below the preset threshold, it is determined to be a potential matching region.

[0074] The direction consistency score is calculated by the proportion of elements with the same direction sign in the sub-region of the global relation direction matrix and the local matrix. The higher the proportion, the stronger the semantic consistency.

[0075] In screening China-Africa ,No "The set of valid elements" Record the direction sign of each element; for the sub-regions of the global relational direction matrix. ,extract The direction sign of the elements at corresponding positions; the number of elements with exactly the same direction sign is counted. The consistency score is calculated using the following formula:

[0076] ;

[0077] In the formula, The number of valid elements that are in the same direction; This represents the total number of valid elements in the local matrix. The consistency score is used to indicate the greater the semantic match between the two regions, and the region is given priority as a candidate for the target region.

[0078] Weighted similarity is calculated by assigning node weights and defining a similarity function. Each ,node and weight The average confidence level of the two is expressed as:

[0079] ;

[0080] In the formula, For semantic weights; It is a similarity function, which can be a Gaussian kernel, cosine similarity, or structural difference function; The first sub-region of the global relation direction matrix Line number Column elements; Sub-region of the global relation direction matrix The Line number Column elements; The weighted similarity score is as follows: the closer it is to 1, the higher the weighted structural similarity, and the better the matching region is.

[0081] In the global relational direction matrix In, according to the effective element range of the local matrix S (i.e. (Covered row / column index), traversing all possible sub-regions For each sub-region Computation and Local Matrices The similarity score is calculated; the similarity scores of all sub-regions are sorted, and the sub-region with the highest score is selected as the optimal matching sub-region; the row / column index corresponding to the optimal matching sub-region is associated with the set of semantic nodes, and the spatial coordinates of the semantic nodes corresponding to these indices in the document image are obtained; a minimum bounding rectangle (or irregular area) is generated with the coordinates of these semantic nodes as the boundary, and this area is the final spatial range of the semantic region to be extracted.

[0082] S4: Receive the structured JSON format OCR result from S2, the set of valid keywords, and the target area coordinate range from S3; process the OCR text block to generate structured semantic content;

[0083] In this step, to address complex scenarios such as field position offsets, cross-column distribution, and text block splitting / misalignment in documents, this invention employs a parameterized definition method to construct a configurable, scalable, and transferable unified extraction strategy. This ensures stable content extraction from documents of different layouts and OCR output formats, as shown in the table below:

[0084] Table 1 Parameter Definition Extraction Behavior Table

[0085]

[0086] By combining the above parameters, a standardized extraction strategy structure is formed, as shown in the following expression:

[0087] ;

[0088] In the formula, To extract the strategy structure, The strategy parameter name; The values ​​to be taken for the corresponding parameters; This represents the total number of parameters.

[0089] Based on the target region coordinate range in S3 and the predefined strategy structure, text block processing is performed according to the following steps to generate the final structured content:

[0090] From the OCR recognition results of the documents, all text blocks that intersect with the coordinates of the target semantic region or are within the tolerance offset range are selected; based on The "confidence weight" parameter filters out text blocks whose OCR recognition confidence is below a threshold; it also removes invalid blocks that are too short; thus obtaining a preliminary set of candidate text blocks.

[0091] According to extraction The candidate text blocks are sorted by coordinates according to the main extraction direction defined in the code. If there are obvious missing contents in the text block after sorting by the main direction (e.g., the "address" field only extracts "Gansu Province" and lacks "Yuzhong County, Lanzhou City"), the extraction range is expanded according to the "auxiliary expansion direction" (e.g., "downward") to supplement adjacent text blocks into the candidate set; thus, an ordered set of text blocks arranged in a fixed order is obtained.

[0092] For adjacent text blocks in an ordered set of text blocks, calculate their spatial distance (e.g., horizontal / vertical spacing) or overlap rate. If the distance is ≤ "merging threshold" (e.g., ≤ 10 pixels) or the overlap rate is > "merging threshold" (e.g., > 0.5), they are identified as "related text blocks" and merged. Merge the text blocks according to the "splicing rules" parameter in the extraction strategy (e.g., "splicing by coordinate order + space separation", merging "Lanzhou Aluminum Plant" and "First Branch Plant" into "Lanzhou Aluminum Plant First Branch Plant"); a preliminary merged text block set is obtained.

[0093] By checking the column coordinates of the text block (such as whether the x-coordinate crosses the preset column boundary), determine whether the field has "cross-column distribution" (e.g., "Username" is displayed in two columns: "Lanzhou" in the left column and "Aluminum Plant" in the right column).

[0094] Based on the "cross-column merging strategy" parameter, the column spacing is automatically detected, and cross-column text blocks are concatenated according to column order (from left to right); for multi-line fields (such as addresses, company names), line breaks are processed according to the "paragraph connection strategy" parameter; thus obtaining a complete set of semantic text blocks.

[0095] Based on the "filtering conditions" parameter, invalid information in the text is removed; if multiple text blocks correspond to the same semantics, a weighted score is calculated based on the "confidence weight", and the text block with the highest confidence is selected as the valid content, resulting in the purified text block;

[0096] The text content in the purified text block is concatenated into a complete string, which serves as the final structured semantic content of the target keyword and is then output to subsequent business processes.

[0097] Therefore, this invention adopts the above-mentioned method for extracting semantic content of cross-format documents based on layout semantic nodes. This method achieves cross-format universality by combining semantic nodes with a global relational direction matrix, improves fault tolerance through parameterization strategies, and enhances scalability through a unified semantic node index. It does not require large-scale labeled data, adopts a non-large model architecture to achieve low-cost and fast deployment, and ensures auditability and reduces mismatch rate through explicit modeling. It covers the entire text processing process through parameterization strategies, efficiently handles complex scenarios such as cross-column and cross-row, and fills the gap of weak adaptability to complex layouts.

[0098] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for extracting semantic content from cross-format documents based on layout semantic nodes, characterized in that, Includes the following steps: S1. Collect target documents of various types and formats, traverse all collected document samples, extract keywords that appear consistently, and obtain a standardized keyword dictionary by standardizing keyword index and filtering semantic nodes. S2 receives the standardized keyword dictionary from S1, performs format and quality checks on the document image to be processed, recognizes the text information in the image through OCR technology, and outputs the OCR result in structured JSON format. Semantic matching is performed on the OCR text block to locate the keywords that actually appear in the document and obtain a set of effective keywords; and the document layout is transformed into a structured semantic matrix based on the set of effective keywords to obtain a global relational direction matrix; the global relational direction matrix is ​​an m×m dimensional matrix, where m is the number of keywords in the standardized keyword dictionary, and the matrix elements represent the relative directional relationship between two corresponding semantic nodes, and the directional relationship is encoded using 4-chain code. The target keywords corresponding to the target semantic block to be extracted are determined, and related keywords that have a fixed positional association with the target keywords are screened. Based on the unified index number of the S1 standardized keyword dictionary, the dimensions and row and column indices of the local relation direction matrix are fully aligned with the global relation direction matrix. The relative positional relationship between the target keywords and adjacent keywords is modeled through the chain code method, and then the elements of the local relation direction matrix are assigned values ​​according to the 4-chain code rule to finally complete the local relation direction matrix of the target semantic block to be extracted. S3: Receive the global relation direction matrix, the set of effective keywords, and the local relation direction matrix from S2; by calculating the similarity between the global relation direction matrix and the local relation direction matrix, find the actual region in the document that matches the target semantics, and obtain the target region coordinate range of the semantic block to be extracted; S4 receives the structured JSON format OCR result from S2, the set of valid keywords, and the target area coordinate range from S3; processes the OCR text block to generate structured semantic content.

2. The method for extracting semantic content from cross-format documents based on layout semantic nodes according to claim 1, characterized in that, In S1, the standardized keyword dictionary includes keyword content, unique ID, and semantic category.

3. The method for extracting semantic content from cross-format documents based on layout semantic nodes according to claim 2, characterized in that, S2 also includes: The document is scanned or photographed to generate RGB / JPG / PNG format images, with varying image resolutions. Inclination angle ; The OCR process yields structured JSON data, with each record containing text content, text block coordinates, and recognition confidence score. The text block is marked as low quality.

4. The method for extracting semantic content from cross-format documents based on layout semantic nodes according to claim 3, characterized in that, In S3, the similarity calculation adopts any one or a combination of the following methods: The matrix structure dissimilarity is calculated based on mean squared error or mean absolute difference, and then converted into a range of values. The smaller the similarity value, the smaller the structural differences. The direction consistency score is obtained by statistically analyzing the proportion of the direction signs of the sub-regions of the global relation direction matrix and the effective elements in the local matrix that are consistent with each other. The higher the proportion, the stronger the semantic consistency. Calculate the weighted sum of node weights and the similarity function. The closer the weighted similarity is to 1, the higher the structural similarity. The node weight is the average of the confidence scores of the two corresponding semantic nodes. The similarity function can be a Gaussian kernel, cosine similarity, or structural difference function.

5. The method for extracting semantic content from cross-format documents based on layout semantic nodes according to claim 4, characterized in that, In S3, the process of determining the actual region in the document that matches the target semantics is as follows: Traverse all possible sub-regions of the global relation direction matrix according to the effective element range of the local matrix, calculate the similarity score between each sub-region and the local matrix and sort them, and select the sub-region with the highest score as the optimal matching sub-region. Associate the sub-region index with the semantic node, obtain the spatial coordinates of the corresponding semantic node, and use these coordinates as the boundary to generate a minimum bounding rectangle or irregular region as the semantic region to be extracted.

6. The method for extracting semantic content from cross-format documents based on layout semantic nodes according to claim 5, characterized in that, The processing of OCR text blocks described in S4 is as follows: Filter text blocks that are completely within the target area and within the tolerance offset range, and filter out low-confidence and short text invalid blocks according to confidence weight; The filtered text blocks are sorted by coordinates according to the predefined main extraction direction; if there are missing text contents after sorting, the extraction range is expanded according to the auxiliary expansion direction to supplement adjacent valid text blocks. Calculate the spatial distance or overlap rate of adjacent text blocks. If the distance is less than or equal to the merging threshold or the overlap rate is greater than the merging threshold, then merge the text blocks. Detect cross-column fields and concatenate them according to the cross-column merging strategy; handle line breaks of multi-line fields according to the paragraph connection strategy. Invalid information is removed based on filtering criteria, and the optimal content is selected from multiple text blocks with the same semantic meaning according to their credibility weights. Finally, they are spliced ​​together to form complete structured semantic content.

7. A computer device, characterized in that, include: A processor configured to be coupled to memory, read and execute instructions and / or program code in the memory to perform the method as described in any one of claims 1-6.

8. A computer-readable medium, characterized in that, The computer-readable medium stores computer program code that, when executed on a computer, causes the computer to perform the method as described in any one of claims 1-6.

Images