International logistics freight forwarding document automatic checking method and system based on semantic analysis

Abstract

The application discloses an international logistics freight forwarding document automatic checking method and system based on semantic analysis, and relates to the technical field of document identification. The international logistics freight forwarding document automatic checking method and system based on semantic analysis comprises the following steps: S1, reading document image data and checking data and performing pretreatment to obtain a single document image; S2, performing text detection, document layout detection, identification correction and layout semantic analysis to generate an anchor point set, a candidate set and a field set; S3, constructing a field attribution relationship graph to obtain a field attribution state, structured data and a candidate field list; and S4, performing neural symbol constraint decoding, evidence back-reading and confidence fusion on the structured data, and performing writing, review and electronic archive filing. The method solves the problem that the existing method relies on keywords triggering, identification confidence or nearest neighbor matching to determine field attribution, and when similar digital fields appear at the same time, the problem that the field attribution is incorrect although the digital identification is correct.

Description

Technical Field

[0001] This invention relates to the field of document recognition technology, specifically to an automatic verification method and system for international logistics freight forwarding documents based on semantic parsing. Background Technology

[0002] With the continuous expansion of international trade, the international logistics and freight forwarding industry plays a crucial information processing role in cross-border transportation organization, document circulation, customer collaboration, and information retention. International logistics and freight forwarding typically involve multiple processes such as transportation entrustment, booking, loading, customs clearance, settlement, and archiving. These processes rely heavily on numerous documents for information transmission and document retention. As enterprises improve their digital management capabilities, more and more logistics companies, freight forwarders, carriers, and customers are exchanging documents through online systems, email, web platforms, or application programming interfaces (APIs). The coexistence of scanned copies of paper documents, electronic documents, and uploaded photos is quite common. Simultaneously, technologies such as electronic document processing, document recognition, information extraction, data verification, and electronic file management are increasingly being applied to logistics and freight forwarding information processing scenarios, gradually shifting traditional document processing methods that rely on manual entry and verification towards automation, structuring, and traceability.

[0003] For example, the invention patent with announcement number CN114359918B discloses a method, apparatus, and computer device for extracting delivery note information. The method includes: acquiring a delivery note image; performing text box detection on the delivery note image based on a first model to obtain the coordinate information of the text boxes in the delivery note image; inputting the coordinate information of the text boxes and the delivery note image into a second model to obtain the text information corresponding to the text boxes; determining a target delivery note template that matches the delivery note image based on the text information, coordinate information, and a template library; generating a reference delivery note based on the text information and the target delivery note template; extracting key information from the reference delivery note to obtain the delivery note information corresponding to the delivery note image, thereby achieving the technical effect of improving the accuracy of delivery note image recognition.

[0004] For example, the invention patent with announcement number CN107633239B discloses a method for classifying and extracting invoice fields based on deep learning and OCR, including the following steps: selecting multiple standard invoice images, extracting the bounding rectangle outline of the official seal image from the standard invoice images as training samples for deep learning to obtain a deep learning model; classifying the bounding rectangle outline of the official seal of the invoice image to be identified through the deep learning model; if the classification is successful, outputting the invoice type of the invoice image to be identified and determining that the invoice image to be identified is a standard invoice image; if the classification fails, determining that the invoice image to be identified is a non-standard invoice image, thus realizing automatic classification of invoices and intelligent extraction of invoice content, providing technical support for rapid retrieval and acquisition of invoice information.

[0005] Existing methods can convert partial text content in documents or invoices into structured information through text box detection, image recognition, template matching, document classification, and field extraction. However, their processing focus is usually on recognizing text content in images, determining document types, or matching template fields. In scenarios where multiple similar numerical fields appear simultaneously in international logistics and freight forwarding documents, existing methods still tend to rely on keyword triggering, recognition confidence, or proximity relationships to determine field attribution. They struggle to fully integrate field titles, page areas, table rows and columns, symbol adjacency, and contextual semantics for comprehensive disambiguation. This results in the numerical content itself being correctly identified, but its corresponding fields are prone to misattribution, thus affecting the accuracy of subsequent field validation, system writing, and review processes.

[0006] Therefore, in order to address the above issues, there is an urgent need for an automatic verification method and system for international logistics freight forwarding documents based on semantic parsing. Summary of the Invention

[0007] Technical problems to be solved To address the shortcomings of existing technologies, this invention provides an automatic verification method and system for international logistics freight forwarding documents based on semantic parsing. This solves the problem that existing methods often rely on keyword triggering, recognition confidence, or nearest neighbor matching to determine field attribution, which can easily lead to correct number recognition but incorrect field attribution when similar numeric fields appear simultaneously.

[0008] Technical solution

[0009] To achieve the above objectives, this invention provides the following technical solution: an automatic verification method for international logistics freight forwarding documents based on semantic parsing, comprising: S1, uploading international logistics freight forwarding document files and reading document image data and verification data, preprocessing the document image data and verification data to obtain a single-page document image; S2, performing text detection, document layout detection, recognition correction, and layout semantic parsing on the single-page document image, generating a set of field title anchor points, a set of field candidates, and a set of multiple candidate fields based on the text line content; S3, constructing a field attribution relationship graph based on the set of field candidates and the set of field title anchor points, obtaining the field attribution status, structured data, and a list of candidate fields; S4, performing neural symbol constraint decoding, evidence rereading, and confidence fusion on the structured data, generating a field processing level, and writing, reviewing, and archiving the freight forwarding document processing system.

[0010] Furthermore, the specific process of uploading international logistics freight forwarding documents and reading document image data and verification data is as follows: Upload international logistics freight forwarding documents through a web page and application programming interface (API); read document image data, which includes: file format identifier, upload timestamp, page number, page pixel matrix, page pixel width value, and page pixel height value; obtain airline code records, IATA port code records, currency code records, and historically confirmed field records through the verification data interface; file format identifiers include portable document format, scanned image, and photographed image.

[0011] Furthermore, the specific process of preprocessing the document image data and verification data to obtain single-page document images is as follows: when the file format is identified as a portable document format, the international logistics freight forwarding document document is split into single-page document images; when the file format is identified as a scanned image or photographed image, the single-page document image is read; the single-page document image is subjected to denoising, skew correction, enhancement, page orientation correction, and page boundary completion; the page pixel width and page pixel height values ​​are standardized using a zero-mean unit variance standardization algorithm; and the grayscale values ​​of the page pixel matrix are normalized using a maximum-minimum-maximum normalization algorithm.

[0012] Furthermore, the specific process of performing text detection, document layout detection, recognition correction, and layout semantic parsing on a single-page document image is as follows: The single-page document image is input into a differentiable binarized text detection model, which outputs the coordinates of the text line regions; the single-page document image is processed using a DETR-based document layout detection method to obtain the coordinates of the table region and the seal occlusion region, and the table region and seal occlusion state are incorporated into the semantic structure expression of the document layout; convolutional recurrent neural network recognition and Transformer context correction are sequentially performed on the image cropping blocks corresponding to the text line region coordinates to obtain the text line content and text line recognition confidence; based on the page number, text line region coordinates, page pixel width value, page pixel height value, and text line content, a... The LayoutReader reading order prediction method and two-dimensional positional encoding are used to generate an original text block list. The original text block list includes text block identifiers, page numbers, text line content, text line area coordinates, text line recognition confidence, and original document cropping blocks. Layout semantic parsing is performed on the original text block list, table area coordinates, and seal occlusion area coordinates to generate the document layout semantic structure. Layout semantic parsing uses a document multimodal pre-trained encoding method and a relational predictive layout analysis method to jointly express text line content, text line area coordinates, table area coordinates, and page area relationships. According to the document layout semantic structure, the header area, detail area, summary area, remarks area, and signature association area are divided, and area hierarchy markers are generated.

[0013] Furthermore, the specific process of generating the field title anchor set, field candidate set, and multi-candidate field set based on the text line content is as follows: Field title recognition is performed on the text line content to obtain the field title anchor set; sequence labeling and span extraction are performed on the numbers, monetary symbols, and code text in the text line content to obtain the field candidate set; the field candidate set includes field candidate values, field candidate coordinates, the region to which the field candidate belongs, adjacent text of the field candidate, field candidate currency, field candidate recognition confidence, and field candidate cropping tiles. Adjacent text of the field candidate is the text line content located on the same page as the field candidate coordinates and within the candidate context window; when there are N numeric field candidates on the same page in the field candidate set, and the adjacent text of the field candidate, the region to which the field candidate belongs, or the field title anchor set corresponds to different field title anchors, the corresponding field candidate set is classified into the multi-candidate field set, preserving the competitive relationship between numeric candidates, and not directly determining field ownership based on keyword triggering results, text line recognition confidence, and nearest neighbor distance.

[0014] Furthermore, the specific process of constructing a field attribution relationship graph based on the field candidate set and the field title anchor point set is as follows: Geometric distance calculation is performed on the field candidate coordinates and field title anchor point coordinates to obtain title distance features; regional consistency analysis is performed on the region to which the field candidate belongs and the region level label to obtain region level features; table row and column relationship analysis is performed on the field candidate coordinates, table region coordinates, and text line region coordinates to obtain table row and column position features; symbolic adjacency analysis is performed on adjacent text and currency of the field candidate to obtain symbolic adjacency features; contextual semantic encoding is performed on adjacent text, field title anchor point set, and text line content of the field candidate to obtain contextual semantic features; joint image quality analysis is performed on the field candidate recognition confidence, seal occlusion region coordinates, and field candidate cropped patch to obtain recognition reliability features; and the title distance is then calculated. The candidate pairing feature vector is composed of features, region-level features, table row and column position features, symbol adjacency features, contextual semantic features, and reliable recognition features. Using field title anchors and field candidates as heterogeneous nodes, spatial relationship edges, row and column relationship edges, table region relationship edges, symbol adjacency edges, and semantic adjacency edges are generated based on the coordinate distance between the field title anchor and the field candidate, horizontal projection overlap, vertical projection overlap, table region affiliation, symbol adjacency state, and semantic similarity state, respectively. This constructs a field affiliation relationship graph, which is a heterogeneous graph containing title anchor nodes, candidate field nodes, and multiple types of relationship edges. The candidate pairing feature vector is input into the field affiliation relationship graph as the edge attribute of the field affiliation relationship edges. Graph attention propagation and biaffine pairing processing are performed on the field affiliation relationship graph to obtain field affiliation matching values ​​and field affiliation separation values.

[0015] Further, the specific process for obtaining the field attribution status, structured data, and candidate field list is as follows: Positive and negative sample pairs of field titles, values, coordinates, and currencies are constructed from historically confirmed field records. Field attribution thresholds and field separation thresholds are obtained using the RankNet pairwise ranking learning method. When the field attribution matching value is greater than or equal to the field attribution threshold, and the field attribution separation value is greater than or equal to the field separation threshold, a field attribution lock flag is generated. When the field attribution matching value is greater than or equal to the field attribution threshold, and the field attribution separation value is less than the field separation threshold, a field attribution competition flag is generated. When the field attribution... When the matching value is less than the field attribution threshold, an unreliable field attribution flag is generated. Structured information is extracted from the bill of lading number, total number of pieces, gross weight, volume, chargeable weight, product name, port of shipment code, port of destination code, unit price, quantity, freight amount, and currency corresponding to the field attribution lock flag, generating key-value pair structured data. The structured data includes field name, field value, confidence level, initial verification status value, field source page number, field source coordinates, field attribution matching value, field attribution status, and original document cutout blocks. Candidate fields corresponding to the field attribution competition flag and the unreliable field attribution flag are retained in the candidate field list.

[0016] Furthermore, the specific process of performing neural symbolic constraint decoding, evidence rereading, and confidence fusion on structured data is as follows: Neural symbolic constraint decoding is performed on the structured data. Combining airline code records, IATA port code records, and currency code records, consistency checks are performed on the reference relationships between field names and field values, code existence relationships, currency binding relationships, and numerical association relationships to obtain the field verification status. The field verification status includes pass status, conflict status, and missing evidence status. For structured data with conflict status and missing evidence status, retrieval-enhanced evidence rereading is performed, retrieving information from the original text block list... The original document cropping block and adjacent text blocks corresponding to the field source coordinates are retrieved. The adjacent text blocks are those with the same page number as the field source and whose text line area coordinates fall within the evidence readback window. The candidate field list is read, and the abnormal field location result is generated. Dempster-Shafer evidence fusion is performed on the text line identification confidence, field attribution matching value, field attribution status, field verification status, and abnormal field location result. A comprehensive confidence and a review candidate set are generated through the distribution-independent conformal prediction method. The review candidate set retains the field candidates corresponding to the field attribution competition marker and the field attribution unreliable marker.

[0017] Further, the specific process of generating field processing levels and writing, reviewing, and archiving electronic documents in the freight forwarding document processing system is as follows: Based on the overall confidence level, review candidate set, and field verification status, a cost-sensitive classification decision method is used to generate field processing levels, including direct write level, manual review level, and no-fill level; when the field processing level is direct write level, the field name and field value are written into the form input box of the freight forwarding document processing system; when the field processing level is manual review level or no-fill level, the corresponding form input box is highlighted in red, the original document clipping block is displayed, and a review task is generated; after review confirmation, the international logistics freight forwarding document files, file format identifiers, upload timestamps, structured data, field ownership status, field verification status, overall confidence level, abnormal field location results, review candidate set, and modification records are used to generate electronic archive records; hash digest calculation and timestamp chain sealing are performed on the electronic archive records, and an archive database is created and stored to verify the integrity of the electronic archive records after archiving.

[0018] The second aspect of this invention provides an automatic verification system for international logistics freight forwarding documents based on semantic parsing, comprising: a data acquisition and preprocessing unit, used to upload international logistics freight forwarding document files and read document image data and verification data, and preprocess the document image data and verification data to obtain a single-page document image; a layout semantic parsing and field candidate generation unit, used to perform text detection, document layout detection, recognition correction and layout semantic parsing on the single-page document image, and generate a set of field title anchor points, a set of field candidates and a set of multiple candidate fields based on the text line content; a field attribution disambiguation unit, used to construct a field attribution relationship graph based on the set of field candidates and the set of field title anchor points, and obtain the field attribution status, structured data and a list of candidate fields; and a confidence verification and archiving unit, used to perform neural symbol constraint decoding, evidence rereading and confidence fusion on the structured data, generate field processing levels and write, review and archive electronically in the freight forwarding document processing system.

[0019] Beneficial effects

[0020] The present invention has the following beneficial effects: (1) This invention expresses the text line content, page area relationship and field candidate relationship in a unified manner through layout semantic parsing, reading order prediction, field title recognition, sequence labeling and span extraction, so that the field title anchor set, field candidate set and multi-candidate field set can be associated in the same processing link, thereby realizing the continuous processing effect of document text from character recognition to field candidate organization, effectively solving the problem of unstable field candidate organization in the prior art that only recognizes text.

[0021] (2) This invention preserves the competitive relationship between candidates of numeric fields on the same page and does not directly determine the field attribution based on keyword triggering results, text line recognition confidence and nearest neighbor distance. This enables similar numeric fields to enter the subsequent field attribution disambiguation process for unified judgment, thereby achieving the effect of delayed judgment and competitive retention of similar numeric fields. This effectively solves the problem in the prior art that the numeric recognition is correct but the field attribution is easily misjudged in advance.

[0022] (3) In this invention, candidate pairing feature vectors are constructed by using title distance features, regional hierarchy features, table row and column position features, symbol adjacency features, contextual semantic features and recognition reliability features. The field attribution status is obtained by combining field attribution relationship graph, graph attention propagation and double affine pairing processing. This achieves the effect of multi-dimensional disambiguation and structured extraction of similar digital fields, effectively solving the problem that it is difficult to accurately determine field attribution by relying only on a single positional relationship or recognition confidence in the prior art.

[0023] (4) In this invention, graph attention propagation, dual affine pairing, RankNet pairwise ranking learning method, Dempster-Shafer evidence fusion and distribution-independent conformal prediction method are respectively bound to single-page document images, text line area coordinates, table area coordinates, field candidate coordinates, field title anchor point set, original document cropping blocks and evidence readback window, so that the field attribution status, field verification status and field processing level are all derived from the locatable document image evidence, thereby reducing field mismatch in the structured extraction process of similar digital fields.

[0024] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0025] Figure 1 The flowchart of the automatic verification method for international logistics freight forwarding documents based on semantic parsing of the present invention is shown below. Figure 2 This is a structural diagram of the semantic parsing-based automatic verification system for international logistics freight forwarding documents of the present invention. Figure 3 This is a comparison chart of field attribution matching values ​​and field attribution separation values ​​in this invention; Figure 4 Decomposition diagram of the contribution of candidate features of competing fields in this invention; Figure 5 This is a flowchart of the field attribution status determination decision process of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Please see Figures 1-5 This invention provides a technical solution: an automatic verification method for international logistics freight forwarding documents based on semantic parsing, comprising the following steps: S1, uploading international logistics freight forwarding document files and reading document image data and verification data, preprocessing the document image data and verification data to obtain a single-page document image; S2, performing text detection, document layout detection, recognition correction, and layout semantic parsing on the single-page document image, generating a set of field title anchor points, a set of field candidates, and a set of multiple candidate fields based on the text line content; S3, constructing a field attribution relationship graph based on the set of field candidates and the set of field title anchor points, obtaining field attribution status, structured data, and a list of candidate fields; S4, performing neural symbol constraint decoding, evidence rereading, and confidence fusion on the structured data, generating field processing levels, and writing, reviewing, and archiving the freight forwarding document processing system.

[0028] Specifically, the process of uploading international logistics freight forwarding documents and reading document image data and verification data is as follows: International logistics freight forwarding documents are uploaded via a web page and application programming interface (API). Document image data is read, including: file format identifier, upload timestamp, page number, page pixel matrix, page pixel width value, and page pixel height value. The file format identifier is derived from the file signature and extension of the uploaded byte stream. The upload timestamp is generated by the server-side request arrival time. The page number is extracted by the document parser according to the physical page order. The page pixel matrix is ​​a two-dimensional array of grayscale values ​​converted from a single-page document image by an image decoder. The page pixel width and page pixel height values ​​are read from the size metadata in the image file header. The verification data interface obtains airline code records, IATA port code records, currency code records, and historically confirmed field records, respectively from the airline master data management system, IATA standard code library, and ISO... The 4217 currency code table and historical business confirmation archive are periodically synchronized. Among them, the IATA port code records are used to represent the origin, destination or transit codes in international air freight forwarding documents. The historical confirmed field records are formed by field records that have been manually reviewed and confirmed, successfully written into the business system and completed electronic archiving. The field range corresponds to the set of field title anchor points, which is used for subsequent positive and negative sample pair construction, field attribution threshold determination and field separation threshold determination. The file format identifier includes portable document format, scanned image and photographed image. This classification is based on the magic number signature of the file header and MIME type.

[0029] This implementation plan achieves unified access, format recognition, structured reading of image data, and preparation of verification reference data for international logistics and freight forwarding documents. It forms a unified data foundation covering document pages, pixel base, document source, and business verification basis, providing stable input for subsequent format semantic parsing, field candidate generation, and field attribution disambiguation. It effectively reduces field recognition and field verification deviations caused by differences in document source, inconsistent page order, missing image basic information, or insufficient verification reference records.

[0030] Specifically, the preprocessing of document image data and verification data to obtain single-page document images is as follows: When the file format is identified as a portable document format, the international logistics freight forwarding document is split into single-page document images. The splitting operation is performed page-by-page rasterization of the page content by a PDF rendering engine. When the file format is identified as a scanned image or photographed image, the single-page document image is read. The single-page document image is then subjected to denoising, skew correction, enhancement, page orientation correction, and page boundary completion. Denoising uses a nonlocal mean filtering algorithm, skew correction utilizes projection contour analysis and Hough transform, enhancement uses a contrast-limited adaptive histogram equalization method, page orientation correction is completed by a pre-trained text orientation classification model, and page boundary completion is achieved through morphological dilation and minimum bounding rectangle fitting. The processed page pixel matrix, page pixel width value, page pixel height value, and text region position are then used as the basis for subsequent text line detection, title distance feature calculation, and table row and column positions. Feature calculation, reliable feature calculation, and field attribution relationship graph construction are performed. The page pixel width and height values ​​are standardized using a zero-mean unit variance standardization algorithm. The mean and standard deviation are obtained by statistically analyzing the page pixel width and height values ​​of all single-page document images in the current batch. If the current batch contains only one page or the standard deviation is zero, the corresponding standardization result is set to zero. The mean and standard deviation of the width and height values ​​are calculated, and the original values ​​are mapped to Z-score values ​​under a standard normal distribution. The grayscale values ​​of the page pixel matrix are normalized using a maximum and minimum value normalization algorithm. The maximum and minimum values ​​are taken from the maximum and minimum grayscale values ​​in the page pixel matrix of the current single-page document image, respectively. The normalization target interval is [0, 1]. When the maximum grayscale value equals the minimum grayscale value, the normalized grayscale value is set to zero, ensuring that the image size, coordinates, and grayscale-related parameters in subsequent field attribution calculations are on a uniform numerical scale.

[0031] This implementation scheme achieves unified image representation, layout quality correction, and numerical scale unification of document pages with different file formats. It forms a single-page document image data foundation with consistent orientation, complete boundaries, controlled noise, and consistent grayscale range. This enables page size, page coordinates, and pixel grayscale to participate in subsequent text line detection, layout semantic parsing, and field attribution disambiguation under the same calculation caliber. It reduces the omission of field candidates, field position offset, and field attribution misjudgment caused by page tilt, abnormal orientation, missing boundaries, image quality fluctuations, and differences in numerical scale.

[0032] Specifically, the process of text detection, document layout detection, recognition and correction, and layout semantic parsing for single-page document images is as follows: The single-page document image is input into a differentiable binarized text detection model, which outputs text line region coordinates. These coordinates are represented by the top-left and bottom-right corner coordinates in the page pixel matrix coordinate system and are bound to the page number. The differentiable binarized text detection model generates a probability map and a threshold map through an adaptive threshold prediction network, and performs differentiable binarization on both to obtain the text line bounding boxes. Based on the original document cropping blocks and field source coordinates corresponding to historically confirmed field records, the differentiable binarized text detection model performs international logistics freight forwarding document layout adaptation, enabling the text line region coordinates to serve as the basic input for subsequent text line content recognition, table row and column position feature calculation, and field attribution relationship graph construction. The single-page document image is then processed using a DETR-based document layout detection method. DETR is an end-to-end target detection method based on a detection transformer. The DETR-based detection method uses an encoder-decoder ensemble prediction. The architecture directly outputs the category labels and bounding box coordinates of the table and seal areas, obtaining the coordinates of the table area and the seal occlusion area, and incorporates the table area and seal occlusion state into the semantic structure of the document layout. For the image cropping blocks corresponding to the text line area coordinates, convolutional recurrent neural network (RNN) recognition and Transformer context correction are sequentially performed to obtain the text line content and text line recognition confidence. The RNN uses convolutional layers to extract local image features and recurrent layers to model character sequence dependencies. Transformer is a sequence modeling technique based on a self-attention mechanism. Transformer correction uses a multi-head self-attention mechanism to perform contextual semantic error correction on the recognized text and outputs a confidence score. Transformer context correction uses the text line content on the same page, the set of field title anchor points, and the text line content within the candidate context window as context input. The text line recognition confidence is obtained by fusing the recognition probability output by the RNN and the text sequence probability after Transformer correction.

[0033] Based on page number sequence, text line area coordinates, page pixel width, page pixel height, and text line content, the LayoutReader reading order prediction method and two-dimensional positional encoding are used to generate an original list of text blocks. LayoutReader is a layout understanding method that combines text content and two-dimensional layout position to predict document reading order. LayoutReader encodes the text block layout and content in a sequence-to-sequence manner to predict the text block arrangement that conforms to human reading order. Two-dimensional positional encoding embeds the center coordinates and width and height of the text lines into continuous vectors, along with the text semantic vectors. The quantity fusion is based on the injection of spatial location priors. The two-dimensional location encoding takes the top left corner of the page as the origin of the coordinate system. The center coordinates of the text line, the width of the text line, and the height of the text line are normalized according to the page pixel width and page pixel height values, respectively, so that the text block position has a uniform expression standard under different page sizes. The original text block list includes the text block identifier, page number, text line content, text line area coordinates, text line recognition confidence, and original document cropping block. The text block identifier is a unique index automatically generated by the system, and the original document cropping block is a sub-image cropped from the original image based on the text line area coordinates.

[0034] The original text block list, table area coordinates, and seal occlusion area coordinates are subjected to layout semantic parsing to generate a document layout semantic structure. The layout semantic parsing process integrates three modalities: visual pixels, text semantics, and layout coordinates, and extracts the global representation of the document through cross-modal attention interaction. The layout semantic parsing adopts a document multimodal pre-trained encoding method and a relational prediction layout analysis method to jointly express text line content, text line area coordinates, table area coordinates, and page area relationships. The document multimodal pre-trained encoding method is self-supervised and trained on a large number of unlabeled documents to learn the alignment relationship between text and layout. The relational prediction layout analysis method constructs a document structure tree by predicting the spatial adjacency, parent-child subordination, and reading order relationships between text blocks, and combines the table area coordinates and seal occlusion area coordinates to predict the table area affiliation relationship and occlusion influence relationship of text blocks. This enables the document layout semantic structure to provide layout basis for field candidate affiliation areas, area hierarchy features, table row and column position features, and reliable identification features. The document layout is divided into header, detail, summary, remarks, and signature / stamp related areas according to the semantic structure of the document format. The signature / stamp related area is a page area where the coordinates of the text line area overlap with the coordinates of the stamp-obscured area, or where the coordinates of the text line area overlap with the coordinates of the stamp-obscured area in either horizontal or vertical projection. A region hierarchy marker is generated for each region. This region hierarchy marker records the nesting depth and subordination type of each region within the document's logical structure, providing structural priors for subsequent disambiguation.

[0035] This implementation scheme achieves joint parsing of text lines, table areas, seal-occupied areas, reading order, and layout hierarchy in single-page document images. This results in a list of original text blocks and a semantic structure of the document layout that includes text content, spatial coordinates, recognition confidence, cropped evidence, and region hierarchy markers. This ensures that the field content in international logistics freight forwarding documents no longer participates in subsequent processing as isolated text. Instead, it enters the field candidate generation and field attribution disambiguation under the joint constraints of page area, table position, visual occlusion, and document logical structure. This reduces field candidate misalignment and similar number field attribution deviations caused by disordered reading order, misjudgment of table areas, seal occlusion interference, or missing layout hierarchy.

[0036] Specifically, the process of generating a set of field title anchor points, a set of candidate fields, and a set of multiple candidate fields based on text line content is as follows: Field title recognition is performed on the text line content to obtain a set of field title anchor points. The field title recognition is determined based on the semantic matching result between the text line content and the corresponding title text in the set of field title anchor points, the page area where the text line area coordinates are located, and the area level marker. The set of field title anchor points includes anchor points for the bill of lading number field, total shipment number field, gross weight field, volume field, chargeable weight field, product name field, port of shipment code field, port of destination code field, unit price field, quantity field, freight amount field, and currency field. Each anchor point records its corresponding title text content, coordinate boundaries, and page number. Sequence labeling and span extraction are performed on numbers, currency symbols, and code text within text lines to obtain a set of candidate fields. Sequence labeling uses the BIOE pattern to identify the start and end positions of number strings, currency symbols, and code text. Span extraction extracts the labeled segments along with their coordinates. The BIOE pattern is a sequence labeling method that identifies start, middle, end, and non-target positions to mark the start and end boundaries of number strings, currency symbols, and code text within the text line content. Number strings, currency symbols, and code text are respectively used as target categories for sequence labeling. Consecutive labeled segments with the same category are merged into the same candidate field. The confidence level for field candidate recognition is determined by... The confidence level of the text line recognition and the probability of the annotation position of the segment to be annotated are jointly determined. The field candidate set includes field candidate value, field candidate coordinates, field candidate region, field candidate adjacent text, field candidate currency, field candidate recognition confidence and field candidate cropping block. The field candidate adjacent text is the text line content that is on the same page number as the field candidate coordinate and is located in the candidate context window. The candidate context window consists of the text line where the field candidate coordinate is located and the text lines that are adjacent to it in the reading order on the same page. The field candidate region is determined according to the page area where its coordinate is located. The field candidate currency is inferred from the adjacent currency symbol or currency code.

[0037] When there are N numeric field candidates on the same page in the field candidate set, and the adjacent text, the region to which the field candidate belongs, or the set of field title anchors correspond to different field title anchors, the corresponding field candidate set is assigned to the multi-candidate field set. N is the number of numeric field candidates that satisfy the field candidate set within the same page number sequence. N is an integer greater than or equal to 2, and N is less than or equal to the total number of field candidates in the field candidate set. The competitive relationship between numeric candidates is preserved. Field attribution is not directly determined based on keyword triggering results, text line recognition confidence, and nearest neighbor distance. The field candidates, corresponding field title anchors, and competitive relationships in the multi-candidate field set are input into the field attribution relationship graph. After generating field attribution matching values ​​and field attribution separation values ​​through graph attention propagation and double affine pairing, the field attribution status is determined. This strategy aims to fully preserve the context of ambiguous candidates, prevent field mismatch caused by premature hard assignment, and provide a complete candidate space for subsequent graph neural network disambiguation.

[0038] This implementation scheme realizes the structured generation of field title anchor point set, field candidate set and multi-candidate field set, forming a candidate expression basis that includes field position, region, adjacent text, currency information, identification confidence and pruning evidence. This ensures that similar numerical fields retain a complete competitive space before entering field attribution disambiguation, avoiding premature field locking caused by keyword triggering results, text line identification confidence or nearest neighbor distance, and improving the reliability of attribution determination for similar numerical fields such as total number of items, gross weight, chargeable weight, volume, unit price, quantity and freight amount.

[0039] Specifically, the process of constructing a field attribution graph based on the field candidate set and the field title anchor point set is as follows: Geometric distance is calculated for the field candidate coordinates and the field title anchor point coordinates to obtain the title distance feature. The geometric distance calculation uses both Chebyshev distance and Canberra distance, and the distance is normalized to the [0, 1] interval based on the page pixel width and page pixel height values ​​to eliminate page size differences. Regional consistency analysis is performed on the region to which the field candidate belongs and the region hierarchy markers to obtain the region hierarchy feature. The regional consistency analysis outputs Boolean or categorical features such as whether the candidate and anchor point belong to the same region, whether they are at the same level, and the region type matching degree. Regions belonging to the same region are considered consistent, regions with direct parent-child hierarchy are considered adjacency, and regions not belonging to the same region and without direct parent-child hierarchy are considered conflict. The table row and column relationship is analyzed on the coordinates of candidate fields, table area coordinates, and text line area coordinates to obtain table row and column position features. The table row and column relationship analysis determines the table row and column indices of the candidate by calculating the intersection-union ratio (IU) of the projections of the candidate and the first row and first column text lines of the table area. The IU is obtained by dividing the overlap length of the candidate's projection interval with the projection interval of the first row or first column text line by the union length of the two. The connection threshold of the row and column relationship edges is determined by the distribution of IU of the projections of similar fields in historically confirmed field records. Symbolic adjacency analysis is performed on the adjacent text and currency of the candidate field to obtain symbolic adjacency features. Symbolic adjacency analysis detects whether amount symbols, currency codes, or field separator symbols appear in the windows before and after the candidate, and encodes them as one-hot vectors. The detection range of symbolic adjacency analysis is the text line where the candidate is located and the text line content in the candidate's context window, and the amount symbols, currency codes, and field separator symbols in the same text line or adjacent text lines in the reading order as the candidate are considered as adjacency objects. Contextual semantic encoding is performed on the adjacent text of the candidate field, the set of field title anchors, and the text line content to obtain contextual semantic features. The contextual semantic encoding uses a pre-trained language model to encode the candidate adjacent text and each title anchor text into sentence vectors, and calculates the cosine similarity as the semantic affinity. The semantic similarity state is determined based on the comparison result of the cosine similarity and the semantic similarity threshold. The semantic similarity threshold is determined by the statistical results of the semantic affinity between the candidate field and the field title anchor in the historically confirmed field records. Image quality joint analysis is performed on the field candidate recognition confidence, the coordinates of the stamp occlusion area, and the field candidate cropped patch to obtain reliable recognition features. The image quality joint analysis calculates the Laplacian variance of the candidate cropped patch to evaluate sharpness, and combines the stamp occlusion area ratio with the recognition confidence to obtain a reliability score. The stamp occlusion area ratio is obtained by dividing the overlapping area of ​​the field candidate cropped patch and the stamp occlusion area coordinates by the area of ​​the field candidate cropped patch. The reliability score is obtained by fusing the sharpness normalization result, the inverse vector of the stamp occlusion area ratio, and the field candidate recognition confidence, with a value range of [0, 1].The title distance feature, region hierarchy feature, table row and column position feature, symbol adjacency feature, context semantic feature and recognition reliability feature are combined into a candidate pair feature vector; during combination, each feature component is L2 normalized to ensure that features from different sources have similar magnitudes in the vector space.

[0040] Using field title anchors and field candidates as heterogeneous nodes, spatial relationship edges, row-column relationship edges, table area relationship edges, symbol adjacency edges, and semantic adjacency edges are generated based on the coordinate distance between the field title anchors and field candidates, horizontal projection overlap, vertical projection overlap, table area affiliation, symbol adjacency status, and semantic similarity status, respectively. This constructs a field affiliation graph, which is a heterogeneous graph containing title anchor nodes, candidate field nodes, and various types of relationship edges. Spatial relationship edges are established based on the coordinate distance ranking result between the field title anchors and field candidates and the bidirectional nearest neighbor rule; row-column relationship edges are established based on the overlap ratio of field candidates and text row areas in horizontal or vertical projection; table area relationship edges are established when the field title anchor and field candidate belong to the same table area; symbol adjacency edges are established when there is a monetary symbol, currency code, or field separator symbol corresponding to the field title anchor in the adjacent window of the candidate; and semantic adjacency edges are established based on the semantic similarity ranking result between the adjacent text of the field candidate and the text of the field title anchor. Candidate pairing feature vectors are input into the field affiliation graph as edge attributes of the field affiliation relationship edges. The graph attention propagation and biaffine pairing processes are performed to obtain field attribution matching values ​​and field attribution separation values. These values ​​are used to distinguish between field attribution locked markers, field attribution competing markers, and field attribution unreliable markers. Graph attention propagation uses a multi-head attention mechanism to aggregate heterogeneous neighbor information to update node representations. It takes the title anchor node representation, candidate field node representation, relation edge type, and candidate pairing feature vector as inputs, enabling information transmission between different field candidates under spatial, row / column, table area, symbolic adjacency, and semantic adjacency relationships. Biaffine pairing inputs the final representations of the title node and candidate nodes into a bilinear affine transformation, outputting scalar matching scores and separation scores. The scalar matching score is normalized to obtain the field attribution matching value, which ranges from [0, 1]. The separation score is obtained by the difference between the normalized matching result corresponding to the maximum matching field and the normalized matching result corresponding to the adjacent competing field, which also ranges from [0, 1]. The field attribution matching value represents the degree of matching between the field candidate and the field corresponding to the field title anchor, while the field attribution separation value represents the degree of distinction between the maximum matching field and the adjacent competing field.

[0041] In this embodiment, Table 1 is a data table of field attribution disambiguation features and attribution status, which records in detail the title distance features, table row and column position features, contextual semantic features, field attribution matching values, field attribution separation values ​​and field attribution status of different field candidate values ​​under the corresponding field title anchor points, which is used to illustrate the influence of candidate pairing feature vectors on field attribution status. Among them, when the candidate value "12" corresponds to the anchor point of the total number segment, the title distance feature is 0.06, the table row and column position feature is 0.94, the contextual semantic feature is 0.90, the field attribution matching value is 0.93, the field attribution separation value is 0.31, and the field attribution status is marked as locked; when the candidate value "12" corresponds to the anchor point of the unit price field, the title distance feature is 0.51, the table row and column position feature is 0.28, the contextual semantic feature is 0.22, the field attribution matching value is 0.26, the field attribution separation value is 0.05, and the field attribution status is marked as unreliable; when the candidate value "128.60" corresponds to the anchor point of the gross weight field, the title distance feature is 0.10, the table row and column position feature is 0.87, the contextual semantic feature is 0.82, the field attribution matching value is 0.88, the field attribution separation value is 0.24, and the field attribution status is marked as locked; When the selected value "128.60" corresponds to the anchor point of the billing weight field, the title distance feature is 0.12, the table row and column position feature is 0.82, the contextual semantic feature is 0.79, the field attribution matching value is 0.86, the field attribution separation value is 0.03, and the field attribution status is marked as field attribution competition. When the candidate value "1.47" corresponds to the anchor point of the volume field, the title distance feature is 0.07, the table row and column position feature is 0.90, the contextual semantic feature is 0.86, the field attribution matching value is 0.91, the field attribution separation value is 0.27, and the field attribution status is marked as field attribution locked. When the candidate value "4200.00" corresponds to the anchor point of the freight amount field, the title distance feature is 0.05, the table row and column position feature is 0.88, the contextual semantic feature is 0.84, the field attribution matching value is 0.92, the field attribution separation value is 0.30, and the field attribution status is marked as field attribution locked. Overall, the data in Table 1 are all within the normalized numerical range, which can reflect the differences in the attribution between the candidate values ​​of the fields and the anchor points of different field titles.

[0042] Table 1. Field Attribution Disambiguation Features and Attribution Status Data 12 Total component digital segment anchor point 0.06 0.94 0.90 0.93，0.31 Field ownership lock mark 12 Unit price field anchor 0.51 0.28 0.22 0.26，0.05 Unreliable field attribution marker 128.60 Gross weight field anchor point 0.10 0.87 0.82 0.88，0.24 Field ownership lock mark 128.60 Billing weight field anchor 0.12 0.82 0.79 0.86，0.03 Field ownership competition flag 1.47 Volume field anchor 0.07 0.90 0.86 0.91，0.27 Field ownership lock mark 4200.00 Anchor point for shipping amount field 0.05 0.88 0.84 0.92，0.30 Field ownership lock mark like Figure 3The chart shows a comparison of field attribution match values ​​and field attribution separation values. As can be seen from Table 1, the field attribution match value indicates the degree of matching between a candidate field and its corresponding field title anchor, while the field attribution separation value indicates the degree of differentiation between the best-matching field and its adjacent competing fields. In the chart, solid bars represent field attribution match values, and diagonally filled bars represent field attribution separation values. Different fill colors correspond to field attribution locked, field attribution competing, and field attribution unreliable markers. When the candidate value "12" corresponds to the total number field anchor, the field attribution matching value is 0.93 and the field attribution separation value is 0.31, both of which support the field attribution locking flag. When the candidate value "12" corresponds to the unit price field anchor, the field attribution matching value is 0.26 and the field attribution separation value is 0.05, indicating that the layout position, table rows and columns, and contextual semantic support between this candidate and the unit price field anchor are insufficient, so the corresponding field attribution is marked as unreliable. When the candidate value "128.60" corresponds to the gross weight field anchor and the billing weight field anchor, the field attribution matching values ​​are 0.88 and 0.86 respectively, which are close, but the field attribution separation value corresponding to the billing weight field anchor is only 0.03, thus triggering the field attribution competition flag. Figure 3 It can intuitively display the differences in the attribution of similar numeric fields under different field title anchors, and demonstrate the process in which field attribution matching value and field attribution separation value jointly participate in the determination of field attribution status.

[0043] like Figure 4 The diagram shows the contribution decomposition of candidate features for competing fields. Taking the candidate value "128.60" as an example, representing the competition between the gross weight field anchor and the billing weight field anchor, the diagram illustrates the contribution of the basic matching value, title distance feature, table row and column position feature, contextual semantic feature, symbolic adjacency feature, and reliable identification feature to the final field assignment matching value. Specifically, the basic matching value for the gross weight field anchor is 0.50, the title distance feature contributes 0.12, the table row and column position feature contributes 0.15, the contextual semantic feature contributes 0.08, the symbolic adjacency feature contributes 0.02, the reliable identification feature contributes 0.01, and the final field assignment matching value is 0.88. Similarly, the basic matching value for the billing weight field anchor is 0.50, the title distance feature contributes 0.10, the table row and column position feature contributes 0.14, the contextual semantic feature contributes 0.07, the symbolic adjacency feature contributes 0.03, the reliable identification feature contributes 0.02, and the final field assignment matching value is 0.86. It can be seen that the candidate value "128.60" has a certain degree of attribution support under both the gross weight field anchor and the billing weight field anchor. However, the final field attribution matching values ​​of the two are quite similar, indicating that relying solely on the field attribution matching value is prone to creating a risk of rigid assignment. Therefore, it is necessary to combine the field attribution separation value to further determine the field attribution status. Figure 4It can illustrate the cumulative impact of different feature components in the candidate pairing feature vector on the field attribution matching value, and provide data support for the formation of field attribution competition markers.

[0044] This implementation scheme achieves multi-dimensional feature quantification, heterogeneous relationship modeling, and attribution status discrimination between field candidates and field title anchors. It forms a similar numerical field disambiguation basis supported by candidate pairing feature vectors, field attribution relationship graphs, field attribution matching values, and field attribution separation values. It can distinguish between true attribution fields, adjacent competing fields, and unreliable candidate fields, avoiding the hard assignment of fields with similar values ​​such as gross weight and billing weight based solely on a single matching result. It improves the accuracy of determining field attribution locking flags, field attribution competing flags, and field attribution unreliable flags, providing an interpretable attribution basis for subsequent structured data generation, candidate field list retention, and confidence verification archiving.

[0045] Specifically, the process of obtaining the field attribution status, structured data, and candidate field list is as follows: Figure 5 The diagram shows the decision-making process for determining field attribution status. Positive and negative sample pairs are constructed from historically confirmed field records, consisting of field titles, field values, field coordinates, and field currencies. Positive sample pairs consist of field titles and their corresponding field values, coordinates, and currencies (verified by manual review) belonging to the same field title anchor point. Negative sample pairs consist of field titles and their corresponding field coordinates and currencies from competing field candidates on the same page, mismatched field values, or fields with different field title anchor points. Field attribution and separation thresholds are obtained using the RankNet pairwise ranking learning method, which is a ranking learning algorithm that learns the relative order of positive and negative sample pairs. Input data includes positive and negative sample pairs and their candidates. Pair feature vectors and constrain positive sample pairs to score higher than negative sample pairs using pairwise cross-entropy loss. Determine field attribution and separation thresholds based on the score ranking distribution in historically confirmed field records. When the field attribution matching value is greater than or equal to the field attribution threshold and the field attribution separation value is greater than or equal to the field separation threshold, a field attribution locked flag is generated. When the field attribution matching value is greater than or equal to the field attribution threshold and the field attribution separation value is less than the field separation threshold, a field attribution competition flag is generated. When the field attribution matching value is less than the field attribution threshold, an unreliable field attribution flag is generated. An unreliable flag means that the matching degree of all candidates is low, and the field value may have identification errors or missing values, requiring key review.

[0046] Structured information is extracted from the bill of lading number, total number of pieces, gross weight, volume, chargeable weight, product name, port of loading code, port of destination code, unit price, quantity, freight amount, and currency corresponding to the field attribution lock mark. These fields constitute the field set corresponding to the field title anchor, generating key-value pair structured data. For cases where the same field name appears with multiple field values ​​in different page numbers, cross-page duplicate field records are generated. The structured data includes field name, field value, confidence level, initial value of verification status, field source page number, field source coordinates, field attribution matching value, field attribution status, and original document cutout blocks. Field candidates corresponding to the field attribution competition mark and the field attribution unreliable mark are retained in the candidate field list. The candidate field list is transmitted to the verification module along with the structured data to provide alternative values ​​for subsequent evidence rereading and manual review.

[0047] This implementation scheme achieves threshold-based determination of field attribution status, structured data generation, and retention of ambiguous candidates. It forms hierarchical processing results corresponding to field attribution lock flags, field attribution competition flags, and field attribution unreliable flags. This allows fields such as bill of lading number, total number of pieces, gross weight, volume, chargeable weight, product name, port of shipment code, port of destination code, unit price, quantity, freight amount, and currency to enter different processing paths according to the degree of attribution credibility. This avoids similar numerical fields being directly written to the wrong field position when the matching distinction is insufficient, improves the reliability of field attribution of structured data, and retains the candidate field list as the basis for evidence rereading and manual review.

[0048] Specifically, the process of performing neural symbolic constraint decoding, evidence rereading, and confidence fusion on structured data is as follows: Neural symbolic constraint decoding is performed on the structured data. Combining airline code records, IATA port code records, and currency code records, consistency checks are performed on the reference relationships between field names and field values, code existence relationships, currency binding relationships, and numerical association relationships to obtain the field verification status. Neural symbolic constraint decoding represents discrete symbolic rules as differentiable energy functions, guiding the model to correct field values ​​that violate constraints during the verification process. The symbolic rules include at least the code existence rules corresponding to airline code records, the code existence rules corresponding to IATA port code records, the currency binding rules corresponding to currency code records, and unit price, number... Numerical association rules between quantity and freight amount; neural symbolic constraint decoding uses field names and values ​​in structured data as decoding objects, generating conflict constraint results or candidate correction results for field values ​​that do not meet the symbol rules; reference relationship verification determines whether the field value references a valid entry in the airline or port code list, and verifies whether the numerical association relationship verification logic equation, such as amount = unit price × quantity, is true. During numerical association relationship verification, the currency field is kept consistent, and the difference between the freight amount and the product of unit price and quantity is controlled within the range of the amount error obtained from the statistical analysis of historically confirmed field records; field verification status includes pass status, conflict status, and missing document status. The pass status indicates that all constraints are satisfied, the conflict status indicates that at least one constraint is not true, and the missing document status indicates that the reference information required for verification is missing.

[0049] For structured data with conflict or missing certificate status, enhanced evidence rereading is performed. The original document cutout tiles and adjacent text blocks corresponding to the field source coordinates are retrieved from the original text block list. A candidate field list is also read. Adjacent text blocks are those with the same page number as the field source and whose text line area coordinates fall within the evidence rereading window. The evidence rereading window is a rectangular area centered on the field source coordinates, extended horizontally and vertically by pixels. The extended pixel distance is determined by statistical results of the distance between the field source coordinates and adjacent text block coordinates in historically confirmed field records. The text block selection range is limited to text blocks within the same page number that fall within the evidence rereading window, ensuring sufficient context is captured. The retrieved cutout tiles and adjacent text blocks are combined to form a visual evidence slice. Abnormal field location results include the abnormal field name, abnormal field value, abnormal source page number, abnormal source coordinates, and abnormal associated candidate fields.

[0050] Dempster-Shafer evidence fusion is performed on text line recognition confidence, field attribution matching value, field attribution status, field verification status, and abnormal field location results. A comprehensive confidence level and a review candidate set are generated using a distribution-independent conformal prediction method. Dempster-Shafer evidence fusion is an uncertainty fusion method based on evidence theory. It maps each evidence source to a basic probability assignment function and synthesizes a unified confidence interval using Dempster combination rules. Specifically, text line recognition confidence and field attribution matching value are mapped to a basic probability assignment supporting direct write levels, while field attribution competition markers, unreliable field attribution markers, conflict status, missing evidence status, and abnormal field location results are mapped to levels supporting manual review or prohibiting fill levels. The basic probability allocation and evidence conflict parts are normalized using Dempster's combination rule. Distribution-independent conformal prediction calculates inconsistency scores based on the calibration set and outputs a prediction set that meets the given error control. The calibration set is formed by historically confirmed field records that did not participate in threshold training. The inconsistency score is obtained by the deviation between the overall confidence score and the manual review confirmation result. The review candidate set consists of field attribution competition markers, field attribution unreliable markers, and field candidates whose inconsistency scores exceed the corresponding control level. The overall confidence score is jointly generated by the Dempster-Shafer evidence fusion result and the distribution-independent conformal prediction output, and its value range is [0, 1]. The review candidate set retains the field candidates corresponding to the field attribution competition markers and the field attribution unreliable markers to prevent similar numerical fields from being covered by a single result.

[0051] This implementation scheme achieves a closed-loop system for quantification and verification decision-making of multi-source uncertain information: by encoding discrete business rules into differentiable energy functions, it automatically detects reference conflicts, currency binding violations, and numerical equation contradictions between field values, and accurately extracts and locates the original image evidence and candidate for conflicting or missing fields. Then, it integrates text recognition confidence, attribution matching strength, and verification status with evidence theory to form a comprehensive confidence and review candidate set with probability boundaries, thereby ensuring the credibility of automatic form filling is controllable and providing contextual visual evidence and competitive candidate retention for manual review, avoiding single mismatch coverage of similar numerical fields.

[0052] Specifically, the process of generating field processing levels and writing, reviewing, and archiving electronic documents in the freight forwarding document processing system is as follows: Based on the comprehensive confidence level, the review candidate set, and the field validation status, a cost-sensitive classification decision method is used to generate field processing levels. These levels include direct write level, manual review level, and no-fill level. The field processing levels control the path of structured data entering the freight forwarding document processing system: write path, manual review path, or no-fill path. Cost-sensitive classification sets a cost matrix based on the field miswrite penalty value and the field misblocking penalty value to determine the level division threshold by minimizing the expected cost, and combines this with whether the field validation status is "passed" and whether the review candidate set is "not yet passed." The field processing level is determined by the level range of the empty field and the overall confidence level. When the field processing level is the direct write level, the field name and field value are written into the corresponding form input box of the freight forwarding document processing system. The corresponding form input box is determined by the correspondence between the field name and the form field of the freight forwarding document processing system. The write operation triggers the front-end form field writing and the back-end document data table to be updated synchronously. When the field processing level is the manual review level or the prohibited fill level, the corresponding form input box is highlighted in red, the original document clipping block is displayed, and a task to be reviewed is generated. The red highlighting is achieved by modifying the CSS style of the form control, and a visual card containing anomaly location and candidate list is displayed in a floating display.

[0053] After review and confirmation, an electronic archive record is generated, consisting of international logistics freight forwarding documents, document format identifiers, upload timestamps, structured data, field ownership status, field verification status, overall confidence level, abnormal field location results, review candidate set, and modification records. The modification records detail the differences in field values ​​before and after manual review, the reviewer, and the operation time. Hash digest calculation and timestamp chaining are performed on the electronic archive record. The hash digest calculation uses the SHA-256 secure hash algorithm. During timestamp chaining, the hash digest of the current electronic archive record, the hash digest of the previous electronic archive record, and the upload timestamp are written together into the current electronic archive record, creating and storing it in the archive database for verifying the integrity of the electronic archive record after archiving.

[0054] This implementation plan establishes a risk-controlled write decision and complete archiving closed loop: based on comprehensive confidence and verification results, with the goal of minimizing business costs, fields are automatically categorized into three levels: direct write, manual review, or prohibited from being filled. While ensuring that high-confidence fields are written to the database immediately, questionable fields are highlighted on the page and displayed with image evidence cards to accurately guide the review operation. After the review is completed, all data and modification traces are sealed in a hash chain to form an immutable electronic archive, balancing the efficiency of automated form filling with the integrity of post-audit traceability.

[0055] like Figure 2As shown, the second aspect of the present invention provides an automatic verification system for international logistics freight forwarding documents based on semantic parsing, comprising: a data acquisition and preprocessing unit, used to upload international logistics freight forwarding document files and read document image data and verification data, and preprocess the document image data and verification data to obtain a single-page document image; a layout semantic parsing and field candidate generation unit, used to perform text detection, document layout detection, recognition correction and layout semantic parsing on the single-page document image, and generate a set of field title anchor points, a set of field candidates and a set of multiple candidate fields based on the text line content; a field attribution disambiguation unit, used to construct a field attribution relationship graph based on the set of field candidates and the set of field title anchor points, and obtain the field attribution status, structured data and a list of candidate fields; and a confidence verification and archiving unit, used to perform neural symbol constraint decoding, evidence rereading and confidence fusion on the structured data, generate field processing levels and write, review and archive electronically in the freight forwarding document processing system.

[0056] In this implementation plan, through the progressive collaboration of the data acquisition and preprocessing unit, the layout semantic parsing and field candidate generation unit, the field attribution disambiguation unit, and the confidence verification and archiving unit, a complete processing chain is formed, from the access of international logistics freight forwarding documents, the generation of single-page document images, the construction of field title anchors and field candidates, the disambiguation of similar numerical fields, the generation of structured data, to the verification and diversion and electronic archive archiving. This enables similar numerical fields such as total number of pieces, gross weight, volume, chargeable weight, unit price, quantity, and freight amount to be reliably assigned under the constraints of the document layout semantic structure and field attribution relationship diagram. This achieves the integrated processing effect of automatic document field identification, field attribution verification, anomaly review, and archive traceability, effectively solving the problems of correct numerical identification but incorrect field attribution, insufficient review basis, and incomplete archive traceability in the existing technology.

[0057] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0058] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for automatic verification of international logistics freight forwarding documents based on semantic parsing, characterized in that, Includes the following steps: S1, upload international logistics freight forwarding documents and read document image data and verification data, preprocess the document image data and verification data to obtain a single-page document image; S2 performs text detection, document layout detection, recognition and correction, and layout semantic parsing on single-page document images, and generates a set of field title anchor points, a set of field candidates, and a set of multiple candidate fields based on the text line content; S3: Construct a field attribution relationship diagram based on the field candidate set and the field title anchor point set to obtain the field attribution status, structured data, and candidate field list; S4 performs neural symbol constraint decoding, evidence rereading, and confidence fusion on structured data to generate field processing levels and write, review, and archive electronic documents in the freight forwarding document processing system.

2. The automatic verification method for international logistics freight forwarding documents based on semantic parsing according to claim 1, characterized in that: The specific process of uploading international logistics freight forwarding documents and reading document image data and verification data is as follows: Upload international logistics and freight forwarding documents via web page and application programming interface (API), and read document image data, including: file format identifier, upload timestamp, page number, page pixel matrix, page pixel width value, and page pixel height value; obtain airline code records, IATA port code records, currency code records, and historically confirmed field records through data verification interface, and file format identifiers include portable document format, scanned image, and photographed image.

3. The automatic verification method for international logistics freight forwarding documents based on semantic parsing according to claim 1, characterized in that: The specific process of preprocessing the document image data and verification data to obtain a single-page document image is as follows: When the file format is identified as a portable document format, the international logistics freight forwarding documents are split into single-page document images; when the file format is identified as a scanned image or photographed image, the single-page document images are read; the single-page document images are then denoised, corrected for deviations, enhanced, have their page orientation corrected, and have their page boundaries filled. The page pixel width and page pixel height values ​​are standardized using a zero-mean unit variance standardization algorithm; The grayscale values ​​of the page pixel matrix are normalized using a maximum and minimum value normalization algorithm.

4. The automatic verification method for international logistics freight forwarding documents based on semantic parsing according to claim 1, characterized in that: The specific process of performing text detection, document layout detection, recognition and correction, and layout semantic parsing on a single-page document image is as follows: A single-page document image is input into a differentiable binarized text detection model, which outputs the coordinates of the text line regions. The single-page document image is then processed using a DETR-based document layout detection method to obtain the coordinates of the table region and the seal occlusion region. The table region and the seal occlusion state are then incorporated into the semantic structure of the document layout. Convolutional recurrent neural network recognition and Transformer context correction are then applied sequentially to the image cropping blocks corresponding to the text line region coordinates to obtain the text line content and text line recognition confidence. Based on page number, text line area coordinates, page pixel width, page pixel height, and text line content, the LayoutReader reading order prediction method and two-dimensional position encoding are used to generate an original text block list. The original text block list includes text block identifier, page number, text line content, text line area coordinates, text line recognition confidence, and original document cropping blocks. The original text block list, table area coordinates, and seal occlusion area coordinates are subjected to layout semantic parsing to generate the document layout semantic structure. The layout semantic parsing adopts the document multimodal pre-trained coding method and the relational prediction layout analysis method to jointly express the text line content, text line area coordinates, table area coordinates, and page area relationships. According to the document layout semantic structure, the header area, detail area, summary area, remarks area, and signature association area are divided, and area hierarchy tags are generated.

5. The automatic verification method for international logistics freight forwarding documents based on semantic parsing according to claim 1, characterized in that: The specific process of generating the set of field title anchor points, the set of field candidates, and the set of multiple candidate fields based on the text line content is as follows: Field title recognition is performed on the text line content to obtain a set of field title anchor points; sequence labeling and span extraction are performed on the numbers, monetary symbols and code text in the text line content to obtain a set of field candidates; the set of field candidates includes field candidate values, field candidate coordinates, field candidate region, adjacent text of field candidates, field candidate currency, field candidate recognition confidence and field candidate clipping tiles, and adjacent text of field candidates is the text line content that is on the same page as the field candidate coordinates and is located within the candidate context window; When there are N numeric field candidates on the same page in the field candidate set, and the adjacent text, the area to which the field candidate belongs, or the set of field title anchor points correspond to different field title anchor points, the corresponding field candidate set will be classified into the multi-candidate field set. The competitive relationship between numeric candidates will be preserved, and the field ownership will not be directly determined according to the keyword triggering result, text line recognition confidence, and nearest neighbor distance.

6. The automatic verification method for international logistics freight forwarding documents based on semantic parsing according to claim 1, characterized in that: The specific process of constructing the field attribution relationship diagram based on the field candidate set and the field title anchor point set is as follows: Geometric distance is calculated between the field candidate coordinates and the field title anchor point coordinates to obtain the title distance feature; regional consistency analysis is performed on the region to which the field candidate belongs and the region level label to obtain the region level feature; The table row and column relationship is analyzed based on the field candidate coordinates, table area coordinates, and text line area coordinates to obtain the table row and column position features; Symbolic adjacency parsing is performed on candidate adjacent text and candidate currency of the field to obtain symbolic adjacency features; Contextual semantic features are obtained by performing contextual semantic encoding on the candidate adjacent text of the field, the set of anchor points of the field title, and the content of the text line. Image quality joint analysis was performed on the field candidate recognition confidence, the coordinates of the seal occlusion area and the field candidate cropping patch to obtain recognition reliable features; the title distance feature, region hierarchy feature, table row and column position feature, symbol adjacency feature, contextual semantic feature and recognition reliable feature were combined into a candidate pairing feature vector; Using field title anchors and field candidates as heterogeneous nodes, spatial relationship edges, row-column relationship edges, table region relationship edges, symbol adjacency edges, and semantic adjacency edges are generated based on the coordinate distance between field title anchors and field candidates, horizontal projection overlap, vertical projection overlap, table region affiliation, symbol adjacency state, and semantic similarity state, respectively. This constructs a field affiliation relationship graph, which is a heterogeneous graph containing title anchor nodes, candidate field nodes, and multiple types of relationship edges. The candidate pairing feature vectors are used as edge attributes of the field affiliation relationship edges and input into the field affiliation relationship graph. Graph attention propagation and double affine pairing processing are performed on the field affiliation relationship graph to obtain field affiliation matching values ​​and field affiliation separation values.

7. The automatic verification method for international logistics freight forwarding documents based on semantic parsing according to claim 1, characterized in that: The specific process for obtaining the field attribution status, structured data, and candidate field list is as follows: Positive and negative sample pairs of field titles, field values, field coordinates, and field currencies are constructed from historically confirmed field records. Field attribution thresholds and field separation thresholds are obtained through the RankNet pairwise ranking learning method. When the field attribution matching value is greater than or equal to the field attribution threshold, and the field attribution separation value is greater than or equal to the field separation threshold, a field attribution lock flag is generated. When the field attribution matching value is greater than or equal to the field attribution threshold, and the field attribution separation value is less than the field separation threshold, a field attribution competition flag is generated. When the field attribution matching value is less than the field attribution threshold, an unreliable field attribution flag is generated. Structured information is extracted from the bill of lading number, total number of pieces, gross weight, volume, chargeable weight, product name, port of loading code, port of destination code, unit price, quantity, freight amount, and currency corresponding to the field attribution lock mark, generating structured data in key-value pair format. The structured data includes field name, field value, confidence level, initial value of verification status, field source page number, field source coordinates, field attribution matching value, field attribution status, and original document cutout block. Candidate fields corresponding to the field attribution competition mark and the field attribution unreliable mark are retained in the candidate field list.

8. The method for automatic verification of international logistics freight forwarding documents based on semantic parsing according to claim 1, characterized in that: The specific process of performing neural symbol constraint decoding, evidence rereading, and confidence fusion on structured data is as follows: Neural symbolic constraint decoding is performed on structured data. Combining airline code records, IATA port code records, and currency code records, consistency checks are performed on the reference relationships, code existence relationships, currency binding relationships, and numerical association relationships between field names and field values ​​to obtain the field verification status. The field verification status includes pass status, conflict status, and missing certificate status. For structured data with conflict or missing certificate status, perform enhanced evidence rereading. Retrieve the original document cutout block and adjacent text block corresponding to the source coordinates of the field from the original text block list. The adjacent text block is the text block with the same page number as the source of the field and whose text line area coordinates fall within the evidence rereading window. Read the candidate field list and generate abnormal field location results. The text line recognition confidence, field attribution matching value, field attribution status, field verification status, and abnormal field location results are fused using Dempster-Shafer evidence fusion. A comprehensive confidence and a verification candidate set are generated using a distribution-independent conformal prediction method. The verification candidate set retains field candidates corresponding to field attribution competition markers and unreliable field attribution markers.

9. The automatic verification method for international logistics freight forwarding documents based on semantic parsing according to claim 1, characterized in that: The specific process of generating field processing levels and writing, reviewing, and archiving electronic documents in the freight forwarding document processing system is as follows: Based on the overall confidence level, the candidate set for review, and the field validation status, a cost-sensitive classification decision method is used to generate field processing levels. Field processing levels include direct write level, manual review level, and no-fill level. When the field processing level is direct write level, the field name and field value are written into the form input box of the freight forwarding document processing system. When the field processing level is manual review level or prohibited from being filled, the corresponding form input box will be highlighted in red, the original document cutout block will be displayed, and a task to be reviewed will be generated. After review and confirmation, the international logistics freight forwarding documents, document format identifiers, upload timestamps, structured data, field ownership status, field verification status, overall confidence level, abnormal field location results, review candidate set, and modification records are generated into electronic archive records. Hash digest calculations and timestamp chaining are performed on the electronic archive records, and an archive database is created and stored to verify the integrity of the electronic archive records after archiving.

10. An automatic verification system for international logistics freight forwarding documents based on semantic parsing, employing the automatic verification method for international logistics freight forwarding documents based on semantic parsing as described in any one of claims 1-9, comprising: The data acquisition and preprocessing unit is used to upload international logistics freight forwarding documents and read document image data and verification data, and preprocess the document image data and verification data to obtain a single-page document image. The layout semantic parsing and field candidate generation unit is used to perform text detection, document layout detection, recognition and correction and layout semantic parsing on single-page document images, and generate a set of field title anchor points, a set of field candidates and a set of multiple candidate fields based on the text line content. The field attribution disambiguation unit is used to construct a field attribution relationship graph based on the field candidate set and the field title anchor point set, and obtain the field attribution status, structured data and candidate field list; The confidence verification and archiving unit is used to perform neural symbol constraint decoding, evidence rereading, and confidence fusion on structured data, generate field processing levels, and write, review, and archive electronic documents in the freight forwarding document processing system.

Images