Legal document fall positioning and quality inspection method and device based on semantic understanding

Abstract

The invention discloses a legal document fall positioning and quality inspection method and device based on semantic understanding, and relates to the technical field of legal review, and the method comprises the steps: analyzing electronic document data, extracting text content, associating the text content with corresponding coordinate metadata, and generating multi-modal document data; performing deep semantic and structural analysis on the multi-modal document data, extracting a text sequence, executing natural language analysis and semantic analysis, obtaining a phrase role labeling result, judging a termination position, and obtaining a semantic boundary; based on the semantic boundary and the coordinate metadata, performing retrieval positioning on the multi-modal document data, extracting coordinate text data and page evidence image segments, obtaining a structured entity and segmented signature blocks through entity recognition and signature detection, and constructing a signature element map; according to the method, automatic verification and accurate fund positioning of the document are realized, the processing efficiency is improved, manual intervention is reduced, and the method is widely applicable to automatic examination and compliance inspection of the electronic document.

Description

Technical Field

[0001] This invention relates to the field of legal review technology, and in particular to a method and apparatus for locating and inspecting the signature of legal documents based on semantic understanding. Background Technology

[0002] With the advancement of judicial digitalization and e-government, legal documents are gradually shifting from paper-based to electronic formats, while a large amount of historical materials are still archived as scanned images. A relatively mature technological chain has been established in the industry for the structured processing of electronic legal documents: on the one hand, searchable text is obtained through PDF parsing or OCR recognition, and layout information is extracted simultaneously to support layout reconstruction and field extraction; on the other hand, layout analysis and information extraction technologies are combined to identify and classify document elements. In this process, natural language processing is widely used for tasks such as named entity recognition, semantic analysis, role element extraction, and referential resolution to improve cross-template and cross-language generalization capabilities; simultaneously, visual algorithms such as signature detection are used to locate and segment image regions, providing support for subsequent quality inspection and evidence preservation.

[0003] In scenarios involving the location and quality inspection of signatures in electronic legal documents with multiple formats and sources, existing methods mostly rely on template rules, layout heuristics, or single-modal detection. When the document format or wording changes, there is still room for improvement in semantic adaptability and location consistency. Furthermore, subtle deviations in the location of signature boundaries or elements can easily accumulate and amplify in subsequent entity recognition, signature segmentation, and rule verification, leading to inconsistencies in the chain of evidence and affecting the interpretability and stability of the review conclusions. Building a robust processing link for adaptive alignment and verification, focusing on the issue of "coupling semantic boundaries and coordinate metadata, and realizing semantic consistency verification of the signing subject and role," is an important direction for development. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a method for locating and inspecting the signature of legal documents based on semantic understanding, which solves the problems of difficulty in stably and accurately locating the signature boundary and the difficulty in automatically verifying the semantic consistency between the signatory and the roles in the document segment.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for locating and quality inspecting signatures on legal documents based on semantic understanding. This method includes: parsing electronic document data, extracting text content and associating it with corresponding coordinate metadata to generate multimodal document data; performing deep semantic and structural analysis on the multimodal document data, extracting text sequences and performing natural language analysis and semantic analysis, obtaining segment role labeling results and determining the termination position, and obtaining semantic boundaries; based on the semantic boundaries and coordinate metadata, performing retrieval and location on the multimodal document data, extracting coordinate text data and page evidence image fragments, and obtaining structured entities and segmented signature blocks through entity recognition and signature detection to construct a signature element map; binding the signature subject in the signature element map with preset roles and encoding them into semantic vectors, generating semantic verification results by calculating the matching degree between the semantic vectors and semantic distribution clusters in the entity referential chain; performing automated verification and calculation on the signature element map and semantic verification results to obtain rule verification results; and summarizing and formatting the semantic verification results and rule verification results to generate a structured review report.

[0007] As a preferred embodiment of the semantic understanding-based legal document signature location and quality inspection method of the present invention, the specific steps for generating multimodal document data are as follows: Electronic document data is uniformly converted into a parsable page sequence, and page images are generated by rendering each page. A two-way mapping from document coordinates to pixel coordinates is established to obtain coordinate metadata. The system performs text recognition and layout element detection on page images to obtain visual text fragments and pixel coordinates. It also parses vector layer text fragments and converts them into pixel coordinates through bidirectional mapping. Spatially overlap the visual text box and coordinate metadata within pixel coordinates to generate candidate pairings and association scores; By using a one-to-one matching algorithm, visual text fragments are bound to coordinate metadata, coordinate text is obtained, and the corresponding page image is cropped into visual evidence. Organize the coordinate text and the layout structure relationship of page elements into multimodal document data.

[0008] As a preferred embodiment of the semantic understanding-based legal document signature location and quality inspection method of the present invention, the vector layer text fragment is obtained by traversing the unified parsable electronic document data page object, parsing the text drawing object and reading the text content and document coordinates, and aggregating them according to line and character spacing rules to form text fragments.

[0009] As a preferred embodiment of the semantic understanding-based legal document signature location and quality inspection method of the present invention, the specific steps for obtaining the semantic boundary are as follows: Read multimodal document data and sort it according to the center point of the text box to form a text sequence, obtain the page spatial structure relationship, and determine the structural elements based on the page spatial structure relationship; Within structural elements, coordinate text is aggregated into segments, and the text sequence is extracted for natural language analysis. The segment text is encoded into semantic representation, the structural features of pixel coordinates are encoded into structural representation, and the semantic representation and structural representation are concatenated into a joint representation to obtain the segment role labeling results. The scanning segment role labeling results take the lower boundary of the pixel coordinates of the last paragraph text as the termination position, and define the boundary between the closing paragraph text and the preceding non-closing paragraph text as the semantic boundary.

[0010] As a preferred embodiment of the semantic understanding-based legal document signature location and quality inspection method of the present invention, the steps of performing retrieval and location on multimodal document data based on semantic boundaries and coordinate metadata, and extracting coordinate text data and page evidence image fragments are as follows. The semantic boundaries are applied to the coordinate metadata to generate a search window. Within the search window, spatial retrieval is performed to recall candidate coordinate texts, and intersection retrieval is performed on the candidate coordinate texts to recall candidate reading sequences. Remove duplicate candidate reading sequences and determine the range of continuous segments of candidate coordinate text. Then, concatenate the candidate coordinate text within the range of continuous segments in order to obtain the continuous text segments of candidate coordinates. The pixel coordinates of the candidate coordinate text are summarized, and the smallest bounding rectangle is selected as the cropping box to crop the image on the page to obtain the evidence image fragment.

[0011] As a preferred embodiment of the semantic understanding-based legal document signature location and quality inspection method of the present invention, the specific steps for obtaining structured entities and segmenting signature blocks through entity recognition and signature detection to construct a signature element map are as follows. The candidate coordinate continuous text segment is segmented into paragraphs and then word segmentation, part-of-speech tagging and named entity recognition are performed in sequence to obtain structured entities. The evidence image fragments are binarized and connected component instance segmented to obtain segmented signature blocks. Structured entities and segmented signature blocks are built into nodes respectively. Candidate entity blocks are generated on the same page and their binding scores are calculated. Then, they are matched one by one to determine the relationship edges. The nodes and relationship edges are associated, scored, and filtered to construct a signature element graph.

[0012] As a preferred embodiment of the semantic understanding-based legal document signature location and quality inspection method of the present invention, the specific steps for generating the semantic verification result are as follows: The signature subject is extracted from the signature element map and compared with the similarity of the text segment and the preset role. The signature subject and entity reference chain are encoded into semantic vectors. Clustering is performed on semantic vectors to form semantic distribution clusters, and the center vector of the semantic distribution clusters is calculated; The matching degree of the distribution clusters is calculated based on the semantic vector of the signature subject and the center vector of each distribution cluster. The cluster with the highest matching degree is selected as the best matching cluster. The maximum matching degree is compared with the preset semantic threshold to generate a semantic verification mark. The maximum value and the semantic verification mark are output as the semantic verification result.

[0013] As a preferred embodiment of the semantic understanding-based legal document signature location and quality inspection method of the present invention, the entity reference chain is obtained by standardizing the entity text of the signature subject, filtering out all entity records with consistent standardized text from the entity records extracted from the segment text, and sequentially connecting them according to the segment sequence number where the entity records are located.

[0014] As a preferred embodiment of the semantic understanding-based legal document signature location and quality inspection method of the present invention, the specific steps for generating the structured review report are as follows: Align and deduplicate the signature element graph and semantic verification results, obtain the list of review objects, and automatically verify and calculate through a fixed set of rules to generate rule verification results; The rule verification results are first grouped and aggregated according to the signature subject identifier, the verification statistics are summarized and the detailed evidence of failure are merged with the semantic verification score and semantic verification mark and formatted output to generate a structured review report.

[0015] Secondly, the present invention provides a legal document signature location and quality inspection device based on semantic understanding, including a document parsing module for parsing electronic document data, extracting text content and associating it with corresponding coordinate metadata, and generating multimodal document data; The semantic boundary module is used to perform deep semantic and structural analysis on multimodal document data, extract text sequences and perform natural language analysis and semantic analysis, obtain segment role labeling results and determine the termination position, and obtain semantic boundaries; The signature map module is used to perform retrieval and location of multimodal document data based on semantic boundaries and coordinate metadata, extract coordinate text data and page evidence image fragments, and obtain structured entities and segmented signature blocks through entity recognition and signature detection to construct a signature element map; The semantic verification module is used to bind the signature subject in the signature element map with the preset role and encode it into a semantic vector. By calculating the matching degree between the semantic vector and the semantic distribution cluster in the entity reference chain, the semantic verification result is generated. The report generation module is used to perform automated verification and calculation on the signature element map and semantic verification results, obtain the rule verification results, summarize and format the semantic verification results and rule verification results, and generate a structured review report.

[0016] The beneficial effects of this invention are as follows: Based on natural language processing technology, electronic document data is parsed and semantically understood to generate multimodal documents containing text, images, and coordinate metadata. Combined with deep semantic analysis and spatial positioning information, the signature location is automatically located and semantic verification and rule validation are performed. Furthermore, through entity recognition and element extraction driven by natural language processing, signature detection, and structured review report generation, the automated review and accurate signature location of electronic documents are achieved, thereby significantly improving processing efficiency, reducing manual intervention costs, and making it widely applicable to automated review and compliance inspection scenarios of electronic documents. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Fig. 1 This is a flowchart of a semantic understanding-based method for locating and inspecting signatures on legal documents.

[0019] Fig. 2 This is a schematic diagram of a semantic understanding-based legal document signature location and quality inspection device.

[0020] Fig. 3 A flowchart for signature positioning and quality inspection methods.

[0021] Fig. 4 A flowchart for generating multimodal document data. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] ReferenceFigs. 1-4 This is one embodiment of the present invention, which provides a method for locating and inspecting the signature of legal documents based on semantic understanding, including the following steps: S1. Parse the electronic document data, extract the text content and associate it with the corresponding coordinate metadata to generate multimodal document data.

[0026] S1.1. Convert electronic document data into a parsable page sequence, render page images page by page, establish a bidirectional mapping from document coordinates to pixel coordinates, and obtain coordinate metadata.

[0027] Furthermore, the electronic document data is standardized and parsed to convert it into a parsable page sequence. Each page is further structured into a page object containing text, images, and other elements, preserving the layout information. Page images are generated page by page according to the original format to ensure complete content presentation. A two-way mapping relationship between document coordinates and pixel coordinates is established between the rendering result and the structured layout, enabling precise location of corresponding text and elements through pixel positions and supporting reverse backtracking. Coordinate metadata containing the bounding boxes of text and image elements is extracted and output, providing a unified spatial benchmark and data support for subsequent element extraction, location verification, and result writing.

[0028] It should be noted that the supplementary mapping is defined as unifying document coordinates and pixel coordinates. The document coordinates take the bottom left of the page as the origin, and the pixel coordinates take the top left of the page image as the origin. A two-way conversion relationship is established according to the rendering ratio and page rotation, and the boundary coordinate metadata of text, images and other elements is output for subsequent positioning, verification and result writing.

[0029] S1.2 Perform text recognition and layout element detection on the page image to obtain visual text fragments and pixel coordinates. At the same time, parse the vector layer text fragments and convert them into pixel coordinates through bidirectional mapping.

[0030] Furthermore, text content on the page is extracted using text recognition technology to obtain visual text fragments and their corresponding pixel coordinates. Vector-layer text fragments on the page are then parsed, and their coordinate data is converted to pixel coordinates using a bidirectional mapping method, ensuring a precise correspondence between the vector-layer text and the pixel positions in the image.

[0031] It should be noted that vector-layer text fragments are obtained by traversing the unified parsable electronic document data page object, parsing the text drawing object, reading the text content and document coordinates, and aggregating them according to line and character spacing rules. When the page does not have vector-layer text fragments, or the vector-layer text fragments fail to parse or are insufficient to cover the main text area, the visual text fragments obtained from character recognition are used as coordinate text candidates, and the pixel coordinates are converted into document coordinates through the inverse mapping of bidirectional mapping to generate the corresponding coordinate metadata. When vector-layer text fragments exist and are parsed successfully, the vector-layer text fragments are used first, and the document coordinates are converted into pixel coordinates through mapping. At the same time, the character recognition results can be used as a supplement for error correction and missing data completion.

[0032] S1.3. Spatially overlap the visual text box and coordinate metadata within pixel coordinates to generate candidate pairings and association scores.

[0033] Furthermore, in the specific operation process, the coordinates and coordinate metadata of the visual text boxes are spatially aligned, overlapping areas are checked, and possible pairings are formed. Based on the degree of spatial overlap, the association score of each pairing is calculated. The association score consists of factors such as overlap coverage, relative position consistency, boundary or center distance, and horizontal and vertical alignment. Each factor is first normalized to zero to one and then weighted and summed. The association score ranges from zero to one, and a matching threshold is set. Only candidate pairings with scores not less than the threshold are proceeded to the next step. If there is a conflict where a visual text box or a coordinate metadata corresponds to multiple candidates, the selection is performed according to one-to-one constraints, prioritizing the highest score. When a local maximum score leads to an overall conflict, a maximum weight matching strategy is used to solve the problem. The matching degree of each candidate pairing is evaluated to determine its rationality and provide relevant pairing information for subsequent steps.

[0034] S1.4. Using a one-to-one matching algorithm, the visual text fragments are bound to the coordinate metadata, the coordinate text is obtained, and the corresponding page image is cropped into visual evidence.

[0035] Furthermore, based on the coordinate metadata and spatial information of the visual text fragments, an algorithm calculates the optimal matching relationship between the two. The coordinate text corresponding to the matched visual text fragment is then obtained and associated with its actual pixel location on the page. Based on the matching result, the corresponding page image is cropped into a visual evidence image, ensuring that the cropped image area matches the bound text fragment, providing accurate visual evidence for subsequent analysis.

[0036] S1.5 Organize the coordinate text and the layout structure relationship of page elements into multimodal document data.

[0037] Furthermore, the coordinate text is spatially aligned with its corresponding layout elements to determine their relative position and structural relationship within the page. By analyzing the layout and hierarchical relationships between layout elements, a complete structural representation is constructed. These elements are then combined with the coordinate text to form multimodal document data. This multimodal document data includes text information, layout elements, and spatial layout, providing comprehensive structured data for subsequent document analysis and processing.

[0038] S2. Perform deep semantic and structural analysis on multimodal document data, extract text sequences and perform natural language analysis and semantic analysis, obtain segment role labeling results and determine the termination position, and obtain semantic boundaries.

[0039] S2.1 Read multimodal document data and sort it according to the center point of the text box to form a text sequence, obtain the page spatial structure relationship, and determine the structural elements based on the page spatial structure relationship.

[0040] Furthermore, the text is sorted according to the center point of the text box, forming an ordered text sequence. By analyzing the spatial layout of the page, the spatial structural relationships of the page are obtained, that is, the relative positions of each element on the page. Based on this structural relationship, the position and hierarchy of key structural elements on the page, such as paragraphs, headings, or other layout elements, are determined, thus providing a clear structural framework for subsequent document processing.

[0041] S2.2. Within the structural elements, coordinate text is aggregated into segments, and the text sequence is extracted for natural language analysis. The segment text is encoded into a semantic representation, the structural features of pixel coordinates are encoded into a structural representation, and the semantic representation and structural representation are concatenated into a joint representation to obtain the segment role labeling results.

[0042] Furthermore, adjacent or related coordinate text is integrated to form a complete sentence segment. Next, the text sequence within the segment is extracted and subjected to natural language analysis to identify grammar, semantics, and interrelationships. The segment text is encoded into a semantic representation, and semantic vectors are generated using natural language processing techniques. Structural features in pixel coordinates are encoded into structural representations to capture the spatial features of the page layout. The semantic and structural representations are concatenated to form a joint representation, and the role labeling results of the segment are obtained through this joint representation to identify the grammatical and semantic roles of the segment within the document.

[0043] It should be noted that the set of role tags includes "main text", "facts", "reasons", "signature", "appendix", etc. The dimensions of the joint representation are composed of semantic representation (from the natural language encoder) and structural representation (composed of pixel coordinate features). At the same time, inference constraints ensure that, for example, the role of "signature" can only appear in the last few lines or the last page area.

[0044] S2.3 The scanning segment role labeling results take the lower boundary of the pixel coordinates of the last paragraph's text as the termination position, and define the boundary between the closing text and the preceding non-closing text as the semantic boundary.

[0045] Furthermore, the lower pixel coordinate boundary of the closing paragraph text is extracted as the marker for the termination position. The boundary between this closing paragraph text and the preceding non-closing paragraph text is defined as the semantic boundary. This semantic boundary clarifies the semantic separation between the closing section and the preceding text, providing a clear structural division for subsequent processing. This process ensures clear boundaries between paragraphs, facilitating further analysis and processing of the document structure. When multiple closing paragraphs are detected, the closing paragraph with the later page number is selected as the termination position; within the same page, the one with the larger y-coordinate takes priority; if a date paragraph and a signature body paragraph exist, the lower boundary of the date paragraph is taken as the termination position.

[0046] S3. Based on semantic boundaries and coordinate metadata, perform retrieval and location on multimodal document data, extract coordinate text data and page evidence image fragments, and obtain structured entities and segmented signature blocks through entity recognition and signature detection to construct a signature element map.

[0047] S3.1. The semantic boundary is applied to the coordinate metadata to generate a search window. Spatial search is performed within the search window to recall candidate coordinate texts, and intersection search is performed on the candidate coordinate texts to recall candidate reading orders.

[0048] Furthermore, by mapping semantic boundaries onto coordinate metadata, a retrieval window is generated, ensuring that this window contains relevant text regions. Within this retrieval window, spatial retrieval is performed to recall candidate coordinate texts—potential text regions related to the semantic boundaries. Intersection retrieval is then performed on the recalled candidate coordinate texts to further filter candidate reading sequences. (For example, if the retrieval window covers the "judgment is as follows" paragraph area, after spatial retrieval recalls multiple text boxes within and around the window, intersection judgment is made between the reading sequences formed by the combinations of candidate text boxes and the window; sequences with high and continuous intersection coverage with the window are retained, while headers, footers, and other texts that barely intersect with the window are removed.) This ensures the relevance and accuracy of the text, obtaining text paragraphs that meet the criteria.

[0049] S3.2. Remove duplicates from the candidate reading order and determine the range of continuous segments of the candidate coordinate text. Then, concatenate the candidate coordinate text within the range of continuous segments in order to obtain the continuous text segment of the candidate coordinates.

[0050] Furthermore, when performing deduplication on the candidate reading order, the coordinate metadata corresponding to each candidate coordinate text in the candidate reading order is compared item by item. The candidate coordinate text with the same coordinate metadata is retained and duplicate entries are removed to obtain the deduplicated candidate reading order. In the deduplicated candidate reading order, the set of consecutively occurring candidate coordinate texts is located according to the order of the candidate reading order to determine the range of continuous segments of candidate coordinate texts. Within the range of continuous segments of candidate coordinate texts, the text content of candidate coordinate texts is extracted in sequence according to the order of the candidate reading order and concatenated sequentially. The concatenated result is used as the continuous text segment of candidate coordinates.

[0051] S3.3 Summarize the pixel coordinates of the candidate coordinate text, and select the smallest bounding rectangle as the cropping box to crop the image on the page to obtain the evidence image fragment.

[0052] Furthermore, the pixel coordinate boundary information corresponding to the candidate coordinate text is read sequentially, and the pixel coordinate boundary information of all candidate coordinate texts is merged to form a unified pixel coordinate range. Within the unified pixel coordinate range, the minimum x-coordinate, maximum x-coordinate, minimum y-coordinate, and maximum y-coordinate of the pixel coordinates of all candidate coordinate texts are calculated respectively. The minimum bounding rectangle is determined using the minimum x-coordinate, maximum x-coordinate, minimum y-coordinate, and maximum y-coordinate. The minimum bounding rectangle is used as the cropping box, and the cropping operation is performed on the page image according to the pixel coordinate area corresponding to the cropping box. The cropped image area is used as the evidence image fragment.

[0053] S3.4. Segment the candidate coordinate continuous text segments by paragraph and perform word segmentation, part-of-speech tagging and named entity recognition in sequence to obtain structured entities. Then, binarize the evidence image fragments and segment them into connected component instances to obtain segmented signature blocks.

[0054] Furthermore, based on paragraph separators, line breaks, and punctuation marks in the candidate coordinate continuous text segments, the segment boundaries are determined, and the candidate coordinate continuous text segments are split into sequentially arranged text segments. For each text segment, word segmentation, part-of-speech tagging, and named entity recognition are performed sequentially. Word segmentation involves scanning the text segment character by character and combining word boundary rules to determine the word sequence. Part-of-speech tagging assigns a part-of-speech category label to each word in the word sequence. Named entity recognition labels entity boundaries and entity categories based on the word sequence and part-of-speech category labels, and outputs structured entities. When performing binarization and connected component instance segmentation on the evidence image fragments, binarization compares the pixel grayscale values ​​of the evidence image fragments with a threshold and converts the pixels into foreground and background pixels to form a binary image. Connected component instance segmentation traverses the connectivity of foreground pixels on the binary image and assigns a unique identifier to each connected region. The pixel set and circumscribed rectangle boundary of the connected region are extracted, and the segmented signature block is output.

[0055] S3.5. Build structured entities and segmented signature blocks into nodes respectively. Generate candidate entity blocks on the same page and calculate the binding score. Then match them one by one to determine the relationship edges. Associate the nodes with the relationship edges, score and filter them to construct a signature element graph.

[0056] Furthermore, structured entity nodes record entity text and entity category, while segmented signature block nodes record the pixel coordinates of the bounding rectangle boundary. Within the same page, the pixel coordinate text box boundary of the structured entity node is combined with the pixel coordinates of the bounding rectangle boundary of the segmented signature block node to generate candidate entity blocks. Spatial relationship calculation is performed on the candidate entity blocks to obtain a binding score. The binding score consists of relative positional relationship, boundary spacing, alignment degree, and coverage. Each item is first normalized to zero to one and then weighted and summarized. The binding score ranges from zero to one, and a filtering threshold is set to filter out low-confidence candidates and retain relationship edges and nodes. When there are multiple candidate conflicts, the relationship edges are determined according to one-to-one matching constraints, with the goal of maximizing the total binding score. In scenarios with limited computing resources, a greedy strategy of prioritizing the maximum score can be adopted, combined with backtracking to resolve conflicts. Spatial relationship calculation includes relative positional relationship calculation, boundary spacing calculation, and alignment degree calculation. One-to-one matching is performed based on the binding score to determine the relationship edges between the structured entity node and the segmented signature block node. The binding score is used as the association scoring result to filter and retain nodes and relationship edges. The filtering results form a signature element graph.

[0057] S4. Bind the signature subject in the signature element map to the preset role and encode it into a semantic vector. Calculate the matching degree between the semantic vector and the semantic distribution cluster in the entity reference chain to generate a semantic verification result.

[0058] S4.1 Extract the signature subject from the signature element map and compare the similarity between the text segment and the preset role, and encode the signature subject and entity reference chain into a semantic vector.

[0059] Furthermore, the process involves filtering to obtain the signature subject and reading the associated text segments; performing text encoding on both the text segments and the preset roles, converting them into semantic vectors. This text encoding includes character sequence organization, word segment generation, and vectorization calculation of mapping the word segments to fixed-dimensional semantic vectors; performing similarity comparison on the semantic vectors, normalizing the semantic vectors of the text segments and each preset role, and calculating the vector dot product to obtain a similarity score, which characterizes the semantic closeness between the text segments and the preset roles; and extracting entity text according to the entity reference chain corresponding to the signature subject in the signature element graph, performing text encoding consistent with the text segments, encoding the signature subject and entity reference chain into semantic vectors, and outputting the semantic vectors of the signature subject and entity reference chain.

[0060] It should be noted that the entity reference chain is obtained by standardizing the entity text of the signing subject, filtering out all entity records with consistent standardized text from the entity records extracted from the segment text, and then connecting them in order according to the segment sequence number where the entity records are located.

[0061] S4.2. Cluster the semantic vectors to form semantic distribution clusters and calculate the center vector of the semantic distribution clusters.

[0062] Furthermore, the semantic vectors of the signature subject and the entity reference chain are aggregated into a semantic vector set. The distance or similarity between the semantic vectors in the semantic vector set is calculated, and a clustering algorithm is used to divide the semantic vector set into multiple semantic distribution clusters. During the execution of the clustering algorithm, the semantic vectors are assigned based on the distance or similarity between the semantic vectors and the center vectors of the semantic distribution clusters, and the center vectors of the semantic distribution clusters are updated after each assignment. When calculating the center vector of the semantic distribution cluster, the average of all semantic vectors in the semantic distribution cluster is calculated according to the dimension, and the average result is used as the center vector of the semantic distribution cluster.

[0063] S4.3 Calculate the matching degree of the distribution clusters based on the semantic vector of the signature subject and the center vector of each distribution cluster. Select the cluster with the highest matching degree as the best matching cluster. Compare the maximum matching degree with the preset semantic threshold to generate a semantic verification mark. Output the maximum value and the semantic verification mark as the semantic verification result.

[0064] Furthermore, vector normalization is performed sequentially on the semantic vector of the signature subject and the center vector of each semantic distribution cluster. Then, the dot product of the normalized semantic vector of the signature subject and the center vector of the normalized semantic distribution cluster is calculated. The normalization calculation expression is as follows: ;

[0065] The expression for calculating the dot product of vectors is: ; in, This represents the semantic vector of the signature entity. Represents the semantic distribution cluster center vector. The vector dimension representing the semantic vector of the signature subject and the center vector of the semantic distribution cluster. Indicates to The result after normalization to L2 norm; Indicates to The result after L2 norm normalization The general notation used to give the definition of the L2 norm for "arbitrary vectors" is... The dimension index of the vector component is used to mark the first dimension of the semantic vector of the signature subject and the center vector of the semantic distribution cluster. Each dimension component; The semantic vector of the signature subject is represented in the th position. Components in each dimension The semantic distribution cluster center vector is represented in the th order. Components in each dimension.

[0066] The vector dot product result is used as the matching degree of the corresponding semantic distribution cluster. The matching degrees of all semantic distribution clusters are compared and the semantic distribution cluster with the highest matching degree is selected as the best matching cluster, and the maximum matching degree is recorded. The maximum matching degree is compared with the preset semantic threshold. When the maximum matching degree is greater than or equal to the preset semantic threshold, a semantic verification mark is generated as passed. When the maximum matching degree is less than the preset semantic threshold, a semantic verification mark is generated as failed. The maximum matching degree and the semantic verification mark are output as the semantic verification result.

[0067] It should be noted that the threshold range [0,1] is scanned on the labeled validation set. By calculating the harmonic mean of the precision and recall of the semantic validation tags under each threshold, the threshold that maximizes the mean value is selected as the preset semantic threshold and will be used in the subsequent semantic validation process to ensure the best matching accuracy.

[0068] S5. Perform automated verification and calculation on the signature element map and semantic verification results, obtain the rule verification results, summarize and format the semantic verification results and rule verification results, and generate a structured review report.

[0069] S5.1 Align and deduplicate the signature element graph and semantic verification results, obtain the list of review objects, and automatically verify and calculate the rules through a fixed rule set to generate rule verification results.

[0070] Furthermore, the signature subjects in the signature element graph are traversed. Based on the consistency of the signature subject text, the corresponding entries are located in the semantic verification results, and the maximum matching degree and semantic verification mark are read. The signature element graph information, the maximum matching degree, and the semantic verification mark are associated with the same signature subject. When deduplicating the associated signature subject set, the consistency of the signature subject text is compared and duplicate signature subjects are removed. The deduplicated signature subject set forms the review object list. When the review object list is automatically verified and calculated using a fixed rule set, the fixed rule set performs existence checks, quantity statistics calculations, and consistency condition comparisons on the structured entity nodes, segmented signature block nodes, and relation edges in the signature element graph. It also performs threshold comparisons and combination condition judgments on the maximum matching degree and semantic verification mark. The verification conclusions output by the fixed rule set and the statistical calculation values ​​are combined to form the rule verification result.

[0071] It should be noted that the fixed rule set includes: performing existence checks, quantity statistics, and consistency comparisons on structured entity nodes, segmented signature block nodes, and relational edges in the signature element graph to ensure the correct graph structure; at the same time, comparing the maximum matching degree with the semantic verification mark with a threshold, and combining the matching degree and the verification mark combination conditions to determine whether the review object meets the preset standards and generate the verification result.

[0072] S5.2. The rule verification results are first grouped and aggregated according to the signature subject identifier, the verification statistics are summarized and the detailed evidence of failure are merged with the semantic verification score and semantic verification mark and formatted output to generate a structured review report.

[0073] Furthermore, the signature entity identifier and corresponding verification item conclusion in each rule verification result are read one by one, and rule verification results with the same signature entity identifier are grouped into the same group. When summarizing the rule verification results within each group, the number of passing and failing entries in the rule verification results are accumulated, and the detailed evidence of failure corresponding to the verification items marked as failing in the rule verification results is collected in the order of verification items to form a set of detailed evidence of failure. When merging the grouped rule verification results with semantic verification scores and semantic verification tags, the semantic verification scores and semantic verification tags are located in the semantic verification results based on the signature entity identifier and written into the same signature entity identifier group record. When formatting the output of the merged content, the signature entity identifier, verification statistics, detailed evidence of failure, semantic verification score, and semantic verification tag are organized into structured fields and output, and the structured field output results serve as a structured review report.

[0074] This embodiment also provides a legal document signature location and quality inspection device based on semantic understanding, including: a document parsing module, used to parse electronic document data, extract text content and associate it with corresponding coordinate metadata, and generate multimodal document data; The semantic boundary module is used to perform deep semantic and structural analysis on multimodal document data, extract text sequences and perform natural language analysis and semantic analysis, obtain segment role labeling results and determine the termination position, and obtain semantic boundaries; The signature map module is used to perform retrieval and location of multimodal document data based on semantic boundaries and coordinate metadata, extract coordinate text data and page evidence image fragments, and obtain structured entities and segmented signature blocks through entity recognition and signature detection to construct a signature element map; The semantic verification module is used to bind the signature subject in the signature element map with the preset role and encode it into a semantic vector. By calculating the matching degree between the semantic vector and the semantic distribution cluster in the entity reference chain, the semantic verification result is generated. The report generation module is used to perform automated verification and calculation on the signature element map and semantic verification results, obtain the rule verification results, summarize and format the semantic verification results and rule verification results, and generate a structured review report.

[0075] In summary, this invention utilizes natural language processing (NLP) technology to parse and semantically understand electronic document data, generating multimodal documents containing text, images, and coordinate metadata. Combined with deep semantic analysis and spatial positioning information, it automatically completes signature location positioning, semantic verification, and rule validation. Furthermore, through NLP-driven entity recognition and element extraction, signature detection, and structured review report generation, it achieves automated review and precise signature location for electronic documents, significantly improving processing efficiency and reducing manual intervention costs. This invention is widely applicable to automated review and compliance checks of electronic documents.

[0076] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for locating and quality inspecting the signature of legal documents based on semantic understanding, characterized by: include, The electronic document data is parsed, the text content is extracted and associated with the corresponding coordinate metadata, and multimodal document data is generated. Deep semantic and structural analysis is performed on multimodal document data to extract text sequences and perform natural language analysis and semantic analysis, obtain segment role labeling results and determine the termination position, and obtain semantic boundaries; Based on semantic boundaries and coordinate metadata, retrieval and location are performed on multimodal document data, coordinate text data and page evidence image fragments are extracted, and structured entities and segmented signature blocks are obtained through entity recognition and signature detection to construct a signature element map; The signature subject in the signature element map is bound to the preset role and encoded into a semantic vector. The semantic verification result is generated by calculating the matching degree between the semantic vector and the semantic distribution cluster in the entity reference chain. Automated verification and calculation are performed on the signature element map and semantic verification results to obtain rule verification results. The semantic verification results and rule verification results are summarized and formatted to generate a structured review report.

2. The method for locating and quality inspecting the signature of legal documents based on semantic understanding as described in claim 1, characterized in that: The specific steps for generating multimodal document data are as follows. Electronic document data is uniformly converted into a parsable page sequence, and page images are generated by rendering each page. A two-way mapping from document coordinates to pixel coordinates is established to obtain coordinate metadata. The system performs text recognition and layout element detection on page images to obtain visual text fragments and pixel coordinates. It also parses vector layer text fragments and converts them into pixel coordinates through bidirectional mapping. Spatially overlap the visual text box and coordinate metadata within pixel coordinates to generate candidate pairings and association scores; By using a one-to-one matching algorithm, visual text fragments are bound to coordinate metadata, coordinate text is obtained, and the corresponding page image is cropped into visual evidence. Organize the coordinate text and the layout structure relationship of page elements into multimodal document data.

3. The method for locating and quality inspecting the signature of legal documents based on semantic understanding as described in claim 2, characterized in that: The vector layer text fragments are obtained by traversing the unified parsable electronic document data page object, parsing the text drawing object and reading the text content and document coordinates, and aggregating them according to line and character spacing rules to form text fragments.

4. The method for locating and quality inspecting the signature of legal documents based on semantic understanding as described in claim 3, characterized in that: The specific steps for obtaining semantic boundaries are as follows: Read multimodal document data and sort it according to the center point of the text box to form a text sequence, obtain the page spatial structure relationship, and determine the structural elements based on the page spatial structure relationship; Within structural elements, coordinate text is aggregated into segments, and the text sequence is extracted for natural language analysis. The segment text is encoded into semantic representation, the structural features of pixel coordinates are encoded into structural representation, and the semantic representation and structural representation are concatenated into a joint representation to obtain the segment role labeling results. The scanning segment role labeling results take the lower boundary of the pixel coordinates of the last paragraph text as the termination position, and define the boundary between the closing paragraph text and the preceding non-closing paragraph text as the semantic boundary.

5. The method for locating and quality inspecting the signature of legal documents based on semantic understanding as described in claim 4, characterized in that: The process of retrieving and locating multimodal document data based on semantic boundaries and coordinate metadata, and extracting coordinate text data and page evidence image fragments, involves the following specific steps. The semantic boundaries are applied to the coordinate metadata to generate a search window. Within the search window, spatial retrieval is performed to recall candidate coordinate texts, and intersection retrieval is performed on the candidate coordinate texts to recall candidate reading sequences. Remove duplicate candidate reading sequences and determine the range of continuous segments of candidate coordinate text. Then, concatenate the candidate coordinate text within the range of continuous segments in order to obtain the continuous text segments of candidate coordinates. The pixel coordinates of the candidate coordinate text are summarized, and the smallest bounding rectangle is selected as the cropping box to crop the image on the page to obtain the evidence image fragment.

6. The method for locating and quality inspecting the signature of legal documents based on semantic understanding as described in claim 5, characterized in that: The process of obtaining structured entities and segmenting signature blocks through entity recognition and signature detection to construct a signature element graph involves the following specific steps: The candidate coordinate continuous text segment is segmented into paragraphs and then word segmentation, part-of-speech tagging and named entity recognition are performed in sequence to obtain structured entities. The evidence image fragments are binarized and connected component instance segmented to obtain segmented signature blocks. Structured entities and segmented signature blocks are built into nodes respectively. Candidate entity blocks are generated on the same page and their binding scores are calculated. Then, they are matched one by one to determine the relationship edges. The nodes and relationship edges are associated, scored, and filtered to construct a signature element graph.

7. The method for locating and quality inspecting the signature of legal documents based on semantic understanding as described in claim 1, characterized in that: The specific steps for generating the semantic verification result are as follows: The signature subject is extracted from the signature element map and compared with the similarity of the text segment and the preset role. The signature subject and entity reference chain are encoded into semantic vectors. Clustering is performed on semantic vectors to form semantic distribution clusters, and the center vector of the semantic distribution clusters is calculated; The matching degree of the distribution clusters is calculated based on the semantic vector of the signature subject and the center vector of each distribution cluster. The cluster with the highest matching degree is selected as the best matching cluster. The maximum matching degree is compared with the preset semantic threshold to generate a semantic verification mark. The maximum value and the semantic verification mark are output as the semantic verification result.

8. The method for locating and quality inspecting the signature of legal documents based on semantic understanding as described in claim 7, characterized in that: The entity reference chain is obtained by standardizing the entity text of the signing subject, filtering out all entity records with consistent standardized text from the entity records extracted from the segment text, and then sequentially connecting them according to the segment sequence number where the entity records are located.

9. The method for locating and quality inspecting the signature of legal documents based on semantic understanding as described in claim 1, characterized in that: The specific steps for generating the structured review report are as follows: Align and deduplicate the signature element graph and semantic verification results, obtain the list of review objects, and automatically verify and calculate through a fixed set of rules to generate rule verification results; The rule verification results are first grouped and aggregated according to the signature subject identifier, the verification statistics are summarized and the detailed evidence of failure are merged with the semantic verification score and semantic verification mark and formatted output to generate a structured review report.

10. A device for locating and inspecting the signature of legal documents based on semantic understanding, based on the method for locating and inspecting the signature of legal documents based on semantic understanding as described in any one of claims 1 to 9, characterized in that: include, The document parsing module is used to parse electronic document data, extract text content and associate it with corresponding coordinate metadata to generate multimodal document data; The semantic boundary module is used to perform deep semantic and structural analysis on multimodal document data, extract text sequences and perform natural language analysis and semantic analysis, obtain segment role labeling results and determine the termination position, and obtain semantic boundaries; The signature map module is used to perform retrieval and location of multimodal document data based on semantic boundaries and coordinate metadata, extract coordinate text data and page evidence image fragments, and obtain structured entities and segmented signature blocks through entity recognition and signature detection to construct a signature element map; The semantic verification module is used to bind the signature subject in the signature element map with the preset role and encode it into a semantic vector. By calculating the matching degree between the semantic vector and the semantic distribution cluster in the entity reference chain, the semantic verification result is generated. The report generation module is used to perform automated verification and calculation on the signature element map and semantic verification results, obtain the rule verification results, summarize and format the semantic verification results and rule verification results, and generate a structured review report.

Images