PDF contract file identification method and system, medium and program product

Abstract

The invention discloses a PDF contract file identification method and system, a medium and a program product, and relates to the technical field of information identification, and the method comprises the steps: carrying out the image analysis of an obtained PDF contract file through employing a deep learning model, generating an editable text, and synchronously identifying the page layout of the document, outputting structured data including page numbers, text paragraphs and coordinate information; performing multi-dimensional feature matching on the text paragraphs according to an adaptive semantic analysis algorithm, a preset contract template library and a dynamic keyword library, and positioning contract core element information; performing entity relationship verification on the content of the contract core element information, correcting an extraction error through dependency syntactic analysis and semantic role labeling, and establishing a contract element data set containing a confidence coefficient weight; and performing serialized packaging on the verified contract element data set to generate a contract element list conforming to electronic signature authentication. According to the invention, the processing efficiency and recognition precision of the PDF contract file are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information recognition, in particular to a PDF contract file recognition method, system, medium and program product. BACKGROUND

[0002] The existing PDF contract file information extraction technology usually relies on fixed template matching or simple rule base, and it is difficult to accurately identify key fields when facing complex formats (such as multi-column layout, picture-text mixed layout, table nesting, etc.). Especially in the extraction process of core elements such as contract amount, contracting party and signing date, recognition errors often occur due to text position offset, semantic ambiguity or format variation, which needs to rely on manual verification, seriously affecting the contract processing efficiency and increasing the operation cost, so there is room for improvement. SUMMARY

[0003] In order to solve the defects of the prior art and improve the processing efficiency and recognition accuracy of the PDF contract file, the present application provides a PDF contract file recognition method, system, medium and program product.

[0004] In the first aspect, the application aims to achieve the following technical solutions: A PDF contract file recognition method, comprising: using a deep learning model to perform image analysis on the obtained PDF contract file, generating editable text and synchronously recognizing the document page layout, and outputting structured data containing page numbers, text paragraphs and coordinate information; According to the self-adaptive semantic analysis algorithm, the preset contract template library and the dynamic keyword library, the text paragraphs are subjected to multi-dimensional feature matching to locate the core element information of the contract; using a natural language understanding model to perform entity relationship verification on the content of the core element information of the contract, correcting and extracting errors through dependency syntax analysis and semantic role labeling, and establishing a contract element data group containing confidence weight; serializing and packaging the verified contract element data group to generate a contract element list conforming to electronic signature authentication.

[0005] By adopting the technical solution, the structured adaptation capability of the PDF contract file can be enhanced, the information extraction accuracy of the PDF contract file can be improved, and the whole-process automation of contract file recognition and processing can be realized. Through the synergistic effect of the deep learning model and the self-adaptive semantic analysis, the core elements of the contract in the complex layout can be accurately positioned to improve the key field recognition accuracy in the PDF contract file. Through the entity relationship verification mechanism of the natural language understanding model, the extraction error can be automatically corrected and the confidence weight can be given, which is convenient for subsequent automatic generation of the contract element list. Finally, the serialization packaging technology is adopted to be compatible with the electronic signature authentication standard to enhance the adaptation capability of the structured data and the contract data flow efficiency, so as to realize the purpose of improving the processing efficiency and recognition accuracy of the PDF contract file.

[0006] In a preferred example of the present application: the deep learning model is used to analyze the obtained PDF contract file, generate editable text and synchronize the document page layout, output structured data containing page numbers, text paragraphs and coordinate information, specifically including: According to the PDF document type, it is judged whether image preprocessing enhancement is needed. The adaptive image enhancement algorithm is used to optimize the image quality of the scanned PDF, and the original image data of the electronic document PDF is directly extracted; A deep learning model based on attention mechanism is used to detect and recognize the preprocessed image, generate initial editable text, and correct the text sequence by connecting the time sequence classification algorithm; Synchronously, a document analysis network is used to analyze the layout of the PDF page, recognize text blocks, tables, titles, paragraph separators and page boundaries, and establish the mapping relationship between the text elements and the visual coordinates; The recognized text content and the layout analysis result are fused to generate multi-dimensional structured data containing page index, paragraph identification and coordinate positioning, and the coordinate information is normalized to adapt to different sizes of documents.

[0007] By adopting the above technical solution, through the dual-path design of scanned PDF image enhancement (such as CLAHE algorithm) and electronic document PDF native analysis, the compatibility problem of PDF files from different sources can be solved; by combining the document analysis network and the coordinate normalization technology, the accurate positioning of text blocks, tables and other elements can be realized to realize high-precision layout restoration; through multi-dimensional structured data, the standardized processing of text elements of the contract file can be realized.

[0008] In a preferred example of the present application: according to the self-adaptive semantic analysis algorithm, the preset contract template library and the dynamic keyword library, the text paragraphs are subjected to multi-dimensional feature matching to locate the core element information of the contract, specifically including: Based on the multi-dimensional structured data, the context text of each paragraph is extracted to form a semantic window to be matched; An adaptive semantic analysis algorithm including a legal term ontology and a contract element relationship graph is constructed to perform entity recognition and candidate element extraction on the semantic window to be matched, and output a key information candidate set including element names, candidate values and appearing coordinates; Pattern matching is performed according to a preset contract template library and a selectable element set to identify mandatory elements and optional elements, and missing elements are marked; Element positioning optimization is performed in combination with a dynamic keyword library, the dynamic keyword library is adaptively updated according to the contract type and historical recognition results, and a TF-IDF weighting algorithm is used to evaluate the importance of keywords; Core element information is positioned, and conflict detection and resolution are performed, logical constraint relationships between elements are processed through a rule engine, and finally contract core element information is positioned.

[0009] By using the above technical solutions, accurate positioning of core element information and optimization of a dynamic keyword library are realized, the keyword library is dynamically updated through a TF-IDF weighting algorithm and feedback of historical recognition results, and the recognition accuracy of high-frequency keywords (such as "liquidated damages" and "delivery time") is improved.

[0010] In a preferred example of the present application, the content of the contract core element information is verified by using a natural language understanding model, extraction errors are corrected through dependency syntax analysis and semantic role labeling, and a contract element data group including confidence weights is established, specifically including: A pre-trained language model is used to encode the extracted contract element information in context, capturing long-distance dependency relationships and semantic information; A dependency syntax analyzer is applied to construct a syntax structure tree between elements, identify grammar patterns such as subject-predicate-object relationships and determinative relationships, and discover potential element associations; The predicate argument structure of each element in the context is extracted through semantic role labeling to supplement implicit element relationship information; A multi-dimensional confidence evaluation function is designed, which includes the frequency, position importance, context consistency and template matching degree of element appearance, and assigns a dynamic confidence weight to each element; Low-confidence elements are determined, an image recognition model is called to re-extract local text of the original PDF page corresponding to the low-confidence elements, and model parameters are iteratively optimized through an active learning strategy; Finally, a contract element data group including element names, candidate values, confidence weights and correction bases is established.

[0011] By adopting the technical scheme, the pre-trained language model is used to encode the contract element information, long-distance dependence and semantic information are captured, the understanding ability of the relationship between contract elements is improved, and the processing efficiency of complex sentence structure is enhanced; through dynamic confidence evaluation and iterative optimization of model parameters, the accuracy and reliability of the extraction result are continuously improved.

[0012] In a preferred example of the present application, the element positioning optimization is performed in combination with a dynamic keyword library, which includes: High-frequency keywords are identified based on historical contract recognition results, and a domain keyword library is established in combination with contract types; Context features of the currently missed elements are extracted, and the keyword set of the dynamic keyword library is updated; The semantic similarity of the misdetected elements and the correct keywords is calculated, and the TF-IDF weight is adjusted; The context text of the low-confidence elements is pushed to manual annotation, and the positive and negative sample libraries of the dynamic keyword library are updated based on the annotation results.

[0013] By adopting the technical scheme, the dynamic updating capability of the keyword library is improved, the context feature extraction of the missed elements (such as “not mentioning ‘penalty’ is marked as missed”) and the semantic similarity calculation of the misdetected elements (such as cosine similarity algorithm) are implemented to realize the adaptive evolution of the keyword library; through the man-machine cooperative optimization mode, the context text of the low-confidence elements is pushed to manual annotation, and the positive and negative sample libraries are updated through the active learning strategy, so that the updating cycle of the dynamic keyword library adapts to the rapid change requirements of the contract clauses.

[0014] In a preferred example of the present application, the verified contract element data group is serialized and packaged to generate a contract element list that meets the electronic signature authentication, which specifically includes: The verified contract element data group is structured and encoded to generate a basic data packet; Based on the requirements of electronic signature authentication, a digital signature field, a timestamp and a hash value are added to the basic data packet; The compliance of the basic data packet is verified so that the basic data packet contains mandatory fields and optional fields, and finally a contract element list that meets the electronic signature authentication is generated.

[0015] By adopting the technical scheme, the safety and integrity of the contract element data are ensured, and the legal effectiveness of the contract file is enhanced; through the electronic signature authentication, the authenticity and non-tamperability of the contract file are ensured, and the security of the contract management is improved to realize the standardized processing of the contract element data.

[0016] In a second aspect, the application aims to achieve the following technical scheme: A PDF contract file recognition system, the system comprising: An image analysis and structuring module for image analysis of an acquired PDF contract file using a deep learning model, generating editable text, and synchronously recognizing document page layout, outputting structured data containing page numbers, text paragraphs, and coordinate information; An element positioning module for multi-dimensional feature matching of the text paragraphs based on an adaptive semantic analysis algorithm, a preset contract template library, and a dynamic keyword library, positioning contract core element information; A semantic verification module for entity relationship verification of the content of the contract core element information using a natural language understanding model, correcting extraction errors through dependency syntax analysis and semantic role labeling, and establishing a contract element data group containing confidence weight; A data packaging module for serializing and packaging the verified contract element data group, generating a contract element list conforming to electronic signature authentication.

[0017] By adopting the above technical solutions, the whole process automation from the input to the output of the contract file is realized; the various modules have clear division of labor and work together, significantly improving the speed and efficiency of contract processing.

[0018] In a preferred example of the present application, the image analysis and structuring module comprises: A document type judgment unit for judging whether image preprocessing enhancement is needed based on the PDF document type, using an adaptive image enhancement algorithm to optimize image quality for scanned PDF, and directly extracting original image data for electronic document PDF; A character recognition unit for character detection and recognition of the preprocessed image using a deep learning model based on attention mechanism, generating initial editable text, and correcting text sequence through connection time sequence classification algorithm; A layout analysis unit for synchronous layout analysis of PDF pages using a document analysis network, recognizing text blocks, tables, titles, paragraph separators, and page boundaries, and establishing a mapping relationship between text elements and visual coordinates; A structured fusion unit for fusing the recognized text content and layout analysis results, generating multi-dimensional structured data containing page index, paragraph identification, and coordinate positioning, wherein the coordinate information is normalized to adapt to documents of different sizes.

[0019] In a third aspect, the application aims to achieve the following technical solutions: A computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps of the above-mentioned PDF contract file recognition method.

[0020] In a fourth aspect, the application aims to achieve the technical solutions as follows: A computer program product comprising computer programs / instructions that, when executed by a processor, implement the steps of a method for identifying a PDF contract file as described above.

[0021] In summary, the present application includes at least one of the following beneficial technical effects: 1. Improving the accuracy and efficiency of extracting key information from PDF contract files, achieving automated processing and reducing the need for manual intervention; being able to flexibly handle different formats and types of contract files, enhancing the adaptability of the system; ensuring the accuracy of extracted information through multiple verification mechanisms; 2. Through dynamic confidence assessment and iterative optimization of model parameters, continuously improving the accuracy and reliability of extraction results; effectively addressing the ambiguity and uncertainty in element extraction, providing more reliable contract element data. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a flowchart of a method for identifying a PDF contract file according to an embodiment of the present application; Figure 2 is a flowchart of step S2 in a method for identifying a PDF contract file according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] The present application will be further described in detail below in conjunction with the accompanying drawings.

[0024] In an embodiment, as shown in Figure 1 A method for identifying a PDF contract file is disclosed, which specifically includes the following steps: S1: Using a deep learning model to analyze the acquired PDF contract file, generating editable text and synchronously identifying the document page layout, and outputting structured data containing page numbers, text paragraphs, and coordinate information.

[0025] In this embodiment, the deep learning model is a text detection model based on the YOLOv8 (You Only Look Once v8) target detection framework combined with the attention mechanism of Transformer, which is used to locate the text area in the PDF page. By introducing multi-head self-attention in the detection head of YOLOv8, the modeling ability for long-distance text relationships is enhanced.

[0026] Specifically, step S1 includes: S11: Determine whether image preprocessing enhancement is needed according to the PDF document type. For scanned PDF, use adaptive image enhancement algorithm to optimize image quality. For electronic document PDF, directly extract original image data.

[0027] In this embodiment, the PDF document type is determined by the PDF metadata (such as / Type, / Subtype fields). If the PDF contains / XObject fields (representing image layers) but no / Font fields (representing text layers), it is determined to be a scanned PDF; if it contains both / Font fields, it is an electronic document PDF (also known as native electronic PDF). Image preprocessing includes using Otsu algorithm to automatically calculate the optimal threshold to separate text from background, and binarization processing to convert grayscale values to black and white images and de-skew correction, where de-skew correction detects the text line skew angle through Radon transform, and uses affine transformation to rotate the image to the horizontal direction, for example: if the detected text skew angle is 5°, the image is rotated counterclockwise by 5° for correction. The adaptive image enhancement algorithm is CLAHE algorithm.

[0028] S12: Use a deep learning model based on attention mechanism to detect and recognize the preprocessed image, generate initial editable text, and correct the text sequence through connection time series classification algorithm.

[0029] In this embodiment, a deep learning model based on attention mechanism (such as improved YOLOv8+Transformer) is used to locate the text area, for example, the deep learning model outputs text box coordinates and confidence score based on the preprocessed contract file image, and filters low-score boxes (such as confidence <0.5); the initial editable text is the original character sequence directly output by the deep learning model, which contains all the original text data detected and recognized from the PDF image, including text content, text position and text confidence, text sequence and text type label; the text content is, for example, the input image is the paragraph "This contract is signed by both parties on October 1, 2023" in the contract text, and the output character sequence is: "This", "contract", "is", "signed", "by", "both", "parties", "on", "October", "1", "2023", "year", "10", "month", "1", "day", "sign", "up".

[0030] Specifically, character recognition converts character fragment sequences into continuous text through connection time series classification algorithm (CTC). For example: CTC decoding of the detected text line "_ _ Hello_World" outputs "Hello World". Combine language model (such as KenLM) to correct and optimize the coherence of the recognition result, for example, correct "10000" to "10,000" and convert "2023" to "2023".

[0031] S13: Synchronize the document analysis network to perform layout analysis on the PDF page, identify text blocks, tables, titles, paragraph separators, and page boundaries, and establish a mapping relationship between text elements and visual coordinates.

[0032] In this embodiment, the document analysis network is a LayoutLMv3 model, which identifies PDF page element types such as text blocks, tables, and titles. The document analysis network outputs the convenient box coordinates and type labels (such as title, body) of all text blocks, and the coordinate mapping refers to converting them into relative coordinates (range [0, 1]) to adapt to documents of different sizes.

[0033] For example, the mapping relationship between text elements and visual coordinates is established, recording the element type, location, and hierarchy, such as the title being located at the top of the page (coordinates: x=0.1, y=0.1, w=0.8, h=0.2), followed by the body.

[0034] S14: Fuse the recognized text content with the layout analysis results to generate multi-dimensional structured data containing page index, paragraph identification, and coordinate positioning, where the coordinate information is normalized to adapt to documents of different sizes.

[0035] Specifically, the normalized coordinate information refers to normalized coordinate conversion, i.e., converting pixel coordinates to relative coordinates (range [0, 1]) to adapt to documents of different sizes. An example of normalized coordinate conversion is: the page width is 1000 pixels, and the top-left corner coordinate of the text block is (100, 200), which is normalized to (x=0.1, y=0.2). The multi-dimensional structured data includes structured data packets that integrate text, coordinate, and page number information.

[0036] S2: According to the adaptive semantic analysis algorithm, the pre-defined contract template library, and the dynamic keyword library, perform multi-dimensional feature matching on the text paragraphs to locate the core element information of the contract.

[0037] In this embodiment, the adaptive semantic analysis algorithm is a dynamic analysis framework combined with a legal terminology ontology and a contract element relationship graph, where the legal terminology ontology is a pre-defined legal concept set (such as "force majeure" and "breach of contract liability"), supporting synonym expansion, such as expanding "natural disasters" to "force majeure". The contract element relationship graph is used to define the logical constraints between elements, such as "Party A" and "Party B" being mutually exclusive, and "contract amount" needing to be associated with "currency". The semantic graph solves the problem of implicit association of contract elements (such as automatically associating "breach of contract compensation" when there is a "breach of contract" clause).

[0038] The pre-set contract template library contains standard clause templates for different contract types, including sales contracts, lease contracts, etc. The content includes a set of required elements and a set of optional elements. The set of required elements includes "contract amount", "signing date" and "performance period", while the set of optional elements includes "confidentiality clause" and "dispute resolution method".

[0039] The dynamic feature word library is a set of keywords that are adaptively updated based on historical contract data and recognition results. It includes high-frequency keyword statistics and TF-IDF weighted calculation. The dynamic feature word library has an update mechanism for correcting missed detection elements and filtering false detection elements. For example, if the frequency of "late payment fee" in a certain contract type exceeds a threshold, it will be added to the keyword library. For filtering false detection elements, the weight of false detection words is reduced after manual annotation and correction (e.g., when "deposit" is mistakenly detected as "earnest money", its TF-IDF value is reduced).

[0040] Specifically, such as Figure 2 As shown, step S2 includes: S21: Based on multi-dimensional structured data, extract the contextual text of each paragraph to form a semantic window to be matched.

[0041] In this embodiment, the semantic window refers to the contextual text fragment surrounding the target paragraph, used to capture the contextual information of the elements. The contextual text is the neighboring text within a certain range before and after the target paragraph; for example, if the target paragraph is "The amount of this contract is RMB One Million Yuan", the semantic window may include the preceding "Agreed by both parties" and the following "Payment in three installments".

[0042] S22: Construct an adaptive semantic analysis algorithm that includes a legal terminology ontology and a contract element relationship graph. Perform entity recognition and candidate element extraction on the semantic window to be matched, and output a candidate set of key information including element name, candidate value and occurrence coordinates.

[0043] In this embodiment, the element name is the standardized field name to be extracted from the contract (such as "Contract Amount", "Signing Date", "Penalty"); the candidate value is the possible value of the element in the text (such as "1 million yuan" or "October 1, 2023"); and the occurrence coordinates are the current element's position information on the PDF page (such as the upper left corner coordinates (x1, y1) and the lower right corner coordinates (x2, y2)). Candidate elements are possible elements extracted from the semantic window and their associated information.

[0044] In this embodiment, after dividing the text into independent paragraphs based on the layout analysis results (obtained in step S13), a semantic window is constructed. The construction of the semantic window includes extracting the neighboring text of the first two paragraphs and the last two paragraphs for each paragraph, forming a semantic window with a length of 5 times the paragraph length, and filtering out non-text elements (such as images, headers and footers) within the window.

[0045] Specifically, the semantic window is subjected to entity recognition using a BERT-CRF joint model, where BERT extracts context features and CRF captures label transition constraints, and the output format is like [(“contract amount”, “RMB one hundred million”, [0.3, 0.4, 0.6, 0.5])].

[0046] S23: Perform pattern matching according to the preset contract template library and the set of optional elements, identify the mandatory elements and optional elements, and mark the missing elements.

[0047] In this embodiment, after contract pattern matching, the weight of each element is adjusted according to the contract type, and the key elements are matched preferentially. For example, in a labor contract, the weight of “salary” is set to 0.8, and the weight of “workplace” is set to 0.5.

[0048] Specifically, according to the contract type (such as “house rental”), the corresponding template is loaded, the mandatory elements (such as “rent” and “lease period”) and the optional elements (such as “furniture list”) are defined, and the semantic similarity (such as cosine similarity) between the candidate elements and the template elements is calculated. For example, the similarity between the candidate element “rent” and the template element “rent” is 0.92, which is determined to be matched. When the missing element label such as “dispute resolution method” is not matched, an alarm is generated. For example, in a house rental contract, if “rental purpose” is not detected, it is marked as a missing element. The specific content of the missing element can be defined according to the contract type.

[0049] Further, the optimization calculation formula of the element weight based on the contract type is as follows: Element priority = where the contract type weight is the priority weight of the corresponding element of the preset contract type, and the denominator is normalized to ensure that the sum of the weights is 1, avoiding numerical deviation, S24: Combine the dynamic keyword library to optimize element positioning. The dynamic keyword library is updated adaptively according to the contract type and historical recognition results, and the TF-IDF weighting algorithm is used to evaluate the importance of keywords.

[0050] In this embodiment, the TF-IDF weighting algorithm is an algorithm for evaluating the importance of keywords, which combines term frequency (TF) and inverse document frequency (IDF).

[0051] Specifically, the missed element processing needs to count the high-frequency words (such as “penalty” appearing 5 times but not detected) in the contract type, and add them to the dynamic keyword library. When correcting the misdetected elements, first calculate the TF-IDF weighted similarity between the misdetected word (such as “deposit”) and the correct word (such as “deposit”): Similarity = TF-IDF_Mischeck * 0.6 + Semantic Similarity * 0.4, if Similarity > 0.8, then reduce the weight of mischeck words, the mischeck word weight adjustment formula is: New mischeck word weight = original mischeck word weight * (1 - similarity adjustment coefficient), wherein the similarity adjustment coefficient is set according to business requirements, such as taking value 0.5.

[0052] The artificial annotation iteration includes pushing the context text of the low-confidence element to an artificial platform (such as an annotation interface), and the artificial correction result is used to update the positive and negative sample library (such as adding “penalty” as a mischeck word). For example, historical contract data shows that “penalty” appears 10 times but is not identified, and the dynamic keyword library will increase its weight to TOP 5.

[0053] Further, step S24 includes: The element positioning optimization combined with the dynamic keyword library includes: S241: Based on the historical contract recognition result, high-frequency keywords are counted, and a domain keyword library is established combined with the contract type.

[0054] In the embodiment, the high-frequency keyword refers to a word (such as “penalty” “delivery time”) with a higher frequency in the historical contract recognition result, which can be obtained by comparing with a preset keyword high-frequency threshold. Different contract types can define different values of the keyword high-frequency threshold based on different keywords, which will not be described here. The keyword exceeding the preset keyword high-frequency threshold is the high-frequency keyword. The contract type is a type divided according to the contract subject or the clause characteristics, such as a sales contract, a labor contract.

[0055] Specifically, the annotated contract text is extracted from the historical contract database, and the text is segmented using a segmentation tool (jieba or HanLP), and the keywords ranked in the top N of the word frequency in each contract type are counted, such as N = 500. Then, non-legal terms (such as “attachment” “page number”) are filtered combined with a legal dictionary (such as Peking University Law Library) to screen keywords, and words strongly related to contract elements are retained, such as price, liability for breach of contract.

[0056] S242: Extract the context features of the current missed elements, and update the keyword set of the dynamic keyword library.

[0057] In the embodiment, the missed element is a contract element (such as “penalty” not appearing in the candidate set) that actually exists but is not recognized by the model; the context feature is the semantic and structural feature of the text around the missed element (such as adjacent keywords, sentence structure).

[0058] Specifically, first, compare the artificial labeling result with the model output, screen out the missed elements, extract the 3 sentences before and after the missed elements, form the context window, get the semantic window of the missed elements, count the high-frequency co-occurrence words in the semantic window of the missed elements, and extract the sentence pattern template. For example, the missed element "penalty" appears in "0.05% per day after the due date is charged as a penalty", and the context "0.05% per day after the due date is charged" is extracted as a feature.

[0059] S243: Calculate the semantic similarity of the misdetected elements and the correct keywords, and adjust the TF-IDF weight.

[0060] In this embodiment, the semantic similarity refers to the semantic matching degree (such as cosine similarity) of the candidate keywords and the correct keywords. After obtaining the calculated semantic similarity result of the misdetected words and the correct words, the adjustment of the TF-IDF weight includes: if the similarity is >0.7, the TF-IDF weight of the misdetected word is reduced (such as the original weight 0.8 is adjusted to 0.4). If the similarity is <0.3, the weight of the correct word is increased (such as the original weight 0.5 is adjusted to 0.8).

[0061] S244: Push the context text of the low-confidence elements to artificial labeling, and update the positive and negative sample libraries of the dynamic keyword library based on the labeling result.

[0062] In this embodiment, the low-confidence elements refer to elements whose confidence output by the model is lower than a threshold value (such as confidence <0.5, and the confidence threshold value is 0.5). The positive and negative sample libraries are used to store correct and incorrect keyword labeling data respectively.

[0063] Specifically, the context text (such as ±5 sentences) of the low-confidence elements is pushed to the artificial labeling platform, and the text is labeled based on the preset labeling rules, wherein the preset labeling rules include whether the labeled elements are correct and whether the labeled context features are correct. For example, whether "penalty" should be corrected to "penalty", and the context feature such as "penalty" appears in the "overdue payment" scene. The correct labeling data is added to the positive sample library, and the incorrect labeling data is added to the negative sample library. Periodically fine-tune the BERT-CRF model using new samples to improve the accuracy of keyword recognition.

[0064] S25: Perform conflict detection and resolution on the located core element information, process the logical constraint relationship between elements through a rule engine, and finally locate the contract core element information.

[0065] In this embodiment, the rule engine refers to a set of predefined logical rules (such as "Party A and Party B cannot be the same principal" and "the contract amount must be greater than zero"); the logical constraint relationship includes mutual exclusion, dependency or numerical range constraint between elements.

[0066] Specifically, a rule is defined in advance: load the business rules related to the contract type, such as "the lease period needs to be ≥ 6 months", and conflict detection is used to identify contradictory elements, such as "the lease period: 3 months" violates the "≥ 6 months" rule. Conflict resolution strategies include priority-based and manual intervention, where the priority of mandatory elements is higher than that of optional elements, and manual intervention is used when automatic resolution is not possible (such as when there are multiple candidate values for "Party A name"), and the system pushes the case to human review.

[0067] For example, when "contract amount: 100 million yuan" and "free use" are detected at the same time, a rule conflict warning is triggered.

[0068] S3: Use natural language understanding models to perform entity relationship verification on the content of the contract core element information, correct extraction errors through dependency syntax analysis and semantic role labeling, and establish a contract element data set containing confidence weights.

[0069] In this embodiment, dependency syntax analysis refers to analyzing the grammatical dependency relationships of words in a sentence (such as subject-predicate-object, and adverbial structure). For example, in the sentence "Party A grants Party B exclusive agency rights", "Party A" is the subject, "grants" is the predicate, and "Party B" is the indirect object, so as to correct the element association errors caused by OCR misidentification. Semantic role labeling refers to extracting the predicate argument structure of elements in the context (such as "time", "location", "amount"). For example, in "the contract is signed in Beijing and is valid for three years", "Beijing" is labeled as "location" and "three years" is labeled as "duration", which supplements the implicit element relationship information, such as the "contract effective conditions" that are not explicitly mentioned. Confidence weight calculation is used to evaluate the credibility of elements by considering multiple factors. The formula for calculating the confidence weight is as follows: Confidence = 0.4 x template matching degree + 0.3 x context consistency + 0.2 x position weight + 0.1 x frequency weight.

[0070] For example, if a certain element matches the template weight 0.7, the context consistency weight 0.8, and the position weight 0.9 in the contract text, the total weight is 0.83.

[0071] Specifically, step S3 includes: S31: Use a pre-trained language model to encode the extracted contract element information in the context, capturing long-range dependencies and semantic information.

[0072] In this embodiment, the pre-trained language model is a general language model trained on a large corpus (the BERT model is used in this embodiment), which has semantic understanding and context association capabilities. Context encoding refers to converting contract element text into high-dimensional vectors to capture long-range dependencies (such as cross-sentence causal logic).

[0073] Specifically, the candidate element list output by the acquisition step S2 is split into independent sentences, each of which is then encoded using a BERT model to generate a sentence vector (e.g., 768 dimensions) and focus on key phrases such as "pay the full amount" through an attention mechanism.

[0074] For example, the encoded vector of the input text "penalty interest rate 0.05% per day" can represent the semantic association of "penalty", "interest rate", and "0.05%".

[0075] S32: Apply a dependency syntax analyzer to construct a syntactic structure tree between elements, identify subject-predicate-object relationships, and identify grammatical patterns such as adverbial phrase relationships, to discover potential element associations.

[0076] In this embodiment, the syntactic structure tree visually displays the dependency relationships between words (e.g., "Party A" is the subject, "pay" is the predicate, and "penalty" is the object). Dependency syntax analysis can be performed using spaCy or Stanford Parser. Relationship extraction includes identifying subject-predicate-object relationships and extracting adverbial relationships, for example, the sentence "penalty interest rate 0.05% per day" is parsed as: "subject: Party A, predicate: pay, object: penalty, adverbial: overdue per day (temporal adverbial), attribute: 0.05% (attribute of penalty)".

[0077] S33: Extract the predicate argument structure of each element in the context through semantic role labeling to supplement implicit element relationship information.

[0078] In this embodiment, implicit element relationship information is supplemented by semantic roles to mention the association that is not explicitly mentioned (e.g., "force majeure" causes "delay in performance"). Semantic role labeling can use AllenNLP or PropBank tools to label semantic roles, and relationship supplementation refers to the case where the text does not explicitly mention "penalty calculation basis", but there is "interest rate 0.05% per day", then it is inferred that the "calculation basis" is "interest rate 0.05% per day".

[0079] For example, the sentence "due to the earthquake, construction was interrupted, and the construction period was extended" is labeled as: "predicate: construction interruption, agent: earthquake (implicit), patient: construction period, semantic roles: cause (earthquake), result (construction period extension)".

[0080] S34: Design a multi-dimensional confidence evaluation function, which includes the frequency of element occurrence, position importance, context consistency, and matching degree with the template, and assigns a dynamic confidence weight to each element.

[0081] In this embodiment, the multi-dimensional confidence function is to comprehensively evaluate the confidence of the element frequency, position, context consistency, template matching degree and other dimensions. The multi-dimensional confidence evaluation function is: Confidence = 0.3 x template matching degree + 0.25 x context consistency + 0.2 x position weight + 0.25 x frequency weight. If the element matching template weight is 0.85, the context consistency weight is 0.92, the position weight is 0.75, and the frequency weight is 0.68, the comprehensive confidence is 0.817.

[0082] Specifically, the frequency refers to the number of times the element appears in the same type of contract, which is calculated by using the TF-IDF improved algorithm: frequency weight = log(1 + number of element occurrences / total number of contracts) x ln(total number of contracts / number of contracts containing the element); the position weight is based on the position weight matrix established by analyzing historical contract big data, and the position weight matrix includes document position and basic weight, wherein the basic weight of the document position in the head of the contract is 0.2, the basic weight of the document position in the text clause area of the contract is 0.5, and the basic weight of the document position in the signing page is 0.8. The actual position weight = basic weight x position significance coefficient (according to the distance attenuation of the element appearance position from the document boundary), and the specific attenuation rules can be customized as needed. The context consistency refers to whether the element value is contradictory to the context (such as "rent: 1 million yuan" and "total price: 50 million yuan" conflict), and the context consistency evaluation is verified by establishing a three-level consistency verification method, which includes semantic consistency, logical consistency and numerical consistency. The semantic consistency is verified by dependency syntax analysis to verify the semantic consistency of the element value and the context verb; the logical consistency needs to check whether the element value meets the preset business rules, such as "lease period ≤ 20 years", and the numerical consistency needs to perform format verification on structured elements such as amount and date. The final context consistency score = semantic weight x 0.4 + logical weight x 0.3 + numerical weight x 0.3. The template matching degree refers to the similarity with the corresponding element in the preset contract template library. In this embodiment, the cosine similarity algorithm is used to calculate the similarity between the candidate value of the contract element and the corresponding element in the preset contract template library. The calculation formula is: template matching degree = (candidate value vector · template value vector) / (||candidate value vector|| x ||template value vector||), wherein the candidate value vector is generated by BERT model coding, and the template value vector is the TF-IDF weighted average vector of historical contract data.

[0083] S35: Determine the low-confidence element, call the image recognition model to re-extract the local text of the original PDF page corresponding to the low-confidence element, and iteratively optimize the model parameters through the active learning strategy.

[0084] In this embodiment, the active learning strategy refers to manually labeling samples with uncertain model confidence (low-confidence features) and iteratively optimizing the model. Local text re-extraction refers to re-extracting text from specific regions (such as table cells) in the PDF.

[0085] Specifically, features with a confidence score less than 0.5 are filtered out, and the coordinates of these features within the PDF contract file are located (e.g., page 2, coordinates [0.4, 0.5, 0.6, 0.7]). An OCR model (e.g., Easy OCR) is then used to re-identify the local text. The active learning process involves adding the corrected data to the training set and using incremental learning to update the BERT model parameters.

[0086] S36: Finally, establish a contract element data group that includes element name, candidate value, confidence weight, and correction basis.

[0087] In this embodiment, the source of element corrections (such as "subject-verb disagreement discovered by dependency parsing") is recorded first, serving as the basis for subsequent corrections. The contract element data group is formatted as structured data.

[0088] Specifically, the data sets are stored in a database, supporting queries by contract number and element name. Modification logs are also recorded (e.g., "Modified to 0.05% / day on 2023-10-01, original value 0.05%").

[0089] S4: Serialize and encapsulate the verified contract element data group to generate a list of contract elements that conforms to electronic signature authentication.

[0090] In this embodiment, the digital signature uses the RSA algorithm for asymmetric encryption of the data packet. The timestamp is generated in milliseconds by the National Time Service Center (e.g., 2023-10-01 T12:34:56.789Z). The hash value is generated as a data fingerprint using the SHA-256 algorithm.

[0091] Specifically, step S4 includes: S41: The verified contract element data group is structured and encoded to generate a basic data package.

[0092] In this embodiment, the basic data package is a standardized data structure containing information such as contract element names, values, coordinates, and confidence levels. Structured coding refers to converting multi-dimensional contract element data into a machine-readable format (such as JSON-LD or XML).

[0093] Specifically, the contract element data group output in step S3 is mapped to predefined data fields, such as {"name": "contract amount", "value": "1 million yuan", "confidence": 0.95}. Standardized encapsulation includes the addition of contract metadata and multilingual extensions.

[0094] For example, the structured data in the final basic data package includes contract subject information (Party A / Party B name, address), core terms (amount, term, liability for breach of contract), and spatial information (clause coordinates, page number).

[0095] S42: Based on the electronic signature authentication requirements, add a digital signature field, a timestamp, and a hash value to the basic data package.

[0096] In this embodiment, the private key is used to encrypt the data package digest (such as SHA-256 hash value) to generate a digital signature. The data package hash value is sent to the timestamp server through an HTTP request, and a timestamp signed by the certification authority is received. The original data package hash value, signature, and timestamp are stored together in the data package header to complete the binding of the hash value.

[0097] S43: Verify the compliance of the basic data package so that the basic data package contains mandatory fields and optional fields, and finally generate a contract element list that meets the electronic signature authentication requirements.

[0098] In this embodiment, the mandatory fields are fields that must exist according to the electronic signature authentication requirements (such as contract subject, signing time). The optional fields are fields that can be extended according to the contract type (such as attachment hash value).

[0099] Specifically, based on JSON Schema, the existence of mandatory fields is verified, and the date format is checked to see if it conforms to YYYY-MM-DDTHH:mm:ssZ and the coordinate value is checked to see if it is within the range [0, 1]. At the same time, legal compliance is checked: automatically compare whether the electronic signature meets the requirements of Article 13 of the Electronic Signature Law (true identity, true intention, and data integrity), generate a compliance report if it meets the requirements, and if it does not meet the requirements, perform an exception prompt, correction until it meets the requirements. Then generate a compliance report.

[0100] Based on the compliance report and the basic data package, the data package that passes the verification is converted into an authentication list in XML or PDF format through XSLT or JSONPath.

[0101] For example, the final output includes a contract element list with an electronic seal picture (SVG format), a timestamp certificate chain, and a data package hash value (for third-party signature verification).

[0102] Further, OpenSSL is used to implement RSA signature generation, and Apache Santuario is used to process XML signatures.

[0103] It should be understood that the sequence numbers of the steps in the above embodiments do not mean the order of execution, and the execution order of the processes should be determined according to their functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0104] In an embodiment, a PDF contract file identification system is provided, which corresponds to the PDF contract file identification method described above.

[0105] A PDF contract file identification system includes an image analysis and structuring module, an element positioning module, a semantic verification module, and a data encapsulation module. The detailed description of each functional module is as follows: The image analysis and structuring module is used to analyze the acquired PDF contract file using a deep learning model, generate editable text, and simultaneously identify the document page layout, output structured data containing page numbers, text paragraphs, and coordinate information; The element positioning module is used to perform multi-dimensional feature matching on text paragraphs based on adaptive semantic analysis algorithms, pre-set contract template libraries, and dynamic keyword libraries, and locate contract core element information; The semantic verification module is used to use natural language understanding models to verify the content of the contract core element information, correct extraction errors through dependency syntax analysis and semantic role labeling, and establish a contract element data group containing confidence weight; The data encapsulation module is used to serialize and encapsulate the verified contract element data group to generate a contract element list that meets the electronic signature authentication requirements.

[0106] Optionally, the image analysis and structuring module includes: The document type judgment unit is used to determine whether image preprocessing enhancement is needed according to the PDF document type, and to optimize the image quality of scanned PDF using adaptive image enhancement algorithms, and to directly extract original image data for electronic document PDF; The character recognition unit is used to use a deep learning model based on attention mechanism to detect and recognize characters in the preprocessed image, generate initial editable text, and correct the text sequence through connection time sequence classification algorithm; The layout analysis unit is used to simultaneously use a document analysis network to analyze the layout of the PDF page, identify text blocks, tables, titles, paragraph separators, and page boundaries, and establish a mapping relationship between text elements and visual coordinates; The structured fusion unit is used to fuse the recognized text content and the layout analysis result to generate multi-dimensional structured data containing page index, paragraph identification, and coordinate positioning, wherein the coordinate information is normalized to adapt to documents of different sizes.

[0107] The specific definition of the PDF contract file recognition system can refer to the definition of the PDF contract file recognition method described above, which will not be repeated here; each module in the PDF contract file recognition system described above can be realized by software, hardware, and a combination thereof, in whole or in part; each module described above can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor invokes and executes the operations corresponding to each module.

[0108] In one embodiment, a computer-readable storage medium is provided, and the computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the following steps are implemented: S1: using a deep learning model to perform image analysis on the obtained PDF contract file, to generate editable text and simultaneously recognize the document page layout, and to output structured data containing page numbers, text paragraphs, and coordinate information; S2: performing multi-dimensional feature matching on the text paragraphs according to an adaptive semantic analysis algorithm, a preset contract template library, and a dynamic keyword library, and locating contract core element information; S3: using a natural language understanding model to perform entity relationship verification on the content of the contract core element information, correcting extraction errors through dependency syntax analysis and semantic role labeling, and establishing a contract element data group containing a confidence weight; S4: serializing and packaging the verified contract element data group to generate a contract element list that meets the electronic signature authentication requirements.

[0109] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0110] In one embodiment, in particular, according to the embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, the embodiments of the present application include a computer program product including computer programs / instructions that, when executed by a processor, implement the steps of the method of identifying a PDF contract file as described. In such embodiments, the computer program can be downloaded and installed from a network by a communication module, and / or installed from a detachable medium. When the computer program is executed by a central processing unit (CPU), various functions defined in the present application are performed.

[0111] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is exemplified, and in actual applications, the above-mentioned functions can be completed by different functional units and modules according to needs, i.e. the internal structure of the device is divided into different functional units or modules to complete all or part of the above-described functions.

[0112] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand; it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of identifying a PDF contract file, characterized by, The application comprises the following steps: Using a deep learning model to analyze the obtained PDF contract file, generating editable text and synchronously identifying the document page layout, outputting structured data containing page numbers, text paragraphs, and coordinate information; According to the adaptive semantic analysis algorithm, the preset contract template library, and the dynamic keyword library, the text paragraphs are subjected to multi-dimensional feature matching to locate the core element information of the contract; Using a natural language understanding model to verify the content of the core element information of the contract, correcting extraction errors through dependency syntax analysis and semantic role labeling, and establishing a contract element data set containing confidence weights; The verified contract element data set is serialized and packaged to generate a contract element list that meets the requirements of electronic signature authentication.

2. The method of claim 1, wherein, The deep learning model is used to analyze the obtained PDF contract file, generate editable text, and synchronously identify the document page layout, output structured data containing page numbers, text paragraphs, and coordinate information, which specifically includes: According to the PDF document type, determine whether image preprocessing enhancement is needed, use adaptive image enhancement algorithm to optimize the image quality of scanned PDF, and directly extract the original image data of electronic document PDF; Using a deep learning model based on attention mechanism to detect and recognize the preprocessed image, generating initial editable text, and correcting the text sequence through connection time series classification algorithm; Synchronously using document analysis network to analyze the layout of PDF page, identifying text block, table, title, paragraph separator and page boundary, establishing the mapping relationship between text element and visual coordinate; Fusing the recognized text content and layout analysis results, generating multi-dimensional structured data containing page index, paragraph identification, and coordinate positioning, and the coordinate information is normalized to adapt to documents of different sizes.

3. The method of claim 2, wherein, According to the adaptive semantic analysis algorithm, the preset contract template library, and the dynamic keyword library, the text paragraphs are subjected to multi-dimensional feature matching to locate the core element information of the contract, which specifically includes: Based on the multi-dimensional structured data, extract the context text of each paragraph to form a semantic window to be matched; Constructing an adaptive semantic analysis algorithm containing legal terminology ontology and contract element relationship graph, performing entity recognition and candidate element extraction on the semantic window to be matched, and outputting a key information candidate set containing element name, candidate value, and appearance coordinate; According to the preset contract template library and optional element set, perform pattern matching to identify mandatory elements and optional elements, and mark missing elements; Combine the dynamic keyword library to optimize element positioning, the dynamic keyword library is updated adaptively according to the contract type and historical recognition results, and the TF-IDF weighting algorithm is used to evaluate the importance of keywords; Perform conflict detection and resolution on the located core element information, process the logical constraint relationship between elements through a rule engine, and finally locate the core element information of the contract.

4. The method of claim 1, wherein, The entity relationship verification of the contract core element information is performed by using a natural language understanding model, extraction errors are corrected through dependency syntax analysis and semantic role labeling, and a contract element data group containing confidence weight is established, specifically including: A pre-trained language model is used to encode the extracted contract element information in context, capturing long-distance dependency relationships and semantic information; An dependency syntax analyzer is applied to construct a syntax structure tree between elements, identify grammar patterns such as subject-predicate-object relationship and center-determining relationship, and find potential element associations; The predicate argument structure of each element in the context is extracted through semantic role labeling to supplement the implicit element relationship information; A multi-dimensional confidence evaluation function is designed, which includes the frequency, position importance, context consistency and template matching degree of the element, and each element is assigned a dynamic confidence weight; Low-confidence elements are determined, an image recognition model is called to re-extract the local text of the original PDF page corresponding to the low-confidence elements, and the model parameters are iteratively optimized through an active learning strategy; Finally, a contract element data group containing element name, candidate value, confidence weight and correction basis is established.

5. The method of claim 3, wherein, The element positioning optimization combined with a dynamic keyword library includes: High-frequency keywords are identified based on historical contract recognition results, and a domain keyword library is established based on contract types; The context features of the currently missed elements are extracted, and the keyword set of the dynamic keyword library is updated; The semantic similarity between the misdetected elements and the correct keywords is calculated, and the TF-IDF weight is adjusted; The context text of low-confidence elements is pushed to manual annotation, and the positive and negative sample library of the dynamic keyword library is updated based on the annotation results.

6. The method of claim 1, wherein, The contract element data group after verification is serialized and packaged to generate a contract element list that meets the electronic signature authentication requirements, specifically including: The contract element data group after verification is structured and encoded to generate a basic data packet; Based on the requirements of electronic signature authentication, a digital signature field, a timestamp and a hash value are added to the basic data packet; The compliance of the basic data packet is verified to ensure that it contains mandatory fields and optional fields, and finally a contract element list that meets the electronic signature authentication requirements is generated.

7. A system for identifying a PDF contract file, characterized by, The system includes: An image analysis and structuring module for performing image analysis on the acquired PDF contract file by using a deep learning model, generating editable text, and synchronously identifying the document page layout to output structured data containing page numbers, text paragraphs and coordinate information; An element positioning module for performing multi-dimensional feature matching on the text paragraphs based on an adaptive semantic analysis algorithm, a pre-set contract template library and a dynamic keyword library to locate contract core element information; A semantic verification module for performing entity relationship verification on the content of the contract core element information by using a natural language understanding model, correcting extraction errors through dependency syntax analysis and semantic role labeling, and establishing a contract element data group containing confidence weight; A data packaging module for serializing and packaging the contract element data group after verification to generate a contract element list that meets the electronic signature authentication requirements.

8. The system for identifying a PDF contract file of claim 7, wherein, The image analysis and structuring module includes: The document type judging unit is used for judging whether image preprocessing enhancement is needed according to the PDF document type, and optimizing the image quality of the scanned PDF by using an adaptive image enhancement algorithm, and directly extracting original image data of the electronic document PDF; The character recognition unit is used for detecting and recognizing the preprocessed image by using a deep learning model based on an attention mechanism, generating initial editable text, and correcting the text sequence by using a connection time sequence classification algorithm. The layout analysis unit is used for synchronously performing layout analysis on the PDF page by using a document analysis network, recognizing text blocks, tables, titles, paragraph separators and page boundaries, and establishing a mapping relationship between the text elements and the visual coordinates. The structured fusion unit is used for fusing the recognized text content and the layout analysis result, and generating multi-dimensional structured data containing page index, paragraph identification and coordinate positioning, wherein the coordinate information is normalized to adapt to documents of different sizes.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the steps of the PDF contract file recognition method according to any one of claims 1 to 6.

10. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the steps of the PDF contract file recognition method according to any one of claims 1 to 6.

Images