Method for automatically checking electronic signature document based on keyword extraction

Abstract

The invention discloses a method for automatically auditing an electronic signature document based on keyword extraction. The method comprises the following steps: configuring document data item auditing rules corresponding to different document types; analyzing the title name information of the to-be-audited document, and extracting a document type keyword; on the basis of the document type keywords, verifying the document type review configuration information, analyzing document data items of the to-be-reviewed document, and for each document data item, extracting a document data item keyword to form a to-be-reviewed data set; performing data item review on the document data item keyword corresponding to each document data item in the to-be-reviewed data set; according to the method, the problem that the signature document cannot be verified and reviewed after being generated can be effectively solved, the purpose of online verification and auditing is achieved, the accuracy of keyword extraction is relatively improved based on the method, and the problems of the document after auditing are greatly reduced; meanwhile, the manual workload in the online verification and auditing stage is reduced, and the document circulation efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of text data extraction and data review, and in particular relates to a method for automatically reviewing electronic signature documents based on keyword extraction. Background Art

[0002] In order to adapt to the needs of green modern digital supply chain, we should carry out the adaptation of engineering materials, control the material performance process, optimize the material performance management, and carry out the electronicization of documents such as arrival acceptance form, commissioning form, and warranty form in combination with the actual business, so as to realize the business process of online creation of business documents, online verification, online signature, online circulation, and online review of performance documents; in order to ensure the correctness of the documents in the process of document generation, manual data verification and confirmation are required before each document is generated; and the generated documents must be signed and sealed before they can be downloaded for review. Sometimes it is found that the content and format of the documents are wrong or do not meet the requirements; not only does it bring a lot of workload to the reviewers, but also the efficiency of document review is relatively low. At present, the electronic signature data review process is all before the document is generated, and in the TF-IDF keyword extraction algorithm, simply measuring the importance of words by word frequency is not accurate and comprehensive, and the number of important words may not be large; and words of different parts of speech, such as nouns and prepositions, are given the same weight. Summary of the invention

[0003] In order to solve the problems existing in the prior art, the present invention proposes a method for automatically reviewing electronic signature documents based on keyword extraction.

[0004] The technical solution of the present invention is as follows: A method for automatically reviewing electronic signature documents based on keyword extraction, the method comprising: Step 1) Configure the document data item review rules corresponding to different document types to form document type review configuration information; Step 2) Parse the header name information of the document to be reviewed, and extract the document type keywords based on the K-means algorithm and TF-IDF algorithm; based on the document type keywords, verify the document type review configuration information. If there is no document data item review rule corresponding to the document to be reviewed, stop the review and return to the rule configuration prompt; otherwise, record the document data item review rule corresponding to the document to be reviewed and enter the data item review; Step 3) Analyze the document data items of the documents to be reviewed. For each document data item, use the K-means algorithm and TF-IDF algorithm to extract the keywords of the document data item to form a data set to be reviewed; Step 4) For the document data item keywords corresponding to each document data item in the to-be-reviewed data set, data item review is performed based on the recorded document data item review rules; for documents to be reviewed where the document data item keywords corresponding to each document data item have passed the data item review, a PDF document is generated and signed; for documents to be reviewed where one or more document data item keywords corresponding to the document data items have failed the data item review, the review is terminated and transferred to manual review.

[0005] Furthermore, the document types include supply documents, miscellaneous material documents, material arrival and handover documents, arrival acceptance documents, commissioning documents and quality assurance documents.

[0006] Furthermore, the specific method of step 2) is as follows: Step 2-1) Analyze the header of the document to be reviewed and obtain the header name information; Step 2-2) Use the K-means algorithm to cluster and extract the text keywords in the header name information, select and calculate the TF-IDF value and part-of-speech weight of the currently clustered and extracted text keywords, and select the first 2-5 as the determined keyword combination to extract the document type keywords.

[0007] Furthermore, the specific method of step 3) is as follows: Step 3-1) Parse the document data items of the document to be reviewed, parse the data horizontally with a row as a unit and a cell as a basic data unit, and store the document data in a table format, with one row as a unit, storing one KV data item or multiple KV data items, and each KV data item corresponds to two cells of the basic data unit; Step 3-2) Determine the number of clusters K of each cell content by the elbow rule, execute the K-means algorithm and calculate the sum of squared errors (SSE) under different cluster numbers K, and determine the number of clusters K of each cell content based on the relationship curve between the sum of squared errors (SSE) and the number of clusters K; Step 3-3) Based on the determined number of clusters K, use the K-means clustering algorithm to cluster the KV data items to obtain new clustered KV content; Step 3-4) Based on the newly clustered KV content, use the TF-IDF algorithm to calculate the word weights, re-select 2-5 KV contents with different weights to form key words, extract the keywords of the document data items, and generate a data set to be reviewed and examined.

[0008] Furthermore, the calculation method of the sum of squared errors SSE is as follows: ; In the formula, p is the sample within the cluster; m ifor C i Centroid point; C i for clusters; i is the i-th cluster, i The value range is [1, k ], k is the number of cluster centers, defined according to the sample data.

[0009] Furthermore, the specific method of step 3-3) is as follows: Step 3-3-1) Calculate the distance between each sample point and the K cluster centers, and assign each sample point to the cluster closest to it. The distance calculation formula is as follows: ; In the formula, L is the distance from the sample to the cluster center; x i is the i-th sample point; is the cluster center point; is the square of the distance to the i-th sample point; Step 3-3-2) Calculate the center vectors of K clusters and use the calculation results as the cluster center of the new cluster. The vector formula is as follows: ; In the formula, is the center vector of the cluster; M k is the number of samples in K clusters; x i is the i-th sample point; P k is a sample in a cluster; Step 3-3-3) Repeat steps 3-3-1) and 3-3-2) until the center vectors of all clusters no longer change, and output the clustered clusters: ; In the formula, P is a collection of clusters; c 1, c 2, c 3, ..., c k They are the 1st, 2nd, 3rd, ..., kth clusters respectively.

[0010] Furthermore, the specific method of step 3-4) is as follows: Step 3-4-1) Calculate the TF-IDF value of the text in all current clusters: TF = n / N IDF =log[ D / (1+ d )] In the formula, TF is the word frequency; n is the number of times the word appears in the text; N is the sum of the number of occurrences of all words; IDF is the prevalence of the word in the corpus; D is the total number of texts in the corpus; d is the number of texts containing the word; Step 3-4-2) Give different part-of-speech weights according to part-of-speech w 1. w 2.…… w n , calculate the weight of the part of speech corresponding to the word value of each KV content, which is used for subsequent word weight addition calculation: Part-of-speech weight = data n*weight w; Word weight = TF*IDF+part-of-speech weight; Step 3-4-3) After the weights of the clustered texts in each cluster are redistributed, the first 2-5 are selected as representative texts, the keywords of the document data items are extracted, and the data set to be reviewed and examined is generated.

[0011] Furthermore, the specific method of step 4) is as follows: Step 4-1) traverse the data set to be reviewed and examined, and match the document data item keyword corresponding to each document data item with the corresponding review rule in the document type review configuration information recorded in turn; Step 4-2) For the documents to be reviewed that have passed the data item review for the document data item keywords corresponding to each document data item, generate a PDF document and sign it; Step 4-3) For documents to be reviewed where the document data item keywords corresponding to one or more document data items have not passed the data item review, the review is terminated and transferred to manual review.

[0012] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes a method for automatically reviewing electronic signature documents based on keyword extraction. The method can effectively solve the problem that the signature documents cannot be verified and reviewed after they are generated, and achieve the purpose of online verification and review. In addition, based on the present invention, the accuracy of keyword extraction is relatively improved, and the problems of documents after review are greatly reduced; at the same time, the manual workload in the online verification and review stage is reduced, and the circulation efficiency of documents is improved.

[0013] The method of the present invention relatively improves the accuracy of extracting corresponding text data information, realizes the automated and intelligent extraction of document review information, realizes the automated rule matching and verification of document key information data sets, effectively solves the tedious work of manual verification in the document generation process, fully realizes the online verification and audit of document files, and greatly improves the accuracy of document generation and the efficiency of circulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Flow chart of a method for automatically reviewing electronic signature documents based on keyword extraction in an embodiment; Figure 2 Schematic diagram of the relationship between the number of clusters K and SSE in the embodiment. DETAILED DESCRIPTION

[0015] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent modifications of the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0016] Example: The present invention provides a method for automatically reviewing electronic signature documents based on keyword extraction, such as Figure 1 As shown, the method comprises the following steps: Step 1) Configure the document data item review rules corresponding to different document types to form document type review configuration information; there are many types of documents involved in the business process, such as supply orders, miscellaneous material documents, material arrival and delivery orders, etc. For each document type, configure the review rules of this type of document data item. For example, configure the delivery date [2024-08-08], and configure \d{4}-\d{2}-\d{2}.

[0017] Step 2) Parse the header name information of the document to be reviewed, and extract the document type keywords based on the K-means algorithm and TF-IDF algorithm; based on the document type keywords, verify the document type review configuration information. If there is no document data item review rule corresponding to the document to be reviewed, stop the review and return to the rule configuration prompt; otherwise, record the document data item review rule corresponding to the document to be reviewed and enter the data item review; Step 3) Analyze the document data items of the documents to be reviewed. For each document data item, use the K-means algorithm and TF-IDF algorithm to extract the keywords of the document data item to form a data set to be reviewed; Step 4) For the document data item keywords corresponding to each document data item in the to-be-reviewed data set, data item review is performed based on the recorded document data item review rules; for documents to be reviewed where the document data item keywords corresponding to each document data item have passed the data item review, a PDF document is generated and signed; for documents to be reviewed where one or more document data item keywords corresponding to the document data items have failed the data item review, the review is terminated and transferred to manual review.

[0018] Furthermore, the document types include supply documents, miscellaneous material documents, material arrival and handover documents, arrival acceptance documents, commissioning documents and quality assurance documents.

[0019] Furthermore, the specific method of step 2) is as follows: Step 2-1) Analyze the header of the document to be reviewed and obtain the header name information; Step 2-2) Use the K-means algorithm to cluster and extract the text keywords in the header name information, select and calculate the TF-IDF value and part-of-speech weight of the currently clustered and extracted text keywords, and select the first 2-5 as the determined keyword combination to extract the document type keywords.

[0020] The following is an example to illustrate the above process. Take the type of sporadic material arrival acceptance and handover form as an example, parse the document name, and use the elbow rule to select the appropriate cluster number K of 6 according to the random 1-9 cluster number K; then use K-means to cluster the text "sporadic material arrival acceptance and handover form" to obtain the clustering result {"sporadic", "materials", "arrival", "handover", "acceptance", "order"}. Design the permission for the clustering result part of speech {noun: 30%, verb: 20%, adjective: 10%, numeral: 10%, quantifier: 5%, pronoun; 5%, adverb: 5%, preposition: 5%, other words (conjunction, auxiliary word, interjection, onomatopoeia, etc.): 10%}, calculate the part of speech weight and TF and IDF according to the clustering result; re-weight the clustering result, that is: {"sporadic", "materials", "handover", "order", "arrival", "acceptance"}, select the keyword "sporadic material handover form" according to the result, and match the corresponding rule information according to the keyword.

[0021] Furthermore, the specific method of step 3) is as follows: Step 3-1) Parse the document data items of the document to be reviewed, parse the data horizontally with a row as a unit and a cell as a basic data unit, and store the document data in a table format, with one row as a unit, storing one KV data item or multiple KV data items, and each KV data item corresponds to two cells of the basic data unit; Step 3-2) Determine the appropriate number of clusters K for each cell content by the elbow rule, execute the K-means algorithm and calculate the sum of squared errors SSE under different cluster numbers K, based on the relationship curve between the sum of squared errors SSE and the number of clusters K, as shown in Figure 2 As shown, the appropriate number of clusters K for each cell content is determined; the specific method is as follows: First, a range of possible values ​​of K (the number of clusters) are chosen.

[0022] For each value of K, the K-means algorithm is run and the sum of squares (SSE) of the distances from each data point (or "sample point") to the center of the cluster to which it belongs is calculated.

[0023] Draw a line graph between the K value and the corresponding SSE.

[0024] Observe the line graph and find a K value that no longer significantly reduces SSE when increasing K value. The point corresponding to this K value forms an "elbow".

[0025] The value of K where the elbow lies is considered to be the optimal number of clusters.

[0026] It should be noted that the elbow rule does not always give a clear elbow point, and sometimes there may be some subjective judgment. In practical applications, factors such as business requirements and computing resources may also need to be considered to determine the final K value.

[0027] Step 3-3) Based on the determined appropriate number of clusters K, use the K-means clustering algorithm to cluster the KV data items to obtain new clustered KV content; Step 3-4) Based on the newly clustered KV content, use the TF-IDF algorithm to calculate the word weights, re-select 2-5 KV contents with different weights to form key words, extract the keywords of the document data items, and generate a data set to be reviewed and examined.

[0028] TF-IDF is a statistical method used to assess the importance of a word to a document in a set of documents or a corpus (another way of saying it: to assess the importance of a word or term relative to other terms in a set of documents or a corpus). The importance of a word increases in direct proportion to the number of times it appears in a document, but decreases in inverse proportion to the frequency of its occurrence in the corpus.

[0029] The main idea of ​​TF-IDF is: if a word appears in an article with a high frequency TF and rarely appears in other articles, it is considered that this word or phrase has good category distinction ability and is suitable for classification. In the task of keyword extraction, TF-IDF can be used to identify important terms in a document. By calculating the TF-IDF value of each term and selecting terms with higher values ​​as keywords, key terms that can represent the content of the document can be automatically extracted from the document.

[0030] There are many ways to calculate term frequency (TF), the most common is to directly count the number of times a term appears in a document. In order to make documents of different lengths comparable, term frequency is usually normalized as follows: TF = n / N in, TF is the word frequency; n is the number of times the word appears in the text; N is the sum of the number of occurrences of all words; Inverse document frequency (IDF) reflects the importance of a term in the entire document collection, and its calculation formula is: IDF =log[ D / (1+ d )] in: IDF is the prevalence of the word in the corpus; D is the total number of texts in the corpus; d is the number of texts containing the word;

[0031] The significance of IDF is that when a term appears in fewer documents, its IDF value is higher, indicating that its discrimination is greater; conversely, when a term appears in most documents, its IDF value is lower, indicating that its discrimination is smaller.

[0032] The method of the present invention introduces part-of-speech weighting on the basis of the TF-IDF algorithm, thereby improving the TF-IDF algorithm. Part-of-speech weighting is the weight of part-of-speech features. For example, in a text, nouns can better represent the topic features of a document than verbs and adjectives. Part-of-speech weighting effectively weighs the weight impact brought by the part-of-speech.

[0033] After word segmentation and clustering, the parts of speech are annotated. Parts of speech are the basic grammatical attributes of vocabulary. The act of part-of-speech tagging is to determine the grammatical role of each word in a given Chinese sentence, determine the part of speech of each word and annotate it. Words that represent places, things, and names are called nouns, those that represent state changes are called verbs, and those that describe or modify nouns are called adjectives. Example sentence: "Sporadic material arrival handover form", the result of part-of-speech tagging of this sentence is as follows: "Sporadic / adjective material / noun arrival / verb handover / verb single / noun".

[0034] Furthermore, the calculation method of the sum of squared errors SSE is as follows: ; In the formula, p is the sample within the cluster; m i for C i Centroid point; C i for clusters; i is the i-th cluster, i The value range is [1, k ], k is the number of cluster centers, defined according to the sample data.

[0035] Furthermore, the specific method of step 3-3) is as follows: Step 3-3-1) Calculate the distance between each sample point and the K cluster centers, and assign each sample point to the cluster closest to it. The distance calculation formula is as follows: ; In the formula, L is the distance from the sample to the cluster center; x i is the i-th sample point; is the cluster center point; is the square of the distance to the i-th sample point; Step 3-3-2) Calculate the center vectors of K clusters and use the calculation results as the cluster center of the new cluster. The vector formula is as follows: ; In the formula, is the center vector of the cluster; is the number of samples in K clusters; x i is the i-th sample point; P k is a sample in a cluster; Step 3-3-3) Repeat steps 3-3-1) and 3-3-2) until the center vectors of all clusters no longer change, and output the clustered clusters: ; In the formula, P is a collection of clusters; c 1, c 2, c 3, ..., c k They are the 1st, 2nd, 3rd, ..., kth clusters respectively.

[0036] Furthermore, the specific method of step 3-4) is as follows: Step 3-4-1) Calculate the TF-IDF value of the text in all current clusters: TF = n / N IDF =log[ D / (1+ d )] In the formula, TF is the word frequency; n is the number of times the word appears in the text; N is the sum of the number of occurrences of all words; IDF is the prevalence of the word in the corpus; D is the total number of texts in the corpus; d is the number of texts containing the word; Step 3-4-2) Give different part-of-speech weights according to part-of-speech w 1. w 2.…… w n , calculate the weight of the part of speech corresponding to the word value of each KV content, which is used for subsequent word weight addition calculation: Part-of-speech weight = data n*weight w; Word weight = TF*IDF+part-of-speech weight; Step 3-4-3) After the weights of the clustered texts in each cluster are redistributed, the first 2-5 are selected as representative texts, the keywords of the document data items are extracted, and the data set to be reviewed and examined is generated.

[0037] Furthermore, the specific method of step 4) is as follows: Step 4-1) traverse the data set to be reviewed and examined, and match the document data item keyword corresponding to each document data item with the corresponding review rule in the document type review configuration information recorded in turn; Step 4-2) For the documents to be reviewed that have passed the data item review for the document data item keywords corresponding to each document data item, generate a PDF document and sign it; Step 4-3) For documents to be reviewed where the document data item keywords corresponding to one or more document data items have not passed the data item review, the review is terminated and transferred to manual review.

[0038] The above description is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a technician familiar with the technical field within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.

Claims

1. A method for automatically reviewing electronic signature documents based on keyword extraction, characterized in that: The method comprises: Step 1) Configure the document data item review rules corresponding to different document types to form document type review configuration information; Step 2) Parse the header name information of the document to be reviewed, and extract the document type keywords based on the K-means algorithm and TF-IDF algorithm; based on the document type keywords, verify the document type review configuration information. If there is no document data item review rule corresponding to the document to be reviewed, stop the review and return to the rule configuration prompt; otherwise, record the document data item review rule corresponding to the document to be reviewed and enter the data item review; Step 3) Analyze the document data items of the documents to be reviewed. For each document data item, use the K-means algorithm and TF-IDF algorithm to extract the keywords of the document data item to form a data set to be reviewed; Step 4) For the document data item keywords corresponding to each document data item in the to-be-reviewed data set, data item review is performed based on the recorded document data item review rules; for documents to be reviewed where the document data item keywords corresponding to each document data item have passed the data item review, a PDF document is generated and signed; for documents to be reviewed where one or more document data item keywords corresponding to the document data items have failed the data item review, the review is terminated and transferred to manual review.

2. The method for automatically reviewing electronic signature documents based on keyword extraction according to claim 1 is characterized in that: The document types include supply documents, miscellaneous material documents, material arrival and handover documents, arrival acceptance documents, commissioning documents and quality assurance documents.

3. The method for automatically reviewing electronic signature documents based on keyword extraction according to claim 1 is characterized in that: The specific method of step 2) is as follows: Step 2-1) Analyze the header of the document to be reviewed and obtain the header name information; Step 2-2) Use the K-means algorithm to cluster and extract the text keywords in the header name information, select and calculate the TF-IDF value and part-of-speech weight of the currently clustered and extracted text keywords, and select the first 2-5 as the determined keyword combination to extract the document type keywords.

4. The method for automatically reviewing electronic signature documents based on keyword extraction according to claim 1 is characterized in that: The specific method of step 3) is as follows: Step 3-1) Parse the document data items of the document to be reviewed, parse the data horizontally with a row as a unit and a cell as a basic data unit, and store the document data in a table format, with one row as a unit, storing one KV data item or multiple KV data items, and each KV data item corresponds to two cells of the basic data unit; Step 3-2) Determine the number of clusters K of each cell content by the elbow rule, execute the K-means algorithm and calculate the sum of squared errors (SSE) under different cluster numbers K, and determine the number of clusters K of each cell content based on the relationship curve between the sum of squared errors (SSE) and the number of clusters K; Step 3-3) Based on the determined number of clusters K, use the K-means clustering algorithm to cluster the KV data items to obtain new clustered KV content; Step 3-4) Based on the newly clustered KV content, use the TF-IDF algorithm to calculate the word weights, re-select 2-5 KV contents with different weights to form key words, extract the keywords of the document data items, and generate a data set to be reviewed and examined.

5. The method for automatically reviewing electronic signature documents based on keyword extraction according to claim 4 is characterized in that: The calculation method of the sum of squared errors SSE is as follows: ； In the formula, p is the sample within the cluster; m i for C i Centroid point; C i for clusters; i is the i-th cluster, i The value range is [1, k ], k is the number of cluster centers, defined according to the sample data.

6. The method for automatically reviewing electronic signature documents based on keyword extraction according to claim 5 is characterized in that: The specific method of step 3-3) is as follows: Step 3-3-1) Calculate the distance between each sample point and the K cluster centers, and assign each sample point to the cluster closest to it. The distance calculation formula is as follows: ； In the formula, L is the distance from the sample to the cluster center; x i is the i-th sample point; is the cluster center point; is the square of the distance to the i-th sample point; Step 3-3-2) Calculate the center vectors of K clusters and use the calculation results as the cluster center of the new cluster. The vector formula is as follows: ； In the formula, is the center vector of the cluster; M k is the number of samples in K clusters; x i is the i-th sample point; P k is a sample in a cluster; Step 3-3-3) Repeat steps 3-3-1) and 3-3-2) until the center vectors of all clusters no longer change, and output the clustered clusters: ； In the formula, P is a collection of clusters; c 1, c 2, c 3, ..., c k They are the 1st, 2nd, 3rd, ..., kth clusters respectively.

7. The method for automatically reviewing electronic signature documents based on keyword extraction according to claim 6 is characterized in that: The specific methods of steps 3-4) are as follows: Step 3-4-1) Calculate the TF-IDF value of the text in all current clusters: TF = n / N IDF =log[ D / (1+ d )] In the formula, TF is the word frequency; n is the number of times the word appears in the text; N is the sum of the number of occurrences of all words; IDF is the prevalence of the word in the corpus; D is the total number of texts in the corpus; d is the number of texts containing the word; Step 3-4-2) Give different part-of-speech weights according to part-of-speech w 1. w 2.…… w n , calculate the weight of the part of speech corresponding to the word value of each KV content, which is used for subsequent word weight addition calculation: Part-of-speech weight = data n*weight w; Word weight = TF*IDF+part-of-speech weight; Step 3-4-3) After the weights of the clustered texts in each cluster are redistributed, the first 2-5 are selected as representative texts, the keywords of the document data items are extracted, and the data set to be reviewed and examined is generated.

8. The method for automatically reviewing electronic signature documents based on keyword extraction according to claim 4 is characterized in that: The specific method of step 4) is as follows: Step 4-1) traverse the data set to be reviewed and examined, and match the document data item keyword corresponding to each document data item with the corresponding review rule in the document type review configuration information recorded in turn; Step 4-2) For the documents to be reviewed that have passed the data item review for the document data item keywords corresponding to each document data item, generate a PDF document and sign it; Step 4-3) For documents to be reviewed where the document data item keywords corresponding to one or more document data items have not passed the data item review, the review is terminated and transferred to manual review.

Images