Document information extraction and assignment method and device

Abstract

The invention provides a document information extraction and assignment method and device, and the method comprises the steps: firstly obtaining original government affair documents, carrying out the original department marking of each original government affair document, obtaining a document information set of each original government affair document, carrying out the primary evaluation of the original government affair documents based on the document information set, and obtaining a document information set of each original government affair document; obtaining an analysis data matrix on the basis of the original government affair document subjected to the primary evaluation, performing secondary evaluation, performing correction operation on the original government affair document on the basis of the secondary evaluation, uniformly moving the defective government affair document to an artificial to-be-processed area, extracting government affair keywords of the original government affair document subjected to the correction operation, and storing the extracted government affair keywords in the artificial to-be-processed area. According to the method, the optimized clustering data set is obtained based on government affair keyword clustering, and finally, the original government affair documents are subjected to ant colony assignment operation, so that the original government affair documents are assigned to the corresponding departments, the document assignment accuracy and processing efficiency are improved, automatic defect screening and error correction are realized, and the method is suitable for large-scale popularization and application. And the intelligence and standardization level of government affair document processing can be obviously enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of document distribution, and in particular to a method and device for extracting and distributing document information. Background Art

[0002] Against the backdrop of the digital transformation of government affairs and the surge in demand for enterprise document management, document information extraction and intelligent assignment technologies are facing multi-dimensional upgrade challenges. The existing technical system mainly relies on manual experience or rule-making to achieve document classification and task assignment. However, this approach has obvious limitations.

[0003] On the one hand, manual experience is often subjective and one-sided, and in the process of allocating government documents based on manual experience, manual allocation errors are prone to occur. However, the allocation of government documents based on rule-making has low allocation accuracy.

[0004] On the other hand, traditional dispatching methods often lack the ability to dynamically correct errors in government documents. In the process of submitting departments to write government documents, it is difficult to avoid errors in government documents such as format errors and text errors. Relying on traditional dispatching methods may cause these erroneous information to continue to be transmitted in the dispatch process, and there is also a lack of a unified and centralized processing process for government documents with larger defects.

[0005] Therefore, the existing document distribution methods have low distribution accuracy when dealing with distribution environments where government documents have a large number of words and there are many distribution departments. They are often unable to screen documents multiple times and correct errors in the documents at the same time during the distribution process. In addition, the existing document distribution methods often lack unified and centralized processing of defective documents. Summary of the Invention

[0006] In view of the above-mentioned problems, in combination with the first aspect of the present invention, an embodiment of the present invention provides a method for extracting and assigning document information, the method comprising: S1: Obtain original government documents and mark each original government document with the original department; S2: Obtain the document information set of each original government document and perform an evaluation on the original government document based on the document information set; S3: Obtain an analysis data matrix based on the original government document that has undergone the first evaluation, conduct a second evaluation on the original government document based on the analysis data matrix, and perform correction operations on the original government document based on the second evaluation; S4: Obtain defective government documents based on steps S1, S2, and S3; S5: extracting government keywords from the original government document that has undergone the correction operation, clustering based on the government keywords to obtain an original clustering dataset, performing clustering evaluation on the original clustering dataset, and generating an optimized clustering dataset; S6: Perform ant colony assignment based on the optimized clustering dataset to assign the original government documents to the corresponding departments; S7: The defective rate of documents submitted by the acquisition department based on defective government documents and the assigned original government documents.

[0007] On the other hand, an embodiment of the present invention also provides a document information extraction and dispatching device, including a processor and a machine-readable storage medium, wherein the machine-readable storage medium is connected to the processor, the machine-readable storage medium is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the machine-readable storage medium to implement the above method.

[0008] Based on the above aspects, the embodiment of the present application determines the submission department of each original government document by marking the original department of each original government document. When marking the original department, it will make a judgment based on the standardized original government document acceptance path, so as to screen the original government document once. The original government document can be screened again through the first evaluation and the second evaluation to screen out defective government documents, and the defective government documents will be uniformly moved to the manual waiting area for manual correction or return. In the process of the second evaluation, the original government document will be corrected to prevent the continuous transmission of errors in the original government document during the dispatch process. Government keywords can be classified through multi-stage clustering to provide support for subsequent ant colony dispatch, so that the ant colony dispatch can find the dispatch path more accurately. The ant colony dispatch can improve the dispatch speed of document dispatch and the accuracy of path dispatch. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a schematic diagram of the execution flow of a document information extraction and dispatching method provided by an embodiment of the present invention; Figure 2 This is a data flow diagram of a document information extraction and distribution method provided by an embodiment of the present invention; Figure 3 It is a schematic diagram of a document information extraction and distribution device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0010] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1 This is a schematic diagram of the execution flow of a document information extraction and dispatching method provided by an embodiment of the present invention. Figure 2 This is a data flow diagram of a document information extraction and dispatching method provided by an embodiment of the present invention. The document information extraction and dispatching method is introduced in detail below.

[0011] Step S1: Obtain original government documents and mark each original government document with the original department.

[0012] In this embodiment, original government documents may include policy documents, service materials, meeting minutes, and statistical data. For policy documents, they may include "Opinions on Promoting the Implementation of XXXXX" in PDF format, and normative documents jointly issued by departments, such as "Public XXXXX Management Measures" in DOCX format. For service materials, they may include telephone application forms submitted by the public, such as "XX Application Form" in JSON format. For meeting minutes, they may include voice-converted text of internal meetings, such as "Minutes of the 2025 **** Work Conference" in TXT format. For statistical data, they may include annual XX revenue and expenditure reports in EXCEL format.

[0013] In a possible implementation, step S1 includes: Step S11: receiving the original government document based on the unified file transfer interface protocol, and standardizing the original government document receiving path using a path hierarchy structure to obtain a standardized original government document receiving path.

[0014] Specifically, the path hierarchy structure is represented by using the first level, second level, third level and fourth level to standardize the original government document acceptance path. The first level is represented by the department name, the second level is represented by the original government document time, the third level is represented by the type of the original government document, and the fourth level is represented by the name of the original government document. The original government document acceptance path is standardized and sorted step by step based on the order of the first level, second level, third level and fourth level.

[0015] In this embodiment, it is assumed that the original government documents are being transmitted. In detail, taking the XX Bureau documents as an example, the XX Bureau documents are received based on standardized file transfer interfaces such as API and SFTP, supporting batch upload and real-time streaming transmission modes, and being compatible with mainstream document formats such as PDF and DOCX. Based on a unified interface protocol, the transmission differences of multi-source documents are eliminated, and a mode such as Kaflka message queue is used to buffer high-concurrency document streams to prevent server overload. Before the XX Bureau staff submits the XX Bureau documents, the storage path of the XX Bureau documents is stored based on the path hierarchy structure to ensure file transmission. The original government document acceptance path received by the transmission interface is a standardized receiving path, and the path hierarchy structure can be expressed as: / Submitting department name / year / document type / file name, where the submitting department name is the first level, the year is the second level, the document type is the third level, and the file name is the fourth level. For example, if the name of the document received by the XX Bureau is "Notice on Adjusting XX Standards.pdf", the path hierarchy structure can be expressed as: / XX Bureau / 2023 / Official Letter / Notice on Adjusting XX Standards.pdf, and the original government document acceptance path obtained by the transmission interface is shown in the above path hierarchy structure.

[0016] Step S12: pre-set the department standard coding library and establish a fault-tolerant mechanism, and match the unique coding of the standardized original government document acceptance path based on the department standard coding library and the fault-tolerant mechanism.

[0017] In this embodiment, the data in the preset department standard code library can be expressed as, for example, XX Bureau code MZ001, XX Bureau code SW002, etc. Each submitted department has a unique independent code, and supports dynamic and continuous expansion by staff.

[0018] In a possible implementation, step S12 includes: Step S121: Establish a regularized expression for the standardized original government document acceptance path, and judge the standardized original government document acceptance path based on the regularized expression.

[0019] In this embodiment, a regular expression of a common path pattern is established to cover typical expressions of the submitted department name.

[0020] Step S121-1: If a non-standardized original government document acceptance path appears, the path segment of the suspected department name is extracted from the non-standardized original government document acceptance path based on the regular expression, and the path segment of the suspected department name is semantically cleaned. After the semantic cleaning is completed, the confidence score with the standard coding library is calculated.

[0021] In this embodiment, typical non-standard scenarios are selected for illustration. For example, in the non-standardized original government document acceptance path, there are problems such as spelling mistakes and redundant characters. Spelling mistakes include "Xzheng Bureau" with chaotic letters and "X証 Bureau" with variant characters. Redundant characters include "X Political Bureau_Temporary" and "X Political Bureau (Archiving)". Based on regular expressions, the path segments of suspected department names with the above-mentioned spelling mistakes and redundant characters are extracted, and semantic cleaning is performed on the path segments of suspected department names. Semantic cleaning includes character normalization, pinyin correction, synonym replacement, etc. For example, for the spelling mistake "Xzheng Bureau", the first pinyin letters XZ are extracted and mapped to "X Political Bureau". For the redundant field of "X Political Bureau_Temporary", it is removed. After semantic cleaning, the edit distance between the cleaned name and the standard name is calculated. The edit distance represents the total number of operation steps for semantic cleaning of the path segment of the suspected department name. For example, the edit distance from "Xzheng Bureau" to "X Political Bureau" is 2. Based on the edit distance and the maximum edit distance, the confidence score is calculated.

[0022] Step S121-1-1, if the confidence score ≥ 80%, then execute Step S13.

[0023] In this embodiment, if the confidence score of the suspected department name "Xzheng Bureau" in the non-standardized original government document acceptance path is 83%, it indicates that the error is small, and the original department can be marked and proceed to the next operation.

[0024] Step S121-1-②, if the confidence score < 80%, then mark the original government document as defective, determine that the original government document of this acceptance path is a defective government document, and move the defective government document to the manual processing area.

[0025] In this embodiment, if the confidence score obtained from the non-standardized original government document acceptance path is 43%, it indicates that the error is large. Based on the error inference of the non-standardized original government document acceptance path, it is inferred that the attitude of the staff writing this document is relatively casual, and there may be more errors in this document. Therefore, it is marked as defective and transferred to the manual processing area to prevent this document from entering the subsequent steps and occupying computing power.

[0026] Step S121-2, if there is no non-standardized original government document acceptance path, then perform a unique coding match on the standardized original government document acceptance path and execute Step S13.

[0027] Step S13, obtain the original department mark corresponding to the original government document based on the unique coding, and match the original government document with each submitted department.

[0028] In a possible implementation manner, Step S1 further includes: Step S1-1, count the number of documents submitted in each batch based on the original department tag. Each batch of original government documents needs to execute steps S1 to S7, and then receive the next batch of original government documents. The upper limit of original government documents submitted by each department in each batch is 100.

[0029] In this embodiment, it is assumed that a municipal government platform needs to process documents submitted by various departments in batches to control the server load. Specifically, if the number of submissions in a single batch by a department exceeds the limit, such as XX Bureau uploads 105 documents at one time, it will be automatically split into the first batch of 100 for immediate processing, and the remaining 5 will be temporarily queued waiting for the next batch. After the next batch, the next batch will be released to enter the processing flow, forming a rolling closed-loop management of "receive-process-release", ensuring stable server throughput in high-concurrency scenarios and preventing server overload.

[0030] Step S1-2: Establish a historical document database for storing original government documents marked by their original departments, and calculate the daily / weekly / monthly change rate of original government documents submitted by various departments based on the original department marks.

[0031] Step S1-3: Create a document submission heat map based on the daily / weekly / monthly change rate. The document submission heat map represents the document submission activity of each department.

[0032] In this embodiment, it is assumed that a provincial government platform establishes a historical document database by integrating the document data submitted by various departments in the past year, which stores government documents marked with the original departments, such as the "****** Draft" of the XX Bureau, the "****** Report" of the XX Bureau, and the "****** Statistics Table" of the XX Committee, covering policy documents, etc. The month-on-month change rate of the department document submission volume is calculated based on the time dimension. For example, the weekly submission volume of the XX Bureau increased from 230 to 265, and the weekly submission volume increased by 15% month-on-month. The monthly submission volume of the XX Bureau decreased from 480 to 442, and the monthly submission volume decreased by 8% month-on-month. The abnormal fluctuation time is also recorded, such as the XX Bureau's concentrated release of policies during the school season. This resulted in a 180% surge in daily submissions. A dynamic document submission heat map was generated based on the above month-on-month data. The horizontal axis is the timeline (day / week / month) and the vertical axis is the department list. The document submission trends of each department are intuitively displayed using light and dark colors, with red indicating high activity and blue indicating low activity. For example, the "******" type documents of the XX Bureau show a dark red peak at the beginning of each month, with an average of 40 documents per day, while "******" type documents are submitted in batches at the end of the year. At the same time, the heat map marks abnormal hot spots. For example, due to a sudden road construction notice, the weekly submission volume of the XX Bureau soared to 90, triggering an orange warning mark. The document submission heat map can provide visual decision support for resource scheduling and efficiency analysis.

[0033] Step S2: Obtain a document information set of each original government document, and perform an evaluation on the original government document based on the document information set.

[0034] In this embodiment, step S2 includes: Step S21, obtain the historical submission document information set of each submitting department in the past 12 months, match the historical submission document information set with the original department label, obtain the historical information set mean and historical information set standard deviation of each submitting department based on the historical submission document information set, obtain the one-time evaluation dynamic threshold of each submitting department based on the historical information set mean and historical information set standard deviation, and the one-time evaluation dynamic threshold is dynamically updated every quarter.

[0035] In this embodiment, the mean of the historical information set is , let the standard deviation of the historical information set be , a dynamic threshold for evaluation can be expressed as The use of a one-time dynamic threshold can distinguish between normal fluctuations and significant anomalies, reducing the misjudgment rate of documents, and the one-time dynamic threshold of each department can be calculated independently to avoid a one-size-fits-all situation.

[0036] Step S22: Obtain the first analysis data, second analysis data, and third analysis data of the original government document. The first analysis data represents all editable text in the original government document, the second analysis data represents the bitmap and vector map embedded in the original government document, and the third analysis data represents the actual number of pages in the original government document based on printing page breaks.

[0037] Specifically, the first analysis data represents all editable text in the original government document, such as all editable text in the document body, attachments, and notes. When obtaining the first analysis data, based on the document structure tree, all text nodes are traversed and the number of Unicode characters, including spaces and punctuation, is counted.

[0038] Step S23: Based on the first analysis data, the second analysis data and the third analysis data, a document information set of the original government document is formed, the document information set is matched with the original department tag, and a comprehensive size factor is calculated based on the document information set to obtain the comprehensive size factor.

[0039] Furthermore, before calculating the comprehensive factor, the text density is obtained based on the first and third analysis data. The text density calculation formula is: [number of characters / number of pages], which reflects the information carrying capacity of each page. The average DPI of each image is obtained based on the second analysis data, and the weight coefficient of each image is set based on the average DPI, such as 300 DPI = 1.0 or 150 DPI = 0.5. The comprehensive size factor is calculated based on the third analysis data, text density, and weight coefficient. The calculation formula is: ; The constant is determined by the size of each image, such as the full-size image coefficient = 1.0 and the small illustration coefficient = 0.3.

[0040] Step S24: Perform an assessment based on the comprehensive size factor and the dynamic threshold of the assessment corresponding to the submitting department.

[0041] In step S24-1, if the comprehensive size factor is within the dynamic threshold range of the one-time assessment of the corresponding submitting department, step S3 is executed.

[0042] Step S24-2: If the comprehensive size factor is outside the dynamic threshold range of the corresponding submitting department's one-time assessment, the original government document corresponding to the comprehensive size factor is marked as defective, identified as a defective government document, and moved to the manual processing area.

[0043] Step S3: obtaining an analysis data matrix based on the original government document that has undergone the first evaluation, performing a second evaluation on the original government document based on the analysis data matrix, and performing a correction operation on the original government document based on the second evaluation.

[0044] In this embodiment, step S3 includes: Step S31 , extracting an analysis data matrix of the original government document that has undergone an evaluation, wherein the analysis data matrix includes text layer data, image layer data, page layer data, and format layer data.

[0045] Step S32, detect the text layer data to obtain the number of text layer errors, detect the picture layer data to obtain the number of picture layer errors, detect the page layer data to obtain the number of page layer errors, and detect the format layer data to obtain the number of format layer errors.

[0046] Specifically, for error detection at the text level, typos at the text level are identified based on a pre-trained language model, and the cosine similarity of text segments is calculated. If the similarity between two consecutive segments is greater than 95%, they are determined to be duplicate paragraphs. For violations of sensitive words, the government sensitive word library is matched, and sensitive word errors in the text level are detected based on the government sensitive word library.

[0047] Specifically, for error detection at the image layer, images with a DPI of less than 300 are identified and judged as blurred images. Key images such as tables and official seals are prioritized for detection. At the same time, the variance of the image gradient histogram is calculated, and images with a variance of less than 50 are judged as blurred images. For watermark errors, the detection mode is the same as that of the image. In addition to the same parts, detection of overlapping watermarks and watermark priorities is added. Error detection at the image layer can be based on edge detection algorithms, such as the Canny operator.

[0048] Specifically, for the error detection of the page number layer, analyze the text density of the page. For pages where the text proportion < 1% and there are no pictures, verify the consistency between the physical page number and the directory annotation. For example, if the directory annotates page 5 as page 5 but it is actually page 6.

[0049] Specifically, for the error detection of the format layer, verify the nesting order of the title levels "I. → (I) → 1. → (1)", and cross-level usage is prohibited. For example, skipping "(I)" and directly using "1.", detect whether the body text font is the standard font, such as whether the body text font is FangSong - GB2312, and whether the font size is the standard font size. For example, if the font size is三号, measure whether the line spacing and page margin are the standard line spacing and standard page margin.

[0050] Step S33, establish an error weight assignment model. The error weight assignment model is expressed as assigning a weight of 0.1 to text layer errors, a weight of 0.2 to picture layer errors, a weight of 0.3 to page number layer errors, and a weight of 0.4 to format layer errors, and introduce the error tolerance factors of each department.

[0051] In this embodiment, the error tolerance factor is expressed as the different document error tolerances of each department. For example, XX Bureau allows a small number of format defects, while the documents of XX Bureau need to be absolutely rigorous.

[0052] Step S34, obtain the error value of the original government document based on the error weight assignment model and the error tolerance factor, and conduct a secondary evaluation of the original government documents submitted by each department based on the error value.

[0053] In this embodiment, assume that the following errors are detected in the document submitted by a certain submitting department: 1 format layer error and 2 page number layer errors. Then the error value is calculated as [(1×0.4 + 2×0.3)×p], where p represents the error tolerance factor corresponding to this submitting department, and the final calculated error value is p.

[0054] Step S34-1, if the error value ≥ the preset department error threshold, then mark the original body text document corresponding to this error value as defective, identify it as a defective government document, and move the defective government document to the manual processing area.

[0055] Step S34-2, if the error value < the preset department error threshold, then perform a correction operation on the original body text document corresponding to this error value.

[0056] Specifically, the correction operation includes: For text-level errors, error types include typos, repeated paragraphs, and sensitive words. The text-level correction operation is expressed as follows: obtaining a dictionary of professional terms and establishing a text error mapping table, confirming the error type based on the contextual semantic matching of the text-level errors, replacing typos based on the text error mapping table, deleting repeated paragraphs based on the contextual semantic matching, and replacing sensitive words with symbols based on the contextual semantic matching.

[0057] For example, if "whisper" appears in the text layer, but "whisper" is a typo, "whisper" is replaced with "low X" based on the text error mapping table; or if a personal ID number appears in the text layer, which is personal privacy and a sensitive word, the sensitive word is replaced based on the symbol, and the replacement process can be expressed as "12345678" → "********".

[0058] For image-level errors, error types include image blur and watermark conflict. The image-level correction operation is represented as follows: reconstructing the blurred image based on the enhanced super-resolution generative adversarial network, focusing on optimizing the text area, correcting watermark conflicts based on the watermark priority rule, and deleting duplicate watermarks.

[0059] Specifically, for general document images of 300-600dpi, the dpi is increased to 600dpi. For severely blurred images less than 150dpi, they are sharpened first and then the resolution is increased. After reconstruction, the peak signal-to-noise ratio of the image needs to be calculated to ensure that the peak signal-to-noise ratio is greater than or equal to 30db.

[0060] For page level errors, the error types include blank pages and disordered page numbers. The page level correction operations are as follows: delete blank pages and adjust page numbers.

[0061] For format-level errors, the error types include title-level errors, font specification errors, and line spacing and margin errors. The format-level correction operations are as follows: correct the title level based on the preset administrative document format and ensure the continuity of the title numbering, forcibly replace the text font and font size based on the preset administrative document format, and correct the line spacing and margins based on the preset administrative document format.

[0062] For example, if "1." is mistakenly used as a secondary title, it should be replaced with "(I)". After the replacement, ensure the continuity of the title numbering.

[0063] The corrected original government document is re-imported into the secondary evaluation operation. If the correction operation is performed ≥3 times, the original text document will be marked as defective and identified as a defective government document. The defective government document will be moved to the manual processing area.

[0064] In step S34-3, if the error value is 0, it means that the original text document corresponding to the error value has completed the correction operation, and the original text document is imported into step S5.

[0065] Step S4: Based on steps S1, S2 and S3, defective government documents are obtained and the defective government documents are uniformly moved to the manual processing area.

[0066] Specifically, defective government documents are moved to a manual processing area, where they are manually repaired or returned. Manually corrected documents can be returned to the processing flow.

[0067] Step S5: extracting government keywords from the original government document that has completed the correction operation, clustering based on the government keywords to obtain an original clustering data set, performing clustering evaluation on the original clustering data set and generating an optimized clustering data set.

[0068] In this embodiment, step S5 includes: Step S51 , calculating the TF-IDF value of each word in each original government document, screening government keywords in each original government document based on the TF-IDF value, and extracting 10-15 government keywords from each original government document.

[0069] In this embodiment, the TF value, i.e., the frequency of a word appearing in the entire document, and the IDF value, i.e., the inverse of the frequency of a word appearing in all documents, are used to reduce the influence of common words. Common words can be eliminated based on the IDF value. The TF-IDF value is calculated by combining the TF value and the IDF value. 10-15 government keywords are extracted from each document. For example, in a document titled "XXXXX Upgrade Plan," the high-frequency words are campus safety (TF=0.12), monitoring equipment (TF=0.1), and fire drill (TF=0.06). The low-IDF common words are notice (TF=0.15, IDF=0.02) and reinforcement (TF=0.1, IDF=0.03). The common words are filtered out, and the government keywords [campus safety, monitoring equipment, fire drill] are extracted. , or in a document named "XXXXX Coverage Statistics", the high-frequency words are vaccination (TF=0.18), coverage (TF=0.09), and parent notification (TF=0.07), and the low-IDF common words are notification (TF=0.13, IDF=0.01), and issuance (TF=0.16, IDF=0.04). The common words are filtered, or in a document named "XX Student XX Subsidy Distribution Details", the high-frequency words are XX subsidy (TF=0.14), XX student (TF=0.11), and application process (TF=0.07), and the low-IDF common words are issuance (TF=0.12, IDF=0.01), and formulation (TF=0.13, IDF=0.03). In addition, the rest of the documents also include campus transportation keywords.

[0070] Step S52: converting the government keywords extracted from each original government document into keyword vectors, calculating keyword similarity based on the keyword vectors, and performing initial clustering on the government keywords based on the keyword similarity to generate an original clustering data set.

[0071] In this embodiment, the three documents mentioned above are used as examples. The generated keyword vectors are: campus safety vector [0.8, 0.2, -0.1], vaccination vector [-0.3, 0.7, 0.5], and XX subsidy vector [0.1, -0.6, 0.9]. Clusters are generated based on keyword similarity, and campus safety and campus transportation are merged based on their high keyword similarity. Based on clustering, cluster C1 (campus) is generated, and the rest are generated as cluster C2 (vaccine) and cluster C3 (subsidy).

[0072] Step S53 , calculating the keyword concentration in each cluster in the original clustering data set, performing clustering evaluation on the original clustering data set based on the keyword concentration, and performing secondary optimization on the original clustering data set.

[0073] Step S54: merge adjacent clusters with keyword concentrations less than 70% in the original clustering data set, split clusters with conflicting government keyword content in the original clustering data set, and generate an optimized clustering data set.

[0074] For example, suppose the keywords in cluster C1 are [campus safety, monitoring equipment, fire drill, traffic diversion, traffic police collaboration], the number of occurrences of the first five words are campus safety (2 times), monitoring equipment (1 time), fire drill (2 times), traffic diversion (3 times), and traffic police collaboration (3 times), with a total word frequency of 6 times. The concentration is 5 / 11 = 45.5%. The concentration is less than 70%, so it needs to be split. After the split is completed, K-Means sub-clustering is performed on cluster C1, dividing it into sub-clusters C1a and sub-cluster C1b, and the concentration is calculated again until the concentration of all clusters is greater than 70%.

[0075] Step S6: Perform an ant colony assignment operation based on the optimized clustering data set to assign the original government documents to corresponding departments.

[0076] In this embodiment, step S6 includes: Step S61 : pre-set dataset matching weights according to different responsibilities of departments, obtain path costs based on the dataset matching weights and the optimized clustering dataset, and initialize the ant colony pheromone.

[0077] For example, the clusters in the optimized clustering data set are "campus", "vaccine", "education", and "transportation". The weights of these four clusters for the XX Bureau can be set to [0.9, 0.1, 0.8, 0.2] respectively.

[0078] Furthermore, the path cost calculation formula is path cost = (1 - dataset matching weight) + correlation coefficient + (1 - historical efficiency score), where the historical efficiency score starts at 0 and continues to increase when the ant colony simulates the distribution of original government documents. The correlation coefficient represents the degree of correlation between different clusters in the optimized clustering dataset and the content in the document to be distributed.

[0079] In step S62, the ant colony simulates the path of dispatching the original government documents to different departments based on the optimized clustering data set; during the simulation process, the path selection value is obtained based on the path cost and the ant colony pheromone, and the ant colony pheromone of the correct dispatch path is continuously improved. The correct dispatch path is represented as the path with the highest path selection value.

[0080] In step S63, when the ant colony selects the same path for three consecutive rounds, it is determined to be the optimal path, and the original government documents are assigned to departments based on this path.

[0081] In this embodiment, taking 10 ants in an ant colony as an example, the 10 ants independently explore the dispatch path and select the path based on the path selection value. The path selection value calculation formula is: [pheromone × path cost²]. If the dispatch is successful, the pheromone is increased by 2. If the dispatch fails, the pheromone is reduced by 0.5. During the dispatch process, if 8 of the ants choose XX bureau and the other 2 choose XX bureau, the pheromone of the path is increased by 2. Then, if XX bureau is chosen for three consecutive rounds, it is determined that dispatching the document to XX bureau is the optimal path, and the document is dispatched to XX bureau. The optimal path of the document is memorized. If there is a document with a similar name to the document, it is preferentially simulated based on the optimal path of the document, thereby improving the speed and accuracy of dispatching each document.

[0082] Step S7, obtaining the defective rate of department-submitted documents based on the defective government documents and the assigned original government documents.

[0083] In this embodiment, by counting the document defect rate of each department, quality control and responsibility traceability are achieved, and departments with high-frequency problems are quickly located. For example, the XX Bureau has exceeded the defect rate due to format errors and promotes root cause rectification. This can effectively reduce the error rate of documents, thereby further improving the efficiency of document processing and reducing the cost of manual review.

[0084] Figure 3 A schematic diagram of a document information extraction and distribution device provided by some embodiments of the present application that can implement the ideas of the present application is shown.

[0085] A document information extraction and dispatching device can be implemented using a general-purpose server or a special-purpose server, both of which can be used to implement the document information extraction and dispatching method of the present application. Although only one server is shown in this application, for convenience, the functions described in this application can be implemented in a distributed manner on multiple similar platforms to balance the processing load.

[0086] For example, a document information extraction and dispatching device may include a network port 100 connected to a network, one or more processors 102 for executing program instructions, a communication bus 104, and storage media 103 in different forms, such as a disk, ROM, or RAM, or any combination thereof. Exemplarily, a document information extraction and dispatching device may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The method of the present application can be implemented according to the above-mentioned program instructions. A document information extraction and dispatching device also includes an input / output (I / O) interface 101 between a computer and other input / output devices.

[0087] For ease of explanation, only one processor is described in a document information extraction and dispatching device. However, it should be noted that a document information extraction and dispatching device in the present application may also include multiple processors, so the steps performed by a processor described in the present application may also be performed jointly or individually by multiple processors. For example, if a processor of a document information extraction and dispatching device performs step A and step B, it should be understood that step A and step B may also be performed jointly by two different processors or individually in one processor. For example, the first processor performs step A, the second processor performs step B, or the first processor and the second processor perform steps A and B together.

[0088] In addition, an embodiment of the present invention further provides a readable storage medium, in which computer-executable instructions are preset. When a processor executes the computer-executable instructions, the above document information extraction and assignment method is implemented.

[0089] The specific usage and function of this embodiment are described below: First, the original government documents are obtained, and the original department is marked for each original government document. Then, the document information set of each original government document is obtained, and the original government document is evaluated based on the document information set. An analysis data matrix is obtained based on the original government document that has undergone the first evaluation. A second evaluation is performed on the original government document based on the analysis data matrix. The original government document is corrected based on the second evaluation. Then, defective government documents are obtained based on steps S1, S2, and S3, and the defective government documents are uniformly moved to a manual waiting area. The defective government documents in the manual waiting area are uniformly repaired or returned by humans. Then, government keywords are extracted from the original government documents that have completed the correction operation, and clustering is performed based on the government keywords to obtain an optimized clustering data set. An ant colony dispatching operation is performed based on the optimized clustering data set to dispatch the original government documents to the corresponding departments, thereby improving the accuracy of document dispatching. In addition, this method can correct errors in government documents during the execution of the dispatching process. In addition, the method can also obtain the defective rate of document submissions by departments based on the defective government documents and the dispatched original government documents, thereby quantitatively reflecting the quality of document submissions from various departments, which is used to promote the improvement of the quality of government documents from various departments.

[0090] It should be understood that in the embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0091] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A document information extraction and distribution method, characterized in that: The method comprises: S1: Obtain original government documents and mark each original government document with the original department; S2: Obtain the document information set of each original government document and perform an evaluation on the original government document based on the document information set; S3: Obtain an analysis data matrix based on the original government document that has undergone the first evaluation, conduct a second evaluation on the original government document based on the analysis data matrix, and perform correction operations on the original government document based on the second evaluation; S4: Based on steps S1, S2, and S3, defective government documents are obtained and uniformly moved to a manual processing area; S5: extracting government keywords from the original government document that has undergone the correction operation, clustering based on the government keywords to obtain an original clustering dataset, performing clustering evaluation on the original clustering dataset, and generating an optimized clustering dataset; S6: Perform ant colony assignment based on the optimized clustering dataset to assign the original government documents to the corresponding departments; S7: The defective rate of documents submitted by the acquisition department based on defective government documents and the assigned original government documents.

2. A document information extraction and distribution method according to claim 1, characterized in that: The method of obtaining an analysis data matrix based on the original government document that has undergone the primary evaluation, performing a secondary evaluation on the original government document based on the analysis data matrix, and performing a correction operation on the original government document based on the secondary evaluation includes: Extracting an analysis data matrix of an original government document that has undergone an evaluation, wherein the analysis data matrix includes text layer data, image layer data, page layer data, and format layer data; Detect text layer data to obtain the number of text layer errors, detect image layer data to obtain the number of image layer errors, detect page layer data to obtain the number of page layer errors, detect format layer data to obtain the number of format layer errors; Establish an error weight distribution model, where the error weight distribution model assigns a weight of 0.1 to text-level errors, 0.2 to image-level errors, 0.3 to page-level errors, and 0.4 to format-level errors, and introduces an error tolerance factor for each department; Based on the error weight distribution model and error tolerance factor, the error value of the original government documents is obtained. Based on the error value, a secondary evaluation is performed on the original government documents submitted by each department: If the error value is greater than or equal to the preset department error threshold, the original text document corresponding to the error value will be marked as defective, identified as a defective government document, and moved to the manual processing area; If the error value is less than the preset department error threshold, the original text document corresponding to the error value is corrected; If the error value is 0, it means that the original text document corresponding to the error value has completed the correction operation, and the original text document is imported into step S5.

3. A document information extraction and distribution method according to claim 2, characterized in that: The correction operation includes: For text-level errors, error types include typos, repeated paragraphs, and sensitive words. The text-level correction operation is as follows: obtaining a professional terminology dictionary and establishing a text error mapping table; confirming the error type based on the contextual semantic matching of the text-level error; replacing typos based on the text error mapping table; deleting repeated paragraphs based on contextual semantic matching; and replacing sensitive words with symbols based on contextual semantic matching. For image-level errors, error types include image blur and watermark conflict. The image-level correction operation is as follows: reconstruct the blurred image based on the enhanced super-resolution generative adversarial network, focus on optimizing the text area, correct watermark conflicts based on the watermark priority rule, and delete duplicate watermarks. For page level errors, the error types include blank pages and disordered page numbers. The page level correction operations are as follows: delete blank pages and adjust page numbers; For format-level errors, the error types include title level errors, font standard errors, and line spacing and margin errors. The format-level correction operations are as follows: based on the preset administrative document format, correct the title level and ensure the continuity of the title numbering; based on the preset administrative document format, forcibly replace the text font and font size; based on the preset administrative document format, correct the line spacing and margins; Re-import the revised original government documents into the secondary evaluation operation; If the correction operation is performed ≥3 times, the original text document will be marked as defective and identified as a defective government document, and the defective government document will be moved to the manual processing area.

4. A document information extraction and distribution method according to claim 1, characterized in that: The extracting of government keywords from the original government document that has completed the correction operation, clustering based on the government keywords, obtaining an original clustering data set, performing clustering evaluation on the original clustering data set and generating an optimized clustering data set includes: Calculate the TF-IDF value of each word in each original government document, filter government keywords in each original government document based on the TF-IDF value, and extract 10-15 government keywords from each original government document; The government keywords extracted from each original government document are converted into keyword vectors, keyword similarity is calculated based on the keyword vectors, and the government keywords are initially clustered based on the keyword similarity to generate an original clustering dataset; Calculate the keyword concentration in each cluster in the original clustering data set, perform clustering evaluation on the original clustering data set based on the keyword concentration, and perform secondary optimization on the original clustering data set; Adjacent clusters with keyword concentration less than 70% in the original clustering data set are merged, and clusters with conflicting government keywords in the original clustering data set are split to generate an optimized clustering data set.

5. A document information extraction and distribution method according to claim 1, characterized in that: The ant colony assignment operation based on the optimized clustering data set to assign the original government documents to the corresponding government departments includes: Preset dataset matching weights based on different departmental responsibilities, and initialize ant colony pheromones based on the dataset matching weights and the path cost obtained from the optimized clustering dataset. The ant colony simulates the paths that distribute original government documents to different departments based on an optimized clustering dataset. During the simulation, the ant colony obtains a path selection value based on the path cost and the ant colony pheromone, and continuously improves the ant colony pheromone of the correctly distributed path. The correctly distributed path is represented as the path with the highest path selection value. When the ant colony selects the same path for three consecutive rounds, it is determined to be the optimal path, and the original government documents are assigned to departments based on this path.

6. A document information extraction and distribution method according to claim 1, characterized in that: The step of obtaining a document information set of each original government document and evaluating the original government document based on the document information set includes: Obtain the historical submission document information set of each submitting department in the past 12 months, match the historical submission document information set with the original department tag, obtain the historical information set mean and historical information set standard deviation of each submitting department based on the historical submission document information set, obtain the primary assessment dynamic threshold of each submitting department based on the historical information set mean and historical information set standard deviation, and dynamically update the primary assessment dynamic threshold every quarter; Obtaining first analysis data, second analysis data, and third analysis data of the original government document, where the first analysis data represents all editable text in the original government document, the second analysis data represents bitmaps and vector graphics embedded in the original government document, and the third analysis data represents the actual number of pages in the original government document based on printed page breaks; A document information set of the original government document is formed based on the first analysis data, the second analysis data, and the third analysis data, the document information set is matched with the original department tag, and a comprehensive size factor is calculated based on the document information set to obtain the comprehensive size factor; An assessment is performed based on the comprehensive size factor and the dynamic threshold of the corresponding submission department: If the comprehensive size factor is within the dynamic threshold range of the one-time assessment of the corresponding submitting department, then execute step S3; If the comprehensive size factor is outside the dynamic threshold range of the corresponding submitting department's one-time assessment, the original government document corresponding to the comprehensive size factor will be marked as defective, identified as a defective government document, and moved to the manual processing area.

7. A document information extraction and distribution method according to claim 1, characterized in that: The obtaining of original government documents and marking each original government document with the original department mark include: Receive original government documents based on a unified file transfer interface protocol and standardize the path for receiving original government documents using a path hierarchy structure; The path hierarchy structure is represented by using a first level, a second level, a third level, and a fourth level to standardize the original government document acceptance path, wherein the first level is represented by the department name, the second level is represented by the original government document time, the third level is represented by the original government document type, and the fourth level is represented by the original government document name. The original government document acceptance path is standardized and sorted level by level based on the order of the first level, the second level, the third level, and the fourth level to obtain a standardized original government document acceptance path; Pre-set departmental standard coding libraries and establish a fault-tolerant mechanism to match the unique coding of the acceptance path of standardized original government documents based on the departmental standard coding libraries and the fault-tolerant mechanism; The original department tag corresponding to the original government document is obtained based on the unique code, so that the original government document is matched with the submitted departments.

8. A document information extraction and distribution method according to claim 7, characterized in that: The matching of the unique coding of the original government document acceptance path based on the department standard coding library and fault tolerance mechanism includes: Establish a regularized expression for the acceptance path of standardized original government documents, and judge the acceptance path of standardized original government documents based on the regularized expression. If a non-standardized original government document acceptance path appears, extract the path segment suspected to be the department name from the non-standardized original government document acceptance path based on the regularized expression, perform semantic cleaning on the path segment suspected to be the department name, and after the semantic cleaning is completed, calculate the confidence score with the standard coding library; If the confidence score is ≥80%, the original government document is automatically associated with the matching unique code, and the original department tag corresponding to the original government document is obtained; If the confidence score is less than 80%, the original government document will be marked as defective, and the original government document of the acceptance path will be identified as a defective government document, and the defective government document will be moved to the manual processing area.

9. A document information extraction and distribution method according to claim 7, characterized in that: The method further comprises: Count the number of documents submitted in each batch based on the original department tag. Each batch of original government documents needs to go through steps S1 to S7. After the end, the next batch of original government documents will be received. The upper limit of the number of original government documents submitted by each department in each batch is 100. Establishing a historical document database for storing original government documents marked by their original departments, and calculating the daily / weekly / monthly change rate of original government documents submitted by various departments based on the original department marks; A document submission heat map is created based on the daily / weekly / monthly change rate, and the document submission heat map represents the document submission activity of each department.

10. A document information extraction and distribution device, characterized in that: The document information extraction and dispatching device includes a processor and a memory, the memory is connected to the processor, the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to implement the document information extraction and dispatching method described in any one of claims 1 to 9.

Images