Scanned document conversion method and system integrated with secret detection

Abstract

The application discloses a scanning version file conversion method and system integrated secret-related detection and belongs to the technical field of digital management, and aims to solve the technical problem of how to realize batch processing of scanning version archives, improve file conversion speed and meet the time requirement of the archive management department on large-scale cabinet type conversion. The method comprises the following steps: traversing all relevant scanning version files in the source file path and the subdirectory specified by a user, loading configured secret-related keywords and matching rules; calling C# to perform file disassembly and recognizing the files through OCR recognition technology to label the files with corresponding secret-related level tags; generating OFD files based on the picture layout data and the extracted text data of the disassembled files through memory stream technology; importing the OFD files and corresponding metadata into an archive library and storing key analysis data into the database of the archive management system; and manually checking file processing records, secret-related inspection results and OFD files.

Description

Technical Field

[0001] This invention relates to the field of digital management technology, specifically to a method and system for converting scanned documents with integrated classified information detection. Background Technology

[0002] Currently, most archives still rely primarily on scanned documents and do not widely utilize OCR technology, resulting in problems such as low retrieval efficiency, the need for manual verification of document confidentiality, and difficulties in integrating with document management systems.

[0003] TIF and PDF are commonly used file formats in the process of digitizing archives. However, OFD (Open Fixed-layout Document), as a national standard for layout file format independently developed in my country, is being increasingly used in the field of electronic archives management. Especially against the backdrop of the country's promotion of the standardization of archives digitization, archives and other institutions need to convert a large number of existing TIF and PDF format archives into OFD format to meet the latest industry standards and management requirements.

[0004] How to achieve batch processing of scanned archives, improve the speed of document conversion, and meet the time requirements of archive management departments for large-scale cabinet conversion is a technical problem that needs to be solved. Summary of the Invention

[0005] The technical objective of this invention is to address the above-mentioned shortcomings by providing a method and system for converting scanned documents with integrated classified information detection, thereby solving the technical problem of how to achieve batch processing of scanned archives, improve document conversion speed, and meet the time requirements of archive management departments for large-scale cabinet conversion.

[0006] In a first aspect, the present invention provides a method for converting scanned documents with integrated classified information detection, comprising the following operations: File import: Iterate through all relevant scanned files in the user-specified source file path and subdirectories, generate a file list, and if classified information detection is required, load the configured classified keywords and matching rules. The scanned files include TIF and PDF formats. Document Disassembly and Recognition: For scanned documents in the document list, C# is used to disassemble the documents. For the image layout data of the disassembled documents, text information is extracted and recognized by OCR technology to form searchable text data. If classified information detection is required, classified information checks are performed based on classified keywords and matching rules. Based on the classified information check results, the documents are labeled with the corresponding classified information level, and the classified information check process is recorded to form a classified information check log. Conversion and synthesis: Based on memory stream technology, the image layout data and extracted text data of the disassembled files are used to generate OFD files without IO operations. The generated OFD files are stored in the output path specified by the user according to the directory hierarchy of the source file path. If classified information detection is required, the classified information level label and classified information detection log are used as metadata, and the OFD files are associated with the metadata. Archiving and storage: Import OFD files and their corresponding metadata into the archive repository, extract key analysis data from the files, and store the key analysis data in the database of the archive management system to provide data support for subsequent retrieval and statistics; Verification and Correction: The process of importing, disassembling and identifying, converting and synthesizing, and archiving the files is recorded to generate file processing records. The file processing records, confidentiality check results, and OFD files are manually reviewed. If errors are found, they are manually corrected. The corrected file data is then re-entered into the conversion, synthesis, or archiving process.

[0007] As a preferred method, the file disassembly and identification includes the following operations: For scanned files in the file list, the underlying file disassembly capability of C# is called to disassemble the files in the memory stream, generating image layout data of the disassembled files. The image layout data includes image structure and text position. Analyze the image layout data of the disassembled file to extract the core information, which includes text and images. The extracted text information is recognized using OCR to form searchable text data. If classified information detection is required, the extracted text data is matched against preset classified keywords based on matching rules to generate classified keyword matching results. The classified keyword matching results include the frequency of keyword occurrence and the position of the text. Based on the frequency of occurrence of classified keywords, the classified level of the document is determined according to preset matching rules, and the document is tagged with the corresponding classified level. The classified information detection process is recorded to form a classified information detection log, and users can manually correct the classified information level of matching errors. The classified information levels include high, medium, low, and none. The classified information detection log includes the reason for classified information and the position of the text.

[0008] As a preferred method, multiple scanned files are processed concurrently using a multi-threaded approach during file disassembly and identification.

[0009] As a preferred approach, when using C# to disassemble files, a backend is built using JavaScript. This backend has stand-alone OCR recognition capabilities and is compatible with domestic platforms.

[0010] Secondly, the present invention provides a scanned document conversion system integrating classified information detection, comprising a file import module, a file disassembly and identification module, a conversion and synthesis module, an archiving and storage module, and an inspection and correction module; The file import module is used to perform the following: iterate through all relevant scanned files in the user-specified source file path and subdirectories, generate a file list, and if classified information detection is required, load the configured classified keywords and matching rules. The scanned files include TIF and PDF formats. The document disassembly and recognition module performs the following: For scanned documents in the document list, it calls C# to disassemble the documents; for the image layout data of the disassembled documents, it extracts text information and recognizes the extracted text information using OCR recognition technology to form searchable text data; if classified information detection is required, it performs classified information checks based on classified keywords and matching rules; based on the classified information check results, it labels the documents with the corresponding classified information level and records the classified information check process to form a classified information check log. The conversion and synthesis module is used to perform the following: Based on memory stream technology, it generates OFD files from the image layout data and extracted text data of the disassembled files. Without IO operations, it stores the generated OFD files to the output path specified by the user according to the directory hierarchy of the source file path. If classified information detection is required, it associates the classified information level label and classified information detection log as metadata with the OFD files. The archiving module is used to perform the following: import OFD files and their corresponding metadata into the archive repository, extract key analysis data from the files, and store the key analysis data in the database of the archive management system to provide data support for subsequent retrieval and statistics; The verification and correction module is used to perform the following: record the file import, file disassembly and identification, conversion and synthesis, and archiving processes to generate file processing records; manually check the file processing records, confidentiality check results, and OFD files; if errors are found, manually correct them; and re-enter the conversion and synthesis or archiving operations after correction.

[0011] Preferably, the file disassembly and identification module is used to perform the following operations: For scanned files in the file list, the underlying file disassembly capability of C# is called to disassemble the files in the memory stream, generating image layout data of the disassembled files. The image layout data includes image structure and text position. Analyze the image layout data of the disassembled file to extract the core information, which includes text and images. The extracted text information is recognized using OCR to form searchable text data. If classified information detection is required, the extracted text data is matched against preset classified keywords based on matching rules to generate classified keyword matching results. The classified keyword matching results include the frequency of keyword occurrence and the position of the text. Based on the frequency of occurrence of classified keywords, the classified level of the document is determined according to preset matching rules, and the document is tagged with the corresponding classified level. The classified information detection process is recorded to form a classified information detection log, and users can manually correct the classified information level of matching errors. The classified information levels include high, medium, low, and none. The classified information detection log includes the reason for classified information and the position of the text.

[0012] Preferably, the file disassembly and recognition module is used to process multiple scanned files concurrently using a multi-threaded approach.

[0013] As a preferred option, when calling C# for file decomposition, the file decomposition and recognition module is used to build a backend using JavaScript. The backend has stand-alone OCR recognition capabilities and is compatible with domestic platforms.

[0014] The scanned document conversion method and system with integrated classified information detection of the present invention have the following advantages: 1. Significantly improved file conversion speed, meeting the time requirements of archives management departments for large-scale format conversion, and added functions such as OCR recognition and classified information detection, facilitating later integration with archives systems and retrieval expansion; 2. Simply select the source file path and output path. If you need to perform classified information detection, configure the classified keywords. After execution, the entire process is automatic, and the final output result is provided without any additional human intervention. It is especially suitable for large-scale conversion scenarios. 3. By combining OCR capabilities, information can be extracted from documents, key locations can be analyzed for classified information detection, and related archives can be added to the database. A retrieval system can be built to improve the efficiency of archive management. 4. Supports local deployment, does not rely on the Internet environment, supports domestically produced equipment, is safe and reliable, and is suitable for government use. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] The invention will be further described below with reference to the accompanying drawings.

[0017] Figure 1This is a flowchart of a scanned document conversion method integrating classified information detection, as shown in Example 1. Detailed Implementation

[0018] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0019] This invention provides a method and system for converting scanned documents with integrated classified information detection, which solves the technical problem of how to achieve batch processing of scanned archives, improve the conversion speed, and meet the time requirements of archive management departments for large-scale cabinet conversion. Example

[0020] This invention provides a method for converting scanned documents with integrated classified information detection, comprising five steps: file import, file disassembly and identification, conversion and synthesis, archiving and storage, and inspection and correction.

[0021] Step S100 File Import: Traverse all relevant scanned files in the user-specified source file path and subdirectories to generate a file list. If classified information detection is required, load the configured classified keywords and matching rules. The scanned files include TIF and PDF formats.

[0022] Among them, classified keywords can be configured manually or imported through a JSON file, and the archives can customize the matching rules themselves.

[0023] Step S200: File Disassembly and Recognition. For scanned files in the file list, C# is used to disassemble the files. For the image layout data of the disassembled files, text information is extracted and recognized using OCR technology to form searchable text data. If classified information detection is required, classified information checks are performed based on classified keywords and matching rules. Based on the classified information check results, the files are tagged with the corresponding classified information level, and the classified information check process is recorded to form a classified information check log.

[0024] As a specific implementation of document disassembly and identification, this step includes the following operations: (1) For scanned files in the file list, the underlying file disassembly capability of C# is called to disassemble the files in the memory stream and generate image layout data of the disassembled files. The image layout data includes image structure and text position. When calling C# to disassemble the files, the backend is built using JavaScript. The backend has stand-alone OCR recognition capability and is compatible with domestic platforms. (2) Analyze the image layout data of the disassembled file and extract the core information in the file, which includes text and images; (3) The extracted text information is recognized by the OCR recognition method to form searchable text data; if classified information detection is required, the extracted text data is matched and compared with the preset classified keywords based on the matching rules to generate classified keyword matching results. The classified keyword matching results include the frequency of keyword occurrence and the position of text. Based on the frequency of occurrence of classified keywords, the classified level of the file is determined according to the preset matching rules, and the file is labeled with the corresponding classified level. The classified information detection process is recorded to form a classified information detection log, and users can manually correct the classified level of the matching error. The classified level includes high, medium, low and none. The classified information detection log includes the reason for classified information and the position of text.

[0025] In this embodiment, selecting the TIF or PDF source file directory will iterate through all files conforming to the TIF or PDF format within that directory (including subdirectories). A multi-threaded architecture allows for concurrent processing of multiple scanned files. Finally, the files are organized hierarchically by directory and stored in the output folder. For example, if the source file path is C: / / Test / / Test.pdf and the output directory is selected as drive D, the final output file path will be D: / / Test / / Test.ofd. This design greatly simplifies the user experience; only a few steps are required, and all steps will run automatically without much intervention. The classified information detection is based on OCR recognition technology, combined with classified keywords. When classified keywords are found in a document, the document will be tagged with a classification level (high / medium / low / none) based on their frequency, leaving a processing trail that allows archivists to view the specific reasons for classification and the text location. If a mismatch is found, it can still be manually corrected before being added back to the database.

[0026] Step S300 Conversion and Composition: Based on memory stream technology, the image layout data and extracted text data of the disassembled files are used to generate OFD files without IO operations. The generated OFD files are stored in the output path specified by the user according to the directory hierarchy of the source file path. If classified information detection is required, the classified information level label and classified information detection log are used as metadata, and the OFD files are associated with the metadata.

[0027] In this embodiment of the OFD synthesis process, a deep analysis function is used to directly decompose TIF and PDF files in the memory stream to analyze the image layout data, and OFD files are generated directly in the memory stream without IO operations, which greatly improves processing efficiency, shortens conversion time, and is more friendly to low-performance machines.

[0028] Step S400: Archive and store the OFD file and its corresponding metadata into the archive library. Extract the key analysis data from the file and store the key analysis data into the database of the archive management system to provide data support for subsequent retrieval and statistics.

[0029] Step S500 Verification and Correction: Record the file import, file disassembly and identification, conversion and synthesis, and archiving process to generate a file processing record. Manually review the file processing record, confidentiality check results, and OFD file. If errors are found, manually correct them. The corrected file data is then re-entered into the conversion and synthesis or archiving operation.

[0030] The method in this embodiment is applied between archives, electronic archives, and databases. The archives are electronic devices that store electronic archives, including TIF, PDF, OFD, and other formatted files.

[0031] Based on the processing procedure disclosed in this embodiment, TIF and PDF format archive files are received. The TIF and PDF file decomposition capabilities provided by C# are utilized, combined with OCR technology for intelligent document recognition. Classified keywords are used to determine whether the document is confidential, and finally, a classification tag is added. JavaScript is used to implement the backend functionality as middleware, calling C#'s underlying capabilities and providing an HTTP interface to the frontend.

[0032] Electronic archives: Domestically produced formatted files are two-layer OFD files exported after system processing and analysis, serving as a replacement for TIF and PDF electronic archives.

[0033] Database: The database of the document management system is used to archive and store analytical data of electronic documents, and can be used for subsequent function expansion.

[0034] Based on the method disclosed in this embodiment, a specific case is given. A certain archive system needs to convert a large number of TIF and PDF archives stored in the archive into OFD format files in batches, but has few staff and a short timeframe. The conversion work can be completed quickly based on the method disclosed in this embodiment. Example

[0035] This invention discloses a scanned document conversion system with integrated classified information detection, comprising a file import module, a file disassembly and identification module, a conversion and synthesis module, an archiving and storage module, and an inspection and correction module.

[0036] The file import module is used to perform the following: iterate through all relevant scanned files in the user-specified source file path and subdirectories, generate a file list, and if classified information detection is required, load the configured classified keywords and matching rules. The scanned files include TIF and PDF formats.

[0037] Among them, classified keywords can be configured manually or imported through a JSON file, and the archives can customize the matching rules themselves.

[0038] The document disassembly and recognition module performs the following: For scanned documents in the document list, it calls C# to disassemble the documents; for the image layout data of the disassembled documents, it extracts text information and recognizes the extracted text information using OCR recognition technology to form searchable text data; if classified information detection is required, it performs classified information checks based on classified keywords and matching rules; based on the classified information check results, it labels the documents with the corresponding classified information level and records the classified information check process to form a classified information check log.

[0039] As a specific implementation of the file disassembly and recognition module, this module is used to perform the following operations: (1) For scanned files in the file list, the underlying file disassembly capability of C# is called to disassemble the files in the memory stream and generate image layout data of the disassembled files. The image layout data includes image structure and text position. When calling C# to disassemble the files, the backend is built using JavaScript. The backend has stand-alone OCR recognition capability and is compatible with domestic platforms. (2) Analyze the image layout data of the disassembled file and extract the core information in the file, which includes text and images; (3) The extracted text information is recognized by the OCR recognition method to form searchable text data; if classified information detection is required, the extracted text data is matched and compared with the preset classified keywords based on the matching rules to generate classified keyword matching results. The classified keyword matching results include the frequency of keyword occurrence and the position of text. Based on the frequency of occurrence of classified keywords, the classified level of the file is determined according to the preset matching rules, and the file is labeled with the corresponding classified level. The classified information detection process is recorded to form a classified information detection log, and users can manually correct the classified level of the matching error. The classified level includes high, medium, low and none. The classified information detection log includes the reason for classified information and the position of text.

[0040] In this embodiment, selecting the TIF or PDF source file directory will iterate through all files conforming to the TIF or PDF format within that directory (including subdirectories). A multi-threaded architecture allows for concurrent processing of multiple scanned files. Finally, the files are organized hierarchically by directory and stored in the output folder. For example, if the source file path is C: / / Test / / Test.pdf and the output directory is selected as drive D, the final output file path will be D: / / Test / / Test.ofd. This design greatly simplifies the user experience; only a few steps are required, and all steps will run automatically without much intervention. The classified information detection is based on OCR recognition technology, combined with classified keywords. When classified keywords are found in a document, the document will be tagged with a classification level (high / medium / low / none) based on their frequency, leaving a processing trail that allows archivists to view the specific reasons for classification and the text location. If a mismatch is found, it can still be manually corrected before being added back to the database.

[0041] The conversion and synthesis module is used to perform the following: Based on memory stream technology, it generates OFD files from the image layout data and extracted text data of the disassembled files. Without IO operations, it stores the generated OFD files to the user-specified output path according to the directory hierarchy of the source file path. If classified information detection is required, it associates the classified information level label and classified information detection log as metadata with the OFD files.

[0042] In this embodiment of the OFD synthesis process, a deep analysis function is used to directly decompose TIF and PDF files in the memory stream to analyze the image layout data, and OFD files are generated directly in the memory stream without IO operations, which greatly improves processing efficiency, shortens conversion time, and is more friendly to low-performance machines.

[0043] The archiving module is used to perform the following: import OFD files and their corresponding metadata into the archive repository, extract key analysis data from the files, and store the key analysis data in the database of the archive management system to provide data support for subsequent retrieval and statistics.

[0044] The verification and correction module is used to perform the following: record the file import, file disassembly and identification, conversion and synthesis, and archiving processes to generate file processing records; manually check the file processing records, confidentiality check results, and OFD files; if errors are found, manually correct them; and re-enter the conversion and synthesis or archiving operations after correction.

[0045] The system in this embodiment can execute the method disclosed in Embodiment 1 to efficiently convert TIF or PDF files into OFD files.

[0046] The above provides a detailed description of the scanned document conversion method and system with integrated classified information detection provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A scanning version file conversion method integrated with classified information detection, characterized in that, This includes the following operations: File import: Iterate through all relevant scanned files in the user-specified source file path and subdirectories, generate a file list, and if classified information detection is required, load the configured classified keywords and matching rules. The scanned files include TIF and PDF formats. Document Disassembly and Recognition: For scanned documents in the document list, C# is used to disassemble the documents. For the image layout data of the disassembled documents, text information is extracted and recognized by OCR technology to form searchable text data. If classified information detection is required, classified information checks are performed based on classified keywords and matching rules. Based on the classified information check results, the documents are labeled with the corresponding classified information level, and the classified information check process is recorded to form a classified information check log. Conversion and synthesis: Based on memory stream technology, the image layout data and extracted text data of the disassembled files are used to generate OFD files without IO operations. The generated OFD files are stored in the output path specified by the user according to the directory hierarchy of the source file path. If classified information detection is required, the classified information level label and classified information detection log are used as metadata, and the OFD files are associated with the metadata. Archiving and storage: Import OFD files and their corresponding metadata into the archive repository, extract key analysis data from the files, and store the key analysis data in the database of the archive management system to provide data support for subsequent retrieval and statistics; Verification and Correction: The process of importing, disassembling and identifying, converting and synthesizing, and archiving the files is recorded to generate file processing records. The file processing records, confidentiality check results, and OFD files are manually reviewed. If errors are found, they are manually corrected. The corrected file data is then re-entered into the conversion, synthesis, or archiving process.

2. The scanning document conversion method integrated with classified information detection according to claim 1, characterized in that, The file disassembly and identification process includes the following steps: For scanned files in the file list, the underlying file disassembly capability of C# is called to disassemble the files in the memory stream, generating image layout data of the disassembled files. The image layout data includes image structure and text position. Analyze the image layout data of the disassembled file to extract the core information, which includes text and images. The extracted text information is recognized using OCR to form searchable text data. If classified information detection is required, the extracted text data is matched against preset classified keywords based on matching rules to generate classified keyword matching results. The classified keyword matching results include the frequency of keyword occurrence and the position of the text. Based on the frequency of occurrence of classified keywords, the classified level of the document is determined according to preset matching rules, and the document is tagged with the corresponding classified level. The classified information detection process is recorded to form a classified information detection log, and users can manually correct the classified information level of matching errors. The classified information levels include high, medium, low, and none. The classified information detection log includes the reason for classified information and the position of the text.

3. The method for converting scanned documents with integrated classified information detection according to claim 1, characterized in that, The file disassembly and identification process uses a multi-threaded approach to process multiple scanned files concurrently.

4. The method for converting scanned documents with integrated classified information detection according to claim 1, characterized in that, When using C# to disassemble files, a backend is built using JavaScript. The backend has stand-alone OCR recognition capabilities and is compatible with domestic platforms.

5. A scanned document conversion system integrating classified information detection, characterized in that, It includes a file import module, a file disassembly and identification module, a conversion and synthesis module, an archiving and storage module, and an inspection and correction module; The file import module is used to perform the following: iterate through all relevant scanned files in the user-specified source file path and subdirectories, generate a file list, and if classified information detection is required, load the configured classified keywords and matching rules. The scanned files include TIF and PDF formats. The document disassembly and recognition module performs the following: For scanned documents in the document list, it calls C# to disassemble the documents; for the image layout data of the disassembled documents, it extracts text information and recognizes the extracted text information using OCR recognition technology to form searchable text data; if classified information detection is required, it performs classified information checks based on classified keywords and matching rules; based on the classified information check results, it labels the documents with the corresponding classified information level and records the classified information check process to form a classified information check log. The conversion and synthesis module is used to perform the following: Based on memory stream technology, it generates OFD files from the image layout data and extracted text data of the disassembled files. Without IO operations, it stores the generated OFD files to the output path specified by the user according to the directory hierarchy of the source file path. If classified information detection is required, it associates the classified information level label and classified information detection log as metadata with the OFD files. The archiving module is used to perform the following: import OFD files and their corresponding metadata into the archive repository, extract key analysis data from the files, and store the key analysis data in the database of the archive management system to provide data support for subsequent retrieval and statistics; The verification and correction module is used to perform the following: record the file import, file disassembly and identification, conversion and synthesis, and archiving processes to generate file processing records; manually check the file processing records, confidentiality check results, and OFD files; if errors are found, manually correct them; and re-enter the conversion and synthesis or archiving operations after correction.

6. The scanned document conversion system with integrated classified information detection according to claim 5, characterized in that, The file disassembly and identification module is used to perform the following operations: For scanned files in the file list, the underlying file disassembly capability of C# is called to disassemble the files in the memory stream, generating image layout data of the disassembled files. The image layout data includes image structure and text position. Analyze the image layout data of the disassembled file to extract the core information, which includes text and images. The extracted text information is recognized using OCR to form searchable text data. If classified information detection is required, the extracted text data is matched against preset classified keywords based on matching rules to generate classified keyword matching results. The classified keyword matching results include the frequency of keyword occurrence and the position of the text. Based on the frequency of occurrence of classified keywords, the classified level of the document is determined according to preset matching rules, and the document is tagged with the corresponding classified level. The classified information detection process is recorded to form a classified information detection log, and users can manually correct the classified information level of matching errors. The classified information levels include high, medium, low, and none. The classified information detection log includes the reason for classified information and the position of the text.

7. The scanned document conversion system with integrated classified information detection according to claim 5, characterized in that, The file disassembly and recognition module is used to process multiple scanned files concurrently using a multi-threaded approach.

8. The scanned document conversion system with integrated classified information detection according to claim 5, characterized in that, When calling C# to perform file disassembly, the file disassembly and recognition module is used to build a backend using JavaScript. The backend has stand-alone OCR recognition capabilities and is compatible with domestic platforms.

Images