Electronic archive automatic archiving method based on custom archiving strategy

Abstract

The invention provides an electronic archive automatic archiving method based on a user-defined archiving strategy. The method comprises the following steps that S1, a to-be-archived electronic document is prepared; s2, configuring an archiving strategy; s3, executing an archiving task according to the archiving strategy; s4, checking the integrity of the to-be-classified tasks; s5, generating an archiving task report; and S6, ending the archiving task. According to the method, the files can be automatically sorted and managed, the files in a computer or a network can be scanned, and the files are classified and archived into specified folders according to preset strategy rules. The method has the advantages of saving time, improving efficiency, reducing errors, being customizable and the like.

Description

Technical Field

[0001] The present application relates to the technical field of electronic archive archiving and processing, and in particular to an automatic archiving method for electronic archives based on a custom archiving strategy. Background Art

[0002] With the rapid development of information technology, the demand for electronic document archiving is becoming increasingly urgent. With the digital transformation strategy proposed to promote industrial digitization and digital industrialization, a large number of electronic documents with archival value have been generated, and the demand for electronic document archiving management and application is urgent. However, most of the current electronic document archiving and electronic document archiving adopt manual sorting methods, which face the problems of low production efficiency, high document classification and archiving costs, and poor accuracy. Based on the current status of electronic archive sorting, an automated solution based on a custom archiving strategy is proposed to realize the application of machine automated archiving technology for various achievements in various industries. Summary of the invention

[0003] In order to solve the technical problems in the background technology, the embodiment of the present application provides an automatic archiving method for electronic archives based on a custom archiving strategy, comprising the following steps: Step S1: Preparation of electronic documents to be archived: Store electronic documents generated by various types of digital document scanning equipment or existing electronic documents in the file system of a designated computer installed with an archiving software system; Step S2: Configure archiving strategy: According to the specific archiving requirements of different digital achievements, corresponding archiving and sorting strategies are configured. One sorting strategy is configured with multiple archiving rules. The archiving strategy and the set of configured archiving rules are stored in the archiving software system database. The archiving strategy is maintained and updated through the strategy configuration function module in the archiving software system. When archiving is executed, the specified strategy scheme can be loaded from the database to execute the archiving work; Step S3: Execute the archiving task according to the archiving strategy: For the results to be archived, after configuring the corresponding archiving strategy according to the archiving requirements, the archived results are automatically archived and sorted. Two modes of automatic archiving are supported: Mode 1: Link with the automated scanner device, set the scanner's scan output path as its to-be-archived directory, dynamically monitor changes in the to-be-archived directory files, and sort and classify the files according to the configured strategy; Mode 2: Specify a static directory to be archived and sort and classify the files according to the configured archiving strategy. This mode is for reclassifying the scanned but unclassified electronic files. Step S4: Check the integrity of the archive task: Check whether the size of the source file directory to be archived is equal to the size of the target directory file to be archived, check whether the total number of source files to be archived is consistent with the number of files in the target directory file to be archived, count Tab_SouceFileIndex, calculate the file OCR recognition rate rr, calculate the number of archived files and the archiving success rate ar, obtain the number of duplicate files in the archived files and a detailed list; rr=(number of successful character recognition in character recognition feature areas / number of character recognition feature areas)*100%; ar=(number of successfully archived files / number of electronic files to be archived)*100%; For OCR recognition failures, we manually annotate each character one by one, and write the training data results incrementally into the recognition data set to improve the text recognition accuracy. Step S5: Generate an archiving task report: Generate an archiving task report, and automatically generate an archiving directory to display in the report display area in PDF format. At the same time, the PDF of the report with digital signature information is output to the directory above the archive output root directory. Step S6: Archiving task ends: After completing an automated organization based on the selected archiving strategy, the archived files are displayed in the archiving software system in the form of an archived electronic file directory tree.

[0004] In one of the embodiments of the present application, in the method for automated archiving of electronic archives based on a custom archiving strategy, in the step S2, configuring the archiving strategy, the attributes of the archiving strategy include a strategy name, a template file list, an archiving description, references or standards, an index directory creation option, and an archiving rule set; The attributes are visually configured through the archiving policy configuration function module and saved in the system database table Tab_policy and subtables Tab_filetemplate and Tab_IndexRules of the archiving software system; The template file list is used to select sample files of all categories of documents to be organized by setting this archiving strategy. The sample files are imported by opening the specified file directory, and their file information is stored in Tab_filetemplate. The option of creating index directory is used in situations where the volume directory needs to be automatically generated. It is an advanced option. The system does not create index directory by default. If you need to automatically generate the volume directory file after finishing, set an Excel template style for the volume directory, set the corresponding referenced template file name and other attribute change names in the cells in the template, and the system will automatically dynamically assign values ​​to generate the volume directory and its attributes. Each sub-item in the archiving rule set is an archiving rule, and each archiving rule corresponds to the archiving rule definition of a type of file.

[0005] In one embodiment of the present application, the method for automatically archiving electronic archives based on a custom archiving strategy, the step S2, configuring the archiving strategy, comprises: Step S2.1, create a new archiving strategy and name it according to its archiving task; Step S2.2, import various types of files to be archived, import the types of files to be archived into the current policy file template library, the system stores the imported files in the policy subfolder under the program running root directory, and according to the order of import, they are T1..Tx; Tx represents the file template file T1, T2...; at the same time, the main attributes of the imported file Tx are identified and stored in the data table Tab_filetemplates; its template file attributes include file identification name, file type, page pixel width, page pixel height, and number of pages; Step S2.3, configure the archiving rule set, formulate one or more rules (R1..Rx) according to the archiving requirements, so as to realize the automatic archiving and sorting of files. Usually, one rule corresponds to the archiving task of one type of file. When the same type of files need to be repeatedly stored in different target archiving directories, two or more archiving rules can be configured for one type of file; The attributes of the archiving rule include the rule name, file source directory, corresponding template file (one of T1..Tx), page pixel width, page pixel height, number of pages, identification name, file matching rule expression, archiving directory, archive file naming rules and file identification features.

[0006] In one embodiment of the present application, the method for automatic archiving of electronic archives based on a custom archiving strategy, step S3, executing the archiving task according to the archiving strategy, comprises: Step S3.1, the system starts, connects to the system standard database SYSDB, initializes global option parameters, initializes the interface, initializes the OCR engine, initializes the barcode and QR code recognition engine, loads the log message module, and completes the authorization and authentication of system users; Step S3.2, select the current archiving strategy. By default, the system loads the most recently used archiving strategy and displays it in the status bar. The archiving strategy can be switched as needed. When switching, it will pop up in the form of a list, providing a human-computer interaction interface for manual switching. Keywords can be entered to query the archiving strategy name, and fuzzy query is supported; Step S3.3, specify the directory to be archived, open the directory where the files to be archived are located, and the system will reinitialize all configuration properties of the current archiving strategy according to the selected archiving strategy, load all properties of the current archiving strategy and the archiving rule set contained therein; Step S3.4, specify the archive output directory, specify the archive output folder directory root directory, which can be a local disk or an external storage device path. When the specified folder directory does not exist, the system will automatically create the specified output directory; Step S3.5, scan and analyze the entire directory file of the directory to be archived, perform a full file scan on the directory to be archived, and accordingly establish a global memory file index table Tab_SouceFileIndex, and obtain a memory table of all file attribute index information. The attributes of the established memory data table include: file name, file path, file size, file MD5 checksum, pixel width, pixel height, total number of pages, archive directory, archive file name, archive file MD5 checksum, and archive status; Step S3.6, automatic archiving and sorting, according to the number of archiving rules N in the archiving strategy, create N archiving rule threads Tarchivethread tasks, and assign the attribute sets of 1..N archiving rules in the archiving strategy as member attributes to the corresponding 1..N threads, and start the multi-threaded archiving processing task, and at the same time create a global task counter TaskCounter to record the number of sub-thread tasks completed, TaskCounter is initialized to 0, and the archiving start time g_starttime is recorded as the current time; In one embodiment of the present application, the method for automatic archiving of electronic archives based on a custom archiving strategy, the step S3.6, automatic archiving and sorting includes: Step S3.6.1, create N Tarchivethread sub-threads, set thread attributes so that they self-destruct after completing the archiving sub-task; Step S3.6.2, traverse each Tarchivethread sub-thread and assign the rule property set to the member variable rule property set Rulepropertys; Step S3.6.3, each child thread Tarchivethread converts the property file rule expression of the rule corresponding to its Rulepropertys into the corresponding SQL statement, executes the query of the SQL statement, quickly retrieves and generates the list of files to be sorted under the archiving rule corresponding to the thread from the memory data index table Tab_SouceFileIndex to a new memory table, that is, the list of files to be archived Tab_SourcefileRule. At the same time, the attribute field recognition value Cx_value, recognition result Cx_state, archive file MD5 check code C_MD5, and archive file size Tab_SourcefileRule.C_FileSize of the new X feature areas are created for the data set Tab_SourcefileRule; Step S3.6.4, traverse all files of Tab_SourcefileRule, extract the bitmap of the feature block according to the feature block features, and perform text recognition through tesseract-ocr or barcode QR code engine, and assign the result of its text recognition to Cx_value, Cx_state of the dataset Tab_SourcefileRule; Step S3.6.5, obtain the archive directory, and generate the archive target directory AimDir according to the archive directory rule expression; Step S3.6.6, obtain the archive file name, and convert it into a new archive file name NewFileName according to the archive file naming rules; Step S3.6.7, archive the file at the target location. According to the file's archive directory and archive file name, copy the original file to the target location. After successful copying, set the successful status of the archive field arcivestate in Tab_SourcefileRule to 1. After the copy is completed, if the archive optimization option is set, the archived file is optimized to form the final archive file, and the final archive file verification code is obtained and updated to Tab_SourcefileRule.C_MD5 and the file size Tab_SourcefileRule.C_FileSize. Finally, the file archive update status message is sent through multi-threaded messages. After receiving the message, the main thread will refresh the progress bar in the log display area; Step S3.6.8, when all files in Tab_SourcefileRule are archived, update the archive results (archive directory, archive file name, archive file MD5 checksum, archive status field) to Tab_SouceFileIndex. The sub-thread task is completed, and a message is sent to the main thread in its before destruction event, the global variable TaskCounter in the main thread is incremented by 1, and the Tarchivethread sub-thread resources are released; Step S3.6.9, when TaskCounter=N is determined in the main thread (all archiving rule tasks are completed), if the option to create an index directory is set, the volume directory needs to be automatically generated according to the template style. After all operations are completed, record g_endtime and calculate the total archiving time T_totaltime=g_endtime-g_starttime.

[0007] Beneficial effects of this application: This method can realize automatic organization and management of files. It can scan files in a computer or network and classify and archive them into designated folders according to preset policy rules. Its beneficial effects are analyzed as follows: Save time: Its automated archiving software and methods can automatically perform tedious file management tasks, saving users time and energy. Users do not need to manually find, move and rename files, just set up rules and the software will automatically complete these tasks.

[0008] Improve efficiency: Through its automated archiving software and methods, digital archives can be organized faster and work efficiency can be improved. The software can classify and archive files according to file type, creation date, modification date and other attributes, making files more orderly and easy to retrieve and access to third-party archive management and archiving software systems.

[0009] Reduce errors: Manual archiving not only requires a lot of manpower, but is also prone to errors, such as accidentally deleting important files, moving files to the wrong folder, etc. Automated archiving software and methods can reduce the occurrence of these errors and ensure the security and accuracy of files.

[0010] Customizability: The automated archiving software and method are highly customizable, and users can set archiving rules according to their needs. This allows the method and the software to be adapted to a variety of different working environments and archiving requirements, and is suitable for factory production of archives in various industries, improving the production efficiency of digital archives. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0012] Figure 1 It is a working step diagram of the automated filing method; Figure 2 It is a logical diagram of a file feature identifier set and file identifier feature attributes; Figure 3 It is a workflow diagram for automated file archiving. DETAILED DESCRIPTION

[0013] The embodiment of the present application provides an automated archiving method for electronic archives based on a custom archiving strategy, which uses OCR recognition technology based on image recognition technology to quickly identify the classification type and document attributes of electronic documents, and realizes automated processing of electronic document classification and organization through automation technology, thereby reducing manual intervention in the digital archiving and organization of archives, reducing production costs, improving production efficiency, and reducing errors caused by manual participation.

[0014] The main implementation steps of the technical solution are as follows: 2. Prepare the digitized electronic files to be sorted and archived. Store the electronic documents generated by various types of digital document scanning equipment or existing electronic documents in the file system of the designated computer system with the archiving software system installed, which can access the media of the storage device, and prepare the electronic documents before archiving.

[0015] 2. Configure the corresponding archiving and sorting strategy according to the requirements of the digital achievement archiving. According to the specific requirements of the specific achievement archiving, configure a corresponding archiving strategy, and configure multiple archiving rules for one strategy. And store the archiving strategy and the set of configured archiving rules in the database of its automated archiving software system for persistent management and calling at any time to meet the task switching of different archiving requirements.

[0016] , start the automatic file archiving task according to the set archiving strategy and its rule sequence. Select the directory to be archived and the directory of the archive target output, load the corresponding archiving strategy to be used, start the sub-thread archiving task based on the number of rules, and start the fully automatic electronic file archiving.

[0017] 2. Check the integrity of the archive task. After the task threads of all the archiving rules of the strategy are completed, check the integrity of the archive, and at the same time count the OCR recognition accuracy and task completion performance indicator data of this archiving task.

[0018] , automatically generate the achievement archiving task report. Generate a PDF quality inspection report of the integrity check and related statistical data of this archiving task, and generate the digital signature information of the archiving report according to the rules to identify the uniqueness of the report.

[0019] , the archiving task is completed. An automated archiving task based on the selected archiving strategy is completed, and the browsing of archiving results, archiving report data viewing, and content and accuracy comparison of files before and after archiving are supported.

[0020] See also Figure 1 The main working steps of the above method are as follows: (1) Step S1: Preparation of electronic documents to be archived Massive electronic files in BMP, JPG, JPEG, PDF, OFD and other formats generated by batch digitization with scanning equipment are not archived and organized as required. Usually, it refers to the original digitized electronic files that are not archived and organized as required by the archiving directory and file organization rules, which are generated by semi-manual or automated scanning equipment in the production process of large-scale archive digitization projects, involving large volumes, diverse document types, and huge manual organization workload.

[0021] (2) Step S2: Configure archiving strategy A specific target type archiving result corresponds to a configuration of an archiving strategy. The configuration strategy consists of an archiving rule set. Multiple rules are configured in the configuration strategy to complete the archiving rules and file organization requirements of each file. The archiving strategy is maintained and updated through the strategy configuration function module in the system, and its configuration information is saved in the system database table. When executing archiving, the specified strategy solution can be loaded from the database to execute the archiving work.

[0022] The properties of an archiving strategy consist of the policy name, template file list (a list of sample files of various types in the documents to be organized), archiving instructions, references or standards, index directory creation options, archiving rule set (a collection of multiple archiving rules), and other properties.

[0023] Its properties are visually configured through the archiving policy configuration function module and saved in the system database table Tab_policy and subtables Tab_filetemplate and Tab_IndexRules of the archiving software system.

[0024] The template file list is used to select sample files of all categories of documents to be organized by setting this archiving strategy. The sample files are imported by opening the specified file directory, and their file information is stored in Tab_filetemplate.

[0025] The option of creating index directory is used in situations where you need to automatically generate the volume directory. It is an advanced option. The system does not create the index directory by default. If you need to automatically generate the volume directory file after finishing, you need to set an Excel template style for the volume directory, set the corresponding referenced template file name and other attribute change names in the cells in the template, and the system will automatically dynamically assign values ​​to generate the volume directory and its attributes.

[0026] Each sub-item in the archiving rule set is an archiving rule (Rule), and each archiving rule corresponds to the archiving rule definition of a type of file.

[0027] The archiving strategy described above is visually configured by its quality inspection strategy configuration function module, and dynamically maintained to meet the goal of archiving new types of achievements. The main steps of the configuration process are as follows: Step S2.1, create a new archiving policy and name it according to its archiving task.

[0028] For example: the new archiving strategy is named "Automatic archiving strategy for the results of the second round of extension".

[0029] Step S2.2, importing various types of files to be archived.

[0030] Import the type of files to be archived (support BMP, JPG, JPEG, PDF, OFD formats) into the current policy file template library. The system stores the imported files in the policy subfolder under the program running root directory, and they are T1..Tx according to the order of import. Tx represents the file template file T1, T2... At the same time, the main attributes of the imported file Tx will be identified and stored in the data table Tab_filetemplates. Its template file attributes include file identification name, file type (BMP, JPG, JPEG, PDF, OFD), page pixel width, page pixel height, number of pages (the number of pages of the template file is automatically read, but it can be edited. When it is manually set to 0, the number of pages is dynamically variable).

[0031] Step S2.3, configure the archiving rule set.

[0032] According to the archiving requirements, one or more rules (R1..Rx) are formulated to realize the automatic archiving and sorting of files. Usually, one rule corresponds to the archiving task of one type of file. When the same type of files need to be stored repeatedly in different target archiving directories, two or more archiving rules can be configured for one type of file.

[0033] The archiving rule includes attributes such as rule name, file source directory, corresponding template file (one of T1..Tx), page pixel width, page pixel height, number of pages, identification name, file matching rule expression, archiving directory, archive file naming rules, and file identification features.

[0034] File source directory: that is, define the directory information to be archived that this rule matches, that is, the relative file directory to the root directory of the files to be archived (the global variable $SourceRoot represents the root directory of the directory to be archived). For example, in the second round of extended archive archiving strategy, the rule name for the archiving of the farmer survey information table is R1, and its source directory attribute is configured as: $SourceRoot / farmer archives, which means that the rule only executes the archiving rule of rule R1 for the files under the "farmer archives" sub-file in the root directory.

[0035] Template file settings, select the template file corresponding to this rule. The drop-down list is the file list configured by the policy to which this rule belongs. After selecting one, the page pixel width, page pixel height, and number of pages attributes of this rule are automatically assigned values ​​from the selected template file Tx.

[0036] File matching rule expression, used to set the files to be matched by the policy. It supports wildcards, regular expressions, built-in functions and other expressions.

[0037] For example, the expression: *.pdf will match all files with the suffix .pdf in the target folder.

[0038] Another example is the expression: ext:pdf uses a function expression to match all files with the suffix .pdf in the target folder.

[0039] The archive directory is the archive directory that defines the archive files under the rule conditions. For example, the default target root directory is $AimRoot. Directory configuration supports changes, functions, and other forms to meet the application scenarios of dynamic directory compilation. For example, if the second round of extended farmer archives requires the use of farmer codes to generate dynamic folders, you can configure it like this: $AimRoot\Farmer Document\$C.value, which means that the value (contractor code) identified by the feature block in the identified file is used as the archive directory. $C.value represents the value identified by the feature block defined in the rule.

[0040] Archive file naming rules define whether archive files are renamed according to the rules. The default value is: $SourceName, that is, the file is not renamed and remains the same as the source file name. Supports multiple combinations of changes and functional renaming. For example, if the rule is configured as: $time, the current time is used as the file name. Another example is: $C1.value-C1.flagname, which means that the file name is in the form of "4302231921-basic table.pdf", that is, the dynamic file name of the identified {contractor code}-basic table.pdf.

[0041] A file feature identification set is a set of information that defines the feature identification area of ​​an electronic file to determine the relevant attributes of the document. A type of file can have multiple feature identification areas. For example, in the document of "Contractor Survey Form" in the second round of contractor extension archiving project in rural land management rights, in order to identify the two features of contractor code and contractor name in the survey form to generate a PDF file with the contractor code as the farmer's archive directory and the survey form archived by name, two file feature identification areas are set in its archive: C1 and C2. C1 is used to set the location area attribute of the contractor code, and C2 is used to set the location area attribute of the contractor name. The file feature identification area includes the following setting properties: page number Cx.pageindex, identification feature block area (Cx.left, Cx.top, Cx.width, Cx.height pixel rectangular area), feature area category Cx.type (printed text, handwritten text, one-dimensional code, two-dimensional code), feature area identification name Cx.cname, feature area identification value Cx.value (the setting value is the character sequence of the feature identification area after OCR recognition). For the file feature identification set and the file identification feature attribute logic diagram, see Figure 2 .

[0042] Archive optimization options provide options for document optimization such as noise reduction, document orientation correction, and file compression. By default, documents are not optimized.

[0043] Denoising mainly processes the archives in the following aspects: Decontamination and restoration of archives: Decontamination and restoration operations are performed on scanned archives. Stains, spots, tears and other defects on archives can be removed and repaired to improve the quality and readability of the image. Color correction: If the color of the archive is biased or faded, use the color correction tool to adjust the color balance, contrast and saturation to make the image look more realistic and clear. Denoising and sharpening: For noise or blur in the scanned image, you can use the denoising and sharpening tools to reduce noise and enhance the details and clarity of the image.

[0044] Document direction correction: correct the document direction according to the recognized text direction identified by the recognized text recognition feature area.

[0045] File compression, compress image files and reduce the resolution of PDF and OFD files to reduce the file size.

[0046] (3) Step S3: Automated archiving and organization For specific results to be archived, after configuring the corresponding archiving strategy according to the archiving requirements, you can automatically archive and organize them. Two modes of automatic archiving are supported: Mode 1: Link with automated scanners and other devices, set the scanner's scan output path as the directory to be archived, dynamically monitor changes in the directory files to be archived, and organize and classify the files according to the configured strategy.

[0047] Mode 2: Specify a static directory to be archived and organize and classify files according to the configured archiving strategy. This mode is for reclassifying scanned but unclassified electronic files.

[0048] There is no difference in technical implementation between the two modes. The main process steps are as follows (see Figure 3 ): Step S3.1, system startup. Connect to the system standard database SYSDB, initialize global option parameters, initialize the interface, initialize the OCR engine (load printed and handwritten sample data sets), initialize the barcode recognition engine, load the log message module, and complete the system user authorization authentication (user login), etc.

[0049] Step S3.2, select the current archiving strategy. By default, the system loads the most recently used archiving strategy and displays it in the status bar. The archiving strategy can be switched as needed. When switching, it will pop up in the form of a list, providing a human-computer interaction interface for manual switching. Keywords can be entered to query the archiving strategy name, and fuzzy query is supported.

[0050] Step S3.3, specify the directory to be archived. Open the directory where the files to be archived are located, and the system will reinitialize all configuration properties of the current archiving strategy according to the selected archiving strategy, and load all properties of the current archiving strategy and the archiving rule set included.

[0051] Step S3.4, specify the archive output directory. Specify the root directory of the archive output folder directory, which can be a local disk or an external storage device path. When the specified folder directory does not exist, the system will automatically create the specified output directory.

[0052] Step S3.5, scan and analyze the entire directory file of the directory to be archived. Perform a full file scan on the directory to be archived, and establish a global memory file index table Tab_SouceFileIndex accordingly to obtain a memory table of all file attribute index information. The attributes of the established memory data table include: file name, file path, file size, file MD5 checksum, pixel width, pixel height, total number of pages, archive directory, archive file name, archive file MD5 checksum, and archive status.

[0053] Step S3.6, automatic archiving and sorting. According to the number of archiving rules N in the archiving strategy, create N archiving rule threads Tarchivethread tasks, and assign the attribute sets of 1..N archiving rules in the archiving strategy as member attributes to the corresponding 1..N threads, and start the multi-threaded archiving processing task. At the same time, create a global task counter TaskCounter to record the number of sub-thread tasks completed, TaskCounter is initialized to 0, and the archiving start time g_starttime is recorded as the current time. The sub-process steps are as follows: Step S3.6.1, create N Tarchivethread child threads, and set them to self-destruct after completion; Step S3.6.2, traverse each Tarchivethread subthread and assign the rule property set (rule name, file source directory, corresponding template file, page pixel width, page pixel height, number of pages, identification name, file matching rule expression, archive directory, archive file naming rule, file identification feature) to the member variable rule property set Rulepropertys; Step S3.6.3, each child thread Tarchivethread converts the property file rule expression of the rule corresponding to its Rulepropertys into the corresponding SQL statement, and executes the SQL statement to quickly retrieve and generate the list of files to be sorted under the archiving rule corresponding to the thread from the memory data index table Tab_SouceFileIndex to a new memory table, that is, the list of files to be archived Tab_SourcefileRule. At the same time, the attribute field identification value Cx_value, identification result Cx_state, archive file MD5 check code C_MD5, and archive file size Tab_SourcefileRule.C_FileSize of new X feature areas are created for the data set Tab_SourcefileRule.

[0054] Step S3.6.4, traverse all files of Tab_SourcefileRule, extract the bitmap of the feature block according to the feature block features, and perform text recognition through the tesseract-ocr engine, and assign the text recognition results to Cx_value and Cx_state of the dataset Tab_SourcefileRule. Loop through each feature recognition block C1..Cx, the detailed steps are as follows: ① Create 8bit Tcharacterbitmap according to the size of Cx.width and Cx.height; ② Get the bitmap TIndexpageBitmap of the page Cx.pageindex where the identification feature of the file to be archived is located. When the file to be archived is a PDF or OFD file, it is necessary to obtain the content of the Cx.pageindex of the PDF or OFD file, convert it into a bitmap format and assign it to TIndexpageBitmap.

[0055] ③Copy the pixel rectangular area corresponding to the identification feature block area (Cx.left, Cx.top, Cx.width, Cx.height) in TindexpageBitmap to the bitmap Tcharacterbitmap; ④Call the tesseract-ocr engine to recognize Tcharacterbitmap. When the recognition is successful, the recognized result string is assigned to Tab_SourcefileRule.Cx_value, and Tab_SourcefileRule.Cx_state is assigned to 1; when it fails, Tab_SourcefileRule.Cx_state is assigned to 0; Step S3.6.5, obtain the archive directory, according to the archive directory rule expression, generate the archive target directory AimDir; Step S3.6.6, obtain the archive file name, and convert it into a new archive file name NewFileName according to the archive file naming rules.

[0056] Step S3.6.7, archive the file at the target location. According to the file's archive directory and archive file name, copy the original file to the target location. After the copy is successful, set the successful status of the archive field arcivestate in Tab_SourcefileRule to 1. After the copy is completed, if the archive optimization option is set, the archived file is optimized to form the final archive file, and the final archive file verification code is obtained and updated to Tab_SourcefileRule.C_MD5, the file size Tab_SourcefileRule.C_FileSize, and finally the file archive update status message is sent through multi-threaded messages. After receiving the message, the main thread will refresh the progress bar in the log display area.

[0057] Step S3.6.8: When all files in Tab_SourcefileRule are archived, the archive results (archive directory, archive file name, archive file MD5 checksum, archive status field) are updated to Tab_SouceFileIndex. The sub-thread task is completed, and a message is sent to the main thread in its before-destruction event. The global variable TaskCounter in the main thread is incremented by 1, and the Tarchivethread sub-thread resources are released.

[0058] Step S3.6.9, when TaskCounter=N is determined in the main thread (all archiving rule tasks are completed), if the option to create an index directory is set, the volume directory needs to be automatically generated according to the template style. After all operations are completed, record g_endtime and calculate the total archiving time T_totaltime=g_endtime-g_starttime.

[0059] (4) Step S4: Archiving task integrity check Check the integrity of the archiving task. Check whether the size of the source file directory to be archived is equal to the size of the target directory to be archived, whether the total number of source files to be archived is consistent with the number of files in the target directory to be archived, count Tab_SouceFileIndex, calculate the file OCR recognition rate rr, calculate the number of archived files and the archiving success rate ar, and obtain the number of duplicate files in the archived files and a detailed list.

[0060] rr=(number of successful character recognition in character recognition feature areas / number of character recognition feature areas)*100%.

[0061] ar=(number of files successfully archived / number of electronic files to be archived)*100%.

[0062] For OCR recognition failures, manual character annotation can be performed one by one, and the training data results can be incrementally written into the recognition data set to improve the text recognition accuracy.

[0063] (5) Step S5: Task completion report Generate an archiving task report, including the total file size of the archive directory, the total number of files, the number of archived files, the number of duplicate files in the archived files and the duplicate file list, file type classification, file feature recognition rate, number of archived files, archiving success rate, archiving start time, archiving end time, total time and other report information. The archive directory is automatically generated and displayed in the report display area in the form of PDF, and the PDF of the report generation digital signature information is output to the upper directory of the archive output root directory, which is named: "Archiving strategy name-report generation time (YYYYMMDDHHMMSS).pdf".

[0064] The digital signature content is a 16-bit MD5 hash value as a report uniqueness check to prevent external tampering. Its generation rule is: Get the report date filedate, get the total file size of the source directory sourcefilesize, get the total file size of the directory after archiving aimfileszie, and convert the above three changed values ​​into character changes s1, s2, s3. The final signature digital string SN = MD5 ("s1" + "," + s2 + "," + s3).

[0065] (6) Step S6: Archiving task completed Complete an automated archiving task based on the selected archiving strategy. Display the archived files in the system in the form of an electronic archive directory tree after archiving, and support visual browsing of archived files for manual instant query or quality verification after processing. At the same time, you can view the archive quality inspection report generated after the completion of this task. At the same time, for those who have set the document optimization option, you can compare and browse the content and accuracy of the files before and after archiving.

[0066] The technical innovations of the above method are as follows: 1. High adaptability of archive archiving. Dynamic configuration based on rules and policies makes it widely adaptable and can meet the requirements of different industries for specific digital archive archiving.

[0067] Second, high degree of automation. Its archiving mode is based on computer automation processing, without any human involvement, and is suitable for the automated archiving factory production mode of engineering massive digital archives.

[0068] 3. High processing efficiency. On the one hand, the archive directory file list is formed to establish a file index memory data table. All calculations and analyses are based on the analysis of the SQL query language of the inner data table. The acquisition of file attributes is only performed once. The file matching rules in the subsequent rules are converted into SQL through expressions and then efficiently retrieved based on the memory table, making the file matching and internal system retrieval classification efficient. On the other hand, the file archiving process is based on the multi-threaded implementation of the rule set, making full use of the current mainstream multi-CPU multi-core concurrent processing capabilities to improve the automation processing efficiency of archiving processing.

[0069] 4. High degree of intelligence. The recognition engine Tesseract used in its core OCR technology application has highly accurate text recognition capabilities. It supports more than a variety of languages ​​and is based on an AI learning technology framework to adapt to new fonts and languages, especially in the training of handwritten text recognition. Its deep learning-based model uses convolutional neural networks (CNNs) for image preprocessing and character recognition, especially in complex backgrounds and handwriting recognition. It has high accuracy, reflecting its intelligence.

[0070] Beneficial effects: This method can realize automatic organization and management of files, scan files in computers or networks, and classify and archive them into designated folders according to preset policy rules. Its beneficial effects are analyzed as follows: Save time: Its automated archiving software and methods can automatically perform tedious file management tasks, saving users time and energy. Users do not need to manually find, move and rename files, just set up rules and the software will automatically complete these tasks.

[0071] Improve efficiency: Through its automated archiving software and methods, digital archives can be organized faster and work efficiency can be improved. The software can classify and archive files according to file type, creation date, modification date and other attributes, making files more orderly and easy to retrieve and access to third-party archive management and archiving software systems.

[0072] Reduce errors: Manual archiving not only requires a lot of manpower, but is also prone to errors, such as accidentally deleting important files, moving files to the wrong folder, etc. Automated archiving software and methods can reduce the occurrence of these errors and ensure the security and accuracy of files.

[0073] Customizability: The automated archiving software and method are highly customizable, and users can set archiving rules according to their needs. This allows the method and the software to be adapted to a variety of different working environments and archiving requirements, and is suitable for factory production of archives in various industries, improving the production efficiency of digital archives.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An automated electronic archive archiving method based on a custom archiving strategy, characterized in that: The steps include: Step S1: Preparation of electronic documents to be archived: Store electronic documents generated by various types of digital document scanning equipment or existing electronic documents in the file system of a designated computer installed with an archiving software system; Step S2: Configure archiving strategy: According to the specific archiving requirements of different digital achievements, corresponding archiving and sorting strategies are configured. One sorting strategy is configured with multiple archiving rules. The archiving strategy and the configured archiving rule set are stored in the archiving software system database. The archiving strategy is maintained and updated through the strategy configuration function module in the archiving software system. When archiving, the specified strategy can be loaded from the database to perform archiving work. Step S3: Execute the archiving task according to the archiving strategy: For the results to be archived, after configuring the corresponding archiving strategy according to the archiving requirements, the archived results are automatically archived and sorted. Two modes of automatic archiving are supported: Mode 1: Link with the automated scanner device, set the scanner's scan output path as its to-be-archived directory, dynamically monitor changes in the to-be-archived directory files, and sort and classify the files according to the configured strategy; Mode 2: Specify a static directory to be archived and sort and classify the files according to the configured archiving strategy. This mode is for reclassifying the scanned but unclassified electronic files. Step S4: Check the integrity of the archive task: Check whether the size of the source file directory to be archived is equal to the size of the target directory file to be archived, check whether the total number of source files to be archived is consistent with the number of files in the target directory file to be archived, count Tab_SouceFileIndex, calculate the file OCR recognition rate rr, calculate the number of archived files and the archiving success rate ar, obtain the number of duplicate files in the archived files and a detailed list; rr=(number of successful character recognition in character recognition feature areas / number of character recognition feature areas)*100%; ar=(number of successfully archived files / number of electronic files to be archived)*100%; For OCR recognition failures, we manually annotate each character one by one, and write the training data results incrementally into the recognition data set to improve the text recognition accuracy. Step S5: Generate an archiving task report: Generate an archiving task report, and automatically generate an archiving directory to display in the report display area in PDF format. At the same time, the PDF of the report with digital signature information is output to the directory above the archive output root directory. Step S6: Archiving task ends: After completing an automated organization based on the selected archiving strategy, the archived files are displayed in the archiving software system in the form of an archived electronic archive directory tree to facilitate manual review of the archived files.

2. The method for automatically archiving electronic files based on a custom archiving strategy according to claim 1, characterized in that: In the step S2, configuring the archiving strategy, the attributes of the archiving strategy include a strategy name, a template file list, an archiving description, references or standards, an index directory creation option, and an archiving rule set; The attributes are visually configured through the archiving policy configuration function module and saved in the system database table Tab_policy and subtables Tab_filetemplate and Tab_IndexRules of the archiving software system; The template file list is used to select sample files of all categories of documents to be organized by setting this archiving strategy. The sample files are imported by opening the specified file directory, and their file information is stored in Tab_filetemplate. The option of creating index directory is used when you need to automatically generate the directory in the volume. It is an advanced option. The system does not create the index directory file by default. If you need to automatically generate the volume directory after finishing, set an Excel template style for the volume directory, set the corresponding referenced template file name and other attribute change names in the cells in the template, and the system will automatically dynamically assign values ​​to generate the volume directory and its attributes, including file name information and page number information; Each sub-item in the archiving rule set is an archiving rule, and each archiving rule corresponds to the archiving rule definition of a type of file.

3. The method for automatically archiving electronic files based on a custom archiving strategy according to claim 1, characterized in that: The step S2, configuring the archiving strategy, comprises: Step S2.1, create a new archiving strategy and name it according to its archiving task; Step S2.2, import various types of files to be archived, import the types of files to be archived into the current policy file template library, the system stores the imported files in the policy subfolder under the program running root directory, and according to the order of import, they are T1..Tx; Tx represents the file template file T1, T2...; at the same time, the main attributes of the imported file Tx are identified and stored in the data table Tab_filetemplates; its template file attributes include file identification name, file type, page pixel width, page pixel height, and number of pages; Step S2.3, configure the archiving rule set, formulate one or more rules (R1..Rx) according to the archiving requirements, so as to realize the automatic archiving and sorting of files. Usually, one rule corresponds to the archiving task of one type of file. When the same type of files need to be repeatedly stored in different target archiving directories, two or more archiving rules can be configured for one type of file; The attributes of the archiving rule include the rule name, file source directory, corresponding template file (one of T1..Tx), page pixel width, page pixel height, number of pages, identification name, file matching rule expression, archiving directory, archive file naming rules and file identification features.

4. The method for automatically archiving electronic files based on a custom archiving strategy according to claim 1, characterized in that: Step S3, executing the archiving task according to the archiving strategy includes: Step S3.1, the system starts, connects to the system standard database SYSDB, initializes global option parameters, initializes the interface, initializes the OCR engine, initializes the barcode and QR code recognition engine, loads the log message module, and completes the authorization and authentication of system users; Step S3.2, select the current archiving strategy. By default, the system loads the most recently used archiving strategy and displays it in the status bar. The archiving strategy can be switched as needed. When switching, it will pop up in the form of a list, providing a human-computer interaction interface for manual switching. Keywords can be entered to query the archiving strategy name, and fuzzy query is supported; Step S3.3, specify the directory to be archived, open the directory where the files to be archived are located, and the system will reinitialize all configuration properties of the current archiving strategy according to the selected archiving strategy, load all properties of the current archiving strategy and the archiving rule set contained therein; Step S3.4, specify the archive output directory, specify the archive output folder directory root directory, which can be a local disk or an external storage device path. When the specified folder directory does not exist, the system will automatically create the specified output directory; Step S3.5, scan and analyze the entire directory file of the directory to be archived, perform a full file scan on the directory to be archived, and accordingly establish a global memory file index table Tab_SouceFileIndex, and obtain a memory table of all file attribute index information. The attributes of the established memory data table include: file name, file path, file size, file MD5 checksum, pixel width, pixel height, total number of pages, archive directory, archive file name, archive file MD5 checksum, and archive status; Step S3.6, automated archiving and sorting, creates N archiving rule threads Tarchivethread tasks according to the number of archiving rules N in the archiving strategy, and assigns the attribute sets of 1..N archiving rules in the archiving strategy as member attributes to the corresponding 1..N threads, and starts the multi-threaded archiving processing task. At the same time, a global task counter TaskCounter is created to record the number of completed sub-thread tasks. TaskCounter is initialized to 0, and the archiving start time g_starttime is recorded as the current time.

5. The method for automatically archiving electronic files based on a custom archiving strategy according to claim 4, characterized in that: The step S3.6, automated filing and sorting includes: Step S3.6.1, create N Tarchivethread sub-threads, set thread attributes so that they self-destruct after completing the archiving sub-task; Step S3.6.2, traverse each Tarchivethread sub-thread and assign the rule property set to the member variable rule property set Rulepropertys of the sub-thread; Step S3.6.3, each child thread Tarchivethread converts the property file rule expression of the rule corresponding to its Rulepropertys into the corresponding SQL statement, executes the query of the SQL statement, quickly retrieves and generates the list of files to be sorted under the archiving rule corresponding to the thread from the memory data index table Tab_SouceFileIndex to a new memory table, that is, the list of files to be archived Tab_SourcefileRule; at the same time, the attribute field recognition value Cx_value, recognition result Cx_state, archive file MD5 check code C_MD5, and archive file size Tab_SourcefileRule.C_FileSize of new X feature areas are created for the data set Tab_SourcefileRule; Step S3.6.4, traverse all files of Tab_SourcefileRule, extract the bitmap of the feature block according to the feature block features, and perform text recognition through tesseract-ocr or barcode QR code engine, and assign the result of its text recognition to Cx_value, Cx_state of the dataset Tab_SourcefileRule; Step S3.6.5, obtain the archive directory, and generate the archive target directory AimDir according to the archive directory rule expression; Step S3.6.6, obtain the archive file name, and convert it into a new archive file name NewFileName according to the archive file naming rules; Step S3.6.7, archive the file at the target location. According to the file's archive directory and archive file name, copy the original file to the target location. After successful copying, set the successful status of the archive field arcivestate in Tab_SourcefileRule to 1. After the copy is completed, if the archive optimization option is set, the archived file is optimized to form the final archive file, and the final archive file verification code is obtained and updated to Tab_SourcefileRule.C_MD5 and the file size Tab_SourcefileRule.C_FileSize. Finally, the file archive update status message is sent through multi-threaded messages. After receiving the message, the main thread will refresh the progress bar in the log display area; Step S3.6.8, when all files in Tab_SourcefileRule are archived, update the archive results (archive directory, archive file name, archive file MD5 checksum, archive status field) to Tab_SouceFileIndex. The sub-thread task is completed, and a message is sent to the main thread in its before destruction event, the global variable TaskCounter in the main thread is incremented by 1, and the Tarchivethread sub-thread resources are released; Step S3.6.9, when TaskCounter=N is determined in the main thread (all archiving rule tasks are completed), if the option to create an index directory is set, the volume directory needs to be automatically generated according to the template style. After all operations are completed, record g_endtime and calculate the total archiving time T_totaltime=g_endtime-g_starttime.

Images