Automatic annotation method on OFD files through machine learning

By applying machine learning automatic labeling method on OFD files, the problem of relying on manual labeling for OFD files in the prior art is solved, and automated labeling is realized, reducing management costs and improving file management efficiency.

CN118734812BActive Publication Date: 2024-11-12JIANGSU ZHONGWEI TECH SOFTWARE SYST
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411215926.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-02
Publication Date
2024-11-12
Estimated Expiration
2044-09-02

AI Technical Summary

Technical Problem

In the prior art, the labeling and classification of OFD files mainly relies on manual labor and cannot be automated, resulting in high management costs and unavailable after the labeling information is separated from the management system, and it is impossible to automatically manage the file content.

Method used

By applying machine learning automatic labeling methods on OFD files, including defining file entities and relationships, parsing XML files, generating automated labeling models, extracting and adding entity and relationship information to OFD expansion files, using the model for automatic labeling, verifying the labeling effect , reduce the workload of manual labeling.

Benefits of technology

It realizes automatic labeling of OFD files, reduces manual labeling costs, improves the efficiency of file management and circulation systems, can quickly obtain file feature information, and reduces the manual labeling workload of key information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118734812B_ABST
    Figure CN118734812B_ABST
Patent Text Reader

Abstract

The present invention proposes a method for automatic labeling on OFD files through machine learning. Key information is pre-made when the application system is constructed, and is written into the OFD extended file package through the application system, so that the OFD file produced by the application system carries the file feature information that the user needs to train. This technology enables automatic learning to lay a solid foundation for machine learning. The present invention fully utilizes the structural advantages of OFD files, quickly obtains the feature information of the files, and automatically carries out learning, which greatly reduces the cost of manual labeling of key information. The labeling results of the OFD files are directly saved in the extended file, and the user can quickly locate them when searching for file-related content, which also improves the efficiency of file maintenance and labeling in the later stage.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and in particular to a method for automatically marking OFD files through machine learning. Background Art

[0002] On October 14, 2016, World Standards Day, the National Standardization Administration of China officially approved and issued the national standard GB / T 33190-2016 "Electronic File Storage and Exchange Format Layout Document" (OFD for short) based on independent technology. OFD, the abbreviation of OpenFixed-layout Document, is my country's independent and controllable electronic file layout document format, and the corresponding international standard is PDF. OFD layout files have fixed layouts, no layout changes, and what you see is what you get. They can be regarded as the "digital paper" of the computer age; they are the ideal document format for electronic document publishing, digital information dissemination and archiving.

[0003] In the prior art, OFD files are manually tagged and identified and classified. OFD file information is annotated in the file management and circulation system and classified according to the tag category. It is impossible to manage the files in a tagged manner based on the file content information itself. The annotated information of the files cannot be obtained after being separated from the corresponding management system, thereby increasing the management cost. Summary of the invention

[0004] The purpose of the present invention is to provide a method for automatic annotation on OFD files through machine learning to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a method for automatic annotation on OFD files by machine learning, characterized in that it comprises the following steps:

[0006] Step S1: define the entities and the relationships between the entities in the content of different types of files;

[0007] Step S2: Read the OFD file, parse the XML file in the OFD file, and determine whether the file complies with the standard;

[0008] Step S3: When the number of read files reaches a preset value, the content information of entities and relationships in the extracted OFD files is processed to generate the data format required for automatic annotation model training;

[0009] Step S4: Match the text content in the file with the entity relationship information one by one, import the processed data into the model for training, and generate an automatic annotation model;

[0010] Step S5: Customize the entity, relationship and other information of different types of files in the OFD file application system, and add a device for writing entity and relationship information in the application system. After reading the file content, use the automatic annotation model to extract entity and relationship information, and add the extracted entity relationship information to the OFD extension file.

[0011] Step S6: extracting standard value and extended value data of OFD file entities and relationship information in the file application system, processing these data into model training data format, and importing them into the automatic annotation model for training;

[0012] Step S7: Use the automated annotation model to annotate the unprocessed files and verify the automated annotation effect of the model.

[0013] Preferably, the definition in step S1 includes entity and relationship information such as index, document number, classification, title, issuance time, document source, unit name, and subordinate relationship between units.

[0014] Preferably, the standards that the file is judged to comply with in step S2 include but are not limited to national standards, industry standards, local standards and custom standards.

[0015] Preferably, if it is necessary to extract information content other than the defined standard entities and relationships in step S5, the extended information to be extracted is preset in the file system, and the extended information is filled into the OFD extended file through the writing device.

[0016] Preferably, when verifying the automatic labeling effect of the model, the labeling results of the automatic labeling model are compared with the manual labeling results, and the accuracy of the labeling of the automatic labeling model is calculated.

[0017] Preferably, the manual annotation result is annotated manually by manually annotating the file. During the manual annotation, the entity category in the extended file can be selected for confirmation, and then the annotation content corresponding to the entity category in the OFD file is added to the extended file.

[0018] Preferably, after the accuracy of the annotation calculated by the automatic annotation model reaches a preset value, the automatic annotation model starts working, and the automatic annotation model parses the OFD file to determine whether there is a corresponding extended information file in the OFD file.

[0019] Preferably, when judging whether there is a corresponding extended information file for the OFD file, if there is a corresponding extended information file for the OFD file, the extended file information content in the OFD file is read, and it is identified whether the annotation content corresponding to the extended file entity class is complete, and the result of the automatic annotation model recognition is saved in the extension file.

[0020] Preferably, when determining whether there is a corresponding extended information file for the OFD file, if there is no corresponding extended information file for the OFD file, or the entity category information of the extended file is incomplete or empty, the OFD file content is read, and the content in the OFD file is analyzed by an automatic annotation model to extract the entity category and the text content and position corresponding to the entity category, and the result of the automatic annotation model recognition is saved in the extension file.

[0021] Preferably, according to the result of the automatic annotation model recognition, when reading with the OFD reader, the automatic annotation result of the automatic annotation model is displayed in the metadata of the OFD file according to the entity name and content.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] (1) The present invention prefabricates key information when building the application system, and writes it into the OFD extension file package through the application system, so that the OFD file produced by the application system carries the file feature information that the user needs to train. This technology enables automatic learning, laying a solid foundation for machine learning;

[0024] (2) The present invention makes full use of the structural advantages of OFD files, quickly obtains the feature information of the files, and automatically conducts learning, which greatly reduces the cost of manual labeling of key information. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Flowchart of the present invention DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Example

[0027] The method for automatic annotation on OFD files by machine learning is characterized by comprising the following steps:

[0028] Step S1: define entities and relationships between entities in different types of documents, where the definition includes index, document number, classification, title, date of issue, document source, unit name, subordinate relationship between units and other entity and relationship information;

[0029] Step S2: reading the OFD file, parsing the XML file in the OFD file, and determining whether the file complies with the standard, wherein the standard includes but is not limited to the national standard, the industry standard, the local standard, and the custom standard;

[0030] Step S3: When the number of read files reaches a preset value, the content information of entities and relationships in the extracted OFD files is processed to generate the data format required for automatic annotation model training;

[0031] Step S4: Match the text content in the file with the entity relationship information one by one, import the processed data into the model for training, and generate an automatic annotation model;

[0032] Step S5: in the OFD file application system, the entity, relationship and other information of different types of files are customized, and a device for writing entity, relationship and other information is added in the application system. If it is necessary to extract information content other than the defined standard entities and relationships, the extended information to be extracted is preset in the file system, and the extended information is filled into the OFD extended file through the writing device. After reading the file content, the entity and relationship information is extracted using the automatic annotation model, and the extracted entity relationship information is added to the OFD extended file;

[0033] Step S6: extracting standard value and extended value data of OFD file entities and relationship information in the file application system, processing these data into model training data format, and importing them into the automatic annotation model for training;

[0034] Step S7: annotate the unprocessed files using the automated annotation model. The manual annotation results are annotated manually. During the manual annotation, the entity category in the extended file can be selected for confirmation. Then, the annotation content corresponding to the entity category in the OFD file is added to the extended file to verify the automated annotation effect of the model.

[0035] Step S8: When verifying the automatic annotation effect of the model, the annotation results of the automatic annotation model are compared with the manual annotation results, and the accuracy of the annotation of the automatic annotation model is calculated;

[0036] Step S9: After the accuracy of the automated annotation model reaches a preset value, the automated annotation model is used to parse the OFD file and determine whether there is a corresponding extended information file for the OFD file. If so, the process proceeds to step A; if not, the process proceeds to step B.

[0037] Step A: when determining whether there is a corresponding extended information file in the OFD file, if there is a corresponding extended information file in the OFD file, read the extended file information content in the OFD file, identify whether the annotation content corresponding to the extended file entity class is complete, and save the result of the automatic annotation model recognition in the extension file;

[0038] Step B: When determining whether there is a corresponding extended information file for the OFD file, if there is no corresponding extended information file for the OFD file, or the entity category information of the extended file is incomplete or empty, read the content of the OFD file, analyze the content in the OFD file through the automatic annotation model, extract the entity category and the text content and position corresponding to the entity category, and save the results of the automatic annotation model recognition to the extension file.

[0039] While the OFD file is being circulated and read in the system, the content information in the OFD file is identified, the file content is identified and classified, and the classification results are annotated to the OFD extension file, reducing the workload of manual classification and annotation. The annotation results of the OFD file are directly saved in the extension file, allowing users to quickly locate the file when searching for file-related content, which also improves the efficiency of later file maintenance and labeling.

[0040] Step S10: Based on the result of the automatic annotation model recognition, when reading with the OFD reader, the automatic annotation result of the automatic annotation model is displayed in the metadata of the OFD file according to the entity name and content.

[0041] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A method for automatic annotation on OFD files by machine learning, characterized in that: The following steps are involved: Step S1: define the entities and the relationships between the entities in the content of different types of files; Step S2: Read the OFD file, parse the XML file in the OFD file, and determine whether the file complies with the standard; Step S3: When the number of read files reaches a preset value, the content information of entities and relationships in the extracted OFD files is processed to generate the data format required for automatic annotation model training; Step S4: Match the text content in the file with the entity relationship information one by one, import the processed data into the model for training, and generate an automatic annotation model; Step S5: Customize the entity and relationship information of different types of files in the OFD file application system, add a device for writing entity and relationship information in the application system, extract entity and relationship information using the automatic annotation model after reading the file content, and add the extracted entity relationship information to the OFD extension file; Step S6: extracting standard value and extended value data of OFD file entities and relationship information in the file application system, processing these data into model training data format, and importing them into the automatic annotation model for training; Step S7: Use the automated annotation model to annotate the unprocessed files and verify the automated annotation effect of the model.

2. The method for automatic annotation on OFD files by machine learning according to claim 1, characterized in that: The definition in step S1 includes index, document number, classification, title, issuance time, document source, unit name, subordinate relationship entities and relationship information between units.

3. The method for automatic annotation on OFD files by machine learning according to claim 1, characterized in that: The standards that the file is judged to comply with in step S2 include national standards, industry standards, local standards and custom standards.

4. The method for automatic annotation on OFD files by machine learning according to claim 1, characterized in that: If it is necessary to extract information content other than the defined standard entities and relationships in step S5, the extended information to be extracted is preset in the file system, and the extended information is filled into the OFD extended file through the writing device.

5. The method for automatic annotation on OFD files by machine learning according to claim 1, characterized in that: When verifying the automatic labeling effect of the model, the labeling results of the automatic labeling model are compared with the manual labeling results, and the accuracy of the labeling of the automatic labeling model is calculated.

6. The method for automatic annotation on OFD files by machine learning according to claim 5, characterized in that: The manual annotation result is annotated by manually annotating the file. During the manual annotation, the entity category in the extension file can be selected for confirmation, and then the annotation content corresponding to the entity category in the OFD file is added to the extension file.

7. The method for automatic annotation on OFD files by machine learning according to claim 5, characterized in that: After the accuracy of the annotation calculated by the automatic annotation model reaches a preset value, the automatic annotation model starts working, and the automatic annotation model parses the OFD file to determine whether there is a corresponding extended information file in the OFD file.

8. The method for automatic annotation on OFD files by machine learning according to claim 7, characterized in that: When determining whether there is a corresponding extended information file for the OFD file, if there is a corresponding extended information file for the OFD file, read the extended file information content in the OFD file, identify whether the annotation content corresponding to the extended file entity class is complete, and save the result of the automatic annotation model recognition in the extension file.

9. The method for automatic annotation on OFD files by machine learning according to claim 7, characterized in that: When determining whether an OFD file has a corresponding extended information file, if the OFD file does not have a corresponding extended information file, or the entity category information of the extended file is incomplete or empty, the OFD file content is read, and the content in the OFD file is analyzed by the automatic annotation model to extract the entity category and the text content and position corresponding to the entity category, and the result of the automatic annotation model recognition is saved in the extension file.

10. The method for automatic annotation on OFD files by machine learning according to claim 7, characterized in that: According to the result of the automatic annotation model recognition, when reading with the OFD reader, the automatic annotation result of the automatic annotation model is displayed in the metadata of the OFD file according to the entity name and content.